[dandraper]

Searchable encryption means you can have encryption and queries (from apps etc) still work for authorized users.

This is all started when I was the CTO of a health-tech and the engineers all had access to patient data. They needed DB access (data migrations, managing backups, reading non-sensitive fields) but not access to the sensitive values. 10m+ patient records - it made me nervous.

Row-level Security (RLS) was a nightmare, it was slow, easy to mess up and could be overridden by DBAs anyway. Plus the application connected to the DB using a service account so RLS was useless for end-user access control.

CipherStash solves the following:

* Hide sensitive data from direct database accessors (authorized or not!)

My original use-case. Direct access to the DB doesn't reveal sensitive data (unless the user also has permission to decrypt a specific record).

* Support or even replace application access controls

Decryption happens in the application layer and is based on end-user identity. That means a gap in application defenses is mitigated by the encryption. If the user can't decrypt the value then they can't access the data.

* Access auditing

Every sensitive data access is recorded (and by who) which is very useful for compliance and incident response. SQL query auditing is great for knowing what queries were run but it doesn't tell you what data was actually returned. Its also blind once data actually leaves the DB and can inadvertently leak sensitive data itself. Using key accesses with non-identifying value IDs to audit decryptions makes the auditing more reliable and super granular.

We have customers who use the tech to prove to their users (often large enterprise) that no internal staff can access the data.

* Agent access to data

The encryption shifts access control decisions down to the data layer on a per-value basis. This is incredibly useful for gating access to agents that need access to data. Every decryption requires authentication (e.g via Supabase Auth, Auth0 etc) and combining it with agent identities (creds issued to agents) means you can not only limit what agents can access, you can also audit exactly what _was_ accessed.

The searchable encryption is the enabler: value level encryption is what makes all of this possible. Searchable encryption means you can have encryption and queries (from apps etc) still work for authorized users.

Hope that helps! I'm so in the weeds I don't know if I explain things that well sometimes!

[catmanjan]

On auditing, is that CipherStash or the application doing the auditing? Seems like your logs would be huge if it contained values of queries

[dandraper]

CipherStash - specifically the key service. There is a lot of data for sure but we only record an identifier for each value and (optionally) the user ID. It compresses well.

[catmanjan]

So an identifier is kept for each column returned by a query?

If the value changes would you be able to get from the id you have audited back to the value that was returned at that point in time?

[dandraper]

Not just each column, each value.

The answer to your question is yes.

Explanation: The identifier is actually for the key that encrypts the value (1 unique key per value).

1. When the value is encrypted for the first time, it gets an ID. 2. When its decrypted, the application requests the key for that ID from the key-server (key-server records that the data was accessed)* 3. When updating, the same data key is used so the ID is persistent*

* Technically the key server doesn't return a key, it generates partial key material that can be used to derive the data key in the app. It can do this at up to 10,000 keys per second. * You can also tag each value with the table/column name and the row-id to link everything together

The 3 values form the "descriptor" of a value: `table/column/id`.

[catmanjan]

So the audit log contains an ID for a key which could decrypt a value - does the audit log also contain the encrypted value itself? If not, how do you go from the audit log back to the original value in 1?

[dandraper]

No, not by default. You could but as you said, that would be a A LOT of data.

It depends on your setup. If you're using Supabase, one way is to send the logs to Clickhouse and use the Clickhouse partner integration to query the audit logs and join it to the actual data.

Keeping only the ids in the audit log means you need to stitch the data together later. This means you don't accidentally leak data via your audit trail!