A referentially intact slice of production, without the PII

Two requirements get sent to staging and they are not the same requirement.

The first is a compliance requirement: no real customer row may sit in an environment that developers and CI and a half-dozen third-party tools can reach. You solve it by masking, which means replacing the identifying columns with substitutes so the database keeps its shape and loses its names.

The second is a testing requirement: the database has to be big enough and strange enough that a query plan or a migration behaves the way it will behave against the full production set. A twenty-row fixture passes that requirement in the same way a three-row fixture passes a uniqueness test, which is to say not at all.

Teams pick one tool for the other job and get disappointed. Mask a full production copy and you inherit full production volume, a slow copy, and a restore that has to run somewhere with room for it. Generate fresh data and you get exactly the volume you asked for but none of the real rows, so the duplicate email your legacy import produced simply is not there to catch your migration. There is a third thing people usually want, and it is a small, connected, scrubbed copy of the actual database. That is what weavori sync now does in one command.

What a naive masking script does to a join

The instinctive way to mask a Postgres database is an UPDATE over the columns you are worried about:

UPDATE customers SET email = 'user' || id || '@masked.invalid';

That specific form is fine, because it derives the replacement from a column that is already unique. The trouble starts when the derivation is random per row:

UPDATE customers SET email = gen_random_uuid()::text || '@masked.invalid';
UPDATE events  SET actor_email = gen_random_uuid()::text || '@masked.invalid';

Now the same person's email, once copied into customers.email and again into the denormalized events.actor_email, receives two different replacements. Nothing failed. Every statement reported UPDATE 1000000. But the join your analytics depend on quietly returns nothing:

SELECT count(*)
FROM events e
JOIN customers c ON c.email = e.actor_email;
 count
-------
     0

Masked, safe, and impossible to query. The dataset passed the security review and failed the usefulness one, and nobody notices until a dashboard goes blank a week later.

Keys are a second trap wearing a different disguise. Try to scrub a primary key and Postgres stops you:

ERROR:  update or delete on table "customers" violates
        foreign key constraint "orders_customer_id_fkey" on table "orders"
DETAIL:  Key (id)=(4217) is still referenced from table "orders".

The usual way around that error is ON UPDATE CASCADE, which rewrites every child key too, or the quieter way: skip the key columns and mask only leaf values, which is how a script ends up leaving the one identifier you most wanted gone untouched because it was also a join column.

Masking the slice

Here is the command, run client-side against two connection strings:

Four flags, and the interesting thing is what does not get touched. The scrubber only rewrites columns that clearly read as personal data (name, email, phone, address, birth date, and the rest of that family of tokens). It never rewrites anything that carries identity or structure in the schema: primary keys, foreign keys, serials, integers, UUIDs, and booleans all pass through unchanged. A column referenced by a foreign key is left alone by construction. That is why the graph is still connected when the copy lands, and why you did not need ON UPDATE CASCADE to get there.

For the columns it does rewrite, the replacement map is built one distinct value to one distinct value. Every occurrence of jane.doe@real-corp.com becomes the same substitute everywhere that column is carried along a foreign key, a NULL stays NULL rather than becoming a fabricated address, and uniqueness that existed before still exists after. Because the map is derived from the --seed value, the same input rows produce the same substitutions every single time, which is the difference between a masked database you can reason about and one you have to re-learn on every run.

Subsetting that stays connected

--subset 10% is not LIMIT per table. Taking the first thousand rows of every table produces a set of orphans, because orders row one thousand references customers row eight hundred and eleven, which the per-table limit threw away. The slice would fail to load against a schema that declares the foreign key, and if you loaded it with the constraints disabled you would be testing a database that cannot exist.

Instead the sampler starts at the root tables and walks the foreign key graph upward until the result stops growing. Every row you keep has its parents pulled in, transitively, so the slice closes over its own references. --subset 1000 works the same way as a row target, and --subset 10% as a proportion. What lands in the target is a smaller database that is internally honest about its keys, which is the only kind of smaller database you can run a real query or a migration against.

Determinism is the part that makes it usable

It is worth being precise about the property you are buying, because "anonymization" gets used loosely in this space and the looseness hides the actual benefit.

A scrub that replaces each value independently, with no stable map, is irreversible and also useless: you cannot join across it, you cannot reproduce the bug that only shows up for one specific customer, and you cannot compare staging to anything. A scrub built from a fixed seed is stable, which is what lets the masked data still answer questions. The same person is the same masked person in every environment and on every re-run, so the joins hold, the reproductions match, and a diff between two environments means something.

That stable, reversible-from-the-seed substitution is pseudonymization rather than true anonymization. For a staging or demo database it is normally the property you actually want. If a specific obligation requires data that can never be mapped back, check that obligation against what a consistent substitution provides before assuming the flag's name settles it.

No replication slot, no extension, no nightly job

Most hosted masking and subsetting pipelines are built on logical decoding. They create a publication, open a replication slot, run a change-capture reader, and usually schedule a nightly sync from a read replica inside a vendor's cloud. That is a reasonable architecture when your requirement is continuous replication of live traffic into a governed store.

It is heavier than the requirement when all you want is a scrubbed slice for staging that you can rebuild on a Tuesday afternoon. The command above speaks the ordinary COPY protocol over two normal connections. No logical replication slot, which means no slot holding transaction logs and no subscriber lag on the source. No extension to install and no superuser to install it as. No always-on service standing between you and the copy. You point it at production, point it at staging, and it is done before the pipeline clears its cache.

What it does not replace

Summary

Masking and subsetting are two halves of one job: get a slice of the real database into an environment where people can query it without exposing a customer and without outgrowing the machine. The failure mode of the do-it-yourself version is silent, a zero-row join, because random per-row replacement desynchronizes the same value across columns and touching a key either errors out or forces a cascade you did not want. weavori sync --subset 10% --anonymize --seed 42 keeps every key intact, closes the slice over its own foreign keys, and makes the substitutions stable enough to reason about, in one client-side command that needs no replication slot, no extension, and no cloud standing behind it.