Privacy-safe test data

The problem with "just anonymize the prod copy"

Every team that handles personal data eventually has the same conversation. Developers need realistic data in staging, CI, or a demo sandbox, and production is off limits. The proposed fix is usually: dump prod, mask the sensitive columns, load it somewhere else.

That pipeline has three failure modes, and none of them are theoretical:

• The pipeline rots. Someone adds a secondary_email or a support_notes column, the masking config never learns about it, and real values flow through untouched. Now your staging environment is a compliance finding.
• Masking does not remove the risk, it reduces it. A masked copy is still derived from real records. Given enough quasi-identifiers, or a join against an outside dataset, rows can be re-identified. That is why privacy frameworks separate pseudonymized data, which is still regulated, from genuinely anonymized data.
• Consistent masking breaks the relationships tests depend on. Scrub a primary key and every child row dangles. Keep the key and you have preserved the most identifying column in the schema. Teams spend weeks on dependency-aware masking to avoid exactly this.

The pragmatic fix is to stop transforming real data and start generating fresh data that has the same structure.

Generate it, don't copy it

weavori generate reads your schema and produces new rows. It never copies production values into the target, so there is no real identity in the output to leak, re-identify, or track through an audit request.

The realism is not sacrificed. Values come from the same machinery that powers every Weavori run:

• ~230 column-name rules across 25+ domains, so first_name, ssn, phone, and address get valid shapes from type-aware generators, not Lorem ipsum.
• pg_stats distribution sampling, so a status column that is 70 percent active in production comes out roughly 70 percent active.
• Foreign keys resolved by construction, parents before children, so joins and constraint tests behave like they do against real data.
• Derived identity columns. Bind an email to the generated name with a formula and it can never collide with a real mailbox:

# weavori.yaml
tables:
  employees:
    columns:
      email:
        formula: "lower(concat(first_name, '.', last_name, '@example.com'))"

Prove it with a query, not a promise

Privacy work needs evidence. Mark PII columns in the schema with COMMENT ON COLUMN, then let the database itself answer the audit question. This is what cookbook recipe 08 ships: a classification inventory straight out of pg_description, plus the generated rows next to it.

SELECT a.attname AS column_name,
       col_description(a.attrelid, a.attnum) AS classification
FROM pg_attribute a
WHERE a.attrelid = 'public.employees'::regclass
  AND a.attnum > 0 AND NOT a.attisdropped
  AND col_description(a.attrelid, a.attnum) IS NOT NULL
ORDER BY a.attnum;

 column_name | classification
-------------+---------------
 first_name  | PII
 last_name   | PII
 email       | PII
 phone       | PII
 ssn         | PII
 address     | PII
 salary      | SENSITIVE

The recipe's sanity check then asserts emails_not_derived = 0. Every email in the table matches the generated pattern, which is the property you want when someone asks how you know nothing real got through.

When you do need real rows

Some jobs require production's actual edge cases: the pathological row that only exists in one tenant, the real correlation your model learned from. For those, weavori sync copies rows in FK-safe order and rewrites recognized PII columns deterministically in the same pass.

Two things worth stating precisely:

• --anonymize performs deterministic pseudonymization, not irreversible anonymization. The same original always maps to the same replacement, and the mapping is a stable substitution. That consistency is what keeps joins, UNIQUE constraints, and repeatable tests intact.
• Because it is pseudonymization, the result may still be personal data under GDPR. Treat it accordingly. The generation path above is the option that keeps real records out of the picture entirely.

sync, including --subset and --anonymize, is part of Pro.

How it runs

Generation runs on your machine or your CI runner. Only license activation and auth credentials involve the server, so the data itself stays inside your infrastructure.

FAQ

Does this make our test data GDPR compliant?

Weavori does not make a legal determination, and no tool can. What it changes is the factual question you have to answer. With a masked production copy you must argue that real personal data was sufficiently transformed. With generated synthetic data, the rows were never about anyone. Your counsel still owns the classification.

Does it find every PII column?

The classifier is deliberately conservative. It rewrites columns it recognizes as identity data and leaves *_id, serial, numeric, and boolean columns alone so foreign keys survive. If your schema stores personal data in unusual columns, put the classification in the schema with COMMENT ON COLUMN and verify it in the audit query, the same way you would verify any masking pipeline.

Is this a replacement for a data governance platform?

No. Weavori produces privacy-safe data; it is not an access-control, discovery, or audit system. Teams that need SSO, governance workflows, and regulated data lifecycle management should expect an enterprise TDM platform. This is the self-serve path for the developers and QA engineers who just need a database they can hand out.

Get started