Realistic synthetic data is more than valid foreign keys

Here is a row that passes every check your schema enforces:

INSERT INTO users (id, first_name, last_name, email, birth_date, age, gender, city, state, zip)
VALUES (8217, 'Jennifer', 'Smith', 'jvsmitty82@example.net', '1994-03-11', 41, 'female', 'Portland', 'OR', '31402');

Every foreign key resolves. Every column is the right type. Nothing violates a constraint. And the row is wrong four different ways.

age is 41 but birth_date is 1994, which makes her 32. The email is a random string that has nothing to do with "Jennifer Smith." Portland, Oregon is not 31402, that ZIP is in Georgia. A test harness reading this row learns nothing about how real data behaves, because real data does not produce rows like this, and yet a generator produced it effortlessly.

This is the gap between referential integrity, which most tools advertise, and semantic integrity, which almost nobody demonstrates. Foreign keys are topology: they say the boxes connect. They say nothing about whether the values inside the boxes hang together, or whether their frequencies look like a population rather than a random draw. Weavori shipped four controls that close that gap, and none of them are in the README. They are --distribution, --null-rate, --since/--until, and automatic cross-column coherence.

Coherence: the values in a row agree

The nonsense row above is coherent by accident of the matcher. Each column was filled independently: age from a plausible range, email from a random localpart, zip from a random five digits. When columns are drawn independently, agreement is impossible.

The fix is to draw the entity once per row and let every column of that entity read from the same draw. Weavori detects when a row describes a person (two or more person-shaped columns: a first name, an email, a birth date, an age, a gender) and generates those columns jointly. The email is derived from the actual first and last name in the row, from a set of real domains. The first name is consistent with the declared gender rather than pulled from a unisex pool. age and birth_date are computed from each other, so they cannot disagree.

The same mechanism covers a place. When a row carries two or more location columns, one city/state/ZIP/country is drawn and every column reads from it, so city='Portland' and state='OR' and a real Oregon ZIP land together. You can see why this matters the moment you filter:

SELECT count(*) FROM users WHERE state = 'OR' AND zip LIKE '97%';

With independent columns this returns a number that is pure noise. With a coherent place draw it returns something close to your real Oregon population share, which is the only version of the query worth running against test data.

Distributions: the population has a shape

Uniform generation gives every allowed value roughly the same chance. A status column that is 90% paid in production comes out a third paid, a third free, a third refunded, and a test that branches on that column exercises the wrong path.

There are two cases and they are not the same.

Categorical values that repeat inherit automatically. When you generate against a live source, Weavori reads PostgreSQL's column statistics and carries the most-common-value frequencies into the output. A status that is 70% active in the source stays 70% active. This is free and needs no flag, but it only applies to values that actually repeat, because that is what a most-common-value list is.

Continuous columns you shape yourself. An age, a price, or a quantity has no repeated values to inherit, so it falls back to a plausible range until you declare a shape. That declaration is --distribution, and it takes a small set of named forms:

uniform flattens a column, zipf makes it heavy-tailed so a few values dominate (which is what real order quantities and page-view counts actually look like), and weighted lets you name the frequencies directly when you know them:

The one honest limit, stated plainly because the word "distribution" invites a wrong assumption: shapes are declared, not inherited, for continuous columns. Weavori does not read your numeric histogram and reproduce it automatically. It reads repeating categorical values from the statistics PostgreSQL already gives it, and for everything else it shapes the column to whatever you specify. That is a different and more limited claim than "your numeric columns match production exactly," so treat --distribution as something you configure against the handful of columns that matter, not as a mirror.

Null rates: the columns that are usually empty

A generator that fills every nullable column produces a database where middle_name is present on every row. Production is the opposite, and the code path that handles a missing value is exactly the one your uniform data never exercises.

--null-rate sets the probability a nullable column is left empty, per column or globally:

--null-rate "users.middle_name=0.25"        # a quarter of rows
--null-rate 0.1                             # global default for nullable columns

Set it on the columns where absence is the interesting case: the optional phone number, the unverified billing address, the deleted_at that should be NULL for almost every row. A test that reads middle_name without a NULL branch will fail against a realistic set and pass against a complete one, and passing is the worse outcome.

Time: the window the data lives in

Load tests, report snapshots, and temporal queries all assume the dataset sits inside a plausible period. Two flags bound every generated timestamp:

--since 2024-01-01 --until 2026-10-01

Both accept a date or a full RFC 3339 timestamp, and the upper bound is inclusive. Inside that window the coherence rules keep ordering sane, so created_at does not postdate updated_at and a child event does not precede the parent record it belongs to. Without a bounded window a "last 90 days of orders" report against synthetic data returns an arbitrary slice of dates scattered over decades, and the query plan it exercises is not the one you are testing.

Reproducibility: the whole run replays

All of the above is worthless in CI if the next run produces a different database. --seed makes a run byte-identical: the same seed against the same schema produces the same rows, the same shapes, and the same coherent entities. A build that fails on the fourteenth attempt can be replayed exactly, because the seed it used is printed at the start of every run even when you did not set one.

Three ways to say the same thing

Every control above can be given three equivalent ways, which is a deliberate design choice rather than duplication: as CLI flags, in a committed weavori.yaml project file, or through the MCP server's generate parameters. The YAML form is the one that turns a 200-character command into a checked-in definition of what your fixture data is:

tables:
  users:
    distribution:
      age: "normal:mean=35,stddev=10"
    null_rate:
      middle_name: 0.25
  orders:
    distribution:
      status: "weighted:paid=0.9,free=0.05,refunded=0.03"
window:
  since: 2024-01-01
  until: 2026-10-01
seed: 42

Once it is a file, "give me a realistic database" is one command, and the definition of realistic is reviewable in a pull request.

What this does not do

Summary

1. Valid is not believable. Foreign-key integrity is topology. A row can satisfy every constraint and still contradict itself four ways.
2. Coherence draws the entity, not the column. Name, email, age, birth date, and gender agree because they come from one per-row draw; city, state, and ZIP do the same.
3. Distributions give the population a shape. Repeating categorical values inherit real frequencies automatically; continuous columns you declare with normal, zipf, lognormal, or weighted.
4. Null rates and a time window make the boring columns and timestamps realistic enough to exercise the code paths and plans you actually ship.
5. --seed replays it all, and weavori.yaml turns the whole recipe into a committed definition of "realistic."

Try it on a table whose numbers matter:

Then run your real aggregate query against it and look at whether the shape holds. That is the difference this closes.