Golden fixtures

Describe an expected result

A golden fixture belongs to a ruleset version and contains an input and an expectation. Assert the full output, with any ignored paths explicitly listed, or an expected failure with a registered code. Fixtures can also carry document assets. Adoption copies a template version's fixtures; a ruleset created from scratch starts with none.

Create fixtures manually or save an eligible completed run as a fixture. Confirm the expected output and supply a rationale. Saving from a run is unavailable when the needed raw content has been purged.

Creating, editing, deleting and re-recording fixture content requires a rationale and writes an audit entry. Input/output contract drift appears as a warning in the editor, is stored on the fixture and appears in approval evidence. The warning does not block the edit.

Frozen versions and deletion

Activation freezes a version's fixtures. Make ordinary fixture changes in a working version and include them in its activation request. Deleting an editable fixture removes its input, expectation, document assets and recorded responses from the current fixture set. It keeps a tombstone and the fixture's run-result history, even if the fixture has already run.

Erasure is an exception to the freeze: run erasure redacts the fixture documents and recorded responses linked to that run, including on a frozen version. A frozen fixture cannot make erased content available again.

Dataset content and extraction recordings

Template fixtures use bounded inline dataset snapshots. Tenant fixtures pin dataset versions. These fixture bindings make the comparison's reference data explicit.

Extraction fixtures replay recorded model responses by default. Record a response before a gate or approval comparison; publication preparation can obtain a missing recording before opening the gate batch. Gate and approval comparison batches replay recordings, rather than recording new answers during the comparison. An approval comparison never creates a recording; a missing required recording is failing fixture evidence. Explicit re-recording requires a rationale and audit entry. Replay avoids a new provider call for the recorded extraction response.

Use synthetic data

The fixture data policy requires synthetic examples only: never put real personal, bank or financial data in a fixture. This is a data-handling rule for authors, not an automatic guarantee that uploaded material contains no personal data.

Fixture document assets and recorded responses are stored outside the ordinary run retention window. Purging a run's raw content is not a fixture-asset retention rule. The linked erasure path described above can redact those assets, and deleting an editable fixture removes its content. Treat long-lived fixtures accordingly; a successful test does not make real customer data suitable for storage there.