CHINZY / FDE and robotics acceptance fieldbook / 05
Configuration drift reproduction
Run the same input against two test configurations with one variable changed. Save config versions and output differences before blaming the model.
Test setup
Freeze samples, versions and ownership before the rehearsal. Keep before/after evidence for an independent reviewer. Budget caps and thresholds in examples are fixtures; agree actual gates with the project owner.
Contrasting examples
Figures and IDs are illustrative fixtures, not product results or contract terms. Save inputs, versions and conditions before repeating an unexpected result.
| Example input | Expected observation |
|---|---|
corpus=V1; top_k=1; source_rank=2 | source_absent |
corpus=V1; top_k=3; source_rank=2 | source_present |
Recording the result
Keep a separate row for each example; one pass cannot stand in for the other. Enter the actual output, evidence location, environment version and review decision. Leave unresolved results open.
Download test worksheet (CSV) ↓