Controlled AI evaluation
State eval details
The evidence behind State's Project Health quality signals: controlled scenarios for update understanding and grounded answers.
Read-only details. Run controlled AI evals from Project Health, where paid-model confirmation, one active run at a time, and the cooldown are enforced. These cases are deliberately constructed test data, not customer or private project content. State records aggregate run metrics and, for failed controlled scenarios, limited scenario IDs and pass/fail metadata. Raw production content and full model transcripts are not published here.
Update understanding
Loading…
Checking the latest recorded run.
Answer quality
Loading…
Checking the latest recorded run.
Failed scenarios
Start here. These are the scenarios that failed in the latest recorded AI eval run.
Scenario catalog
The scenario catalog is the product contract. Each case describes the setup, expected behavior, and why that behavior matters.
Recent run history
Project Health persists aggregate metrics by suite, model, build, and run time. Failed controlled scenarios can also retain limited failure metadata; individual model transcripts stay out of this public portfolio view.
Loading recent runs…