A maintained summary beat chat history, until it started deciding things.
As my AI chats grew longer, I kept re-explaining things we’d already discussed and noticing details it had dropped or gotten wrong. That made me wonder whether the project notes I kept as a project manager would hold up if AI maintained them instead.
Using notes from simulated exercises in Claude, I compared raw history, stronger reconstruction instructions and a maintained summary. The summary worked best, but it still recorded conclusions the notes didn’t support. The worst one:
The question shifted from “how do we keep context?” to “who decides what counts as true?”
State keeps a current, human-approved picture of a project as new information arrives.
Notes and documents come in as Evidence, which State keeps unchanged. Slack intake from approved channels is built too, but isn’t connected in this demo. AI compares each piece of Evidence with Current State, the project’s approved picture, and with the open Questions, then suggests what, if anything, should change.
- InputEvidenceCaptured and kept unchanged
- AIAI interpretsSuggests changes, Questions, or no effect
- SoftwareSoftware checksBlocks invalid proposals
- PersonPerson decides, when neededApproves, adjusts, tracks a Question, or dismisses
- Current StateAuthorized changes only. History records why.
- QuestionsOpened or answered, without changing Current State.
- No changeEvidence kept. Includes notes that needed no Review.
Why not just use a Claude Project?
A Claude Project could probably cover my own need. State let me explore a different problem: keeping an explicit current understanding as evidence arrives, with the decision about what becomes authoritative kept outside the model.
AI can interpret what changed. It can’t decide what becomes true.
Interpretation needed to move quickly, but project truth couldn’t depend on a model sounding confident. I kept the decision about what becomes official outside the model, so its interpretation could be reviewed rather than automatically accepted.
That made review burden the real tradeoff. If every detail needed a decision, review would add work without adding control. So State first filters out Evidence that doesn’t change anything consequential, keeping it without a Review, then groups related changes into decision-sized Reviews.
Protecting Current State turned out to be only half the job.
Unknown has to stay unknown.
The 0% failure became a requirement. Constructed eval scenarios check that a note saying the target is still unknown leaves Current State alone, and that Ask says no figure has been established and points to the open Question. They don’t reproduce my original experiment; they pin down behavior State has to keep.
Current State can be right while Ask misrepresents it.
Ask can’t change anything, so I first treated it as the lower-risk part of State. I’d noted one risk there: a safeguard meant to stop Ask overstating unapproved information might distort approved information. QA reproduced it, and found the opposite failure too.
| Item | What the record established | What Ask said |
|---|---|---|
| Move budget | Approved, and accepted into Current State | “not yet approved” |
| Vendor data retention | A vendor’s unconfirmed claim, tracked as an open Question | “on record” |
Each failure got a fix and a regression test, but later evaluation kept finding new variations, and repeated runs still fail occasionally. Correcting wording after the fact isn’t enough. The proposed next step is to track each claim’s authority explicitly as the answer is written.
A test can be wrong when the behavior is right.
One eval expected Legal’s answer to an open contract question to resolve the Question and change nothing else. State also proposed recording what the answer established, which the Evidence supported and State’s rules allowed. I revised the expectation to accept it, and added a check that fails any update implying the whole contract review is complete. The goal was a test that matched correct behavior, not a higher pass rate.
Approving changes isn’t enough. The distinction has to hold everywhere the record is used.
I started with one boundary: AI interprets, and people decide what becomes true. Testing showed it also has to hold wherever the record is read. A correct Current State can still be misrepresented in an answer, so approved facts, pending proposals and unknowns have to stay distinct in every answer and summary. In Ask, that part isn’t solved yet.