Your agent doesn’t remember it got something wrong yesterday. If you want it not to happen again, what has to change is the system it works in. How to turn a repeated mistake into a change that stays, and why writing it in a file isn’t always enough.
There’s a feeling anyone who works with AI agents knows. You correct a mistake, it thanks you, it fixes it. The next day, in another session, it makes exactly the same one. And you explain it again, with a little less patience than the first time.
It isn’t bad faith or lack of intelligence. It just doesn’t remember. Each conversation starts almost from scratch, and what you learned yesterday, it didn’t. If you want the mistake not to come back, the memory has to live outside the agent: in the system it works in.
I learned this with UXMachine at the beginning of September. The only mandatory check before merging a change was that the code was well written, with no syntax errors. On 2 September, the tests failed on all ten branches the agents were working on that day, and nothing was blocked, because nobody had said that failing tests should stop a merge. Two things changed. Now a change only goes in with all tests green, checked against the real version. And every check has to prove, with a failure made on purpose, that it really turns red when it should.
The third loop
In this series I separate three loops. The agent’s loop, which lasts seconds: it calls a tool, reads what comes back and carries on. The work loop, which I wrote about in the previous article: someone outside checks the result and sends it back if it fails. And the system loop, which lasts days or weeks: when a failure repeats, it becomes a permanent change.
A simple way to see it: the work loop is marking the exam; the system loop is changing how the next one is prepared. The first fixes what the agent has just done. The second fixes what makes it go wrong. It’s the only one of the three that makes next week cheaper than this one.
Hashimoto’s idea: make sure it doesn’t happen again
Mitchell Hashimoto has given this habit a name: harness engineering. Every time an agent makes a mistake, you take some time to engineer a solution so that it never makes that mistake again.
He describes two ways of doing it:
- Write it down. Add a line to the instructions file the agent reads at the start of every session, AGENTS.md. In one of his, he says, each line comes from a bad agent behaviour, and almost all of them went away.
- Give it a tool. For example, a small program that takes screenshots or runs only the tests that matter. Then note it in AGENTS.md so the agent knows it exists.
A written rule is a wish
It’s worth not being too optimistic here. Writing a line in an instructions file is the easiest thing, and not always the most effective.
A study published in February, Evaluating AGENTS.md, measured it across several agents and models. Instruction files, in general, did not increase the number of tasks solved, and made each task more than 20% more expensive on average. The agents did follow the instructions; what didn’t help were general descriptions of the project. The authors’ conclusion is cautious: these files are useful to write down what your project does differently from the usual, and everything else is worth measuring before you trust it.
Ichigo puts it with an image I really like. “Don’t touch the migrations”, written in a text file, is a wish: the agent will read it and almost always follow it. A rule that takes away its permission to touch them is a wall: it will follow it even if it forgets. That’s why he proposes this order when something fails:
- First, structure: make the mistake impossible to make.
- Then, an automated check that always catches it.
- Text in the instructions, only where judgement is really needed.
- And a person who reviews, at the end.
The reason is almost accounting. A line in the instructions is paid for in every session, and the agent can overlook it. An automated check, once built, costs nothing.
Fix it where it’s produced, not where it shows
There’s a very common mistake of approach: fixing the failure exactly where it was seen. In the AI-First Guide I put it this way: if something comes back after you fixed it, it was fixed where it was observed, not where it’s produced. And “a failure fixed where it was observed is hidden, not fixed”.
The guide suggests one simple question before you close something: if this same thing showed up somewhere else, would anything stop it? If the answer is no, it isn’t finished yet.
And there’s a counterweight, also in the guide: a check that has never gone red is either untested or unnecessary. That’s why every new check should prove, at least once, that it catches the failure it exists for.
From prompts to a way of working
At first I thought the trick was asking better. Now I think it’s deciding beforehand.
In the guide, every task says “what gets done, what isn’t touched, how it’s checked and when it stops”. And before starting, one line: “what would have to happen for me to drop it?”. The rest follows from there:
- What it can touch. Don’t ask the agent not to touch something: take away the permission. “What a role can touch isn’t told to it; it’s limited.”
- Who decides what. For each role, two columns: what it decides and what it doesn’t. The second is the hard one, and the only one that helps when you’re in a hurry.
- When it stops. A stop condition written before starting. A deadline is not a stop condition.
- Who looks before it goes out. A delivery is accepted on the real result, not on what the agent says it did. And if you can’t look at it yourself, someone has to, and it can’t be the agent.
Knowing what each agent can reach
To put up a wall, you first need to know where it’s needed. With several agents on the same computer that isn’t obvious: each one reads its own files, tools and keys, and sometimes the others’.
That’s why I’m building RosmarOps. It reads the configuration of the agents on your computer and draws a map of what each one can reach. It doesn’t change your agents or set limits: it shows you where they are today, so you decide where the walls go.
Three questions for the next time an agent repeats a mistake
- Where is the mistake produced, and not just where did I see it?
- Can I make it impossible, or have a check always catch it, before writing it into the instructions?
- If it shows up somewhere else, would anything stop it?
What rules don’t fix
A system that learns avoids repeating mistakes. But there’s a cost that doesn’t show up as any mistake: building faster than you understand. That’s what the last article in the series is about.
Sources: Mitchell Hashimoto, “My AI Adoption Journey” (5 February 2026); Gloaguen, Mündler-Sasahara, Müller, Raychev and Vechev, “Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?”, arXiv:2602.11988 (12 February 2026); Albert Garcia Pujadas, AI-First Guide (in Spanish); Ichigo’s thread on X (5 November 2026).
Written with the help of AI assistants. The examples, the interpretation and the decision to publish are mine.