The improvement people notice after a few weeks of running an AI agent almost never comes from the model itself. It comes from a feedback loop someone built around it, on purpose or by accident.
An AI agent doesn’t get better because it ran again. It gets better when someone captures what it got wrong, stores that durably, and makes sure the next run actually sees it, a feedback loop, not the model retraining itself. This guide covers the real mechanism, what a repeated mistake costs, a five-step build sequence with a filled-in log and a ready-to-paste instruction block, and a five-question check for whether your loop is actually working.
Picture a marketing ops manager who runs the same AI agent every Monday morning to turn last week’s campaign numbers into a one-page summary. Three weeks in, she tells a colleague it’s “gotten better,” fewer strange formatting choices, fewer numbers pulled from the wrong column. She isn’t wrong that something changed. She’s wrong about why.
Between those two runs, the underlying model didn’t change at all. The version answering her prompt in week four is running the exact same weights it was running in week one. What actually changed is everything around it: she tightened her instructions after the first run’s formatting mess, she started pasting in last week’s good summary as an example to match, and she stopped feeding it the messy raw export that confused it early on. The system she built around the agent improved. The agent itself stayed exactly what it was.
That distinction matters more than it sounds like it should, because “learns from experience” is doing at least three different jobs depending on who’s saying it, and almost nobody bothers separating them.
| What changes | Who controls it | How common is this for you |
|---|---|---|
| Pretraining. The base model’s weights, on a fresh training run | The model provider (Anthropic, OpenAI, and so on), on their own schedule | Not something you do. You just receive a new model version occasionally |
| Fine-tuning or RLHF. The weights of an existing model, deliberately retrained on new examples | A team with machine-learning infrastructure, for a specific, narrow task | Rare for most businesses: it needs volume, budget, and specialist skill |
| Feedback loop and memory. Nothing in the model. What’s stored in instructions, context, or a log it can reference | You, or whoever owns the workflow, every time output gets reviewed | This is the one available to almost everyone reading this |
The third row is what “getting better from experience” means for the vast majority of business AI-agent use. Source: Anthropic’s own engineering account of context engineering distinguishes model weights from context state. [1]
Every company selling an AI agent will tell you it “learns” from how you use it. Almost none of them mean the model’s weights are changing. Anthropic’s own engineering team, describing how they build agents like Claude Code, draws this line explicitly: the useful lever isn’t retraining the model, it’s curating and maintaining the smallest set of high-signal tokens available to it at each step.[1] Put plainly, what you feed an agent and what it’s allowed to remember matters far more than anything happening inside the model itself.
If you’ve read what a CLAUDE.md or MEMORY.md file actually does, this will sound familiar. It’s the same idea: a plain text file sitting outside the model, doing the remembering that people assume the model does on its own, a pattern we’ve covered in more technical detail in our explainer on what CLAUDE.md and MEMORY.md files actually do.
Getting better almost never means the model changed; it means something got written down and fed back in.
The rest of this guide is about that system: what actually has to exist around an off-the-shelf agent for “getting better” to be true, rather than something you’re hoping happens on its own.
“It’s excellent for brainstorming and first drafts,” a corporate lawyer told researchers studying enterprise AI adoption in 2025, “but it doesn’t retain knowledge of client preferences or learn from previous edits. It repeats the same mistakes and requires extensive context input for each session.” Her firm had spent $50,000 on a dedicated contract-analysis tool. For anything that actually mattered, she went back to a general chatbot instead, and did the remembering herself, every single time.[2]
That’s the cost, stated plainly by someone paying it. An agent that repeats a known mistake doesn’t only waste the few minutes it takes to fix one output. It caps how much anyone is willing to hand it next. Say a support lead sets up an agent to draft renewal emails, and it keeps recommending a loyalty discount that was quietly retired two quarters ago. Nobody ever told the agent otherwise, so nothing it can see rules the offer out. She catches it once, fixes it, moves on. She catches it again the following week, then the week after that. By the sixth week she’s reading every email in full before it goes out, roughly the amount of work she adopted the agent to avoid in the first place. She hasn’t stopped using the tool. She’s stopped trusting it, and those are two very different things.
Three specific costs stack up when an agent keeps making the same mistake, and none of them show up on a usage dashboard:
| Role | What one repeated mistake actually costs |
|---|---|
| Marketer | A wasted afternoon rebuilding one asset. Embarrassing, but contained to that afternoon |
| Finance or operations | A quietly wrong number that compounds across a reporting period before anyone notices it |
| Sales or support | An error that reaches an actual customer directly, not just an internal reviewer |
The mechanism is identical in every row. What differs is how long you can afford not to notice.
That’s also why support teams tend to notice a broken feedback loop fastest, and revert to full manual review soonest: their repeated mistake reaches someone outside the building within hours, not at the end of a quarter. A marketer or an ops analyst gets to notice quietly and fix it before anyone outside the team sees the mess. The mechanism behind the mistake is the same in all three rows above. The clock you’re working against is not.
Widen the lens past any one person’s inbox and the pattern holds at the level of entire companies. Researchers who studied roughly 300 enterprise AI deployments and interviewed leaders at 52 organizations in 2025 found that 95% of the generative AI pilots they tracked showed no measurable financial return, against $30 to $40 billion in combined investment.[2] Their stated conclusion is worth quoting directly, because it’s not the headline most coverage led with: “The core barrier to scaling is not infrastructure, regulation, or talent. It is learning. Most GenAI systems do not retain feedback, adapt to context, or improve over time.”[2]
One lawyer’s contract tool and a $30 billion research finding are the same mechanism, showing up at two different scales. In both cases, an agent kept working exactly as well, or as poorly, as it did on day one, because nothing around it was set up to change that.
So if the model’s weights aren’t moving, what’s actually happening when an agent’s output gets visibly better after a few weeks of use? Two things, usually stacked on top of each other, and it helps to name them separately because they fail in different ways.
The first is memory: something the agent produced, or something you told it, gets stored somewhere outside the conversation and gets pulled back in later. Anthropic’s engineering team calls one version of this “structured note-taking,” and describes watching an agent maintain its own working notes across a long task, then read them back after a context reset and continue exactly where it left off, without ever being told to structure memory that way.[1] The same team names three specific techniques for holding a long-running agent together this way: compaction, which compresses old context into a summary instead of dropping it; structured note-taking, a persistent file the agent reads and writes as it works; and sub-agent architecture, handing a self-contained piece of a job to a fresh agent that reports back a short, distilled summary rather than its entire working history.[1] Skip all three and an agent given more and more raw history doesn’t get wiser with it. It actually gets worse at using what it has, a separate problem worth understanding on its own, which we cover in our explainer on context rot.
The second piece is the feedback loop itself: a human, or a defined process, deciding what counts as a mistake, writing that down somewhere durable, and making sure it reaches the next run’s instructions. Memory without a feedback loop is just a pile of notes nobody curates. What makes the loop actually matter is a person deciding which notes are worth keeping and making sure they resurface next time.
The clearest published demonstration of this, and it’s worth knowing this is peer-reviewed research rather than a vendor’s marketing claim, comes from a 2023 paper called Reflexion. Researchers gave an AI agent a coding task, let it fail, then had it write a short verbal reflection on what went wrong and store that reflection in a memory buffer it could reference on the next attempt. No retraining, at any point.[3] The agent’s pass rate on a standard coding benchmark climbed to 91%, up from a baseline around 80%, purely by remembering its own prior failures in plain language and reading them back before trying again.[3]
That result is the whole argument, in miniature. Nothing about the model changed between the failed attempt and the successful one. What changed is that a specific failure got written down in a form the agent could use, and something made sure it got read again before the next try. Scale that down to an agent drafting renewal emails or sorting expense receipts, and the mechanism is identical: capture what went wrong, store it somewhere durable, feed it back in before the next run.
None of this runs itself, and it’s worth being explicit about where the judgment still sits, because “feedback loop” can sound like something that closes on its own once you’ve set it up. It doesn’t.
| What the loop does | What you still own | How it gets checked |
|---|---|---|
| Captures and stores what went wrong, or worked well, on a given run | Decides what actually counts as “better” or “wrong” in the first place | A named person reviews a sample of flagged cases on a fixed schedule, not only when something looks off |
| Surfaces a pattern across several runs, the same error recurring two or three times | Decides whether that pattern is worth changing the standing instructions over, or is a one-off | Changes to the standing instructions get a second pair of eyes before they ship |
| Sounds more confident and fluent the more examples it has seen | Decides whether that confidence reflects real improvement or just better-rehearsed phrasing | Spot-checks the actual outcome (was the number right, was the offer current), not just how polished the output reads |
The loop does the remembering. A person still decides what “better” means and whether the agent has actually gotten there.
A feedback loop nobody reviews is just an unread inbox with extra steps.
Here’s the sequence, and every step below is something one person can start this week without buying new software or waiting on anyone’s approval.
The order matters. Skipping step one is the most common reason steps two through five don’t hold up.
Step one is the one people skip, because it feels like the least urgent. Write one sentence describing what a good output looks like for this specific task, specific enough that two different people would agree on whether a given run met it. “A good renewal email” isn’t specific enough to check against. “A renewal email that names the two currently active discount tiers by name and never references a retired one” is.
Step two is where the actual mechanism lives. Every time you fix or reject something the agent produced, log it somewhere durable: what happened, what you changed it to, and why. A corrections log doesn’t need software behind it. A shared spreadsheet works, as long as someone actually opens it.
| Date | What the agent did | What should have happened | Fixed in the instructions? |
|---|---|---|---|
| Sept 2 | Recommended the 15% loyalty discount | That tier was retired last quarter; only Standard (10%) and Founding Member (20%) are active | Yes, added both current tier names to the standing instructions |
| Sept 9 | Same retired discount resurfaced in a follow-up email | It had pulled the tier name from an old reference thread, not the new instructions | Not yet, flagged for Friday’s review |
| Sept 16 | No mention of the retired discount | n/a | Confirmed fixed, closing the item |
This is the whole mechanism in one spreadsheet. The middle row matters most: catching a repeat before anyone decides it’s fixed.
Step three turns a logged pattern into something the agent will actually see. Not every entry in the log deserves to become a standing instruction, most are one-offs, but once the same correction shows up two or three times, it’s earned a place in the instructions the agent reads before every run.
Short, dated, specific. This is what step three actually looks like once the log above has a few entries in it.
Picture an operations manager running an agent that sorts a folder of receipts into expense categories every Friday. In the first month, the same three mistakes keep showing up: office supplies get filed as software, a client dinner gets filed as personal, a subscription renewal lands in two categories at once. She starts a version of the log above, nothing more sophisticated than a shared spreadsheet. After four weekly reviews, two of the three mistakes stop appearing entirely. The third takes longer to fix, because it turns out the underlying receipts really are ambiguous, not something a corrected instruction alone can resolve, and that’s a useful thing to find out too.
Step four is the one that keeps the other three from quietly stopping. Pick a fixed day, weekly or every other week, and have one named person actually read the log and decide whether anything needs to change. Who that person is depends on team size: on a five-person team it’s whoever owns the workflow; on a larger one, it’s worth naming a single accountable reviewer rather than leaving it to whoever notices, because “whoever notices” quietly turns into nobody.
Step five doesn’t go away once the first four are working. Before an agent’s output reaches a customer, a filing, or a decision that costs money, someone who isn’t the agent looks at it. That checkpoint is what keeps an improving system from turning into an unsupervised one.
A correction that never reaches the next prompt fixed one output, not the agent.
Deciding which task deserves this level of setup in the first place is a separate question from building the loop itself, and it’s the subject of our guide to building your first AI-enabled business workflow. And if you’re trying to get a whole team running this consistently, rather than one person’s spreadsheet, that’s usually where structured training earns its place, same as with any new practice. Not because nobody can work out a corrections log alone, but because five people left to invent it end up with five different logs and five different definitions of “better.”
A feedback loop can exist on paper, a log, a review meeting on the calendar, and still not be doing anything. What decides that isn’t whether the ritual happens, but whether the agent’s actual mistakes change over time.
| # | Question |
|---|---|
| 1 | Has a specific, named mistake actually stopped recurring in the last month? |
| 2 | Does at least one person read the corrections log on a fixed schedule, not only when something breaks? |
| 3 | Have the standing instructions changed in the last month because of something the log showed? |
| 4 | If you turned this agent off tomorrow, would anyone notice a drop in quality, not just a drop in speed? |
| 5 | Can you name the last correction that made it into the instructions, including the date? |
Score one point for every honest yes. A “sort of” counts as no.
| Score | What it means |
|---|---|
| 4 to 5 | The loop is real. Corrections are reaching the agent, and something is measurably different because of it |
| 2 to 3 | Half-built. You’re probably capturing feedback somewhere but not reliably feeding it back in. Revisit step three above |
| 0 to 1 | There’s no loop yet, just a tool being used repeatedly. Worth naming that honestly before handing it anything bigger |
This is a scored self-check, not a certification. Score it honestly and act on what it tells you.
Notice that none of those five questions ask how often the agent gets used. Usage is the easiest number to pull and the least useful one to trust. What actually matters is persistence, whether a fix from three weeks ago is still holding, and impact, whether the real output is better, not just whether people are still opening the tool.
A few patterns show up often enough, even on teams that mean well, to name directly:
One more thing worth watching for, and it’s the one that’s easiest to miss: an agent’s output can get more fluent and more confident without getting more accurate. More examples to draw on, more polished phrasing, a tone that reads as more assured, none of that is the same as the actual numbers being right or the actual offer being current. The five-question check above exists partly to catch that gap. “Sounds better” and “is better” are not the same claim, and only one of them is worth trusting.
None of this requires new software or a data science team. It takes picking one recurring task this week, writing down the next mistake that agent makes somewhere you’ll actually reread, and putting one name next to “reviews this on Fridays.” That’s the entire mechanism behind every example in this guide, run by hand, on purpose, until it’s just how the team works. For a broader look at what separates someone who’s actually built this kind of habit around AI from someone who’s just a heavy user, see our piece on what makes an AI-powered professional different from everyone else using AI.
Not in the way a person does. A person’s brain changes with repetition whether or not anyone writes it down. An off-the-shelf AI agent, the kind most businesses use, doesn’t update its underlying model from a single conversation or a week of use. What looks like learning is almost always an engineered feedback loop: someone capturing what went wrong, storing it somewhere the agent will see again, and feeding it back in before the next run. The improvement is real. The mechanism isn’t biological, and it doesn’t happen without someone building it.
Learning, in the strict sense researchers use the word, means the model’s weights change through training, fine-tuning, or reinforcement learning from human feedback. Remembering means something is stored as text outside the model, a note, a log entry, a retrieved example, and gets pulled back into context on a later run. Almost everything people call “an agent learning” day to day is actually remembering, combined with a person deciding what’s worth keeping. Not a lesser version of learning, just a different mechanism with a different owner.
Five steps, in order: define what “better” means for the specific task in one concrete sentence, capture every correction in a durable log, fold repeated patterns, not every single instance, into the agent’s standing instructions, review that log on a fixed schedule with one named person responsible, and keep a human checkpoint before any output reaches a customer or a costly decision. The full build sequence, including a filled-in example log and a ready-to-paste instruction block, is covered in the how-to-build-it section above.
The correction never makes it anywhere the agent will see it again. Someone notices the mistake, fixes that one output, and moves on, but the fix lives in their head or in a chat thread instead of in a log or the agent’s instructions. The next run has no way to know anything went wrong last time, so it repeats the identical mistake with the identical amount of confidence. This is the single most common failure pattern behind agents that get used constantly and never seem to improve.
For nearly every business use case, no. Fine-tuning changes a model’s actual weights and requires real machine-learning infrastructure, a sizable set of labeled examples, and ongoing maintenance, worthwhile for a narrow, high-volume, stable task, and overkill for almost everything else. The simpler path, context and memory engineering, gets most of the practical benefit: better instructions, a corrections log, retrieved examples of past successes, without touching the model itself. Most teams asking this question are better served by the feedback loop in this guide than by a fine-tuning project.
The distinction between model training and engineered memory in this guide is checked directly against Anthropic’s own engineering documentation on context management for Claude agents, and against the peer-reviewed Reflexion paper on agent self-reflection (NeurIPS 2023). The MIT NANDA “GenAI Divide” figures, including the corporate lawyer’s quote, are taken directly from the published report’s own text, not from secondary coverage of it. The corrections-log, build-sequence, and scoring-checklist examples are Future Factors’ own, built for this guide rather than borrowed from any single outside source.
Weekly tips. Real examples. Practical help for busy professionals.
We care about your data, check out our privacy policy.