Nothing announces it. The answers just get slightly less right, in a tone that sounds exactly as confident as the good ones did.
You’ve probably had a long AI conversation that started sharp and ended vague, without anything obvious going wrong. That’s context rot: performance degrading as input grows, even on tasks the model handles perfectly at shorter lengths. It’s a different problem from filling up the context window, which is a hard limit with an error message. Rot has no error message. This piece covers what’s actually happening, the four signs it has started, and a specific way to carry work into a new conversation without losing the useful parts of the old one.
Picture a contracts manager three hours into one conversation about a supplier agreement. She’s uploaded the current draft and the previous one, worked through the indemnity language, changed her mind twice about the liability cap, and asked for a plain-English summary of the termination clause a few messages back.
She asks for that summary again, tidied up for a colleague. What comes back says the notice period is 60 days. The current draft says 30. The old draft said 60.
Both numbers were in the conversation. Nothing was hallucinated, exactly. The model picked the wrong one, and the answer arrived with the same steady, competent tone as everything else in the thread. That’s the part that makes this dangerous: there is no signal. A degraded answer looks exactly like a good answer.
Context rot is the name for that: output quality dropping as the amount of text in the conversation grows, on tasks the same model handles reliably when the input is short. The term comes from a technical write-up published by Chroma in July 2025, which tested 18 models across four deliberately trivial tasks and reports the same pattern across all of them.[1]
A long thread isn’t memory. It’s competition.
The word “rot” is doing some work here and it’s slightly misleading, so it’s worth being precise. Nothing decays over time. A conversation left open for a week is no worse than one you finish in an hour. What matters is how much text is in it, and how much of that text looks a bit like the thing you’re asking about.
One distinction to get straight before anything else, because these two problems get run together constantly and they need opposite responses.
So a model advertising a million tokens of context is telling you what it can hold, not how well it reasons across all of it. Those are separate claims, and only the first one is a specification.
The intuition most people carry is that a model reads its context the way you’d read a folder: everything’s in there, so everything’s equally available. The 10,000th word should be as usable as the 100th.
It isn’t, and the clearest demonstration in the Chroma work is almost comically simple. Take a question the model can answer perfectly when you give it just the relevant few hundred words. Now give it the same question with the same answer still present, but buried in about 113,000 tokens of surrounding conversation. Every model family tested did worse on the second version.[1]
The information was never missing. What changed was how much work it took to find it.
That’s the mechanism, and it’s more useful than “the model forgets.” In a short exchange, the model does one job: answer. In a long one, it does three, all at once, in a single pass.
Short prompts require step three only. Long threads require all three in a single pass, and steps one and two get harder as the thread grows. Described from the mechanism set out in the Chroma report.
Step two is where most of the damage happens, and it’s the one nobody plans for. The report separates two kinds of extra text, and the difference matters a lot for how you work.
Irrelevant content is unrelated to what you’re asking. A distractor is topically close but wrong: it looks like an answer without being one. The abandoned draft of your contract is a distractor. So is the positioning line you rejected in week one but never deleted. So is the earlier version of the number you’ve since corrected.
A single distractor was enough to reduce accuracy against a clean baseline in their tests, and four made it worse.[1] Which is uncomfortable, because a working conversation naturally accumulates distractors. Every draft you improve on leaves its predecessor sitting in the context.
Two other findings from the same work are worth knowing because they’re counterintuitive enough to change how you’d act.
The first: models did better when the surrounding text was shuffled into incoherence than when it flowed logically. The researchers say plainly they can’t explain why. It does suggest that a long, well-structured document in your context is not the safe input it feels like.
The second is about how models fail, and it differs by family. In their distractor tests, Claude models were the most likely to abstain and say they couldn’t find an answer, while GPT models produced the highest rate of confident, wrong answers.[1] Neither is better. But if you work mostly in one of them, you should know which failure you’re more likely to be handed, because one of them is much easier to miss.
Since there’s no error message, you’re relying on symptoms. Four are worth knowing, and three of them are easy to misread as the model just being unhelpful today.
| What you notice | What’s likely happening | What to do about it |
|---|---|---|
| It reuses a detail you corrected earlier: an old number, a name you changed, a decision you reversed | The superseded version is still sitting in the context and reads as relevant. Classic distractor. | Start a fresh conversation and carry forward only the current version. Correcting it again adds a third candidate. |
| Answers get more general. You asked something specific and got something that would fit any company | Retrieval is failing before reasoning starts, so it’s answering from general knowledge instead of your material. | Restate the specific input in the current message rather than pointing back to it. “As we discussed” is doing no work. |
| It says it can’t find something you’re certain you provided | Abstention under ambiguity. More common in some model families than others, and it’s the honest failure mode. | Treat this as useful. It’s telling you the context is too crowded, which the confident wrong answer never does. |
| Re-asking the same question in the same thread gets you a differently-wrong answer each time | Multiple plausible candidates in the context and no stable way to choose between them. | Stop re-asking. The thread is the problem, and every attempt adds to it. |
A diagnostic table for a degrading conversation. The failure patterns are drawn from the Chroma report; the mapping to everyday work is ours.
The second row is the one I’d watch hardest, because it looks like the tool being lazy rather than the thread being crowded. If you find yourself thinking “it’s gone generic on me,” check the length of the conversation before you rewrite the prompt.
What isn’t a reliable sign: where in the conversation you put the important thing. It’s tempting to assume the middle is where information goes to be forgotten, and on the main retrieval task in the Chroma work, testing eleven different positions produced no notable variation.[1] Position did matter on a separate copying task, where accuracy was better when the unique item sat near the beginning. So it’s real in some settings and absent in others, which means it’s not something to build a habit on.
The advice you’ll see most often is “start a new chat,” which is right and almost useless on its own, because the reason people don’t is that starting fresh means losing forty minutes of setup they don’t want to type again.
So the actual skill is not starting over. It’s deciding what travels.
Move the decisions, not the conversation.
A conversation contains two kinds of thing. Decisions and current facts, which you need. And the route you took to reach them, including everything you tried and rejected, which is exactly the material that turns into distractors. Asking the model to “summarise this chat” carries both, because a good summary of a conversation includes the wrong turns.
Here’s the version that works, written out in full so you can lift it. The specifics are a contract review, but the six headings transfer to anything.
| Heading | What you write under it |
|---|---|
| What I’m doing | Reviewing the current supplier agreement (v4, dated 12 August) before it goes to legal on Friday. |
| The current facts | Notice period 30 days. Liability cap at 12 months of fees. Governing law England and Wales. These are current and override anything else. |
| What’s already settled | Indemnity wording is agreed and not up for discussion. Do not revisit it. |
| What I’ve ruled out | We are not pursuing the mutual-termination option. If it comes up in the document, flag it, don’t develop it. |
| What I need next | A plain-English summary of clauses 7 to 9 for a non-legal colleague, under 200 words. |
| The rule | If a figure isn’t in the attached v4, tell me you can’t find it rather than filling it in. |
A filled-in carry-forward brief. The fourth row is the one people skip and the one doing the most work: naming what you rejected stops the model treating it as still live.
Two or three minutes to write, and it replaces a thread that had grown to forty exchanges. The fourth row is the important one. Telling the model what you ruled out is different from not mentioning it, because if the rejected option appears anywhere in the attached documents, silence leaves it looking like a live option.
When to reset at all, rather than push on:
Start fresh when the goal changes, not when the thread gets long.
And because this is a failure mode with no warning light, the check has to be structural rather than instinctive. Where a long conversation feeds into work that matters, write the split down before you start.
| What AI does | What you still own | How it gets checked |
|---|---|---|
| Pulls the relevant clauses and drafts a plain-English summary of each | Whether the summary is a fair reading, and what the commercial consequence is | Two figures picked at random get traced back to the source document. Not the ones that look wrong. Two at random. |
| Compares two versions and lists what changed | Which of those changes actually matters to your side of the deal | Confirm the version dates in the answer match the files you attached. This is where old-draft contamination shows up first. |
A worked example of the ownership split for a long review. The random spot-check is the point: checking only what looks wrong misses the answers that look right.
If the carry-forward brief works, an obvious question follows: why retype it every time?
You don’t have to. Most current assistants let you keep standing context somewhere outside the conversation. A project or workspace that holds reference files. A saved instruction set that applies to every new chat. In coding-adjacent tools this shows up as a plain markdown file in the folder, often named something like CLAUDE.md or MEMORY.md, that the assistant reads at the start of each session. Different names, same underlying idea.
The advantage isn’t storage. It’s that a file is edited, and a conversation is only ever appended to.
When the notice period changes from 60 days to 30, you open the file and change the number. The 60 is gone. In a thread, the 60 stays in there forever, sitting quietly as a candidate answer. That single difference is most of the value.
What belongs in a standing context file, if you set one up: who you are and what you’re working on, the facts that change rarely, the standing rules about what you want and don’t want, and any vocabulary specific to your organisation. What doesn’t belong: anything about one particular task, and anything you might change this week. A context file that grows without ever being pruned rots for exactly the same reason a thread does.
There’s a related habit worth building alongside this, which is keeping the prompts that worked somewhere reusable rather than rediscovering them. We’ve written separately about building a prompt library for a team, and about setting up Claude Cowork so it works from files rather than one long chat, which is the same principle applied to a whole workflow.
Worth saying what none of this fixes. A context file doesn’t stop the model degrading on genuinely long inputs; a 200-page document is still a 200-page document. It reduces how much of the crowding you’re causing yourself, which in ordinary work is most of it.
Start here. Next time an answer goes vague on you, don’t rewrite the prompt. Open a new conversation, write six lines under those headings, and see whether the answer comes back sharp. If it does, you’ve found your problem, and it was never the prompt.
It’s the drop in answer quality that happens as the amount of text in an AI conversation grows, even on tasks the same model handles reliably when the input is short. The name comes from a July 2025 technical write-up by Chroma, which tested 18 models on deliberately simple tasks like retrieving a fact or copying text back, and found performance degraded with input length across all of them. The word “rot” is slightly misleading, because nothing decays over time. A conversation left open for a week is no worse than one finished in an hour. What matters is the volume of text in the context, and particularly how much of it resembles the thing you’re currently asking about.
The degradation showed up across every model family tested, including GPT, Claude, Gemini and Qwen, so it isn’t a quirk of one product. What differs is how models fail rather than whether they do. In the distractor tests, Claude models were the most likely to abstain and say they couldn’t find an answer, while GPT models produced the highest rate of confident but incorrect responses. That difference matters more than it sounds, because a refusal is visible and a confident wrong answer isn’t. If you work mostly in one tool, it’s worth knowing which failure you’re more likely to be handed, and adjusting how hard you check accordingly.
There’s no error message, so you’re reading symptoms. Four are worth watching: it reuses a detail you already corrected, such as a superseded number or a decision you reversed; answers get noticeably more generic, in a way that would fit any company rather than yours; it claims it can’t find something you’re certain you provided; or re-asking the same question produces a differently-wrong answer each time. The second is the easiest to misread, because it looks like the model being unhelpful rather than the thread being crowded. If a conversation has gone vague on you, check its length before you start rewriting the prompt.
It fixes the crowding, but only if you’re careful about what you carry over. Asking the model to summarise the old thread and pasting that in defeats the purpose, because a good summary of a conversation includes the routes you took and abandoned, and those abandoned versions are precisely what causes the problem. What travels well is decisions and current facts: what you’re doing, what’s true now, what’s already settled, what you’ve explicitly ruled out, what you need next, and any rule about what to do when it can’t find something. Six headings, two or three minutes, and it replaces forty exchanges.
They help with the part of the problem you create yourself, which in day-to-day work is most of it. The real advantage of keeping standing context in a file rather than a thread is that a file can be edited and a conversation can only be added to. When a fact changes, you overwrite it and the old version is gone; in a thread the old version stays in the context indefinitely as a competing candidate. What they don’t fix is degradation on genuinely long inputs. If you hand a model a 200-page document, that’s still a long input regardless of how tidy your standing context is. And a memory file that only ever grows will eventually cause the same crowding, so it needs pruning like anything else.
Everything in this article about how models behave at length comes from the Chroma technical report “Context Rot: How Increasing Input Tokens Impacts LLM Performance” (Hong, Troynikov and Huber, July 2025), which was read in full on 27 August 2026 rather than summarised from coverage of it. That mattered here: several widely-circulated write-ups of this work explain the effect as information being lost in the middle of the context, and the report’s own retrieval experiment tested eleven needle positions and found no notable variation by position on that task. Presenting position as the mechanism would have been repeating a claim the source does not make. A figure that appears in several secondary summaries, describing a 20 to 50% accuracy drop between 10,000 and 100,000 tokens, could not be located in that form in the report itself and is therefore not used here. The report is published by a vendor whose product addresses retrieval, which is worth knowing, though the code to reproduce the results is public. The mapping from these findings to everyday office work is our interpretation, not theirs.
Weekly tips. Real examples. Practical help for busy professionals.
We care about your data, check out our privacy policy.