Why long AI conversations get worse, and what to do about it

Every long AI chat I have follows the same arc. The first ten messages are sharp. Around message thirty, the model starts re-answering questions I asked an hour ago. Somewhere past fifty it forgets the constraint I set at the start, contradicts its own earlier advice, and calls a function by a name we renamed on Tuesday. Nothing about the model changed. The only thing that changed is the length of the chat.

This is usually explained as “you hit the context limit,” and that is part of it. But most conversations degrade well before any hard limit, and understanding why tells you exactly what to do about it.

What a long chat actually sends

A chat interface hides a blunt mechanic: every time you send a message, the entire conversation goes with it. The model has no memory between turns. It rereads everything, every time: your questions, its answers, the dead ends, the draft you rejected, the tangent about error handling that went nowhere.

Three things go wrong as that history grows.

First, the hard ceiling. Every model reads a fixed number of tokens per request, and a week of back-and-forth gets there faster than you’d think. What happens at the boundary depends on the tool, and it’s rarely announced: older messages get dropped or compressed, and the model quietly loses the instructions you gave first, which are usually the ones that mattered most.

Second, dilution. Long before the ceiling, the useful parts of the history are competing with the useless parts. The paragraph where you defined the requirements sits next to four abandoned approaches and the answer you explicitly corrected. The model weighs all of it. Relevant detail buried in the middle of a long context is exactly the detail models are worst at retrieving.

Third, poisoning. A wrong answer doesn’t leave the conversation when you correct it. It stays in the history and keeps getting reread, and models have a strong pull toward consistency with their own earlier output. One confidently wrong turn at message twenty keeps steering message sixty. This is why a chat that took a bad turn rarely fully recovers, no matter how carefully you correct it.

The usual fixes trade one loss for another

I tried all the standard workarounds before building around the problem.

Starting a fresh chat works, but you pay the re-explanation tax: ten minutes rebuilding context the old chat already had, and the two genuinely good replies from the old thread are gone, buried in a conversation you’ll never scroll through again.

Asking the model to summarize the chat and carrying the summary forward is lossy in the worst way. The summary keeps the conclusions and drops the reasoning, and any mistake in it becomes the new ground truth for everything after.

And selectively re-pasting the good parts into a new chat is the most honest workaround, because it’s the correct instinct: curate the context by hand. You’re just doing it with scroll, copy, and paste, in an interface built on the assumption that you’d never need to.

Short threads, on purpose

The fix I’ve landed on isn’t a bigger context window. It’s keeping every thread short enough that dilution and poisoning never get a foothold, without losing the work that made the thread valuable. That takes three moves that linear chat doesn’t have.

The first is branching at the moment a tangent appears. On a canvas, each message is a node, and when a side question comes up (say, “wait, how should we handle the legacy rows?”) I don’t type it into the main thread. I fork from the node where it came up, and the tangent becomes its own path with the full history behind that point already in scope. The original thread continues untouched, still on topic. Each branch ends up holding one question, which means each branch stays short, which means each branch stays sharp. The chat never gets long because no single path ever does.

The second is referencing instead of re-pasting. When a branch needs one specific thing from elsewhere in the workspace (the schema from another chat, the requirements node from Monday), an @ reference pulls exactly that node into context. Not the whole other conversation. One node, chosen on purpose.

The third is merging at the end. After a few days of this, a workspace holds several short branches with several settled answers: one worked out the schema, one settled the rollout order, one handled the legacy rows. A merge pulls the ends of those paths into scope for one message, and I ask a normal question: “write the full migration plan.” The model sees the three conclusions I selected, not the two hundred messages behind them. It’s the fresh-chat trick, except nothing gets lost and nothing gets retyped.

A side benefit of short, separate branches: the model is chosen per message, so I can have GPT-5.5 review a plan that Claude Opus 4.8 drafted, from the same fork point, with identical context. Fresh eyes without a second subscription tab. I do most of my debugging this way, one hypothesis per branch.

What this doesn’t fix

Branching does not extend the context limit. A branch you keep growing for two hundred messages degrades like any other chat; the point is that you stop needing one. The curation is also still yours to do: NodePad makes what-the-model-sees visible and choosable, but it won’t decide for you what matters. There are no hidden summaries or injected memory, on purpose. And the interface is honestly not what your hands expect from a chat app. Working on a canvas takes a few sessions to feel natural, and if your AI conversations are short and disposable, a plain linear chat is the simpler tool and you should keep using it.

But if your real work keeps outliving your chats, the pattern is worth stealing even before the tool: keep threads short, one question each, and assemble answers from the pieces you kept on purpose.

The whole canvas is free to try on community models, no card required. Start a workspace and fork the next tangent instead of following it.