Smart zone, dumb zone: size coding agents' work to fit

A coding agent gets worse long before its context window is full. The early part of a session is the smart zone, where the model follows instructions and keeps track of what it has read. After that comes the dumb zone, where the same model starts to forget, repeat itself and guess. The decline is gradual and raises no error, so you rarely catch it in the middle of a task. The fix comes earlier. Size each task before the session starts so it fits inside one fresh smart zone, however large the context window is.

Matt Pocock's AI Hero dictionary illustrates the term with a sentence most agent users will recognise: "It nailed the first three components and just butchered the fourth." By the fourth component, the agent was also carrying everything it had read and written for the first three.

We plan and size all our agent work against a smart zone of about 150k tokens, and we do the sizing with Pocock's open-source skills.

Key takeaways

• Bigger windows don't remove the decline. In NoLiMa, GPT-4o fell from 99.3% on short inputs to 69.7% at 32k tokens.
• Most practitioner rules of thumb put the line between about 75k and 150k tokens; a few go further on 1M-token models. We plan against roughly 150k and treat it as a budget that moves with each model generation.
• Size the task, not the session. A spec describes the destination, and vertical-slice tickets each fit one fresh session.
• When a ticket runs past its budget, stop and split it. Pushing through and re-explaining only adds more context.

The smart zone is the stretch of a session in which an agent works well. Dex Horthy of HumanLayer uses "dumb zone" for what follows. Gergely Orosz, who credits Horthy with coining "context engineering", summarises it in The Pragmatic Engineer: "You hit the 'dumb zone' when its performance starts to degrade." Matt Pocock's AI Hero dictionary defines both terms as working vocabulary for developers.

The cause sits in how transformer models read. Each token attends to the tokens before it, so the number of relationships grows quadratically with the length of the input.

Anthropic's guide to context engineering describes "n² pairwise relationships for n tokens" and calls the model's limited capacity to track them an "attention budget".

Pocock's dictionary entry on attention relationships puts numbers on it: 1,000 tokens make roughly 1 million relationships, and 100,000 tokens make roughly 10 billion. The model's attention gets spread across far more pairs, and each piece of your instruction gets a thinner share.

Anthropic concludes that context is "a finite resource with diminishing marginal returns" and recommends aiming for "the smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome."

Nothing breaks at a precise token count. In the words of the dictionary's entry on attention degradation, "There's no error and no threshold; each turn is only slightly worse than the last."

Pocock's smart zone entry lists the signs that you're in the dumb zone:

• The agent forgets an instruction you gave twenty turns ago.
• It repeats a mistake it had already corrected.
• It confidently asserts something the context contradicts.

Nor does the smart zone grow with the window. A model that can hold 1M tokens doesn't have a smart zone of 1M tokens, or of any fixed fraction of it.

A context window is how much a model can hold. How much it can use well is a smaller number, and benchmark after benchmark shows it falling as input grows, often far below the advertised limit.

• RULER (NVIDIA, COLM 2024): of 17 long-context models that claim at least 32k tokens, "only half of them can maintain satisfactory performance at the length of 32K".
• NoLiMa (Adobe Research and LMU Munich, ICML 2025): of 13 models that claim at least 128k tokens, "11 models drop below 50% of their strong short-length baselines" at 32k. GPT-4o dropped "from an almost-perfect baseline of 99.3% to 69.7%".
• Lost in the Middle (Liu et al., Stanford, TACL 2024): models use information less reliably when it sits in the middle of a long context than when it sits at the start or end.
• Context Rot (Chroma, July 2025): across 18 models, "model performance consistently degrades with increasing input length". Even a single distractor reduced performance, and on LongMemEval, focused prompts of about 300 tokens beat full prompts of about 113k tokens for every model family tested.

Coding agents carry a second load on top of retrieval, because they have to follow many instructions at once. Jaroslawicz et al. found that even the best models they tested reach only 68% accuracy when given 500 simultaneous instructions. A long agentic session piles up instructions from the system prompt, project rules, your requests and your corrections.

HumanLayer tested this in practice. When Opus 4.6 with a 1M-token window became the Claude Code default, they saw instruction-following degrade even well below 200k tokens and reverted to Opus 4.5.

They also changed their context warning. It used to fire at 40% of Sonnet's 168k usable window (200k minus 32k reserved for output). For long-context models it now fires at a fixed 100k tokens, which is only 10% of a 1M window.

They concluded that "long-context models degrade at all context lengths, not just long ones" and that "more context isn't more capability - the instruction budget doesn't scale with the context window."

Epoch AI reported in June 2025 that the input length at which top models reach 80% accuracy "has risen by over 250x in the past 9 months", while noting that "accuracy declines as input grows". On the 8-needle 1M variant of MRCR v2, Anthropic reports that "Opus 4.6 scores 76%, whereas Sonnet 4.5 scores just 18.5%".

Practitioners see the same drift. The thresholds in the next section have moved up with each model generation. But a retrieval benchmark measures whether a model can find planted facts. It doesn't measure whether an agent keeps following a dozen project conventions while it edits its fortieth file. That's why we still side with HumanLayer: "Context isolation beats context expansion."

Practitioners give different numbers, and nobody has measured any of them. These are rules of thumb from people who run agents every day:

The numbers drift upwards as models improve, and most of them sit well below the advertised windows. The Pragmatic Engineer article sums up the shared instinct: "As a rule of thumb, the less of the context window that is used, the better the outcomes are."

At illumiqon we plan against a smart zone of about 150k tokens. It is the size limit for a ticket. If a ticket looks unlikely to finish inside it in a fresh session, we split it before anyone starts. We treat the figure as a budget that moves with each model generation and revisit it when we switch models.

You can't reliably notice the dumb zone from inside it, so size the work when you decide what a session is for, before it starts.

Pocock's dictionary has the best line on this: "The smart zone is a budget, and unrelated work spends it." A quick question about a different module, a debugging detour and a long log you pasted to check one line all eat into the same budget the main task needs. Even routine tool output counts. HumanLayer measured that output from passing tests can use 2–3% of the context window to convey a result that needs fewer than 10 tokens, noting that "every line of PASS src/utils/helper.test.ts is waste."

So each session gets one task, and the planning question becomes whether that task fits.

If a task is bigger than one smart zone, split it. Developers have always split large work into smaller pieces; the smart zone just gives the pieces a size limit.

Pocock's vocabulary splits work into two layers. A spec describes the destination, what you're building and why. Tickets describe the journey, the steps that get you there, and each ticket is sized to fit one session.

The obvious way to cut a feature is by layer: database first, then the API, then the UI. Pocock's /to-tickets skill rules that out. Each slice must be "vertical, NOT a horizontal slice of one layer". Cut by layer, nothing works end to end until the last layer lands, so you get feedback far too late.

Instead, each ticket should be a vertical slice, or a tracer bullet in the sense of The Pragmatic Programmer: a thin path through every layer that you can run and check on its own. Each slice is sized to fit one fresh smart zone, and each one tells you early whether the whole design holds up.

Plans change once code meets reality. When they do, you don't patch tickets one by one. You close the tickets nobody has built yet, revise the spec, and generate fresh tickets from it. The spec is the durable record of decisions; the tickets are just its current route.

We use mattpocock/skills, which Pocock describes as "My agent skills that I use every day to do real engineering - not vibe coding." They're open source, and together they cover the path from a vague idea to merged code.

/grill-me puts grilling into practice. The agent interviews you in rounds about what you want to build, following a design tree. Each round asks about the frontier, meaning the decisions whose prerequisites are already settled. The grilling ends when the frontier is empty, so no decision is left for the agent to guess at later.

/grill-with-docs does the same and also updates the project's domain glossary and architecture decision records as decisions land. Later sessions then start from documentation rather than from a long chat history.

Grilling costs a round of questions, so we don't grill everything. Our rule of thumb is to skip it when the agent can't get the answer wrong in a way that matters, and to grill when a wrong guess is expensive or someone else's call.

We skip grilling when:

• The answer already exists. The codebase, docs or a ticket holds it, and the agent can look it up: "Upgrade React Query from 5.40 to 5.52."
• There's a pattern to copy. A sibling feature made the decision already: "Add a CSV export to Invoices, the same as Orders has."
• It's mechanical. Behaviour stays the same and only the structure changes: "Rename fetchUser to getUser everywhere."
• It's measurable. A number or a test says whether it worked: "Add an index on orders.user_id; the query is slow."
• It's cheap to undo. Redoing it costs less than one round of questions: "Make the toast disappear after 5 seconds instead of 3."

We grill when:

• It deletes or rewrites existing data. Old records have to fit the new rule, or be lost. "Make usernames case-insensitive." What happens to "Tom" and "tom"?
• It touches money. Invoices, taxes and refunds have legal weight. "Offer a yearly plan at 20% off." What about upgrades mid-month, or refunds?
• It touches security or sessions. Each choice trades convenience against safety. "Let users change their email." Verify the new address first? Notify the old one?
• It exposes something about users. Privacy defaults are hard to walk back. "Add a 'Last seen' indicator to profiles." Who sees it, and can users hide it?
• Outsiders depend on it. Once it's public, changing it breaks someone else. "Add pagination to the /orders API." Cursor or offset? Is it a breaking change?
• Users read it or receive it. Wording, timing and tone are our call, not the agent's. "Send a welcome email on signup." When, and what does it say?

When we're unsure, we ask the agent: "What would you have to guess at?" If the answer is nothing, or we'd accept every guess, we skip grilling. If we'd dispute even one guess, we grill, usually one round of two or three questions. The tasks we skip grilling for are usually small enough to fit one session without a spec or tickets at all.

/to-spec turns the grilling conversation into an issue without interviewing you again. The spec covers the problem, the solution, a long numbered list of user stories, implementation decisions, testing decisions and what's out of scope. It deliberately leaves out file paths and code snippets, because those go stale as soon as the code moves.

/to-tickets breaks the spec into tracer-bullet tickets. Its rules are explicit: "Each slice cuts a narrow but COMPLETE path through every layer (schema, API, UI, tests)", "A completed slice is demoable or verifiable on its own", and "Each slice is sized to fit in a single fresh context window".

Each ticket declares which other tickets block it. Prefactoring comes first; the skill quotes Kent Beck's "Make the change easy, then make the easy change." The skill also quizzes you on granularity before it creates anything, which is where you catch a ticket that's still too big. For wide refactors that can't be sliced vertically, it uses expand–contract instead: add the new path, migrate callers, then remove the old one.

/implement builds one ticket. It uses /tdd to write tests first, at test seams it agrees with you up front. It then runs /code-review, which checks the change against project standards and against the spec in two parallel subagents, and commits.

Between planning and building, we clear the session rather than compact it. The spec and tickets are handoff artefacts that already hold every decision, so the planning conversation is disposable. As the dictionary puts it, compaction "tries to carry the essentials across; clearing bets they're already written down somewhere better." Carrying a long planning conversation into implementation would spend much of the smart zone before the first line of code.

/implement-spec runs a whole spec. "The tickets are not a list of steps. They are a task graph with blocking relationships between them." Implementer subagents each work on one unblocked ticket in their own worktree, a merger subagent brings the work together, and each finished ticket unblocks the ones waiting on it. Code review runs once at the end.

The orchestrating session passes context pointers rather than the content itself, so its own context stays small.

At every phase boundary you choose what carries over into the next session. Pocock's dictionary gives the options:

1. Continue if smart-zone budget is left and the current context is relevant to the next step.
2. Clear if the context is disposable because artefacts already hold what matters.
3. Hand off to move work to a fresh session, another agent or a colleague, through a written artefact.
4. Use a subagent for noisy work such as a broad search or an automated review. The subagent "will burn its own context window on the noise" and reports back only the result.
5. Compact otherwise, ideally "at a phase boundary, after the plan is settled, not mid-task."

/handoff writes a portable Markdown document to the operating system's temp directory. It points to existing artefacts instead of copying them ("Do not duplicate content already captured in other artifacts (specs, plans, ADRs, issues, commits, diffs).") and lists the skills the next session should use. The receiving session starts small and knows where to look.

Most oversized tickets can be spotted before anyone starts them. While planning, be suspicious of a ticket that:

• touches every layer horizontally, such as "add all the database tables" or "build every screen",
• can't be demonstrated or verified on its own,
• needs the agent to read most of the codebase before it can start.

While running, watch for the dumb-zone signs: forgotten instructions, repeated mistakes, confident claims the context contradicts. Keep the token count visible too, for example in your agent's status line, and notice when a session passes the budget you planned for.

You can estimate a ticket before it starts from what the session will have to load: the spec and the ticket, the files and modules the agent must read, and the test and log output it will produce along the way. Planning is expensive too, which is why planning and building get separate sessions.

The size is also testable after the fact. The AI Hero ticket entry gives both directions: "If sessions on your tickets routinely degrade before the work is done, the tickets are too big; split them. If each session spends most of its context on setup before doing five minutes of work, they're too small; merge them."

When a session degrades before its ticket is done, stop. The tempting response is to push through and re-explain, which only adds more context. Close the session, split the ticket into smaller vertical slices, and start each one fresh.

If you keep one line from this Post, make it Pocock's: "Plan around the smart zone, not the window."

If you size agent work differently, we'd like to hear how: get in touch.

• Matt Pocock, "Smart zone", AI Hero AI Coding Dictionary, read 31 October 2026: https://www.aihero.dev/ai-coding-dictionary/smart-zone
• Matt Pocock, AI Hero AI Coding Dictionary, further entries read 31 October 2026: attention-relationship, attention-degradation, spec, ticket, clearing, compaction, handoff, handoff-artifact, subagent, context-pointer, grilling
• Matt Pocock, AI Coding Crash Course, AI Hero (paid workshop; lesson "Smart Zone / Dumb Zone"), read October 2026: https://www.aihero.dev/workshops/ai-coding-crash-course
• Matt Pocock, mattpocock/skills, GitHub: https://github.com/mattpocock/skills
• Dex Horthy, "Getting AI to Work in Complex Codebases" (Advanced Context Engineering), HumanLayer, August 2025: https://github.com/humanlayer/advanced-context-engineering-for-coding-agents/blob/main/ace-fca.md
• Dex Horthy, "Context-Efficient Backpressure for Coding Agents", HumanLayer, 9 December 2025: https://www.humanlayer.dev/blog/context-efficient-backpressure
• Kyle (@0xblacklight), "Long-Context Isn't the Answer", HumanLayer, 23 March 2026: https://www.humanlayer.dev/blog/long-context-isnt-the-answer
• Gergely Orosz, "Context engineering with Dex Horthy", The Pragmatic Engineer, 15 July 2026: https://newsletter.pragmaticengineer.com/p/context-engineering-with-dex-horthy
• Hsieh et al., "RULER: What's the Real Context Size of Your Long-Context Language Models?", NVIDIA, COLM 2024: https://arxiv.org/abs/2404.06654
• Modarressi et al., "NoLiMa: Long-Context Evaluation Beyond Literal Matching", Adobe Research and LMU Munich, ICML 2025: https://arxiv.org/abs/2502.05167
• Liu et al., "Lost in the Middle: How Language Models Use Long Contexts", Stanford, TACL 2024: https://arxiv.org/abs/2307.03172
• Hong, Troynikov and Huber, "Context Rot", Chroma, 14 July 2025: https://www.trychroma.com/research/context-rot
• Anthropic, "Effective context engineering for AI agents", 29 September 2025: https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents
• Jaroslawicz et al., "How Many Instructions Can LLMs Follow at Once?", July 2025: https://arxiv.org/abs/2507.11538
• Burnham and Adamczewski, "Context windows", Epoch AI, 25 June 2025: https://epoch.ai/data-insights/context-windows
• Anthropic, "Introducing Claude Opus 4.6", 5 February 2026: https://www.anthropic.com/news/claude-opus-4-6

Thomas Schmidt is the founder of illumiqon GmbH, an Austrian company that builds software with coding agents. This Post was researched and drafted with AI assistance (Claude Code) and edited and fact-checked by the author against the sources listed above. Legal notice · Privacy