Skip to content

Chunk Size Might Beat Model Power

A note-to-self: an experiment I want to run, and the reasoning behind it.

The thing to try

Run OpenClaw + Minimax against Claude Code. The hypothesis: a better workflow and smaller chunk sizes can win even when the underlying model is weaker.

What I'm really testing isn't a model — it's whether how you structure the work matters as much as which model you throw at it. That's the part I'd find more interesting than another benchmark.

And it's not pure speculation. LangChain ran almost exactly this: holding the model fixed at gpt-5.2-codex and changing nothing but the harness, their coding agent went from Top 30 to Top 5 on Terminal Bench 2.0, 52.8 to 66.5 (Improving Deep Agents with harness engineering, LangChain). That same 66.5% edged out a stronger model under a weaker harness — "A test run with Claude Opus 4.6 scored 59.6% with an earlier harness version". LangChain's anatomy writeup makes the general version of the point: "Opus 4.6 in Claude Code scores far below Opus 4.6 in other harnesses" (The Anatomy of an Agent Harness, LangChain). So my hypothesis has already been partially observed, with numbers.

The honest caveat is that the harness lever weakens as the model improves. The same teams that show big harness wins also report tearing harness machinery out once stronger models arrived. Anthropic's harness for long-running builds dropped its sprint construct — "I started by removing the sprint construct entirely. The sprint structure had helped to decompose work into chunks for the model to work coherently. Given the improvements in Opus 4.6, there was good reason to believe that the model could natively handle the job without this sort of decomposition" (Harness design for long-running apps, Anthropic). The anatomy writeup says the same thing about harnesses in general: "That suggests harnesses should matter less over time." That doesn't sink the experiment — it dates it. The weaker-model-plus-better-harness edge has a shelf life, which is exactly why it's worth measuring now.

Why this might work: small tasks as context engineering

Spinning up a fresh agent for a small, immediate task with a clear boundary is, effectively, a form of context engineering. The fresh agent only carries the context it needs for that one task, so the effective context size stays small. This is the same move Anthropic recommends: specialized sub-agents "handle focused tasks with clean context windows," returning a distilled summary, and good context engineering means "finding the smallest possible set of high-signal tokens" (Effective context engineering, Anthropic).

The bet — the part I actually want to test — is that smaller context means sharper behavior. There's evidence pointing that way: Chroma's Context Rot study finds that "even under these minimal conditions, model performance degrades as input length increases," and that across models "we see significantly higher performance on focused prompts compared to full prompts" (Context Rot, Chroma). That's the direction I'm betting on, but it's the assumption the whole experiment is meant to check, not something I'm treating as settled.

The cost is that you pay for context loading repeatedly. Every fresh agent has to reload context to get going, and that takes tokens. So there's a tension: divide the work too finely and the overhead of repeatedly loading overlapping context becomes its own problem. I haven't seen anyone frame that reload cost as the thing that bounds how small a chunk should be — that part is my own synthesis — but the per-spin-up cost is real and concrete. Anthropic's long-running harness has every fresh coding agent re-run pwd, "read the git logs and progress files to get up to speed," and read the features list before doing any work (Effective harnesses, Anthropic).

The art is balancing the size of each task against the number of tasks. That's a familiar problem — it's the same thing an engineering leader does when breaking down work, and the kind of thing a senior engineer is often just better at than a junior one. (That last analogy is my own intuition, not something I can point to evidence for.) Worth noting that the right chunk size isn't a constant. It moves with model strength — the same reason Anthropic could drop the sprint construct for Opus 4.6, and what Karpathy noticed around December 2025 when "the generated chunks got larger, more coherent, and more reliable" (Sequoia Ascent 2026). Which loops straight back to the model-vs-harness question: as the model gets stronger, the optimal chunk grows and the harness does less.

One thing worth flagging, because it cuts against a popular analogy: human cognitive load and agent context are not the same axis. Anthropic's own framing leans the other way — it says that, like humans with limited working memory, "LLMs have an "attention budget"" they draw on. A finite budget existing on both sides is real, but it doesn't mean the units or the size intuitions transfer. The way tokens are measured doesn't map cleanly onto how much a person can hold in their head, and the shape of the degradation differs too: Chroma finds agent decay is "surprising and non-uniform," driven by position and needle-question similarity rather than a clean how-much-can-you-hold ceiling. So intuitions about chunk size that come from managing humans don't transfer one-to-one to managing agents.

Harness engineering: rules, not flows

The other piece I keep circling back to is harness engineering — defining the rules, the do's and don'ts an agent operates under.

I should be upfront that I'm narrowing the term. In its broadest use, a harness is "every piece of code, configuration, and execution logic that isn't the model itself" — system prompts, tools, sandbox, orchestration, hooks — and rules are just one component of that (The Anatomy of an Agent Harness, LangChain). What I'm interested in is the rules layer specifically.

At a glance, writing those rules looks like writing a workflow definition. But the key difference is this: a harness doesn't necessarily need to describe a flow in any formal sense. It focuses on rules. You constrain behavior without prescribing the exact sequence of steps, and let the agent find its own path within those constraints. In Anthropic's vocabulary, that's choosing the agent side over the workflow side: workflows "are systems where LLMs and tools are orchestrated through predefined code paths," whereas agents "dynamically direct their own processes" (Building effective agents, Anthropic).

And there's hard evidence that the rules-not-flows side actually wins. Anthropic's weak-to-strong researcher work found that "less imposed structure leads to better performance," and that a fixed pipeline "underperforms giving AARs no workflow at all" — rigid steps stop the agent from adapting its process to the problem in front of it (Automated weak-to-strong researcher, Anthropic).

That distinction is the part I keep coming back to.

Sources