Skip to content

From Prompt to Context to Harness: Notes on the Agent OS

Coding with AI keeps shifting under us — from prompt engineering, to context engineering, to now harness engineering — and it's genuinely hard to keep up. People are starting to talk about moving from vibe coding to agentic engineering. Here's how I've come to think about that progression.

The thread running through all of it: a harness is how you finally deliver what prompt and context engineering were always reaching for. Alignment on what to build, and continuity of context across a long task — you get those by building mechanics around the model instead of hoping the model holds everything in its head. The earlier stages were us asking the model to be disciplined. The harness stage is us building the discipline into the environment.

Prompt to context

With more capable models, you no longer need to hand-write a well-formatted prompt that carries all the information and instructions up front. Models can ask interview questions or gather resources on their own. This isn't just my impression — Anthropic frames context engineering as "the natural progression of prompt engineering", and notes that the exact formatting of prompts is likely becoming less important as models improve. The work moves from crafting the perfect instruction to curating the right set of tokens.

But as models gather more and more information, their context — their brain capacity — becomes the bottleneck. And the failure modes here aren't one thing; they're at least three distinct things, which I find worth separating because they have different fixes:

  • Forgetting. As the context window fills, the model gets worse at recalling what's in it. Chroma's Context Rot research names exactly this: recall degrades as input tokens grow, independent of whether the fact is still sitting right there in the window.
  • Wrapping up early. Models sometimes declare a task done before it is. Anthropic has a precise name for this too — "context anxiety": Claude Sonnet 4.5 "would wrap up tasks prematurely as it sensed its context limit approaching." (Tellingly, the same harness on Opus 4.5 didn't show it — the behavior was a model property, not a harness one.)
  • Drifting from the goal. The model wanders off the original objective on a long task — my own observation, and one I don't have a clean named-phenomenon citation for the way the other two have.

Three problems, not one blob of the model gets flaky. That distinction matters because the harness fixes them differently.

Context to harness

I joined the crowd when this evolution happened. I started building workflows of my own — with skills, rules, and memories — so the model always plans ahead, breaks down tasks, persists artifacts, keeps track of progress across sessions, and validates its own work before calling something complete.

That, to me, is the gist of a harness: not a single skill or plugin, but a set of mechanics that make the model perform reliably. A few of the concrete moves, each mapped back to the stage it's quietly solving:

  • Interviewing the user (/grill-me, /brainstorming) to lock down what to build — using a harness to get what prompt engineering was after, alignment, without depending on a perfect opening prompt.
  • Persisting artifacts as files, so context and progress survive /clear, /compact, and the jump between subagents — using a harness to get what context engineering was after. There's a clean worked example of this in Anthropic's long-running-agents harness: an initializer agent writes an init.sh, a progress file, a JSON feature list (all features initially marked as "failing"), and a first git commit; every later session begins with "no memory of what came before" and rebuilds its bearings from the git log plus the progress file. The durable files are the memory.
  • Verifying its own work, which is what something like the Ralph Loop is built for.

A note on that last one, because the naive version — just let the model check itself — doesn't hold up. Anthropic's harness-design write-up found that when you ask agents to grade their own output, they tend to respond by "confidently praising the work—even when, to a human observer, the quality is obviously mediocre." The lever that actually works is separating the agent doing the work from the agent judging it — a generator/evaluator split, GAN-style. So real verification usually isn't self-review; it's an external judge wired into the loop. The Ralph Loop itself, for the record, traces back to Ryan Carson and Geoffrey Huntley (per Addy Osmani's self-improving coding agents); at the harness level it "intercepts the model's exit attempt via a hook and reinjects the original prompt in a clean context window," per LangChain's Anatomy of an Agent Harness — which only works because durable files hold the state between resets.

What I'm folding into "harness" — and where that's contested

I should be honest that I'm packing a lot into one word. When I say the harness is "skills, rules, and memories," I'm drawing the boundary more loosely than some people who've thought about this harder.

On one side, Sarah Wooders backs the instinct that memory isn't a plugin, it's the harness: managing context, and therefore memory, is a core capability and responsibility of the harness, not something bolted on. That's the part of my claim I'd defend.

But two of my own sources draw the line in a place that cuts against me. LangChain's "Anatomy of an Agent Harness" defines the harness as "every piece of code, configuration, and execution logic that isn't the model itself" — system prompts, tools, sandbox, orchestration, hooks. By that definition the harness is the whole apparatus and skills are just primitives inside it; "a set of mechanics" is too small. And their "Continual learning for AI agents" goes the other way and splits agent systems into three layers — model, harness, and context — where context "sits outside the harness" and "consists of things like instructions, skills, even tools," also commonly called memory. Which is to say: the exact things I fold in, they fold out.

I don't think this is just terminology. The useful distinction the layered view buys you is where learning happens: you can improve the harness code (offline, at the agent level) or update context/skills (per user or tenant, sometimes in the hot path) without touching the other. So I'll scope my term: when I say "harness" loosely I mean the whole operating apparatus around the model — and when the boundary matters, the harness-vs-context split is the sharper tool.

Harness as the agent OS

One key component of a harness is the operating environment — where models gather context, execute tasks, and validate work. Another way to look at it: the operating system for agents.

The terms that follow are my own coinage; no source I've read uses them. But each pole maps cleanly onto a real system. From what I can tell there are two shapes:

  • Envelop mode: the agent is contained inside an OS. One OS can hold one or more agent instances, but each instance lives in exactly one OS — one-to-many, OS to agent.
  • Sit-aside mode: the agent acts through an OS, like a proxy. One OS can serve many agents, and one agent can talk to many OS — many-to-many.

The benefit of envelop mode is that the OS is the primitive. Agents can communicate freely inside it without leaking out, and the OS gives you an elegant place to monitor and safeguard everything they do. The cost is that each OS is relatively heavy and somewhat one-shot — it has to ship the agent a filesystem, network, the works. Rivet's agentOS is a clean example of this pole: it runs agents in-process on Wasm and V8 isolates (the same isolation tech behind Google Chrome), mounts arbitrary storage backends — S3, SQLite, Google Drive — as a plain directory tree, and enforces per-agent network, filesystem, CPU, and memory limits. The OS is the container, and it's cheap enough (~6 ms cold starts in their numbers) that "heavy" is relative.

Sit-aside mode decouples the agent from the OS. An agent can talk to several OS of different types — one that only does file operations, another that only does network search — so the OS sits closer to an MCP or tool provider, and each can scale up or down independently. Anthropic's Managed Agents is the worked example here: they decouple the "brain" (Claude plus its harness) from the "hands" (sandboxes and tools, called as execute(name, input) → string) and the "session" (the event log). Hands become "cattle" — interchangeable, provisioned on demand, and, because no hand is bound to a brain, passable from one brain to another. That's my many-to-many, OS-as-tool-provider mode, made concrete.

Here's where I have to refine my own framing rather than defend it. I presented envelop and sit-aside as two co-equal options. The Managed Agents post doesn't read that way. They started coupled — everything in one container, the "pet" model — and moved to decoupled specifically because coupling caused real pain: a container crash meant the whole session was lost, connecting to a customer's own VPC was awkward, and putting a full brain in every container was expensive. Decoupling dropped their p50 time-to-first-token by roughly 60% and p95 by over 90%. So the industry isn't treating these as symmetric picks; decoupling is the documented direction of travel, away from coupling's failure modes.

That actually sharpens my one real worry about sit-aside mode, which is the microservices curse. Once an agent talks to many OS, just understanding what it depends on gets hard, and restricting an agent's tools takes more work than dropping it in a container that simply lacks the tool. You can manage it with a configuration or agent-management layer that defines the accessible tools, OS, and MCPs per agent type — but that's more moving parts, the same tax microservices always charge. So the tradeoff isn't envelop-vs-sit-aside as a free choice; it's that decoupling buys you reliability and scale at the cost of a coordination problem you then have to engineer around.

Roughly: envelop mode is Docker or a VM — easy to stand up for ad-hoc or small-scale use, on-demand like cloud. Sit-aside mode is more like a cloud provider or SaaS, where you hand your agents to the platform to run.

Skills and tools on the fly

Models already write scripts and code on the fly to solve tasks. And frameworks are already distilling that into something durable. NousResearch's Hermes advertises a built-in learning loop that "creates skills from experience, improves them during use, nudges itself to persist knowledge." It's not the only one — OpenClaw does offline "dreaming" to rewrite its own SOUL.md, a pattern LangChain's continual-learning post catalogs as context-layer learning, i.e. an agent self-modifying its own skills without a human author in the loop. (Hermes is also model-agnostic enough to run on MiniMax, GLM, Kimi, and a dozen others — the skill-creation loop isn't tied to one vendor's model.)

So my guess: with more capable models and more compute, models will increasingly create their own skills and tools, and a user will rarely need to install or write them by hand. I'll keep the hedge — this is a belief, not a result.

If that holds, providers like Claude Code could maintain a database of skills and tools, or outsource it to a third party. Instead of installing ahead of time, Claude Code could figure out just-in-time which skill, tool, or MCP to use by consulting such a platform — or ask it to provide one, drawn from the pool or created from scratch. This isn't a wild extrapolation; it's a named open question in the literature I'm reading. The Anatomy post closes by listing exactly this among its open problems: "harnesses that dynamically assemble the right tools and context just-in-time for a given task instead of being pre-configured."

The hard part for such a platform is answering one question: what is the best skill or tool for this task? That implies a wide collection of assets and, more importantly, enough experience or benchmark results to know whether a given skill actually fits. When I first wrote this down I assumed no such benchmark existed yet. I was wrong — SkillsBench bills itself as the first benchmark for how well agents use skills, measuring both skill effectiveness and agent behavior, with composition tasks (two or more skills) deliberately tuned so state-of-the-art lands under 50%. That's roughly the ranking layer a skill marketplace would need, and it already exists in early form.

A few properties such a platform would have (only the first is a hard constraint; the other two are what that constraint buys you):

  • It has to be fast — so live benchmarking or evaluation per request is off the table. The ranking has to be mostly precomputed.
  • It can take feedback from Claude Code on results. This loop isn't hypothetical: Anthropic reports that you can use Claude Code to automatically optimize your tools against an evaluation, using held-out test sets to avoid overfitting. The agent improving the tools is already a working pattern.
  • It can save each requested task into its eval set, growing the benchmark over time so future ranking gets better.

Whether the right design is "rank from a pool" or "just generate a fresh skill every time," I don't know yet. But the pieces — autonomous skill creation, a skills benchmark, an agent-optimizes-tools loop, and a named JIT-assembly problem — are all already on the table. That's the part that makes me think this is closer than it looks.

Sources