Skip to content

Make Regenerating Code Safe, Not Generated Code Good

I've been reading blogs and listening to podcasts on vibe coding recently, and trying it a few times on my own. It's amazing so far — but I've only used it on smaller tasks: writing a script, analyzing a few files. I'm very skeptical about applying vibe coding to large projects, where the context is exponentially greater and a lot of information is hard to extract just by reading code. That second part is especially true for existing projects and microservices. Worse, the existing documentation might be wrong and misleading because it's gone stale.

I'm not alone in drawing that line. In his Sequoia Ascent talk, Karpathy splits "vibe coding" — which "raises the floor," letting almost anyone build software by describing what they want, mostly for prototypes and personal tools — from "agentic engineering," the professional discipline of coordinating fallible agents while preserving correctness, security, taste, and maintainability (Sequoia Ascent 2026 summary). The small-task magic and the large-project grind are two different jobs.

A colleague made a good point: at that scale, the main challenge is no longer prompt engineering but context engineering — which Anthropic frames as "the natural progression of prompt engineering," moving beyond the initial prompt to manage the whole state available to the model (Effective context engineering, Anthropic). I'm pessimistic even about that reframing. My bet is that LLMs won't evolve to handle the context of large projects well, because the interconnection between components blows up exponentially. And there's hard evidence that bigger, more interconnected context actively hurts: Chroma's Context Rot study finds that "even under these minimal conditions, model performance degrades as input length increases," and that models do markedly better on focused inputs than on full inputs padded with irrelevant context (Context Rot, Chroma). Just stuffing the whole interconnected codebase into a larger context window isn't a strategy — Google's framework team makes the same point: leaning on "ever-larger context windows in foundation models" is the prevailing instinct, but "simply giving agents more space to paste text can not be the single scaling strategy" (Architecting a context-aware multi-agent framework, Google).

This shows up in a few ways:

  • The last 20% is the expensive part. Getting a piece of code to 80% pleasant takes 20% of the effort, but pushing for the remaining 20% of completeness or correctness takes 80%. The deepest reason: 100% accuracy isn't a designed use case for an LLM. Karpathy puts a finer mechanism on this — "jagged intelligence" (capability is non-uniform, spiking where outputs are verifiable) and the distinction that traditional software automates what can be specified, while LLMs automate what can be verified (Sequoia Ascent 2026). The last mile grinds precisely where outputs aren't cheaply verifiable. On top of that, a convoluted codebase mixed with generated code and accumulated debt make the stretch worse. You can watch this happen even on the small tasks: Simon Willison vibe-codes whole single-file SwiftUI menu-bar apps in almost no time, then openly admits he's "unqualified to evaluate if the numbers and charts... are credible or accurate" — one of his apps reported wrong memory numbers and he isn't confident it's reporting correctly even after the model adjusted it (Vibe coding SwiftUI apps, Simon Willison). That's the last 20%, in miniature.
  • Generated code is harder to review. It tends to be lengthy and touches more parts than necessary — something humans are less likely to do, both because of our limited mental capacity and because we're actively discouraged from sprawling diffs. So humans either spend more time reviewing the generated code as-is, or spend more time splitting it apart. The industry is already conceding this point: acai.sh built a dashboard for requirement-level review — "No more file-by-file GitHub PR reviews," you "review requirements instead," because conventional diff review doesn't scale to large agent-generated PRs (acai.sh).
  • Generated code accumulates debt faster. Between the review burden and generally lower quality, the debt piles up quickly. Matt Pocock says it almost word-for-word: "AI agents can accelerate software entropy, making codebases complex and hard to change at an unprecedented rate" (mattpocock/skills).

My take: change what you optimize for

Instead of spending time trying to make generated code better, make regenerating code safer.

Worth saying up front: Pocock's remedy for that accelerating entropy is the opposite of mine. He invests in codebase architecture daily — a /improve-codebase-architecture skill to rescue the ball of mud — and warns that process-owning tools like Spec-Kit, GSD, and BMAD "take away control" (mattpocock/skills). I think that fight is worth having on a human-maintained codebase, but it gets harder to win as more of the code is machine-written. So I want to lean the other way: not better architecture per unit, but cheaper disposal of the unit.

Take microservices. Imagine each one is very well defined and dead simple, doing only one thing. Then it's much easier to rewrite the entire service when you need to. And when you need a slightly different API — say, some new filtering logic — instead of iterating on the existing one, always generate a new one. Yes, that means a significant amount of duplicate code. But code generation is cheap, and reviewing code that is purely an addition to what exists is easy. Cleaning up is simple too: just remove the dead code.

I'll be honest that this is an extension, not an established idea. The closest precedent is pets vs. cattle, whose actual core — as its author insists — is "the disposability of servers": you treat them as interchangeable and replaceable rather than hand-tended (Pets vs Cattle). I'm borrowing that vocabulary but pushing it down a level, from servers to source code. Nobody I've read actually argues for deliberately duplicating source. The decoupling instinct is everywhere, though. Anthropic's managed-agents writeup decouples components so that "each agent component became an interface with few assumptions about the others, allowing independent failure or replacement" — explicitly cattle, not pets (Scaling Managed Agents, Anthropic). The Ralph-loop crowd resets agent memory between iterations: "By resetting its memory each iteration, the agent avoids accumulating confusion from prior tasks and stays focused," and "The result is cleaner code and fewer hallucinations" (Self-Improving Coding Agents). All of these stop at the task or service boundary; I'm asking what happens if the unit you're willing to throw away is the code itself.

Taking a step back, this is really about splitting code into smaller, isolated units, so the blast radius of any rewrite is small and controlled. Clear contracts are what make that safe. Anthropic's harness work describes a "sprint contract": before each sprint, "the generator and evaluator negotiated a sprint contract: agreeing on what 'done' looked like for that chunk of work before any code was written" (Harness design for long-running apps, Anthropic). Same idea, one layer down: a well-defined contract per unit is what lets you regenerate behind it without fear.

The honest objection to "regeneration is cheap, so just regenerate" is that generation isn't free or deterministic. acai.sh frames it as a thought experiment: if software were "generated instantly," always producing "same deterministic output," and cost "nothing," then "your criteria for acceptability is really the only thing of value. The spec." But generation is none of those things — which is the whole reason QA and validation become the new bottleneck (acai.sh). So my bet only holds where the unit is small enough that regeneration is genuinely cheap and the contract is tight enough that you can verify the result. Outside that envelope, "just regenerate" is wishful.

The prompts become the documentation

Here's the part I actually care about most, and it's where my intro about stale docs was heading the whole time.

If you push this far enough, the inter-component connections get exponentially more complicated — to the point where no single human could understand all of them. But I'd argue that's already the case for most large projects today. The difference is what you treat as the source of truth. Once the unit is generated rather than hand-written, the durable artifact stops being the human-written documentation — which goes stale the moment it's written — and becomes the instructions that generated the unit. With clearly defined contracts, a machine can digest those just fine. Karpathy's OpenClaw example is the clean version of this: an installer can be "a block of instructions given to an agent, rather than a brittle shell script" — the instructions are the artifact, the code is just derivable output (Sequoia Ascent 2026).

I have to engage the strongest counter here, because it cuts against my first instinct. My draft phrasing was "look at the prompts, not the docs" — and the spec-driven camp would say that's only half right. acai.sh argues that "Docs and unstructured specs can get you very, very far. Much farther than prompts alone," and that "Spec-writing is where the act of software engineering really happens" (acai.sh). GitHub's Spec Kit agrees in spirit: spec-driven development "flips the script on traditional software development," where for decades "code has been king — specifications were just scaffolding we built and discarded," and now "specifications become executable" — but the durable artifact is a structured, versioned specification, not an ad-hoc prompt log (Spec Kit).

They're right, and it sharpens my point rather than refuting it. A raw prompt drifts and is non-deterministic, same as the code it produces — pinning your source of truth to a chat transcript is no better than pinning it to stale Markdown. So the claim I actually want to make is the structured version: the source of truth becomes the spec or instructions that each unit is regenerated from, versioned and reviewed like code, not the prose documentation a human wrote once and never touched again. That's the artifact worth maintaining. Everything downstream of it is cattle.

Sources