Skip to content

The Filesystem Is Agent Infrastructure

Three things have been on my mind lately, and they may be connected — they're all about the substrate agents run on, not the agents themselves.

The filesystem as critical infrastructure

In a world of agents, the filesystem becomes a critical piece of infrastructure. It's where agents keep persistent long-term memory, how they communicate with each other, and what they treat as the source of truth. Git layers a history of iteration on top of that.

I'm not the only one who landed here. LangChain's anatomy of an agent harness derives the same primitives independently: the filesystem gives agents durable storage, it's a "natural collaboration surface" where multiple agents and humans coordinate through shared files, and "Git adds versioning to the filesystem so agents can track work, rollback errors, and branch experiments." The agent-to-agent channel isn't hypothetical, either — in Anthropic's long-running harness, communication "was handled via files": one agent writes a file, another reads it and responds in that file or a new one. And the source-of-truth claim has a clean implementation in memweave, which treats Markdown on disk as canonical — "The files are the source of truth" while the index is "always a derived cache", version-controllable with git diff like any other code.

Once the filesystem gets large, its metadata becomes the thing that decides whether you can search and retrieve efficiently on demand. The naive approach is to organize metadata as a parallel structure mirroring the filesystem hierarchy, which gets you log(n) lookups. But this is really just information retrieval — and we've used indices and databases for decades to retrieve data efficiently with rich queries. The hard part isn't the database; it's keeping the actual filesystem in sync with the metadata. Plain Unix filesystem semantics aren't enough for that.

That sync problem already has a known design answer: don't make the metadata an authoritative parallel store, make it a derived, rebuildable cache keyed by content. memweave does exactly this — its central idea is to "separate storage from search", with a SQLite index that can be rebuilt from the files and SHA-256 fingerprints so only changed chunks re-embed. Mintlify's ChromaFs shows the metadata mirrors the hierarchy approach running in production: it intercepts grep, cat, ls, and find and translates them into queries against a Chroma database, stores the whole file tree as a gzipped JSON metadata document, and uses the database as a coarse filter before a fine in-memory pass. The abstract argument has working hardware behind it.

There's also a scalability angle, though here I'm extrapolating past anything in my reading: a single physical disk won't hold all the data, so you end up with network-attached storage, and at that point efficient management of metadata and data retrieval stops being optional. That's the classic distributed-filesystem story, and I'm assuming it carries over to agent workloads rather than citing a study that says so.

Is a tree even the right shape?

Here's the question I keep coming back to: is the tree-based filesystem actually the best fit for agents? Traditional filesystems are human-friendly because they mirror how humans organize things. But agents aren't humans.

I want to be precise about two layers that are easy to conflate. Consider a flat, path-addressed namespace: no directories, every file uniquely identified by a path string, where a directory is just a special kind of file. The identifier is still the name — the path — exactly like a normal filesystem; what's gone is the rigid tree of nested directories. This is roughly the shape of a large, flat path-addressed namespace I worked with at a previous job, where a path is the only handle you ever need.

Now go one level down, to physical storage. There's no reason a file has to be a single contiguous object. A file can be a logical representation — multiple chunks of data that physically sit on one or more network disks, with a metadata file (or a hierarchy of metadata files) responsible for grouping the chunks that belong together. Those chunks can be content-addressed: identified and deduplicated by a hash of their bytes, the way Git objects or IPFS blocks are. That's the distinction the storage world already draws — the namespace you query is path-addressed, the blob store underneath is content-addressed — and it's worth keeping straight, because they solve different problems. Path-addressing gives you a stable name to ask for; content-addressing gives you dedup and integrity for the bytes behind that name.

I should flag the strongest counter-argument, because it cuts at the is a tree wrong? framing directly. PageIndex argues the hierarchical tree is precisely what an LLM navigates best: it builds a tree index meant to mirror how human experts navigate documents, and it explicitly lists "No Chunking" as a feature, organizing content into "natural sections, not artificial chunks." That's the opposite of what I'm describing on both axes — tree good, chunking bad. But I think we're talking about different layers. PageIndex is about the reasoning and retrieval structure an LLM walks at query time, where a clean table-of-contents tree genuinely helps. I'm about the physical storage and addressing layout underneath, where chunking is an implementation detail the agent never reasons over directly. You can keep PageIndex's tree for navigation and still let a chunked, content-addressed store hold the bytes. Proxy-Pointer RAG is close to this synthesis already: its chunks carry "Structural Metadata Pointers" — doc ID, node ID, title, line ranges — that point back to the full contiguous section, so the chunk is never the unit of meaning; the metadata reassembles it. That's my metadata-file-groups-the-chunks idea, realized in a retrieval pipeline.

This maps onto how agents already load things. An agent first loads only the descriptors of all available skills; a skill's content loads only when the agent decides to use it; its references and scripts load later still, when the agent actually reaches for them. That's a metadata-then-content hierarchy, and it isn't my invention — it's the established mechanism behind progressive disclosure. Skills, in the harness literature, solve "too many tools or MCP servers loaded into context" via progressive disclosure, loading skill front-matter rather than every tool at agent start. Anthropic frames the same move as just-in-time context: agents "maintain lightweight identifiers (file paths, stored queries, web links, etc.) and use these references to dynamically load data into context at runtime". And someone has already built the storage layer to match: OpenViking processes context into an "L0/L1/L2 three-tier structure, loaded on demand" — abstract, then overview, then full detail — fused with a filesystem paradigm. The metadata-then-content store I'm describing isn't speculative; it's shipping.

We have a benchmark gap for skills and agents — and it's starting to close

The second thing I've been thinking about is evaluation. People train agents, or claim their skills are useful — but whether those skills and agents actually transfer to similar or general tasks is mostly unknown. From what I could see when I first wrote this, it was word-of-mouth, with no public benchmark to point at. Contrast that with LLMs, where you can run multiple models against the same dataset and get comparable scores.

I have to correct myself here, because my own reading since then contradicts the strong version of that claim. SkillsBench now bills itself as the "first benchmark for evaluating how well AI agents use skills" — public on Hugging Face, Apache 2.0, and run against named models (GPT-5.5, Claude Opus 4.8, Gemini 3.1 Pro, and others). That's close to the comparable-scores-across-models setup I said was missing. It isn't alone: Terminal Bench is a public agent benchmark with a leaderboard, concrete enough that LangChain reports moving an agent "Top 30 to Top 5" by changing only the harness. So the honest version of my claim is narrower: the gap was real, and it's only now starting to be filled.

But starting is the operative word, and the part I actually care about is still open. By its own design, SkillsBench targets "tasks requiring skill composition (2+ skills)" where state-of-the-art still sits under 50% — it measures task success on a fixed, curated task set, the same way SWE-Bench or Terminal Bench do. That is not the same as measuring transfer: whether a skill that works here generalizes to a task it was never written for. And none of these benchmarks give you a cost-optimized selection over the (model, skill, agent) space. An open, transparent evaluation framework is the foundation for what comes after: it's what would let you monetize skills and agents, and it's what would tell you which direction to iterate. It would also give developers a real answer to is this combination of model, skill, and agent good enough for this task? — and, more importantly, how to optimize cost across that space.

A framework like this could also solve what I'd call skill pollution and agent pollution. Instead of manually installing skills, the framework could search for, adopt, and adapt skills and agents to fit the task at hand. A user sets a budget for a task, and the framework automatically finds a balanced (model, skill, agent) solution — trading off against time and energy(?).

Parts of this are recognized as open, and parts already work one layer down. LangChain's harness anatomy lists "harnesses that dynamically assemble the right tools and context just-in-time for a given task instead of being pre-configured" as a live open problem — that's the auto-adopt half of my proposal, framed as unsolved. And benchmark-driven self-improvement at the harness level is already running: AutoAgent and the Meta-Harness pattern use a coding agent to read traces and improve the harness, hill-climbing on a benchmark score. The new contribution I'm pointing at isn't optimize with a benchmark — that exists — it's adding a cost/budget dimension across the (model, skill, agent) space, which the harness-optimization work doesn't.

Save the prompt, not the code

The third idea: prompts versus generated code.

At a previous job I worked on a large monorepo where generated code wasn't checked in. It was produced on the fly at build time. So the analogous question for agents is: can we save only the prompts and context, and generate the program or software on demand?

The obvious objection is that LLMs are unpredictable — the generated program might not come out the same twice. But step back. If we have clear and comprehensive exit criteria — not just error cases, but latency, performance, correctness — does it actually matter what the code looks like? The code gets translated to machine code by a compiler anyway.

The save the spec, generate the software half of this already exists as a shipping methodology. GitHub's Spec Kit defines spec-driven development as the point where specifications "become executable", "directly generating working implementations rather than just guiding them." The piece those tools keep is the source code; the spec generates it, but the code stays the durable artifact you read and maintain. My step is the more radical one they don't take: if the code is always regenerated on the fly, drop the source representation entirely. Take it further still — could the model generate machine code directly? What's the benefit of a programming language as the intermediate output? We need that intermediate form mainly so a human can review and maintain the code. Remove the human-review requirement and the intermediate form loses its reason to exist. (The machine-code-as-output idea is the most speculative thing here; I haven't seen anyone actually do it, and it may founder on debuggability long before correctness.)

The thing I'm least sure about is cost. Saving generated software saves model cost and time at the expense of disk(?). Saving only prompts means redoing the generation, which can be expensive. So what if we save the thinking process too? A lot of the cost of producing code comes from making mistakes, fixing them, and looping through validation — on top of the raw tool cost. If we could save all of those intermediate artifacts — capturing the right reasoning path without the error-then-fix detour, caching tool results — maybe regeneration stops being the expensive option.

Save the reasoning, not just the result is a live research direction. Google's ReasoningBank is a "memory mechanism for agents that learns from both successful and failed trajectories, with reasoning stored as memory content", and it proposes "memory-aware test-time scaling" — treating accumulated experience as a new scaling dimension. But it cuts against me on the exact point I was most confident about. I wanted to strip the error-then-fix detour and keep only the clean path. ReasoningBank deliberately keeps the failed trajectories, because the failures are signal. Trace-based continual learning makes the same bet: all of those flows run on traces, "the full execution path of what an agent did", detours included. So the open question sharpens into something I genuinely can't answer yet: is the error-then-fix loop pure waste to be cached away, or is it information the next generation needs to avoid repeating the same mistakes? If the latter, save only the clean path is exactly the wrong compression.

If there's a thread through all three, it's that the agent is only as good as the substrate it stands on — and that substrate still feels mostly inherited from the human-tooling era rather than designed for agents on purpose.

Sources