Budgeted Reads

The problem #

Agents burn their context on whole files and repeated reads, and naive RAG chunking loses structure. Models are already good at grep, tail and reading by line: Cursor writes over-long tool output to files for agents to read on demand, and in runs that called an MCP tool, dynamic discovery of tool descriptions cut total tokens by 46.9%.

So the problem is not “build another retrieval system”, but to give reading on top of files a budgeted, checkable contract.

How it works #

• Files first. Standard reads (including line ranges) are always available; Budgeted Reads is an optional extension on top and does not replace the file system.
• Deterministic outlines (free): the heading tree of Markdown, code symbols (tree-sitter), keys of JSON and YAML, headers and row counts of CSV. No LLM is needed.
• Budget and degradation. A request carries budget_tokens; if it does not fit, it degrades to a coarser view instead of truncating silently; the response states the actual view, the token count and whether it is generated.
• Search returns snippets and pointers, not whole files.
• Evidence pointers: nas://<node>/<tree>/<path>@<commit>#L12-L18, stable and verifiable against a commit; dereference returns the exact content.
• What changed since last time: returns the diff between two commits instead of the whole file.
• An optional LLM summary layer: as a cache, marked generated: true and with a staleness marker.

Start small #

• Add nothing beyond nas_read: let the agent read the old way, with rg and sed -n.
• Then add a small outline tool for Markdown and code.
• Start evidence pointers as “path@commit#line”; no new service is needed.

Interface sketch #

Illustrative only. Field names may change before specification v0.3.

POST /v1/files/read
{ "path": "/agents/cleo/memory/notes/customer-acme.md",
  "view": "outline", "budget_tokens": 400 }

200 OK
{ "view": "outline", "tokens": 212, "generated": false, "truncated": false,
  "content": "# Acme\n## Renewal (L12-L18)\n## Contacts (L19-L31)\n## Open issues (L32-L40)",
  "evidence": [
    { "pointer": "nas://home-box/agents/cleo/memory/notes/customer-acme.md@9f2c1de#L1-L40" }
  ] }

What the specification asks for #

• A response MUST state the view actually returned and its token count; generative views MUST be marked.
• When the budget is too small, the NAS MUST degrade to a coarser view and MUST NOT truncate silently.
• Standard file reads (including ranges) MUST remain available.
• Evidence pointers MUST be verifiable against a commit.
• Search MUST by default return snippets and pointers instead of whole files.

Borrowed, and what is new #

Progressive disclosure and grep are not new in themselves. What is new: making budgets and evidence pointers a read contract at the storage layer, so answers can be checked: you can get from a sentence back to the lines of the commit it came from.

• Cursor: dynamic context discovery
• Agent Skills (progressive disclosure)
• Letta Context Repositories
• RAPTOR

Honest limits #

• Tokenizers differ between models, so token counts are estimates.
• It only works if clients adopt it.
• Summaries go stale and can be wrong, so they are only a cache and always marked.

Open questions #

• How should budget error across different model tokenizers be handled?
• How is summary quality evaluated?

Questions #

Related concepts #

• Agent Memory Notes you can read and edit are the truth. Indexes are a cache you can throw away.
• Agent Home A directory per role, with history. Plain files you can take with you.
• Flight Recorder A per-run, hash-linked record that includes what the run read.