Path + Document: a Unix-like VFS and chunked-graph documents.

2026-07-11 · by Dennis Gubsky · ~12 min read

loomcycle has always been honest about what it stored. Memory keys things by (scope, key). Volumes by name. Channels by topic. The agent dialect is opaque ids. That worked when an agent's job was "do work, write some state, finish" and the operator never had to look. It started to feel wrong as soon as the launch publishing plan ran for three weeks across this exact session, as a single linear Markdown file that I had to rewrite by hand every time a publication moved buckets.

Two related gaps were sitting on the roadmap. First, agents and humans had no shared, human-readable namespace for the things the runtime stored. An agent could write a Memory key called launch/day-5/mastodon and that worked, but a Volume could not have a / in its name, and a Document (none of which existed yet) had no name at all, just a UUID. Three resources, three naming worlds, no ls. Second, "Document" was the obvious primitive to build on top of the existing Memory + SQL Memory work: a chunked-graph thing where each chunk is a first-class unit with hierarchy and type and edges, the agent's natural shape, the surface that finally makes the launch plan something an agent and I can co-author cleanly.

Today's v1.4.0 ships both. The headline:

The rest of this post: what each primitive is, why the inode/dirent split is the right shape, how Document's content/structure split composes with SQL Memory, the on-every-transport piece, and a concrete walkthrough of modeling the launch publishing plan as a Document at /docs/launch.

Path: an inode/dirent split for the agent runtime

A Memory key, a Volume name, and a Document UUID are three unrelated namespaces with no shared structure. You can't ls them together. You can't say "everything under /projects/acme". You can't rename a thing without rewriting every reference to it. The temptation was to add a path column to each resource table and call it done.

That would have re-implemented the same mapping three times, and it wouldn't have expressed directories, cross-kind listings, or atomic subtree moves. The shape that gets all three is the one the kernel has been shipping since 1969: inode/dirent separation. A resource keeps its permanent id (the inode). A directory entry maps (parent_path, name) → resource (the dirent). The two live in different tables. Renaming a thing is a dirent update that never touches the resource. One tree spans all the resource kinds. The runtime VFS, applied to substrate primitives instead of disk blocks.

The dirents table sits in the runtime store, alongside volumedefs and agentdefs. Each row carries the tenant, the scope (agent / user / tenant), the parent path, the leaf name, a kind enum (document / volume_mount / memory_entry), and a resource_ref JSON pointer at the backing resource. The composite key (tenant, scope, scope_id, parent_path, name) indexes lookups. Both backends (sqlite and postgres) carry the table.

The six ops, and the things they don't do

Paths reject .. (not resolved, rejected at the boundary) and forbid inner slashes inside a segment, which is the logical analog of the host-path confinement loomcycle already does in sandbox.go. Segments are [a-zA-Z0-9._-]+, max 64 segments, max 1024 chars total. Trees are per-(tenant, scope, scope_id); cross-tenant resolution returns opaque 404. Scope at the wire is agent (default), user, or tenant, matching Memory's convention.

A dirent is a name, not an authority grant

The trust posture took the most thought. Resolving /docs/launch to a Document id does not, by itself, let you read that Document. The resource's own scope and tenant check still applies. A dirent is a name. Authorization is unchanged. The risk Path introduces is integrity (a wrong-name mapping pointing at the wrong resource), not confidentiality (no widened access). That distinction is what made it acceptable to ship the unified tree from day one across Memory, Volumes, and Documents without rebuilding any of the existing scope discipline.

Resources opt in to a name

Path itself never creates resources. Each resource opts in to a name at creation time:

• Memory: Memory.set { ..., path: "/notes/today" } registers a memory_entry dirent alongside the K/V write. Memory.get { path: ... } resolves it. Keys without a path keep their flat semantics (existing keys are not auto-promoted; that would be inappropriate migration scope creep).
• Volumes: VolumeDef.create { ..., mount_at: "/vol/repo" } registers a volume_mount dirent. Default mount is /vol/<name> when mount_at is omitted. Existing Volumes get implicit mounts at /vol/<name> lazily on first resolve or ls against the path (no migration; the dirent materializes idempotently).
• Documents: Document.create_document { ..., path: "/docs/launch" } registers a document dirent. get_document accepts either id or path.

SQL Memory stays out of the tree. A per-scope SQL database is not a named resource; SELECT FROM chunks does not compose with ls. It is the wrong abstraction for the path tree, likely forever.

Document: chunked-graph storage on Memory + SQL Memory

A Document is a tree of chunks. Each chunk is a first-class unit with a UUID, a position in the hierarchy, an optional type (supertag-like: publication / review-finding / architect-output), structured fields, an optional status, a Markdown body, and a revision integer that grows by one on every update. Edges are first-class too: a separate chunk_edges row connects two chunks with a kind string (promotes / targets / implements), supporting fast bidirectional lookup.

The load-bearing design decision is where to put the data.

Content in Memory, structure in SQL Memory

Chunk content (title, body, fields) lives in Memory, keyed by the chunk's UUID. Chunk structure (document_id, parent_id, position, type, status, title, revision, plus the edges table and the type-definitions table) lives in SQL Memory across four tables: documents, chunks, chunk_edges, chunk_types.

Why the split. Three reasons:

1. Different access patterns. A chunk body is fetched whole, lazily, when a user opens that chunk. Structure is queried in bulk every time the Web UI renders a tree or filters by status. SQL is the right shape for the second; key-value blob storage is the right shape for the first.
2. Audit discipline. Memory's existing audit captures content changes. SQL Memory's audit captures structure changes. Two streams: who edited what content, who restructured what relations.
3. Backup composition. Memory's snapshot captures bodies. SQL Memory's snapshot (RFC AA Phase 3e) captures structure. Either alone is incomplete; together they survive cross-instance restore (RFC X).

Documents require SQL Memory enabled (LOOMCYCLE_SQLMEM_ENABLED=1). Scope is agent or user in v1.4.0; tenant scope is deferred because SQL Memory itself does not yet have a tenant scope. Tenant isolation still applies (it rides on the SQL Memory scope key).

The 13 ops

Same shape as every other loomcycle tool: {tool:"document", input:{op:..., ...}}. The ops, grouped:

Three behaviors worth calling out, because they matter at the trust boundary:

• Optimistic revision concurrency on update. update_chunk takes the current revision; a stale revision returns a conflict (the agent re-reads, re-applies, retries). The model can't silently overwrite a concurrent edit. The Web UI uses the same dance.
• Atomic deletes. delete_document and delete_chunk run the whole cascade (chunks, edges in both directions, type defs scoped to the doc) inside one SQL Memory transaction. A crash mid-delete leaves a consistent state, not a half-deleted graph. Edge cleanup is bidirectional: removing a chunk also removes incoming cross-document edges, so no dangling references.
• Endpoint validation on edges. link_chunks validates both endpoints exist before writing. delete_chunk refuses a document's root chunk (the cascade is rooted there; deleting it should go through delete_document). move_chunk guards against cycles (moving an ancestor under its descendant).

The query shape

query_chunks is the agent's main read path and the one that earns its keep. Three layers:

// Structured filters (the common case)
{"tool":"document",
 "input":{"op":"query_chunks",
          "document_id":"...",
          "type":"publication",
          "status":"draft",
          "limit":20}}

// Path-joined filter: find Documents under a path
{"tool":"document",
 "input":{"op":"query_chunks",
          "under_path":"/docs/launches/",
          "type":"publication"}}

// Raw SQL escape hatch, gated by the SQL Memory validator
{"tool":"document",
 "input":{"op":"query_chunks",
          "sql":"SELECT id, title, status FROM chunks
                 WHERE type='publication'
                 AND fields->>'platform' = 'mastodon'
                 ORDER BY fields->>'date'"}}

The under_path filter is the Path↔Document glue. Pass a directory and the query restricts to Documents at or under that path. The raw sql: route goes through SQL Memory's existing statement validator (RFC AA Phase 1's allowlist for the modernc.org/sqlite driver: no ATTACH / VACUUM / PRAGMA / quoted-load_extension / multi-statement smuggling; writes refused from a read-only op). The agent can write joins and aggregates across the chunk graph and get exactly what it needs.

On every transport: humans co-author with agents

The third piece of v1.4.0 is the part that took the longest to design. Path and Document are agent-facing primitives, but the launch publishing plan and the RFC archive are things I need to edit, not just agents. The first cut had Document as an in-band MCP tool only, callable from inside a run. The second cut, after using it for an hour, was that this is wrong: the right shape is a primitive a human or a UI can drive directly, off-run, against the same scoped namespace agents build.

Both primitives are now first-class on every transport:

All four surfaces dispatch through one op-discriminated Connector method per tool, reusing the existing SubstrateRequest / SubstrateResponse shape. The cross-transport pattern is the one RFC AI established for interactive sessions.

The trust posture for the off-run surface is the load-bearing constraint here. Scope and tenant are resolved server-side from the authenticated principal, never the wire. An off-run call with scope:"user" keys on the principal's subject, which is exactly how an in-band agent op behaves for the same scope. So an external UI authenticated as user_id=alice reads and writes the same user-scoped namespace as the agents running for user_id=alice. They interoperate by default. There is no way for the wire to forge a scope_id.

Both surfaces are tenant-confined under ScopeTenant. substrate:admin also satisfies the scope check, for operator-driven cross-tenant administration.

Modeling the launch publishing plan as a Document

Concrete enough that you can run it. The script lives at examples/; here is the path-shaped narrative.

Create the Document, give it a name in the tree, and define the chunk types this plan uses:

const doc = await client.document({
  op: "create_document",
  title: "Launch publishing plan",
  scope: "user",
  path: "/docs/launches/v1.0",
})

await client.document({
  op: "define_type",
  document_id: doc.document_id,
  name: "publication",
  fields: [
    { name: "platform", type: "enum", values: ["mastodon","bluesky","x","linkedin"] },
    { name: "date",     type: "date"   },
    { name: "status",   type: "enum", values: ["draft","scheduled","published"] },
  ],
})

Add a day, then a publication under that day, and link the publication to the blog post chunk it promotes:

const day5 = await client.document({
  op: "create_chunk",
  document_id: doc.document_id,
  parent_id: doc.root_chunk_id,
  type: "day",
  title: "Day 5 — Friday 2026-06-19",
})

const pub = await client.document({
  op: "create_chunk",
  document_id: doc.document_id,
  parent_id: day5.id,
  type: "publication",
  title: "Mastodon — interactive terminal digest",
  body:  "An interactive terminal in your agent runtime's Web UI: ...",
  fields: { platform: "mastodon", date: "2026-06-19", status: "published" },
})

await client.document({
  op: "link_chunks",
  from_id: pub.id,
  to_id: blogPostChunkId,
  kind: "promotes",
})

Query the workflow. "What publications are still draft?" used to be a manual grep; now:

const drafts = await client.document({
  op: "query_chunks",
  document_id: doc.document_id,
  type: "publication",
  status: "draft",
})

// Or path-joined: "everything under any launch document"
const allLaunchDrafts = await client.document({
  op: "query_chunks",
  under_path: "/docs/launches/",
  type: "publication",
  status: "draft",
})

List the same tree from Path to verify the Document slots in cleanly:

await client.path({ op: "ls", path: "/docs/launches/" })
// → [{kind:"document", name:"v1.0", resource_ref:{document_id:"..."}}]

await client.path({ op: "resolve", path: "/docs/launches/v1.0" })
// → {kind:"document", resource_ref:{document_id:"..."}, full_path:"/docs/launches/v1.0"}

// Move the Document later (rename or re-parent; document_id unchanged)
await client.path({ op: "mv", from: "/docs/launches/v1.0", to: "/docs/archive/v1.0" })

The macOS bundle semantics: a Document at /docs/launches/v1.0 lists as a directory in Path (the dirent has kind: document) AND resolves as one resource. The Web UI will render it expandable: click the directory, see the chunks underneath.

Compatibility and the adapter bump

Additive at the runtime layer. No breaking changes. New HTTP endpoints, gRPC RPCs (riding the existing SubstrateRequest / SubstrateResponse shape), MCP meta-tools; nothing consumed those surfaces before, so no wire break. The dirents table is a new migration on both backends; deployments that don't use Path see zero behavior change because resources only get a dirent when they opt in via path: or mount_at:.

The adapters bump in this release. @loomcycle/[email protected] and python-v1.4.0 add client.path() and client.document(). The two adapter lines realign on 1.4.0 the way they realigned on 1.1.1, so a single release tag publishes both. v1.3.0 carried the no-change-adapter-version pattern (Bashbox was an in-band tool, no wire surface), so the previous adapters at 1.1.1 stay compatible with v1.4.0 against the parts they already speak; only Path / Document operations require the bump.

The Path + Document core (internal/tools/builtin/pathtool.go, document.go) plus the dirents runtime-store table shipped on main ahead of this tag in PRs #538 through #542. v1.4.0 is the first release to carry them as a cut tag.

How to enable it

storage: ...
runtime: ...

env:
  # Document requires SQL Memory (RFC AA, shipped v1.2.0):
  LOOMCYCLE_SQLMEM_ENABLED: "1"
  LOOMCYCLE_SQLMEM_ROOT: "/work/sqlmem"

agents:
  my-document-agent:
    allowed_tools: [Memory, Channel, Path, Document]
    sql_scopes: [user]   # Document needs SQL Memory scopes

# Try the off-run surface directly:
git clone https://github.com/denn-gubsky/loomcycle
cd loomcycle
./loomcycle serve   # in one terminal

# in another terminal, talk to the off-run Path tool:
curl -X POST http://localhost:8080/v1/_path \
  -H "Authorization: Bearer $OPERATOR_TOKEN" \
  -d '{"op":"ls","path":"/","scope":"user"}'

# Or the TS adapter:
npm i @loomcycle/[email protected]
node -e "
const { LoomcycleClient } = require('@loomcycle/client')
const c = new LoomcycleClient({ baseUrl, token })
c.path({op:'ls', path:'/', scope:'user'}).then(console.log)
"

Full release notes in REVISIONS.md. Operator topics: docs/PATH.md and docs/DOCUMENTS.md. The Context op=help in-runtime help has path and document topics for agent-facing reference.

No breaking changes; deployments that don't use Path or Document see zero behavior change. v1.0 through v1.3.0 users upgrade safely. The adapter bump (@loomcycle/[email protected] and [email protected]) is required only for code that calls client.path() or client.document(); older code keeps working unchanged.

Companion reading: v1.3.0, Bashbox (the in-process shell sandbox), v1.2.0, SQL Memory (the substrate Document depends on), v1.1.0, Filesystem Volumes (the per-agent ro/rw roots Path mounts).