Loomcycle's data plane opens up.

2026-08-25 · by Dennis Gubsky · v1.54.0 → v1.55.1 · ~14 min read

An agent runtime's memory is only useful to the extent other things can reach it. Loomcycle has been shipping richer and richer internal memory surfaces for months: a per-scope memory that holds facts and notes and documents in one search, bi-temporal validity so a correction never destroys the record, provenance on every write. Inside a run, an agent has all of it. Outside a run, until this week, most of it was a private detail of the runtime.

That changes. Loomcycle now publishes a documented HTTP contract over its memory and document surfaces, exposes those same surfaces to a peer loomcycle as a native backend, streams every write to subscribers as a change feed, and gives the Python adapter the memory shape it was missing. And on top of that, documents become federated: a document on one loomcycle can bind to a peer document on another, and the two can be reconciled bidirectionally with a body-plus-tags-plus-hierarchy sync that keeps the losing side's overwritten body in its own history.

This post walks the two arcs. RFC CD opens the surfaces to anyone with a bearer. RFC CE lets loomcycle instances share documents between themselves.

Part 1: an HTTP contract you can hand to a code generator

The first surface is the one that removes the most friction. Every loomcycle instance now serves an OpenAPI 3.1 spec at GET /v1/openapi.yaml (and /v1/openapi.json for tools that prefer JSON), documenting the whole external data surface: the memory REST family, the POST /v1/_memory/search unified-search endpoint, the /v1/_document and /v1/_path op-dispatch (modeled as an OpenAPI oneOf on the op field so a generator produces one method per op with distinct request shapes), and the asset GET for image binaries.

The contract is public. The data is not. Both endpoints serve without authentication because the spec itself is non-secret, but every operation it describes is bearer-gated. So an operator can point a code generator at any loomcycle instance and get a typed SDK without holding a token; the generated SDK then attaches a bearer at request time.

curl https://app.loomcycle.cloud/v1/openapi.yaml -o loomcycle.yaml
openapi-generator generate -i loomcycle.yaml -g python -o ./client-py
# or -g typescript-axios, or -g go, or any of the 60+ targets

A self-contained Swagger UI console lives at /v1/docs: vendored, no CDN, works air-gapped. Paste a bearer via the Authorize button, click Try it out on any operation, get real responses from your live instance. It's the fastest path from "read the spec" to "watch the endpoint answer."

A drift test pins the spec's op enums to the tool op sets in code, so the contract cannot silently fall behind the implementation. If a new op lands on the Document tool and no one adds it to the spec, the build fails.

Full guide including auth semantics and generator recipes: docs/EXTERNAL_API.md.

Part 2: a peer loomcycle as a memory backend

The second surface is more interesting from an architecture perspective. Loomcycle's memory layer has always been pluggable: an agent's memory is served by a Backend implementation, and the operator picks which one (in-process default, external mem-layer service, etc.) at config time. As of v1.54.0, one of those options is another loomcycle instance.

memory_backends:
  team-shared:
    kind: remote
    base_url: https://app.loomcycle.cloud
    api_key_env: TEAM_SHARED_TOKEN
    fallback_on_error: true

That is enough. Bind an AgentDef to memory_backend: team-shared, and every memory operation the agent makes (get, set, list, delete, search, stats) round-trips to the peer's /v1/_memory/*. The peer embeds server-side, runs the same search, applies its own tenant fold. From the agent's point of view, nothing changed. From the operator's point of view, this instance's agents share a memory with the peer's tenants.

Three properties matter for the security posture:

The auth is a credential-allowlisted api_key_env. Not ${...}-interpolated at YAML parse time. Not an infrastructure secret pulled from a well-known variable. A name that must appear on the operator's credential allowlist, resolved at request time from the process environment or a CredentialDef. The runtime never persists the bearer value on the config; if the allowlist doesn't include the name, the backend refuses to start.

The peer host is dialed through the SSRF guard. Same guard as any other outbound HTTP call from the runtime. The base URL cannot be an internal address, cannot resolve to an internal address after DNS, cannot smuggle a redirect to one. If you point base_url at http://169.254.169.254/, the runtime refuses.

fallback_on_error degrades to local. A network hiccup or a peer restart does not crash the caller's memory operation; the backend falls back to the local memory for the call, logs, and keeps trying the peer on subsequent calls. Set to false if you want the caller to see the error directly.

An operator can also author a remote backend from the Web UI's Integrations tab. Same shape, no YAML edit required.

Part 3: a change feed you can subscribe to

The third surface is a change data capture stream. Every write to memory or documents (in-run tool call, MCP call, HTTP call, gRPC call, everything that reaches the store) can emit a change event, and any consumer inside or outside the runtime can subscribe.

The feed is opt-in and value-free by design. Opt-in because the emit adds cost; value-free because the events carry keys (scope, scope_id, key/id, kind, op, tenant) but not the row's body. A subscriber that needs the value fetches it. That keeps the feed small, keeps secrets out of a channel that fans out to many subscribers, and lets consumers filter cheaply.

Enable it with LOOMCYCLE_MEMORY_CHANGES_ENABLED (there is a parallel switch for the document feed), and two consumer shapes become available.

SSE subscription

curl -N -H "Authorization: Bearer $TOKEN" \
  https://app.loomcycle.cloud/v1/_memory/changes

event: change
data: {"scope":"user","scope_id":"alice","key":"pref.diet","op":"set", ...}

event: change
data: {"scope":"agent","scope_id":"consolidator","key":"doc.chunk:...","op":"delete", ...}

A Server-Sent-Events stream. Cursor-based; each event carries a monotonic ID. Reconnect with Last-Event-ID to resume. Tenant-scoped: an operator sees their own tenant's events; admin sees all.

HMAC-signed webhook

change_subscriptions:
  audit-pipeline:
    target: https://audit.internal/loomcycle-changes
    events: [memory, document]
    hmac_secret_env: AUDIT_HMAC_SECRET

A config-declared subscription. The runtime POSTs to the target URL with an HMAC-SHA256 signature over the raw body (same shape RFC H input webhooks use, on the inbound side). Cursor is persisted, so an outage on the receiver's side doesn't drop events; delivery is at-least-once and the cursor advances only on 2xx. The SSRF guard applies to the target URL the same way it applies to any outbound.

This is operator observability. It is gated at substrate:tenant, not member-accessible, because the metadata itself (which subjects a tenant is writing about, what keys, when) is more revealing than any single value. An audit pipeline, a data-lake ingester, a real-time analytics feed, a mirror to a warm standby: these are the natural consumers.

Part 4: Python parity on memory

A small but load-bearing addition. The Python adapter (pip install loomcycle) has always been gRPC-only, and the memory tool did not have a gRPC RPC. Its adapter surface was HTTP-only, which Python did not carry. So a Python caller could reach documents and paths through the adapter but had to hand-roll HTTP requests to reach memory.

v1.54.0 adds a generic Memory RPC on the gRPC surface, riding the same op-dispatch shape as documents and paths, and exposes it as client.memory(op="...", ...) in the Python adapter. Every op the HTTP surface has is now on gRPC and in Python. The three transports (HTTP, gRPC, MCP) plus the two adapters (TypeScript, Python) are one uniform surface.

from loomcycle import Client
client = Client(endpoint="app.loomcycle.cloud:443", token=os.environ["TOKEN"])

# recall: the "what have I been told about X" call
result = client.memory(op="recall", scope="user", query="which medicine do I use")

# search: the wider call, across sources
result = client.memory(op="search", scope="user", query="statin", sources=["facts"])

Part 5: documents federate between instances

This is the other half of the release, and it changes what a document can be. Until v1.54.0, a loomcycle document was a local object: authored on one instance, stored on one instance, read on one instance. If two loomcycles wanted the same document, an operator exported it as Markdown from one and imported it into the other by hand, and any subsequent edits diverged silently.

RFC CE makes documents federated. A local document binds to a peer document, and the two can be reconciled in either direction with a real merge model.

Declaring peers, or authoring them at runtime

document_sources:
  team-canonical:
    base_url: https://app.loomcycle.cloud
    api_key_env: TEAM_CANONICAL_TOKEN

A static peer declared in loomcycle.yaml. Or the runtime-authored equivalent through the new DocumentSourceDef substrate: a versioned, tenant-scoped Def with the same create/fork/get/list/retire/promote/verify shape as MemoryBackendDef. Every transport supports it (HTTP, gRPC, MCP, TypeScript, Python), and it is a first-class substrate primitive, not a config bolt-on. Authoring a source at runtime means a tenant operator can add a peer without a config redeploy.

When set_remote and sync resolve a source name, they check the tenant's substrate DocumentSourceDefs first, then the static YAML, then any shared cross-tenant substrate. Tenant-authored wins.

Binding a document to a peer

Document op=set_remote
   id: <local document id>
   source: team-canonical
   remote_path: /docs/architecture

One local document, one peer document, one binding. Bindings are one-to-one. A local document can be bound to at most one peer document, and vice versa. The binding is a substrate fact: it persists across restarts, survives snapshots, and is auditable through the store.

Reconciling: pull, push, or preview first

Reconciliation runs through op=sync. The direction argument picks the flow.

• direction: pull (default) copies the peer's chunks into the local document. Diverged chunks are updated in place; the overwritten body is kept in the local chunk's history (retire-not-delete). Peer-only chunks are created locally. Local-only unkeyed chunks are excluded (they don't have a stable identity to reconcile against) and counted in the response.
• direction: push writes local chunks up to the peer. Same rules, other direction. The peer keeps the overwritten body in its own history.

The key on which reconciliation runs is natural_key. A chunk without one is excluded and counted; only keyed chunks reconcile. That is deliberate: natural_key is the identity contract for a chunk that means the same thing on both sides. A section titled "Draft notes" on your instance is not the same as a section titled "Draft notes" on the peer just because they have the same title.

What travels with a keyed chunk:

• Body. The chunk's Markdown content, verbatim. A diverged body updates in place; the previous body lands in chunk history.
• Tags. The full tag set, replacing the target's tag set.
• Hierarchy. The chunk lands under its keyed parent (matched by natural_key) at the same sibling position it had on the source side.
• Manual cross-reference links. Explicit link_chunks edges between keyed chunks. Parser-derived [[name]] edges regenerate automatically from the body on the target side; they aren't copied.

diff_remote: a dry run

Before you sync, op=diff_remote is a read-only dry-run that classifies keyed chunks into six buckets, plus unkeyed and edge-drift counts:

Neither side is touched. Every classification is deterministic, and the response is what you'd see a pull or a push produce if you ran it. So the intended flow for a nontrivial sync is: diff_remote first, read the change set, then sync in the direction that matches what you want to happen.

The eight review findings, worth naming

The RFC CE line ran through a full-feature adversarial code review before ship, and eight defects came out of it. All fixed with fail-before regression tests. Two are worth naming because they define the guarantees:

Sync now converges on a refuted chunk (no create-churn). A chunk whose write-time judge refuted it on one side would sync to the peer, get refuted there, get re-synced, in a loop. The fix reads the refuted flag at reconciliation time and treats a refuted chunk as absent-for-sync-purposes, so nothing bounces.

A tags-only sync no longer writes a duplicate-body revision. If only tags differed and the body was byte-identical, the naive implementation would rewrite the body during the tag update, which would show up in chunk history as a body change. The fix path-splits: tag updates go through set_tags; the body is left alone.

The other six: DocumentSourceDef's HTTP route was gated at substrate:admin instead of substrate:tenant (fixed to match gRPC/MCP parity); title/type/status edits weren't propagating; diff_remote's reparent prediction didn't match what a real sync would do; a retired source could still resolve; the runtime + static source validators were accepting different values.

Composing: what this makes possible

A worked example. A team runs its own loomcycle at app.acme.internal and syncs its architecture RFCs from the loomcycle.cloud reference deployment.

Step one, at the acme deployment: an operator authors a DocumentSourceDef pointing at app.loomcycle.cloud with a credential-allowlisted API key. The source is versioned, tenant-scoped, and reachable from the Integrations tab in the Web UI.

Step two: a local document at /docs/loomcycle-architecture gets bound to the peer's /loomcycle/docs/architecture via set_remote. The binding persists.

Step three: an operator schedules a nightly sync direction: pull. Every morning the local copy reflects whatever the peer changed, with the overwritten local bodies preserved in chunk history so nothing is lost.

Step four: acme's internal agents read the synced document by its local Path. They cite it in their own memory writes. The source_quote provenance chain shows the exact synced revision the agent was reading. Six weeks from now, an audit trace on any fact leads back to the exact document revision that was live when the fact was written.

Step five: an acme developer improves a chunk of the architecture document locally, then pushes it back with sync direction: push. The upstream peer accepts the change; its own chunk history captures the previous body. The reference deployment's readers see the improved version on their next read.

None of these steps is a bespoke integration. Every one uses primitives that shipped as part of the substrate. That is the point of doing it this way.

Trust properties, restated

Because the surface is now external, the trust properties are worth stating in one place:

• The bearer is the boundary. Every op on /v1/_memory/*, /v1/_document, /v1/_path requires a bearer minted by an OperatorTokenDef. Tenant is server-derived, never wire-supplied.
• Isolated user tokens stay confined. An isolated substrate:user token cannot reach the tenant-scoped external endpoints. It sees only its own data through the surfaces it does reach.
• Members reach the tenant plane, minus operator carve-outs. Non-isolated user tokens (RFC CB) read and write the shared tenant Library, Documents, and Memory.
• All outbound calls go through the SSRF guard. The remote memory backend, the change-feed webhook, the document federation sync: none of them can dial an internal address by mistake.
• The change feed is operator observability. Gated at substrate:tenant, not member-accessible. Metadata about who is writing what and when is more revealing than any single value.
• Config-declared credentials are names. api_key_env holds an env-var name resolved at request time, not a value interpolated at parse time. Secrets never appear in config files.

What's next

Two things are queued in the RFC CD / CE lines.

MCP-side change-feed publisher. The SSE and webhook shapes cover HTTP consumers and config-declared subscribers. An MCP consumer would land as a new op on the change surface, useful mainly for embedded agents that want to react to another agent's writes.

Federated documents with a subset selector. Today sync reconciles the whole document. Real deployments will want "sync only chunks tagged public" or "sync only chunks under this subtree." The tag- and path-scoped variants land in a v1.54.x point release once the selector shape is settled.

Companion reading: agentic memory for the internal surface the external contract now exposes; one thousand PRs later, loomcycle.cloud is live for the public deployment you can point the SDK at; data retention for what happens to the mirrored data over time. Reference: docs/EXTERNAL_API.md, and try the console live at app.loomcycle.cloud/v1/docs.