Search providers, first-class.

2026-07-06 · by Dennis Gubsky · v1.15.0 → v1.15.1 · ~14 min read

Two weeks ago I published a survey of thirteen web-search APIs. The finding that mattered: Brave killed the free tier this year. What used to be a friendly 2K-5K free monthly cap is now metered from query one at $5 per thousand queries. Five to fifty times more expensive than Serper (Google SERP scraper, $0.10-$0.30 per thousand at Pro tier), eight times more than DataForSEO Standard queue at $0.60 per thousand, and infinitely more than a self-hosted SearXNG on a five-dollar VPS. I filed the survey into RFC AR (tenant credentials + search catalog) and moved on.

RFC BB is the shipped answer. The WebSearch tool stops being a Brave-only client. In its place: a proper multi-provider fallback circuit, five drivers ship in the same release, per-agent provider selection, and a routing view that tells the operator which provider is actually serving each tier. This week I switched my own homelab off Brave-first and paid roughly nothing to search. Everyone on the tenant continues to Search The Web as before.

The problem, restated

Every agent runtime that ships a WebSearch tool has this conversation. The tool grew up bound to one provider because one provider had a good free tier and clean JSON. Then that provider changed its pricing, or its rate limits, or its JSON, or all three. Now every downstream deployment that used the runtime's default is stuck migrating the same wire integration in parallel. The runtime says "swap Brave for Serper in your WEBSEARCH_PROVIDER= env var" and calls that vendor-neutral.

That's not vendor-neutral. That's the fastest-to-implement version of vendor-neutrality that leaves every deployment on one provider at a time, and any hiccup on that provider (503, rate limit, key rotation) fails every search across every tenant simultaneously. Real vendor-neutrality is a fallback circuit: enable N providers, order them, walk them in order, take the first success. Which is exactly the shape loomcycle already used for LLM providers (RFC J's provider fallback). RFC BB brings the same shape to search.

The connector package

A new internal/search package, structured like internal/providers is for LLMs. The Provider interface has four methods:

// internal/search — provider interface
type Provider interface {
    ID() string                                    // "brave", "serper", "exa", "tavily", "searxng"
    Search(ctx, query, opts) ([]Result, error)     // pure HTTP + JSON parsing
    KeyEnvName() string                            // "BRAVE_API_KEY", "SERPER_API_KEY", ...
    Probe(ctx) error                               // optional reachability check
}

type Result struct {
    Title   string
    URL     string
    Snippet string
}

Five drivers ship in the same release: Brave (own index), Serper (Google SERP scraper, cheapest at scale), Exa (own neural index; the semantic-search complement), Tavily (RAG-purpose-built), SearXNG (self-hosted; aggregates 70+ engines; no API key needed). Each is a pure-HTTP client that normalizes the provider's JSON to the common []Result shape. The drivers don't own credential resolution; that stays in the WebSearch tool where it can consult the RFC AR CredentialDef store and RFC AX operator-key restrictions. The connector package doesn't import internal/providers or the tools layer, so it's cheap to test with fake HTTP transports.

A Resolver sits above the drivers. Its job is small: given the agent's cascade (from AgentDef.search_providers or the global search_priority), walk the list in order, hand each caller a driver, respect a last-outcome cooldown so a recently-failing provider doesn't get retried on the very next query. There's no active probing of paid providers (Serper / Exa / Tavily / Brave); a probe query costs money, so reachability is inferred from last-outcome. SearXNG's probe is free (no key) and does happen.

Config surface

Two new top-level fields in loomcycle.yaml. Both go under the existing pattern for provider config; the shape is deliberately close to how provider_priority works for LLMs.

# loomcycle.yaml — RFC BB search config
search_providers:
  brave:  {enabled: true}
  serper: {enabled: true}
  exa:    {enabled: true}
  tavily: {enabled: false}
  searxng:
    enabled: true
    base_url: https://searxng.internal:8080     # required

search_priority: [searxng, serper, brave, exa]

Validation runs at boot: the enabled set is checked against search.KnownProviderIDs(), SearXNG's base_url is required when it's enabled, and both search_priority and every agent's search_providers must reference only enabled providers. A typo like search_priority: [searxngg] is caught at boot, not at the first WebSearch call. Same discipline as provider_priority.

Per-agent selection sits on the AgentDef:

# AgentDef fragment
name: research-agent
search_providers: [exa, brave]                  # narrower than global

An agent's search_providers overrides the global search_priority. Empty falls back to the global order. Sub-agents get their own def's list stamped on the loop context at spawn (WithSearchProviders mirrors WithAgentTools), so a research sub-agent can prefer Exa's neural index while its parent supervisor sticks with the global default.

A key detail: search_providers participates in the AgentDef's content_sha256. That's deliberate. Unlike the RFC BA skills: allowlist (which is authority-only and doesn't identify the agent), the provider list changes what the agent is. A different cascade may return different results, so a different cascade means a different content-addressed identity. The substrate overlay round-trips it: static YAML agent and its substrate fork hash identically as long as search_providers stays the same.

The fallback circuit

The WebSearch tool used to be a single Brave call. Now it walks the resolver's cascade:

// tools/websearch.go (paraphrased)
for _, providerID := range Cascade(tools.SearchProviders(ctx)) {
    driver := registry.Get(providerID)
    apiKey := credentials.ResolveKeyOrOperator(ctx, driver.KeyEnvName())
    if apiKey == "" && driver.KeyEnvName() != "" {
        continue                    // un-keyable, skip silently
    }
    results, err := driver.Search(ctx, query, opts)
    if err != nil || len(results) == 0 {
        resolver.MarkStalled(providerID, err)   // short cooldown
        continue                    // fall over to next
    }
    return render(results, providerID)          // success — model sees the text
}
return NoProvidersAvailable

Three failure classes are handled distinctly. A provider error (network, 5xx, quota exceeded) marks the provider stalled with a short cooldown and falls over. An empty result is treated the same as a provider error: no useful text; try the next. An un-keyable provider (no credential for it in the resolver's chain) is skipped silently; not a failure, just not applicable in this run's context.

Credential resolution is the interesting piece. ResolveKeyOrOperator is the RFC AR / AX resolver: it checks the caller's tenant-scoped CredentialDef store first, then falls through to the operator's host env unless RFC AX's providers:operator-key restriction is set to deny operator-key access for this tenant. A tenant that stores its own SERPER_API_KEY as a CredentialDef gets its runs served by that key; the operator's key is never consulted. A tenant that stores no key falls through to the operator's host key (or fails if operator-key restriction is on). Same shape as the LLM provider drivers.

Back-compat: pre-RFC-BB WebSearch worked on a single BRAVE_API_KEY. The v1.15.0 default config includes Brave at the top of search_priority, so a deployment that never touches the new config keeps working exactly as before. Adding search_priority: [serper, brave] to loomcycle.yaml is the whole opt-in for the fallback shape.

The routing view

GET /v1/_routing already exposes the LLM provider cascade per user_tier × tier so an operator can answer "which model does chat resolve to." RFC BB adds a search block to the same response.

// GET /v1/_routing — search block excerpt
{
  "search": [
    { "id": "searxng", "keyable": true,  "available": true,  "selected": true,  "reachable": true },
    { "id": "serper",  "keyable": true,  "available": true,  "selected": false, "reachable": true },
    { "id": "brave",   "keyable": true,  "available": true,  "selected": false, "reachable": true },
    { "id": "exa",     "keyable": false, "available": false, "selected": false, "reachable": null }
  ]
}

Four fields per entry. keyable means the caller has a resolvable credential (keyless provider like SearXNG counts as keyable; a paid provider without a CredentialDef or host key does not). available means keyable AND recently reachable. selected is the current pick from the cascade walk. reachable is the last-outcome status: true if the last query succeeded, false if it failed, null if never queried. Admin-only: a last_error field on failed entries carries the underlying error text for debugging.

The Web UI Routing page renders this as a "search providers" section beneath the LLM cascade. Same visual shape: availability dots, a "selected" badge, a "no key" marker. Data-driven by field presence, so a restricted tenant's filtered payload shows the same fields with fewer rows (the routing view already respects RFC AX tenant restrictions). The Settings → Credentials key-name combobox picks up the new provider keys (SERPER_API_KEY, EXA_API_KEY, TAVILY_API_KEY; BRAVE_API_KEY was already there).

Provenance footer (opt-in)

The WebSearch result renders identically no matter which provider served it. That's the point: swap Brave for SearXNG for Serper and the model gets the same numbered result text; downstream summarizers, extractors, and rerankers keep working. But it also means a fallover is invisible to the model and to the operator watching the transcript. The routing view's selected only shows the current pick, not what served a specific call. If the primary was Brave and Brave 503'd and SearXNG served the result, nothing in the output says so.

v1.15.1 adds an opt-in provenance footer. Set LOOMCYCLE_WEBSEARCH_PROVENANCE=1 and a WebSearch result appends a footer line naming the winning provider:

[1] Half of your Show HN comments arrive in 7.2 hours — jonno.nz
    An analysis of 41,301 Show HN posts shows the comment half-life...
[2] <more results>
[3] <more results>

(via searxng)                              # when the primary served
(via brave — searxng, serper fell over)    # when earlier providers were tried

Off by default (byte-identical output for downstream parsers). Two forms: the plain (via <provider>) when the primary served, and the annotated form when earlier providers were tried and passed over (errored or empty). The fallback loop already knows the winning ID and tracks passed-over providers, so the footer is a render-time detail with no runtime cost. Useful for the operator diagnosing "why did this query take four seconds" or the agent that needs to know when its neural-search primary got bypassed for a keyword-search fallover.

What the config table looks like now

Five drivers, three cost tiers, one keyless option:

All five sit behind the same fallback circuit. A homelab operator can list SearXNG first and take Serper as a fallback for queries SearXNG's engines don't answer well. A cost-sensitive tenant can list Serper first with Brave as backup. A research agent can list Exa first for the neural index and fall over to a keyword provider for cases where the neural search returns nothing useful. The choice is per-tenant + per-agent, not per-deployment.

The composition with earlier substrate

RFC BB reuses three earlier primitives without adding new complexity to them:

RFC AR CredentialDef. Provider API keys ride the same store as everything else. Store SERPER_API_KEY at tenant scope with the CredentialDef tool; encrypted at rest with envelope AES-256-GCM under a per-tenant DEK; auditable; rotatable. A tenant admin manages their own search keys the same way they manage their own LLM keys. The operator's host key is a fallback, not a default, and tenants that want to be independent already are.

RFC AX providers:operator-key restriction. A restricted tenant can be denied access to the operator's host search keys, forcing them onto their own CredentialDef or onto keyless SearXNG. This closes the same operator-cost-exposure hole for search that RFC AX closed for LLMs.

The routing view. Search sits next to LLM providers in the routing response, so the same operator dashboard that answers "which model does chat resolve to" answers "which search provider is currently serving." One view, two provider families, no separate diagnostic surface.

What isn't reused: the search cascade doesn't compose with the RFC AK Document primitive or the RFC AL Path primitive. Those are storage layers; the search connector is a call-out. Deliberately kept separate.

The RFC AR follow-up

Two weeks ago I drafted RFC AR (Tenant credentials + external search-provider catalog) and rejected a `SearchProviderDef` primitive on the grounds that MCPServerDef plus a SkillDef alias would cover the DX gain without adding a new Def family. RFC BB proves that read half-right and half-wrong.

Half-right: search_providers ended up as a first-class config concern, not a Def. Providers are declared in loomcycle.yaml and picked per-agent via AgentDef.search_providers. There's no SearchProviderDef substrate table, no op=create / op=fork / op=retire. Provider identity is a string ID from the enabled set, checked at config-boot. Same shape as LLM provider IDs.

Half-wrong: I predicted the search catalog would be a bundled MCPServerDef list per provider. It isn't. Each driver is in-process Go code inside the runtime binary. Two reasons: MCP wrappers for search providers were mostly community projects with inconsistent maintenance; putting the drivers in-process gives loomcycle version-controlled reliability and one place to fix a driver bug. And an MCPServerDef per provider would need a separate Docker container per, which is fine on TrueNAS but excessive for the homelab-single-binary case. Compiled drivers, config-picked at runtime, MCPServerDef-shaped for tool discoverability but not literally MCP: that's the shape.

What ships next

In this three-day arc, RFC BB is the middle piece. On Saturday: RFC BA on-demand skills (v1.14.0/v1.14.1) shipped. Today (Sunday, v1.15.0/v1.15.1) RFC BB. Tomorrow: RFC BC client-executed tools (v1.16.0), a new substrate primitive that lets a browser or IDE register its own tools that the agent calls as if they were built in. LoomBoard's Chrome extension is the first customer, shipping the same day.

Companion reading: the CredentialDef release digest for the encrypted-secrets substrate that stores the search API keys, and the search-providers help topic for the operator-side config walkthrough.