The system
How the system worksThe DirectoryDispatchThe LedgerAI modelsAI agentsYour installationThe appThe research libraryWhat changes for your businessUse it from your softwareEverything it can do
Messaging
iMessageWhatsApp
Ads and leads
AdsCreativeLeads
Devices
Devices
Industries
RealtorsRestaurantsHome services
Explore
How it worksPermissions and limitsCompareSet upQuestionsText me

Cloudflare OS XL part 1: search and retrieval this build does not have

Cloudflare OS XL part 1: search and retrieval this build does not have

Part 1 of Cloudflare OS XL, an inventory of the Cloudflare platform this build does not have installed.

This build holds 1,171 published articles, several thousand atomized claims, a source ledger, an audit chain, a lead table and a directory of roughly nine hundred callable rows. Every one of those is searched the same way: a SQL LIKE '%term%' against D1, or an exact-key lookup in KV.

That works when you know the word. It fails completely when you know the idea. Ask this build "which articles argue that a gate must measure the object it claims to measure" and there is no query that answers it, because the sentence that makes the argument may not contain any of those words. The corpus knows the answer. The build cannot reach it.

Three Cloudflare products close that, and none of them are installed.

Vectorize

Vectorize is Cloudflare's vector database, bound directly into a Worker. You create an index with a fixed dimensionality and metric, write vectors with metadata, and query by nearest neighbour.

shell
wrangler vectorize create loop-corpus --dimensions=768 --metric=cosine
Line 1, Column 1Shell Script
toml
[[vectorize]]
binding = "CORPUS"
index_name = "loop-corpus"
Line 3, Column 1toml

The embedding model is already here — Workers AI is bound on both the Pages project and the sibling Worker, and @cf/baai/bge-base-en-v1.5 produces 768-dimension vectors without leaving the account. So the whole loop is inside Cloudflare: read the article from D1, embed it with the AI binding, upsert into Vectorize with the slug and claim id as metadata, query it from the same Worker.

What it changes here, concretely:

  • Claim-level retrieval. The unit is not the article, it is the claim. Every claim already has an id, a tier and a text field. Embedding claims rather than articles means a search returns the specific assertion, which is the addressable object this build is built around, and metadata filtering lets a query say "only claims at tier human or rct".
  • Duplicate detection at the write path. Before an article publishes, the write path could ask whether any existing claim is within a cosine distance of the incoming one. The corpus has grown by swarm passes; some of it says the same thing twice in different words, and there is currently no mechanism that could know.
  • Lead matching. The lead table and the content corpus are unrelated tables today. With both embedded, "which article should this clinic receive" becomes a query rather than a guess.
  • The directory. Nine hundred tool rows with descriptions is exactly the retrieval problem vector search is for. An agent looking for the right capability currently reads a list.

Vectorize is metadata-filterable and namespace-partitioned, so one index can hold claims, articles, leads and directory rows without them contaminating each other's results.

Verdict: install. This is the single highest-value absent product in the account, and everything it needs — Workers AI, D1, the claim structure — is already in place.

AI Search, formerly AutoRAG

The product this build's directory still refers to as AutoRAG has been renamed Cloudflare AI Search. It is the managed version of the pipeline described above: point it at an R2 bucket, and Cloudflare crawls it, chunks it, embeds it, stores the vectors, keeps them in sync as the bucket changes, and exposes both a raw search and an aiSearch that returns a generated answer with citations.

The difference from Vectorize is ownership of the pipeline. With Vectorize you write the chunker, choose the model, handle re-embedding on edit, and own the freshness problem. With AI Search, Cloudflare owns all of it and you own a bucket.

For this build the two are not competitors, they are different jobs:

  • AI Search suits the reference material — the vendor documentation absorbed into R2, the Grok docs pulled verbatim from llms.txt, the Workspace and Wrangler surfaces, the absorbed repositories. That content is written once, read often, and nobody needs claim-level addressability into it. Turning that bucket into an AI Search index gives every agent a documentation oracle with citations for near zero code.
  • Vectorize suits the corpus — articles and claims — because the retrieval unit has to be the claim id, the metadata filter has to be the evidence tier, and the write path has to control exactly when a vector is refreshed.

There is also a third property worth noting: AI Search exposes an MCP server. The documentation oracle becomes a tool any model client can attach to without this build writing the bridge.

Verdict: install, for the reference bucket only. Do not point it at the article corpus; that content needs the control Vectorize gives.

D1 read replication and the Sessions API

This one is not retrieval, it is the same problem from the other side: the corpus is read globally and written from one place.

D1 supports read replicas. Replicas are created and placed automatically; the application opts in per request by starting a session, which is what preserves sequential consistency — read-your-writes — across a set of queries that might otherwise land on a replica that has not caught up yet.

js
const session = env.DB.withSession('first-primary');
const { results } = await session.prepare('SELECT ...').all();
// bookmark travels with the response; the next request resumes the session
Line 3, Column 1JavaScript

The shape of this build's traffic is exactly the shape read replication is for. The content spine is read on every page render, every API article fetch, every sitemap build, every feed. It is written by a handful of agents. Today every one of those reads crosses to wherever the primary lives.

The cost of adopting it is real but bounded: read paths must be audited to decide which ones need read-your-writes and which are happy with an eventually consistent replica. The article render is happy. The write path's own read-back after a PUT is not, and must carry the bookmark.

Verdict: install, after an audit of the read paths. It is a configuration change and a code change in one place, and it is free.

What this part does not recommend

There is a fourth option that looks adjacent and is not: putting the corpus in an external vector store and reaching it over HTTP. It would work. It would also put a network hop, a second vendor, a second credential and a second failure mode into the hot path of every page render, in exchange for nothing this account cannot already do inside its own bindings. The reason to run on one platform is that the bindings do not go down separately from the Worker.

Verdicts

ProductWhat it replaces hereVerdict
VectorizeLIKE '%term%' over 1,171 articles; no claim-level retrieval at allinstall
AI Search (AutoRAG)Agents reading absorbed vendor docs by grepping filesinstall — reference bucket only
D1 read replicationEvery global read crossing to the primaryinstall — after read-path audit
External vector storeNothing. It adds a vendor and a hopno

Next: Part 2 — the ledger as a queryable table.

References (3)
  1. Cloudflare Vectorize documentation“Build full-stack AI applications with Vectorize, Cloudflare's vector database.”
  2. Cloudflare AI Search documentation“Index your content and query it with natural language from a Workers binding, REST API, or MCP server.”
  3. Cloudflare D1 documentation“Build serverless SQL databases on Cloudflare's global network and query them from Workers and Pages projects.”