VALID v0.1 · version-aware retrieval · 2026

VALID: Answer from the docs for the generation you run
and refuse rather than borrow from another

Design document · Siddharth Rao · October 2026 Client: Google data center products and services documentation

Disclaimer. This is a portfolio design document. Google has not engaged, sponsored or endorsed this work. The sample content is adapted from the Cloud TPU documentation, which Google publishes under the Creative Commons Attribution 4.0 licence¹. The adapted versions are modified from the originals. The question set, runs, similarity scores and costs in this document are modelled, not measured. Stakeholders are job roles, not real people. The two readers in §3 are fictional.

58%

standard RAG accuracy on
version-sensitive questions

TPU generations, three
right answers to one question

model calls to decide
which content is in scope

Abstract

Ask a documentation assistant how many chips are in one Cloud TPU host or VM and the right answer depends on the generation: 4 for v5p, 8 for v6e, 4 for TPU7x²³⁴. Ask which software version to pass when creating a TPU7x VM and the right answer is that the Cloud TPU API doesn't support TPU7x at all⁵. Retrieval ranks text by meaning, not by generation, so it returns all of these side by side. In a published benchmark, standard RAG answered only 58% of version-sensitive questions correctly⁶.

VALID (Version-Aware Lookup In DITA) is an agent that answers only from content profiled for the reader's generation. The scope of every chunk comes from what writers already maintain: profiling attributes, map cascading and a subject scheme⁸¹⁰. Firestore filters on scope before the vector search runs¹². Gemini writes the answer, and plain Python checks that every citation is in scope and every value appears in its source. If no in-scope content exists, VALID says so and opens a gap issue. It never widens the search to another generation.

VALID is rebuilt from VaultRAG, a deployed RAG service with guardrails that refuse when unsure²⁵. It keeps that service's retrieval stack and refusal design, and replaces plain-text chunking with DITA-aware chunking that carries scope.

retrieval-augmented generationDITA profilingDITAVALsubject schememetadata pre-filterFirestore vector searchBM25groundingrefusalGeminiAgent Development KitCloud Run

§1 Client Brief

The client is Google's data center products and services documentation. The sample content is the Cloud TPU documentation for three hardware generations: TPU v5p, TPU v6e (Trillium) and TPU7x (Ironwood)²³⁴, plus the TPU software versions page that covers all of them⁵. The MANIFEST project converted this content to DITA, with generation-specific content marked by profiling attributes and built per generation with DITAVAL²⁷.

It is a good test for two reasons. First, the generations share most of their concepts and procedures, so retrieval can't tell them apart by meaning alone. Second, the differences are small, exact and easy to get wrong: a chip count, a version string, a supported API. An answer that is almost right is a wrong answer here.

Term	Meaning in this document
Generation	One TPU hardware version: v5p, v6e or TPU7x. VALID's scope dimension. For networking content, the same role is played by a platform or a software release.
Profiling attribute	A DITA attribute that marks content as applying to a condition. `@platform`, `@product`, `@audience` and `@otherprops` are used for filtering⁹. Here, `platform="tpu7x"`.
DITAVAL	A file that tells a DITA processor which profiled content to include or exclude for one build⁸.
Cascading	Profiling attributes on a map entry apply to the topics and entries under it¹⁰. A whole TPU7x branch is scoped once, in the map.
Subject scheme	A DITA map that defines the allowed values of an attribute¹¹. It is VALID's list of generations and their other names, such as Ironwood for TPU7x.
Chunk	One retrievable unit: a DITA section, a table, or a group of task steps, with its text, its vector and its scope.
Scope	The set of generations a chunk applies to, resolved from the map and the element. Stored as one true or false field per generation.
Pre-filter	A filter that runs before the nearest-neighbour search, so only in-scope chunks can be returned¹².

Figure 1

System context: as-is state

The scope exists in the DITA source as an attribute. Publishing turns it into a separate page, and the index sees only text. The answer in the red box is fluent and cited, and still wrong for the reader's generation (§2.1).

§2 Problem Statement

A documentation assistant is only worth its cost if readers trust its answers enough not to ask a person. Hardware generations make that hard. Most of the text is shared, the differences are small, and a retriever can't see which generation a sentence belongs to.

2.1 One question, three right answers

Figure 2 shows three questions a reader might ask about a Cloud TPU VM. Each has a different correct answer for each generation, as Google publishes them today.

Figure 2

Three questions, three generations: answers as published

From the v5p, v6e and TPU7x pages and the TPU software versions page²³⁴⁵, checked on 20 October 2026. The red cell is not a value at all. It is a different procedure.

A real example

The TPU software versions page lists one version string per generation for JAX and PyTorch: v2-alpha-tpuv5 for v5p and v2-alpha-tpuv6e for v6e. On the same page, it says the Cloud TPU API is not supported on TPU7x and later, and points to Compute Engine or GKE instead⁵. A reader who asks "which version do I use for my Ironwood VM?" is closest in meaning to the version table. A retriever returns the table. A model then does what it was asked and picks a version string.

Figure 3 shows why. These are the five chunks nearest in meaning to that question, in an index with no scope. The two that are correct for TPU7x rank fourth and fifth.

Figure 3

What similarity sees: the five nearest chunks for a TPU7x question (modelled)

in scope for TPU7xv6ev5pshared text with a generation-specific value

Illustrative scores, not measured. The pattern is the point: the wrong generations rank first because the question uses the same words as their rows. The third chunk is a shared step that states a v6e value without saying so. A filter on scope can't catch it unless someone marks it. VALID finds it at index time (§5.2).

2.2 How often RAG gets it wrong, and what it costs

The VersionRAG study measured this failure on versioned technical documentation: Apache Spark and Bootstrap release notes, and Node.js reference pages, 34 documents and 100 curated questions⁶. Standard RAG answered 58% correctly. On questions about what changed between versions, it answered 25% correctly (Figure 4).

Figure 4

Accuracy on version-sensitive questions, by retrieval approach

standard (naive) RAGGraphRAGVersionRAG, version-aware

Data from Huwiler, Stockinger and Fürst, VersionRAG, Table 5, with DeepSeek-R1 70B as the model⁶. Different content from this design, the same failure. The version-aware approach built its index with about 97% fewer tokens than GraphRAG⁶. VALID goes one step further: in DITA, the version structure already exists, so it needs no model to build it.

The cost of an almost-right answer is not the answer. It is what the reader does next. In the 2025 Stack Overflow Developer Survey, the most common frustration with AI tools was answers that are almost right, but not quite, at 66%. Three in four developers said they still ask a person when they don't trust an AI answer⁷. Each wrong-generation answer turns a self-service answer back into a question for an engineer.

Figure 5

What developers do with AI answers they don't trust

Stack Overflow 2025 Developer Survey, AI section⁷. Between 29,163 and 33,244 respondents per question. The survey is about AI tools in general, not documentation assistants. It shows the reader's fallback when an answer can't be trusted.

2.3 Why the current options fail

There are six common ways to make an assistant respect versions. Each one either leaves wrong-generation text in front of the model, or rebuilds structure the DITA source already has.

Option	What it does	What it misses	Result
Standard RAG on published pages	Indexes HTML. Returns the top-k chunks by similarity.	No scope field exists. Wrong generations rank first (Figure 3).	58% accurate
Tell the model the generation	Adds "the reader uses TPU7x" to the prompt.	The context still holds v6e and v5p chunks. The model must choose, and can choose wrong.	Unchecked
Boost or rerank by version	Raises in-scope chunks in the ranking.	Wrong-generation chunks drop lower but stay in the context.	Softer, not safe
Filter after the search	Runs kNN on everything, then drops out-of-scope results.	Can leave one result or none. The easy fix is to fall back to unfiltered results.	Fails open
One index per generation	Builds a separate index for each build output.	Copies shared content into every index. Questions that compare generations need several searches. Unscoped values still slip in.	Costly to run
Model-built version graph (VersionRAG)	Extracts versions and changes from documents with a model, then routes queries⁶.	Builds with a model what DITA authors already wrote. Every edge is a model guess.	90%, at a build cost
VALID: scope from the source, filter first	Reads scope from profiling, cascading and the subject scheme. Pre-filters on scope. Checks every citation. Refuses when nothing in scope answers.	Depends on correct profiling. Content that states a value without a scope must be marked, so VALID reports it.	Right or refused

Problem statement

Readers of the Cloud TPU documentation ask questions whose correct answer depends on the TPU generation. A standard documentation assistant retrieves by meaning, so it returns content from every generation and can answer with a value or procedure from the wrong one. Readers who can't trust the answer ask a person instead, and the assistant saves nothing. The DITA source already records which content applies to which generation, but the assistant never sees it.

§3 User Profiles

Five roles at Google decide whether VALID is useful. Two readers at Google use its answers.

Documentation lead, Cloud TPUSponsor

GoalAn assistant that answers TPU questions so engineers don't have to.

Measured byQuestions answered without a person. Cost per 1,000 questions.

TodayCan't show that assistant answers are right for the reader's generation.

ConcernAn assistant that is fast, fluent and wrong, under the documentation team's name.

SuccessZero wrong-generation answers on the test set, with cost logged for every question.

Technical writer, Cloud TPUMain user

GoalAnswers that use the profiling she already maintains.

Measured byGap issues closed. Scope warnings at index time.

TodayProfiles content for DITAVAL builds. The assistant ignores it.

ConcernBeing blamed for answers built from content she didn't scope.

SuccessEvery answer cites topics she wrote, in scope. Every refusal tells her what is missing.

Information architectStructure owner

GoalOne list of generations, used the same way in builds and in answers.

Measured byProfiling values not in the subject scheme. Unscoped chunks with generation values.

TodayOwns the subject scheme. Finds profiling gaps only when a build looks wrong.

ConcernA second, private list of versions inside an AI tool.

SuccessVALID reads her subject scheme and reports every gap to her as an issue.

Cloud TPU software engineerContributor

GoalFewer chat questions that the docs already answer.

Measured byQuestions sent to the team channel each week.

TodayCorrects assistant answers that used a v6e command for TPU7x.

ConcernA wrong command that a customer runs in production.

SuccessEvery command in an answer appears, unchanged, in an in-scope source.

Customer support engineer, Cloud TPUDownstream

GoalPaste a cited answer into a case and trust it.

Measured byCases reopened because the documented answer was for another generation.

TodayChecks every assistant answer against the generation page by hand.

ConcernSending a customer a fix for hardware they don't have.

SuccessEach answer states its generation and links only to pages for that generation.

Readers

Priya NairReader

RoleML engineer at Google, moving a training job from v6e to TPU7x.

GoalCreate TPU7x capacity the first time, without the v6e steps she knows.

TodayGets the v6e command from the assistant, runs it, and it fails.

SuccessAn answer that says "for TPU7x" and gives the TPU7x procedure.

Tomás SilvaReader

RolePlatform engineer at Google who provisions TPU VMs for three teams.

GoalPlan host counts for v5p, v6e and TPU7x in one sitting.

TodayAsks "how many chips per host?" and gets one number with no generation.

SuccessAsked which generation he means, or given one cited answer per generation.

§4 User Stories

Each story has acceptance criteria that a tool or check in §5 tests.

US-1 · Documentation lead

As the documentation lead, I want every answer's scope, citations and cost logged, so that I can show the assistant is right before I fund it further.

Accepted when: log_question writes one row per question with the resolved scope, cited chunk IDs, check results, tokens and cost; the trap-set report in §7 reads only that table.

US-2 · Technical writer

As the Cloud TPU writer, I want answers built only from content I profiled for the reader's generation, so that my profiling decides what readers see.

Accepted when: retrieve_in_scope sends an equality pre-filter on scope with every query; check_answer fails any answer that cites a chunk outside the scope.

US-3 · Information architect

As the information architect, I want VALID to take its generations from my subject scheme and report unscoped generation values, so that there is one source of truth for scope.

Accepted when: resolve_profiles fails the index build on any profiling value not in the subject scheme; audit_scope opens one issue per unscoped chunk that contains a generation-specific value.

US-4 · Software engineer

As a Cloud TPU software engineer, I want every command and flag in an answer to appear in an in-scope source, so that no reader runs a command from another generation.

Accepted when: check_answer matches every command, flag and version string, character for character, against the cited chunk text.

US-5 · Support engineer

As a support engineer, I want each answer to name its generation and link only to pages for it, so that I can send it to a customer without checking it again.

Accepted when: the answer schema requires a scope field; every link resolves to a chunk whose scope includes it.

US-6 · Priya, reader

As an ML engineer moving to TPU7x, I want the TPU7x procedure when I say "Ironwood", so that I don't run the v6e command I already know.

Accepted when: resolve_scope maps "Ironwood" to tpu7x from the subject scheme; no v6e or v5p chunk reaches the model.

US-7 · Tomás, reader

As a platform engineer, I want to be asked which generation I mean when my question doesn't say, so that I never get one generation's number presented as the answer.

Accepted when: a question with no scope and a scope-dependent answer returns a choice of generations, not an answer; a question naming two generations gets one cited answer for each.

Figure 6

Stories mapped to the agent tools

deterministic toolcheck that decides pass or failmodel call

Two stories touch the model. Both are accepted by check_answer, not by the model. embed_chunks serves every story and isn't tested by any of them.

§5 Proposed Design

VALID v0.1 is an agent built with Agent Development Kit (ADK), Google's open-source agent framework¹⁹. It uses Gemini 3.8 Flash (gemini-3.8-flash)¹⁵ with structured output, so the answer must match a JSON Schema¹⁸, and Gemini Embedding 2 (gemini-embedding-2) for vectors¹⁴. Plain Python tools, called through function calling²⁰, do everything else. They decide the scope, run the filtered search and check the answer. The checks decide pass or fail, and the agent can't override them.

VALID has two parts. The index runs on every docs merge: it reads the DITA source, resolves the scope of each chunk and writes it to Firestore. The answer runs on every question: it decides the scope, searches only inside it and checks the result.

5.1 How it works

Figure 7 is the same context as Figure 1, with VALID added. The assistant now reads the DITA source, where the scope still exists, instead of the published pages.

Figure 7

System context: VALID added

existing source or storedeterministic tool or outputmodelcheck that decidesgap or retry path

The model sits behind the filter. By the time Gemini sees a chunk, the chunk is already in scope, and by the time a reader sees the answer, every citation has been checked against that scope.

5.2 Scope comes from the source

VALID doesn't ask a model which generation a chunk belongs to. Writers already answered that question when they profiled the content for DITAVAL builds. resolve_profiles reads three things, in this order:

DITA feature	What VALID reads	Index result
Map cascading	`platform="tpu7x"` on a `<topicref>` applies to every topic under it¹⁰	The whole TPU7x branch is in scope for TPU7x only
Element profiling	`platform` on a step, row or section inside a shared topic	That chunk's scope narrows to the listed values
No attribute	An omitted attribute can't evaluate to "exclude", so the content is in every build⁸	In scope for every generation, and checked by `audit_scope`
Subject scheme	The allowed `@platform` values and their other names¹¹	One field per value; any other value fails the build

The third row is the risk. DITA treats unprofiled content as shared, which is right for a build and dangerous for an answer. A shared step that says --version=v2-alpha-tpuv6e appears in every build, and in Figure 3 it ranked third for a TPU7x question. audit_scope finds these: it looks for any value that belongs to one generation (a version string, a machine type, a key value) inside a chunk that is in scope for all of them. Each finding becomes an issue for the information architect. The writer fixes the profiling, and the next build reads the fix.

Figure 8

From map and topic to scope fields on each chunk

TPU7xv6eshared, no attributeshared text with a one-generation value

Each chunk stores one true or false field per generation in the subject scheme. Cascading decides most of them. The last row is in scope for everything, so the filter alone can't stop it. The audit catches it before any reader can.

5.3 Filter first, then search

Firestore vector search can combine a nearest-neighbour query with other filters, so the filter runs before the search. The filtered query needs a composite vector index¹². VALID uses only equality filters on the scope fields, the pattern the Firestore documentation shows (where("in_tpu7x", "==", True)), with one composite index per generation. It doesn't depend on range or array filters inside a vector query, which the documentation doesn't describe.

Figure 9

Filter after the search compared with filter before it, for a TPU7x question

The left design is what most RAG stacks do when metadata is added late. The right design is the one rule VALID can't break: the scope filter is part of every query, and no code path removes it.

Exact terms matter in this content. Version strings, machine types such as tpu7x-standard-4t, and flags are rare tokens that dense vectors handle poorly. VALID also runs a BM25 search with SQLite FTS5²⁴ over the same scope, and merges the two lists with reciprocal rank fusion³⁰. This is the hybrid retrieval pattern from QueryForge²⁶, with the scope filter applied to both lists.

5.4 Agent tools

VALID has nine tools. Three run at index time and six at question time. One calls the model.

Index time · on every docs merge

1. Resolve profiles `resolve_profiles`

Parses the maps and topics with lxml. Applies map cascading, then element profiling, to give each chunk its scope. Chunks follow DITA structure: one section, one table or one group of steps. Fails the build on any value not in the subject scheme.

output: chunks with text, source path, line and one scope field per generation

2. Audit the scope `audit_scope` check

Finds shared chunks that contain a value owned by one generation: a version string, a machine type, or the resolved value of a generation key. Opens one issue per finding for the information architect.

output: scope findings with file and line; issues in GitHub

3. Embed and store `embed_chunks`

Embeds changed chunks with Gemini Embedding 2 at 1,536 dimensions, a recommended size under Firestore's 2,048 limit¹²¹⁴. Writes vectors and scope fields to Firestore, and text and scope to the BM25 index.

output: Firestore documents and FTS5 rows; only changed chunks are re-embedded

Question time · on every question

4. Decide the scope `resolve_scope`

Matches the question against the subject scheme: generation names, other names such as Trillium and Ironwood, accelerator types and machine types. No model call. Returns one generation, several, or none.

output: scope list and the words that matched

5. Search inside the scope `retrieve_in_scope`

Embeds the question, runs Firestore find_nearest with the scope pre-filter, runs BM25 with the same scope, and fuses the lists. For two generations, it runs once per generation and labels each list.

output: up to 5 chunks per generation, every one in scope, with scores

6. Write the answer `generate_answer` model

Calls Gemini 3.8 Flash with structured output. The schema requires a scope field and a citation on every step, and the citation must come from an enum of the retrieved chunk IDs, so Gemini can't cite anything it wasn't given. AlignR uses the same enum method for keys and sources²⁹.

output: JSON answer with steps, citations and scope

7. Check the answer `check_answer` check

Three plain Python checks. Every citation is in scope. Every number, unit, flag, command and version string appears in its cited chunk. No value owned by an out-of-scope generation appears anywhere in the answer. The grounding check comes from KnowledgeFlow²⁸.

output: pass, or the failed steps with reasons

8. Refuse or ask `refuse_or_ask`

No scope and a scope-dependent question: asks the reader to choose a generation. Scope but nothing in scope, or checks still failing: refuses, says what is missing, and opens a gap issue for the writer.

output: a choice for the reader, or a refusal and a gap issue

9. Log the question `log_question`

Writes one row per question to BigQuery: question, scope, chunk IDs and scopes, check results, outcome, tokens and cost.

output: rows in the question log

5.5 Agent loop

The model decides how to phrase the answer. The checks decide whether it can be sent. A failed step goes back to Gemini with its reason, up to two times, with the same in-scope chunks. VALID never widens the search to fix a failure.

Figure 10

Agent loop for one question

Three ways out, and only one is an answer. A question to the reader and a refusal are designed outcomes, as in VaultRAG²⁵. Each refusal names the missing content, so it becomes work for a writer instead of a dead end.

5.6 One question, end to end

Figure 11 follows Priya's question from §3. It is the run in the second simulator scenario (§6).

Figure 11

Sequence: "I'm on Ironwood. Which --version do I pass when I create the VM?"

The model's first draft carried a --version flag from its own training, not from the context. The filter can't stop that. The grounding check can. Two model calls, zero chunks from another generation.

5.7 Sample output

This is the answer VALID sends for Priya's question, as JSON from the answer schema. The ask page renders it with the scope at the top.

Figure 12

Answer object after the checks pass

{
  "scope": ["tpu7x"],  "matched": "Ironwood",  // from the subject scheme, not the model
  "answer": "The Cloud TPU API doesn't support TPU7x, so the software versions page lists no version for it.",
  "steps": [
    { "text": "Create TPU7x capacity with Compute Engine or Google Kubernetes Engine (GKE).",
      "cite": "runtimes#tpu7x-note" },
    { "text": "Use the tpu7x-standard-4t machine type. Each TPU7x VM has 4 chips.",
      "cite": "tpu7x#vm" },
    { "text": "Use JAX or PyTorch. TensorFlow isn't supported on TPU7x.",
      "cite": "tpu7x#frameworks" }
  ],
  "not_used": "v6e and v5p version strings: out of scope for this question",
  "checks": { "citations_in_scope": "3/3", "values_grounded": "4/4", "out_of_scope_values": 0 },
  "run": { "model": "gemini-3.8-flash", "calls": 2, "tokens_in": 6410, "tokens_out": 690 }
}

ScopeDecided by resolve_scope from the subject scheme. The model fills the field but its enum allows only the resolved value.

CitationsEach cite comes from an enum of retrieved chunk IDs, so the model can't cite a chunk it didn't see.

Not usedTells the reader that other generations exist and were excluded on purpose. It doesn't repeat their values.

Values from the TPU7x page and the TPU software versions page²⁵. Token counts are illustrative.

5.8 What VALID keeps from VaultRAG

VaultRAG is a deployed, voice-first RAG service for manufacturing procedures. It runs on Cloud Run and Firestore Vector Search, refuses at five guardrail stages, and passed 17 of 20 queries in its own guardrail evaluation, with the three misses analysed²⁵. VALID keeps its spine and replaces what was specific to plain text.

VaultRAG part	In VALID	Why
Firestore Vector Search, cosine, normalised vectors	Kept plus a scope pre-filter	The store already supports filters before kNN¹²
Scope guard and confidence threshold (similarity floors 0.62 and 0.70)	Kept recalibrated on the TPU question set	Out-of-domain questions still need a refusal
Citation enforcement with one strict retry	Changed to citation enums plus scope and grounding checks	A regex for "Section 18.4" can't prove a citation is in scope
Query normalisation with a model	Changed to subject scheme matching, no model	The scope must be deterministic and owned by the IA
Heading-pattern chunker for plain text	Changed to DITA-aware chunks with scope	DITA structure gives clean chunk edges and the scope
Wider retrieval when a hazard keyword fires	Dropped	Widening is the move VALID forbids
Voice input, air-gapped production path	Dropped	A second job. The Cloud TPU docs are public.

5.9 Where it runs

Part	Technology	Free tier or licence
Agent framework	Agent Development Kit¹⁹	Apache 2.0
Answer model	Gemini 3.8 Flash (`gemini-3.8-flash`), structured output	Free of charge on the Gemini API free tier¹⁶
Embeddings	Gemini Embedding 2 (`gemini-embedding-2`), 1,536 dimensions¹⁴	Gemini API; priced on the same page¹⁶
Vector store and scope filter	Firestore vector search with composite vector indexes¹²	50,000 reads and 20,000 writes a day; one read per 100 kNN index entries¹³
Exact-term search	SQLite FTS5 with BM25²⁴	Public domain
Profiling and audit	Python and lxml	Open source
Index trigger	GitHub Actions	Free for public repositories²³
Agent runtime	Cloud Run service, `us-central1`	180,000 vCPU-seconds and 2 million requests a month²¹
Question log	BigQuery	10 GiB storage and 1 TiB of queries a month²²

Figure 13

Model cost per 1,000 questions (modelled)

Prices from the Gemini API pricing page¹⁶. Token counts and the revision rate are assumptions: five chunks of about 500 tokens plus instructions, and one revision in ten questions. Query embeddings are not included.

Figure 14

Questions a day inside the free quotas (modelled)

Free quotas from the Firestore and Cloud Run pricing pages¹³²¹. Reads assume 5 documents plus 2 batches of kNN index entries. Run time assumes 3 vCPU-seconds a question, mostly waiting on Gemini.

Data handling. The sample content is public, so the Gemini API free tier is acceptable for the POC. Free-tier content is used to improve Google products, and paid-tier content is not, so private documentation needs the paid tier¹⁶. The free tier is offered in the EEA, Switzerland and the UK¹⁷. The POC runs in us-central1, because the Cloud Run free tier is defined on Tier 1 pricing²¹.

Known limits. VALID is only as right as the profiling. It finds unscoped values that it can recognise, such as version strings and key values, but not a sentence that is wrong for one generation in plain words. Questions about what changed between generations are out of scope for the MVP; VALID answers each generation separately and doesn't compute the difference.

§6 Answer Simulator

Three questions. The first passes cleanly. In the second, the filter keeps out the wrong generations and a check stops a flag the model invented. In the third, the question has no generation, so VALID asks.

valid · answer simulator

idle

Sample question

// choose a sample question and run it

Scripted demo. No live model call. Scores, token counts and timings are illustrative.

§7 Success Metrics

The MVP is measured on a trap set: questions whose correct answer differs between v5p, v6e and TPU7x. The set has 60 scoped questions: four facts (the three in Figure 2, plus supported frameworks) for each of three generations, in five wordings each. It also has 10 questions with no generation, and 10 that name two generations. Each question has a known answer and known source chunks. The same set runs against an unfiltered baseline built from the same chunks.

Metric	How it is measured	MVP target
Wrong-generation answers that reach a reader	Trap-set answers whose value or procedure belongs to another generation	0
Citations in scope	`check_answer`: cited chunks whose scope includes the resolved scope	100%
Values grounded in the cited chunk	`check_answer`: numbers, units, flags and commands found in their source	100%
Scoped questions answered correctly	Correct answers out of 60, against the unfiltered baseline on the same set	54 or more (90%); the rest refused, none wrong
Unscoped questions handled	Questions with no generation that get a choice, not an answer	10 of 10
Unscoped generation values in the index	`audit_scope` findings left open after IA review	0
Time to answer	Question received to answer sent, median	Under 5 seconds
Questions with cost logged	Rows in the question log with tokens and cost	100%

Figure 15

Trap-set scorecard format: baseline compared with VALID

right, cited in scopeasked the readerrefused with a gap issuewrong generation

Sample scorecard format with illustrative values, not results. The baseline's scoped accuracy is set near the 58% that VersionRAG measured for standard RAG⁶. The shape to look for: VALID turns every red bar into green, blue or amber. It may answer less, but it is never wrong about the generation.

Figure 16

Sample index report on a docs merge

Index build · merge 4c2e91a · cloud-tpu docs set1 scope finding

412chunks scoped

3generations in scheme

18chunks re-embedded

1unscoped value

Scope

Cascaded from the map: 377 chunks. Narrowed by element profiling: 21. Shared, no attribute: 14.
Profiling values not in the subject scheme: 0.

Finding

create-tpu-vm.dita, line 41: shared step contains v2-alpha-tpuv6e, owned by v6e. Issue #87 opened for the information architect.

Indexes

Firestore: 18 documents written. Composite vector indexes: in_v5p, in_v6e, in_tpu7x. BM25: 18 rows replaced. Run time 41 s.

The report tells the information architect where profiling is missing. One line to fix, and the next merge closes the issue. Counts are illustrative.

Sources

Checked on 20 October 2026.

Google Developers, "Site Policies." developers.google.com/site-policies
Google Cloud, "TPU7x (Ironwood)," last updated 19 October 2026. docs.cloud.google.com/tpu/docs/tpu7x
Google Cloud, "TPU v6e," last updated 13 October 2026. docs.cloud.google.com/tpu/docs/v6e
Google Cloud, "TPU v5p," last updated 13 October 2026. docs.cloud.google.com/tpu/docs/v5p
Google Cloud, "TPU software versions," last updated 9 October 2026. docs.cloud.google.com/tpu/docs/runtimes
D. Huwiler, K. Stockinger and J. Fürst, "VersionRAG: Version-Aware Retrieval-Augmented Generation for Evolving Documents," arXiv:2510.08109. arxiv.org/html/2510.08109v1
Stack Overflow, "2025 Developer Survey: AI." survey.stackoverflow.co/2025/ai
DITA 1.3 specification, "Filtering based on metadata attributes." dita-lang.org/1.3/dita/archspec/base/filtering
DITA 1.3 specification, "Conditional processing attributes." dita-lang.org/1.3/dita/archspec/base/conditional-processing-attributes
DITA 1.3 specification, "Cascading of metadata attributes in a DITA map." dita-lang.org/1.3/dita/archspec/base/cascading-in-a-ditamap
DITA 1.3 specification, "Defining controlled values for attributes." dita-lang.org/1.3/dita/archspec/base/controlled-values-for-attributes
Google Cloud, "Search with vector embeddings" (Firestore), last updated 19 October 2026. docs.cloud.google.com/firestore/native/docs/vector-search
Google Cloud, "Firestore pricing." cloud.google.com/firestore/pricing
Google AI for Developers, "Embeddings," last updated 9 October 2026. ai.google.dev/gemini-api/docs/embeddings
Google AI for Developers, "Gemini models," last updated 19 October 2026. ai.google.dev/gemini-api/docs/models
Google AI for Developers, "Gemini Developer API pricing." ai.google.dev/gemini-api/docs/pricing
Google AI for Developers, "Billing," last updated 28 September 2026. ai.google.dev/gemini-api/docs/billing
Google AI for Developers, "Structured outputs." ai.google.dev/gemini-api/docs/structured-output
Google, Agent Development Kit. github.com/google/adk-python
Google AI for Developers, "Function calling with the Gemini API." ai.google.dev/gemini-api/docs/function-calling
Google Cloud, "Cloud Run pricing." cloud.google.com/run/pricing
Google Cloud, "BigQuery pricing." cloud.google.com/bigquery/pricing
GitHub Docs, "GitHub Actions billing." docs.github.com/en/billing/concepts/product-billing/github-actions
SQLite, "FTS5 Extension: the bm25() function." sqlite.org/fts5.html
Siddharth Rao, "VaultRAG," repository and design document. github.com/raosiddharthp/vaultrag
Siddharth Rao, "QueryForge," design document. github.com/raosiddharthp/queryforge
Siddharth Rao, "MANIFEST," design document. siddharthraop.github.io/Manifest
Siddharth Rao, "KnowledgeFlow," design document. siddharthraop.github.io/KnowledgeFlow
Siddharth Rao, "AlignR," design document. siddharthraop.github.io/AlignR
G. V. Cormack, C. L. A. Clarke and S. Büttcher, "Reciprocal Rank Fusion outperforms Condorcet and individual Rank Learning Methods," SIGIR 2009. plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf