A working definition

Inspectable retrieval

/ɪnˈspɛktəbəl rɪˈtriːvəl/

The ability to inspect what a retrieval system returned, where it came from, and the query and configuration used to retrieve it.

Before judging an answer, look at the evidence the system retrieved.

What inspection reveals

The result

Which passages matched, in what order, and with which scores.

The source

The document, chunk, and location behind each passage. A relevant-sounding sentence needs its context.

The request

The query, filters, and retrieval parameters used to produce the result.

The configuration

The embedding model, index, and ranking settings needed to make comparisons meaningful.

Replay makes retrieval drift visible

Keep the original query, parameters and ranked results in a query log. After your collection changes, rerun the query against its current state and compare the records: which passages moved, disappeared or appeared? Changed results are the evidence you are looking for.

The log provides a comparison baseline. Reconstructing an earlier collection state is a separate task that requires retaining its data and retrieval configuration.

A retrieval problem, up close

Debug RAG retrieval: right topic, missing answer.

A passage can sound relevant without answering the question. Check what was retrieved before changing the generation prompt.

The separate Refund policy says, “Refund requests are accepted within 14 days of purchase.” In this example, that is the evidence the retriever needed to return.

Check that the refund document was ingested, that its chunk keeps the deadline with its context, and that filters did not exclude it. Keep the original result record, then rerun the query after the change. Compare the sources and ranking to check whether the refund passage now appears. Inspection identifies the problem; replay helps check the fix.