How I Built Agent (4): What Does Agent Know? RAG and a Knowledge Base
When I imagined Agent helping with my own projects, I came across an awkwardly simple question: how would it know what I had written on my website or in my documentation? A language model cannot be assumed to have current knowledge of those materials. Worse still, it can invent a missing answer with impressive confidence.
Search first, answer second
I began exploring retrieval-augmented generation, or RAG. The system first finds relevant passages from available documents and only then uses them to produce an answer. Vector search can look for related meaning rather than merely matching exact words.
At first, I was fascinated by how it could find a useful paragraph even when my question used different wording. I soon realised that similarity is not evidence. Two passages can concern the same topic while actually contradicting one another.
Who is allowed to see what?
Security creates another problem. A search engine may find the right passage, but it must not decide whether a particular user is authorised to receive it. Access rights need to be checked against a trusted source, separately from whatever happened to appear in the index.
I also need to know whether a document still exists, whether it is current and whether it has been withdrawn. A knowledge base is not merely a pile of text. It must preserve provenance and the status of its sources.
Sometimes the best answer is no answer
That led me to a lesson which hardly sounds like an exciting advance in AI: a good assistant must be capable of admitting uncertainty. If the available evidence is insufficient, it should not cover the gap with a beautifully composed paragraph.
With Agent I care not only about how natural an answer sounds but also about whether it can be supported and whether retrieving it exposes material that should remain private. These qualities are less visible than a clever chat interface, but much more important.
Next, I will explain why a single public/private switch is not enough.







