How to build a chatbot knowledge base that doesn't give wrong answers

A chatbot knowledge base is the text your chatbot or AI voice agent answers from. It holds the business's services and prices, hours, location, booking rules, policies and contact details. When a customer asks something, the platform searches that text, hands the closest few pieces to the language model, and the model answers from them. If the pieces are wrong, vague or missing, the answer will be too, however good the prompt is.

This guide is for people who build bots for businesses on Vapi, Retell AI, ElevenLabs, Chatbase, Botpress, Bolna or anything similar. It comes from putting a voice agent on live calls for a client and tracing every wrong answer back to its cause. Almost every one traced back to the knowledge base.

How a chatbot actually uses its knowledge base

Most platforms do the same four things:

1. Cut your document into chunks of a few hundred to a few thousand characters.
2. Turn each chunk into a vector, a numeric fingerprint of its meaning.
3. Search. When the customer asks something, find the chunks whose fingerprint is closest to the question.
4. Hand over only those chunks to the model, and let it answer.

Retell AI, for example, gives the model 3 chunks per turn by default. You can set it anywhere from 1 to 10. Vapi uses a query tool, and its docs say the system prompt has to tell the agent when to use it. Otherwise the agent may answer from general knowledge instead. ElevenLabs lets you skip the search entirely and put a document of up to roughly 300,000 characters straight into the agent's context.

The step that causes the trouble is the first one. Your document gets cut wherever the platform decides, and each chunk has to make sense on its own. A chunk doesn't know what the heading above it said, or which row of the table it came from.

Why chatbots give wrong answers from a knowledge base

1. A number without its owner gets quoted for the wrong thing

This is the one that cost us. A caller asked a resort's agent the price of a day package. The agent said 3,000 rupees. That was a villa's price. The retrieved text gave the amount without naming the villa, so the agent attached it to the package the caller had asked about.

The same thing happens with a dental clinic's fee table ("Whitening ... $350", then "Implants ... from $2,400"), a salon's price list or a builder's configuration sheet. Once a table is chunked, a row loses its column headings and a price loses its product.

2. Where the knowledge base is silent, the bot guesses

Ask a bot whether pets are allowed and, if nothing in the knowledge base mentions pets, it will often answer anyway, politely and wrongly. Callers ask what businesses of that kind always get asked. Is the exam fee credited to the treatment? Does the flat's price include parking? Is the call-out fee waived if I go ahead? If the website never says, the bot has nothing to read.

3. Two pages disagree

The home page says "Free consultation for all new patients." The fees page says "New patient exam $89." Load both and the knowledge base now holds two answers, and the bot will give one or the other depending on which chunk it retrieves.

4. Facts that differ on purpose get merged

The clinic is open nine to six, except the orthodontist, who only comes in on Tuesdays and Fridays. A summary, whether written by you or by ChatGPT, turns that into "open nine to six". Now someone books braces on a Wednesday.

How to create a knowledge base for an AI agent, step by step

Step 1: List what the business sells

Write down every product, service, package or offer a customer can buy, using the business's own names. Include the ones only some customers can buy, such as trade rates, corporate packages and school-group prices. Their numbers collide with the public ones unless each gets its own section.

Step 2: One section per product, one heading per topic

Give each product its own headings: price, what's included, timing, who it's for, how to book. The heading carries the product name. Retrieval often returns a chunk with its heading attached, so the heading does real work.

Step 3: Name the product in every sentence that holds a number

This is the rule that matters most. Write "In-chair teeth whitening costs 350 dollars per session", never "It costs 350" or "Price: $350". Do the same for times, sizes and capacities. It reads clunkily, and it survives being cut into chunks.

Step 4: List products by name only

Callers ask "what treatments do you do?", so you need a list. Keep prices out of it. A list that mixes names and prices is exactly the table that gets mangled. A list of names alone can't be misquoted.

Step 5: Write a sentence for every likely question, including the unknowns

For each question a caller is likely to ask, write the fact, a plain no, or "has not been confirmed". A sentence like "Whether pets are allowed at the resort has not been confirmed" stops the guess. It also leaves you a list of questions to put to your client.

Step 6: Settle contradictions before loading

Where two pages disagree, pick the true one with your client and delete the other. Don't let the bot pick for you on every call.

Step 7: Keep scoped facts scoped

If check-in is 1 PM for most rooms and 12 PM for two weekend offers, write both, each naming the rooms it applies to. Never collapse them into "check-in is at 1 PM".

Step 8: One file, in the website's language

Keep it in one plain Markdown or text file, in the language the business uses on its website. One file is easy to replace when prices change. It also moves between platforms, and a small one can go into full context where the platform allows it.

A knowledge base template you can copy

Here's a short example for a clinic. The business is made up, but the pattern is the one that held up on live calls.

## Treatments at Brightsmile Dental
Brightsmile Dental offers six treatments: check-up and clean, fillings,
root canal treatment, in-chair teeth whitening, dental implants and clear aligners.

## In-chair teeth whitening: price
In-chair teeth whitening costs 350 dollars per session.

## Dental implants: price
Dental implants cost from 2,400 dollars per implant. The dental implant
price does not include the crown, which is billed separately.

## Clear aligners: who provides them
Clear aligners are provided by the orthodontist, who sees patients on
Tuesdays and Fridays only.

## New patient exam: fee
The new patient exam costs 89 dollars. Whether the new patient exam fee
is credited toward treatment has not been confirmed.

Notice what is missing: a table, the word "it", and any number standing on its own.

Full context or retrieval?

A small business's knowledge base written this way comes to roughly 10,000 tokens. If your platform lets you put the whole document into the prompt instead of searching it (ElevenLabs does, up to about 300,000 characters), try that first. With no retrieval step, the wrong chunk can never be picked. Where you do have to use retrieval, everything above is what keeps it honest. On Retell you can also raise the number of chunks per turn while you test.

Test it before the client does

Before launch, ask the bot these questions, preferably in a real call or chat and not the platform's test panel:

• The price of every product, one after another, by name. The wrong-owner problem shows up when similar products are asked back to back.
• Something the website never says. If the bot answers it confidently, it is guessing.
• The exceptions. The orthodontist on a Wednesday, check-in on the weekend offer, the group rate.
• Both sides of any contradiction you settled. Make sure the old answer is really gone.

Doing this for every client

Doing this by hand works. It is how we built our first client's knowledge base, over several rounds of test calls. It stops scaling after a handful of clients, and each file goes stale the day a client changes a price.

KB Builder does it from the website. It finds what the business sells and works through the questions that kind of business gets asked. It writes every sentence with its product named, shows you where pages contradict each other, and gives you the list of what the site never says. You get a Markdown file that loads into Vapi, Retell, ElevenLabs, Chatbase, Botpress or Bolna. See the comparison of Vapi and Retell if you are still choosing a platform.