The ElevenLabs Agents knowledge base: how it works, and how to load your knowledge base into it

ElevenLabs Agents can give your agent a knowledge base in two ways. It can place the whole document in the agent's prompt, so the agent sees every word on every turn, or it can search the document with RAG and pass the agent only the pieces closest to the caller's question. Which one you choose decides most of what can go wrong.

This page is written from ElevenLabs' own documentation, checked on 25 September 2026. KB Builder is not affiliated with ElevenLabs. Our own live calls so far ran on a different platform, so the advice below applies what we learned there; it has not been tested on ElevenLabs calls yet.

How the ElevenLabs knowledge base works

	What the docs say
Where it lives	A workspace knowledge base, with documents attached to each agent. One document can be attached to many agents.
What you can add	Files (PDF, Word .docx, .txt, Markdown .md, HTML, EPUB), a web page or a whole site by crawl or sitemap, or typed text
File size	Up to 20 MB per file
Whole document in the prompt	Only if its extracted text fits, up to roughly 300,000 characters
RAG index size per workspace	Free 1 MB, Starter 2 MB, Creator 20 MB, Pro 100 MB, Scale 500 MB, Business 1 GB
Smallest document RAG will index	500 bytes; anything smaller goes into the prompt
Extra delay with RAG	About 250 milliseconds per response
Extra charge	None stated for the knowledge base; the language model's tokens are billed as usual

Each document has a usage mode. Prompt always puts it in the prompt if it fits. Auto, the default, uses RAG when RAG is switched on and the document is indexed, and otherwise the full text. Folders always go through RAG.

When RAG is on, ElevenLabs rewrites the caller's question for search, then retrieves up to 20 chunks by default, filtered by how close they are to the question (a maximum vector distance of 0.6) and by a total length cap. You can change the embedding model, the number of chunks and the distance under the RAG settings. The chunk size is not stated in the docs.

Where it goes wrong on calls

In prompt mode, very little goes wrong with retrieval, because there is none. The agent reads the whole file. What can still go wrong is the file itself: a price written in a table far from its product, or a question the file never answers, which the agent then answers from general knowledge.

With RAG, the agent sees only a handful of chunks. Each one was cut from your document wherever the platform decided. If a chunk holds "from ₹78 lakh, possession December 2027" and the name of the apartment type sits in the chunk before it, the agent has a price with no owner, and it will attach it to whatever the caller just asked about: the 3 BHK instead of the 2 BHK. This is the failure that cost us two wrong prices on live calls.

Look-alike names meet in the same search. Two unit types, two treatments or two plans with similar names score close to each other, and the agent may be handed the wrong one's details.

How to load a KB Builder file into ElevenLabs

A KB Builder knowledge base is a Markdown file written in short sections that each name the product or service they are about, so a chunk still makes sense when it arrives alone.

1. Open the agent's configuration and find the Knowledge base section.
2. Click Add document and upload the .md file.
3. If the file is under about 300,000 characters (most small businesses are well under), set its usage mode to Prompt. The agent then has the whole file on every turn.
4. If it is larger, or you'd rather save tokens, switch on Use RAG and leave the document on Auto. Wait for indexing to finish before testing.
5. Save the agent and make a few test calls that ask for prices of look-alike products.

When the business's website changes, rebuild the file in KB Builder and replace the document's file rather than adding a second one. The document keeps its ID, so the agent stays connected, and two versions never compete in search.

Lines to add to the agent's prompt

These come from our own live calls. Add them to the system prompt as they are, or reword them in your agent's voice:

Answer questions about the business only from the knowledge base. If it doesn't have the answer, say you will check, and give the business's phone number.
Give a price, time or rule only from a sentence that names the product or service the caller asked about.
Whenever you give a price, say the name of the product or service with it.
If the caller hasn't said which product or service they mean, ask before answering.
When the knowledge base says something has not been confirmed, say exactly that. Never guess.
Don't promise messages, callbacks or bookings unless the knowledge base says the business offers them.

What ElevenLabs says about structuring documents

The docs advise breaking large documents into "smaller, focused pieces" and reviewing call transcripts for topics callers struggle with. A KB Builder knowledge base does the first: every section is short and self-contained, which is what RAG needs.