Build a Knowledge Base and Run Recall Tests

The knowledge base processes files, web pages, and notes into searchable chunks. It is suited for answering “what does the material say,” and it does not mean the model permanently remembers the entire document.

Create and import materials

1

1. Open [Knowledge Base] → [Create Knowledge Base]

Fill in an easily recognizable name, and choose an available embedding model. The embedding model turns the materials into searchable representations; it is not the same as the main model used for conversations.

2

2. Select a source of materials

You can add files, folders, notes, or URLs. Common document formats include PDF, DOCX, Markdown, Excel, TXT, and CSV; scanned files may require OCR to extract the text.

3

3. Wait for processing to complete

Open the file details to view the preview and chunking. If the title is missing, garbled, or table structure is lost, first organize the original file, then process it again.

4

4. Run a recall test

Test with questions real users would ask, not just file titles. Check whether the returned chunks are relevant and contain the full context, then decide whether to bind it to the Agent.

When creating a knowledge base, first enter a name and select an available embedding model.

① All three travel-related documents are ready; ② Click [Recall Test] at the top to verify actual questions.

① Enter questions that are actually asked in real work; ② Check the matched materials, chunk content, and relevance.

Validate recall results with real questions

After the materials show as [Ready], test them with questions that would really come up at work. For example, in a policy knowledge base you could ask “Can overseas car rentals be reimbursed?” Then check whether the returned content comes from the correct material and includes enough context.

What the results show	Next step
Matched the correct material, and the chunk is enough to answer the question	Can be bound for Agent use
Matched the correct material, but the chunk was truncated	First check the structure of the original text, then adjust the chunk length
Matched old policies or irrelevant materials	Clean up outdated materials, and add clearer titles and content
No results at all	Check the material status and how the question is phrased; do not blindly increase the number of returned results first

After the recall test passes, verify the full Q&A in the Agent. This helps distinguish between “could not find the material” and “found the material but answered poorly.”

How to understand RAG settings

RAG means “retrieve materials first, then let the model answer.” Common settings control chunk length, overlap, number of returned results, and relevance threshold.

Configuration	Effect	Suggested starting point	When to adjust
Chunk length	Decides how much content each retrieved chunk contains	Start with the page’s default value	When chunks often cut off sentences or mix too many topics together
Chunk overlap	Keeps the connection between adjacent chunks	Keep a small amount of overlap	When clauses span multiple chunks and context is often broken
Number of returned results	How many candidate chunks to provide at once	Start with a small number of results	Increase it when key materials are missed; decrease it when there is too much noise
Relevance threshold	Filters out irrelevant content	Decide using recall tests	Raise it when there are many irrelevant results; lower it when correct chunks are filtered out

Only adjust RAG settings according to the material structure when the recall results are unstable.

Use case: building an internal policy Q&A knowledge base

Organize the current policies by department, and include the topic and effective scope in the file names. After importing, use real questions such as “How should overtime hotel expenses be handled?” and “Who needs to approve leave during probation?” for recall testing. Once the chunks are confirmed to be accurate, bind only this knowledge base to the “Policy Q&A” Agent, and require it to mention the material name when answering; if content is missing, explicitly say it was not found.

The knowledge base does not automatically determine whether a file is outdated. When policies, prices, or processes change, update or remove old materials and run the recall test again.

Why are there still no results after importing?

Check the file processing status, the embedding model connection, and the document preview. If a scanned PDF has no extractable text, you need to configure OCR first or replace it with a searchable version.