Model and Retrieval Settings

The knowledge base first parses the materials and splits them into chunks, then finds the content in those chunks most likely to answer the question. The embedding model, reranking model, and Top K determine “how to search, how to rank, and how many to keep in the end”; they cannot make up for missing body text or incorrect chunking.

When tuning for the first time, first run one recall test with the current settings. After that, change only one parameter each round, and always retest with the same set of questions, so you can tell which setting caused the change.

First understand the retrieval chain

BM25 can always work on its own; after configuring an embedding model, vector retrieval is added, and reranking is an optional step after merging candidates.

Every layer in the diagram affects the final result: the material content determines whether there is an answer, parsing and chunking determine whether the answer is complete, and retrieval and reranking determine whether the correct chunk can be ranked near the front.

Three common combinations

Combination	Actual retrieval method	What kind of materials it suits	When to upgrade
Do not use an embedding model	BM25 keyword search only	Lots of clause numbers, product names, and proper nouns	When using a different wording makes it hard to hit, add an embedding model
Embedding model	Run BM25 and vector retrieval in parallel, then merge the candidate results	The user's wording differs greatly from the original material	When the correct chunk can be retrieved but the ranking is unstable, add a reranking model
Embedding model + reranking model	Re-score and rerank after hybrid retrieval	Candidate chunks are similar and require stable ranking	First keep the threshold at 0.0, then adjust it gradually according to the noise

The knowledge base can work even without configuring an embedding model. Only after selecting a reranking model will the settings panel show [Similarity Threshold].

Open settings and confirm the current configuration

Open the left navigation [Knowledge Base] → select a knowledge base → click [Settings] in the upper right.

The default section includes [Document Processing], [Embedding Model], [Reranking Model], and [Top K]. For the first test, it is recommended to keep Top K at 6 and temporarily not raise the similarity threshold.

First confirm the model and Top K, then go to [Advanced Settings] to check chunking.

Recommended starting point

Configuration item	Initial interface values	Suggested starting point	Function and notes
Embedding model	Do not use	Keep it unused at first when keywords can reliably match	After adding it, keyword and vector retrieval will run at the same time; billing and data processing for cloud models depend on the service provider
Reranking model	Do not use	Enable it only when the correct chunk can be retrieved but the ranking is unstable	This adds one more model scoring step and more waiting time
Top K	6, optional 1–50	Keep 6 for now	Too small may miss the answer; too large brings more noise and takes up more conversation context
Similarity threshold	0.0, shown only when reranking is configured	Start from 0.0	Only filters low-scoring results after reranking; setting it too high may remove the correct chunk as well

Establish a baseline with fixed questions

Before you begin, prepare 3 to 5 real questions with clear answers. The questions should cover proper nouns in the materials, colloquial wording, and situations that require context to answer.

1

1. First check the material body and Chunks

Confirm that the answer really exists in the material, and that the conditions and conclusion were not split incorrectly. If there is a problem at this layer, handle parsing or chunking first; do not rush to adjust the model.

2

2. Complete the first recall test

Open [Recall Test], enter the prepared questions one by one, and record the hit source, chunk content, order, and time taken.

Do not just look at whether there is a result; also confirm that the source is correct, the chunk is complete, and the ranking is reasonable.

3

3. Find the layer where the problem lies

• No correct chunk at all: first check the material content, parsing, and chunking.
• Keywords can find it, but a different wording cannot: try an embedding model.
• The correct chunk can appear, but it often ranks behind others: try a reranking model.
• The correct chunk is filtered out: lower the similarity threshold.
• The first few results are all relevant, but the answer is still incomplete: slightly increase Top K again.

4

4. Change only one item each round

For example, adjust Top K first and save it; do not change the embedding model and chunk size at the same time. If old material chunks or existing vectors are involved, re-index or rebuild as described below.

5

5. Retest with the same set of questions

Compare the source, chunk completeness, ranking, and time taken before and after the adjustment. If there is no improvement, restore the previous settings and test the next item.

How to adjust when chunking is incomplete

Expand [Advanced Settings], and you can see [Smart Chunking], [Delimiter], [Chunk Size], and [Overlap Size].

For general materials, you can start with smart chunking enabled, chunk size 1024, and overlap size 200.

Configuration item	Initial interface values	When to adjust	Common side effects
Smart chunking	Enable	Keep it enabled when the material has clear headings and paragraph structure	After turning it off, split only by delimiters
Delimiter	\n\n	Adjust when the material has stable custom paragraph boundaries	When smart chunking is turned off, the delimiter cannot be empty
Chunk size	1024 tokens	Decrease it when one chunk mixes multiple topics; increase it when conditions and conclusions are always split apart	Too large increases noise; too small loses context
Overlap size	200 tokens	Increase it slightly when key information often crosses chunk boundaries	It must be smaller than the chunk size; if too large, it will create duplicate content

Chunking settings only affect content added afterward. To make existing materials use the new settings, execute [Re-index] in the material row menu, then retest with the same set of questions.

Changing models and rebuilding the knowledge base

When enabling an embedding model for a knowledge base that previously used only BM25, you can directly complete vector indexing. When changing the embedding model for a knowledge base that already has vectors, the interface will enter the [Rebuild Knowledge Base] process, because vectors generated by different embedding models cannot be mixed.

Before starting the rebuild, first confirm that the new embedding model can be called normally. After rebuilding, you need to re-establish the recall baseline; do not change the model and modify chunking in the same round, or you will not be able to tell where the change in results came from.

If you need to complete embeddings locally, open [Settings] → [Local Models], download an available model in the [Embedding Model] section, and then return to the knowledge base settings to select it.

First complete the local model download, then return to the knowledge base to select it and build the index.

Tuning loop

Fixed questions → check results → identify the layer → adjust one item → reindex if needed → retest.

Only keep changes that stably improve the fixed test questions each time. If the result does not improve, restore the previous set of settings instead of stacking more changes.

A complete example

Xiaolin maintains a set of employee travel expense rules. The material says “lodging expense standard,” but employees often ask “how much can be reimbursed at most for a hotel stay?”

1. He first tested with the default settings and found that queries using the original wording could hit, but colloquial wording was unstable.
2. He configured an embedding model and completed indexing, then retested with the same questions.
3. The correct chunk could appear stably, but occasionally ranked behind others, so he configured a reranking model as well.
4. He kept Top K at 6 and the threshold at 0.0, and only gradually raised the threshold when it was confirmed that irrelevant results were obvious.

The completion criterion is: the same lodging standard can be found within the first few results for three different phrasings, and the chunk contains both the applicable conditions and the reimbursement limit.

Common questions

Can the knowledge base still search without using an embedding model?

Yes. The knowledge base will use BM25 keyword retrieval, which is suitable for clause numbers, proper nouns, and queries close to the original wording.

Why can’t I see [Similarity Threshold]?

Only after selecting a reranking model will the settings panel show [Similarity Threshold].

Why didn’t old materials change after modifying the chunk size?

Chunking settings only affect content added afterward. Please execute [Re-index] for the existing materials, then retest with the same questions.

The correct chunk never appears at all; should I first increase Top K?

First check the data main text and Chunks. If parsing or splitting is wrong, increasing Top K will only return more incorrect or incomplete fragments.

Continue reading

Knowledge base basics

First build an overall understanding of how the knowledge base works.

Document parsing and OCR

When the body text is missing or recognition is incorrect, troubleshoot from the material processing layer.

Add and organize materials

Learn about re-indexing, material status, and content maintenance.

Check materials and recall

Continue practicing recall tests and result judgment.