> For the complete documentation index, see [llms.txt](https://docs.cherryai.com.cn/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.cherryai.com.cn/docs/en-us/knowledge-base/document-preprocessing.md).

# Document Parsing and OCR

The knowledge base can only retrieve text that has already been parsed. For scanned PDFs, double-column papers, complex tables, and image-based pages, first make sure the main text is parsed correctly, then adjust the model and retrieval parameters.

{% hint style="info" %}
To judge whether parsing is acceptable, do not look at the words “Import successful”; check whether the main text order, key tables, amounts, dates, and scanned text can all be read correctly.
{% endhint %}

## First determine the document type

| Document type                                     | Recommended starting point   | Required checks                                      |
| ------------------------------------------------- | ---------------------------- | ---------------------------------------------------- |
| Markdown, TXT, HTML                               | Read by default              | Heading levels, encoding, line breaks                |
| PDF, DOCX, PPTX with copyable text                | Use default processing first | Paragraph order, headers and footers, tables         |
| Scanned PDFs, screenshots, image-based pages      | Local or system OCR          | Recognition language, amounts, dates, serial numbers |
| Multicolumn, formula-heavy, or complex-table PDFs | Dedicated document processor | Reading order, table structure, footnotes            |

## Where parsing sits in the retrieval pipeline

<figure><img src="/files/6e98934885eff518d13a4942ff027d90687e8866" alt="资料经过解析与 OCR、切分、关键词和向量检索后进入回答的知识库检索架构图"><figcaption><p>Parsing errors continue downstream to chunking and retrieval; downstream models cannot recover content that has already been lost from the main text.</p></figcaption></figure>

## Configure and validate one sample document

{% stepper %}
{% step %}

### 1. Select a representative sample

Do not import the entire batch first. Pick one document that best exposes problems, such as a scanned PDF with tables or a double-column manual.
{% endstep %}

{% step %}

### 2. Configure processing capabilities

Open \[Settings] → \[Document Processing], and configure the document parsing service and OCR as needed. Cloud services usually require an API key or service address; local capabilities may require downloading a model first.

<figure><img src="/files/1c3fa785aca74bd301e868ff807803bbba24e769" alt="文档处理设置中的文件解析与 OCR 服务配置"><figcaption><p>First configure the services you plan to use so they are available, then return to the knowledge base and select a processor.</p></figcaption></figure>
{% endstep %}

{% step %}

### 3. Import and wait until ready

Add the sample document to the knowledge base. After processing is complete, open the main text and check the headings, paragraphs, page numbers, tables, and OCR text.
{% endstep %}

{% step %}

### 4. Check the chunks

Make sure key conditions and conclusions are not split apart; headers, footers, and the table of contents should not take up too many chunks as repeated noise.

<figure><img src="/files/0359b26559db3bea6e323bf599d74c081ad91c93" alt="知识库高级设置中的智能分段、分隔符、分段大小和重叠大小"><figcaption><p>Only check chunking after the main text is correct; parsing errors cannot be fixed by increasing chunk size.</p></figcaption></figure>
{% endstep %}

{% step %}

### 5. Retest with real questions

In \[Retrieval Test], enter a question whose answer is in this document. The result should include the correct source, complete conditions, and key numbers.
{% endstep %}

{% step %}

### 6. Fix the setup before batch importing

After the sample passes, batch import documents of the same type. After changing the processor, OCR, or chunking settings, run \[Reindex] on old documents and repeat the test.
{% endstep %}
{% endstepper %}

{% hint style="warning" %}
Switching processors or OCR will not automatically fix old documents that have already been indexed. You must reindex the relevant items to compare the old and new results.
{% endhint %}

## How to choose a processor and OCR

| Choice                         | When to use                                  | Advantages                               | Notes                                                                 |
| ------------------------------ | -------------------------------------------- | ---------------------------------------- | --------------------------------------------------------------------- |
| Read by default                | Common text-based formats                    | Few settings, fast speed                 | Complex layouts and scanned pages may lose content                    |
| System OCR                     | Supported by the system and clear images     | No extra API key required, fast speed    | Accuracy depends on the operating system, language, and image quality |
| Local PaddleOCR                | Offline recognition needed                   | Documents do not leave the local machine | Local models need to be downloaded before first use                   |
| Cloud or self-hosted processor | Double-column, complex tables, many formulas | Usually stronger at layout analysis      | Cloud solutions receive the document content for processing           |

{% hint style="danger" %}
Before using a cloud document processor for sensitive data, first confirm the service terms, data retention policy, and account permissions. Fully offline use requires local capabilities for parsing, OCR, embeddings, reranking, and chat.
{% endhint %}

## How to troubleshoot typical issues

| Symptom                                  | Check first                             | Action to take                                                            |
| ---------------------------------------- | --------------------------------------- | ------------------------------------------------------------------------- |
| Main text is empty or very short         | Whether the file is a scanned document  | Enable OCR or switch processors                                           |
| Double-column text is interleaved        | Reading order of the main text          | Use a processor that is good at layout analysis                           |
| Tables become scattered text             | Table headers, row-column relationships | Switch processors, or organize key rules into Markdown notes              |
| Headers and footers repeat over and over | Duplicate noise in chunks               | Clean up the source file or change the parser; do not just increase Top K |
| OCR gets numbers wrong                   | Amounts, dates, serial numbers          | Improve image clarity and manually verify high-risk fields                |

## Configuration notes

| Configuration item     | Recommended starting point                                            | When to adjust                                            | Actions after adjustment                   |
| ---------------------- | --------------------------------------------------------------------- | --------------------------------------------------------- | ------------------------------------------ |
| Document processor     | Use default processing first                                          | Wrong text order, missing tables, scanned pages are blank | Reindex the sample document                |
| OCR                    | Clear scanned documents should use local or system capabilities first | Image-based pages have no text or many typos              | Reindex and verify key fields              |
| Chunk size and overlap | Keep the knowledge base default settings first                        | Conditions and conclusions are split apart                | Change only one item per round and reindex |
| Validation question    | 3–5 real questions                                                    | After changing the processor, OCR, or chunking            | Use the same question set for comparison   |

## User case

Xiaolin imported a double-column travel policy PDF. The status was already ready, but the main text interleaved the left and right columns, and the approval conditions in the retrieval results were incomplete. Instead of first increasing Top K, he switched to a processor better suited for layout analysis, reindexed the same file, and then checked the main text and chunks.

The completion standard is: the approval conditions are readable in the original order, the amounts and dates are correct, and fixed questions can retrieve chunks containing the complete conditions.

## FAQ

<details>

<summary>If the main text is correct, do we still need to check chunks?</summary>

Yes. Correct main text only means parsing passed; conditions and conclusions may still be split apart during chunking.

</details>

<details>

<summary>Can increasing Top K fix parsing issues?</summary>

No. Top K only controls how many chunks are returned; it cannot restore content that has already been lost or misordered in the main text.

</details>

<details>

<summary>Why did changing processors not change the result?</summary>

The old documents are still using the original index. Run \[Reindex] on the relevant items, then retest with the same questions.

</details>

## Continue reading

<table data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>Add and organize materials</strong></td><td>Select the source and check the processing status.</td><td><a href="/pages/53062e948442e1c25a4e6ad7b4ec9baf548deafc">/pages/53062e948442e1c25a4e6ad7b4ec9baf548deafc</a></td></tr><tr><td><strong>Check materials and recall</strong></td><td>Use fixed questions to validate parsing and chunking.</td><td><a href="/pages/7c7a6d04d1e906eb7a89261dabeedb48874a9b8e">/pages/7c7a6d04d1e906eb7a89261dabeedb48874a9b8e</a></td></tr><tr><td><strong>Data, privacy, and maintenance</strong></td><td>Confirm the data boundaries between local and cloud.</td><td><a href="/pages/79d7d1d4aa8046a5b156be2497b3f18795dd88a8">/pages/79d7d1d4aa8046a5b156be2497b3f18795dd88a8</a></td></tr></tbody></table>


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.cherryai.com.cn/docs/en-us/knowledge-base/document-preprocessing.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.