Document Parsing and OCR

The knowledge base can only retrieve text that has already been parsed. For scanned PDFs, two-column papers, complex tables, and image-based pages, first make sure the body text is parsed correctly, then adjust the model and retrieval parameters.

To judge whether parsing is acceptable, do not look at the words “Import successful”; instead, check whether the body text order, key tables, amounts, dates, and scanned text can be read correctly.

First identify the document type

Document type	Recommended starting point	Must-check items
Markdown, TXT, HTML	Read by default	Heading levels, encoding, line breaks
Copyable-text PDFs, DOCX, PPTX	Use default processing first	Paragraph order, headers and footers, tables
Scanned PDFs, screenshots, image-based pages	Local or system OCR	Recognized language, amounts, dates, IDs
Multi-column, formula-heavy, or complex-table PDFs	Dedicated document processor	Reading order, table structure, footnotes

Where parsing sits in the retrieval pipeline

Parsing errors continue into chunking and retrieval; downstream models cannot recover content already lost from the body text.

Configure and validate a sample document

1

1. Choose a representative sample

Do not import the entire batch first. Choose a document that best exposes problems, such as a scanned PDF with tables or a two-column manual.

2

2. Configure processing capabilities

Open [Settings] → [Document Processing], and configure the document parsing service and OCR as needed. Cloud services usually require an API key or service address; local capabilities may require downloading models first.

First configure the services you plan to use so they are available, then return to the knowledge base and select the processor.

3

3. Import and wait until ready

Add the sample document to the knowledge base. After processing is complete, open the body text and check headings, paragraphs, page numbers, tables, and OCR text.

4

4. Check Chunks

Make sure key conditions and conclusions are not split apart; headers, footers, and the table of contents should not take up too many chunks due to repetition.

Check chunking after the body text is correct; parsing errors cannot be fixed by increasing Chunk size.

5

5. Retest with real questions

In [Retrieval Test], enter a question whose answer is in this document. The result should include the correct source, complete conditions, and key numbers.

6

6. Lock in the setup before bulk import

After the sample passes, import documents of the same type in batches. If the processor, OCR, or chunking settings change, run [Reindex] on the old documents and repeat the test.

Switching processors or OCR will not automatically fix already indexed old documents. You must reindex the relevant items before comparing old and new results.

How to choose a processor and OCR

Choose	Applicable scenarios	Advantages	Notes
Read by default	Common text-based formats	Few settings, fast speed	Complex layouts and scanned pages may lose content
System OCR	Supported by the system and images are clear	No extra API key required, fast	Accuracy depends on the operating system, language, and image quality
Local PaddleOCR	Offline recognition is required	The document never leaves the local machine	Local models must be downloaded before first use
Cloud or self-hosted processors	Two-column, complex tables, or many formulas	Layout analysis is usually stronger	Cloud solutions will receive the document content for processing

Before using a cloud document processor for sensitive materials, confirm the service terms, data retention policy, and account permissions. Fully offline use requires local capabilities for parsing, OCR, embeddings, reranking, and chat.

How to locate typical problems

Symptom	Check first	Handling direction
The body text is empty or very short	Is the file a scanned document	Enable OCR or change the processor
Two-column text is interleaved	Body text reading order	Use a processor that is good at layout analysis
Tables turn into scattered text	Header row, row/column relationships	Change the processor, or organize the key rules into a Markdown note
Headers and footers appear repeatedly	Repeated noise in Chunks	Clean up the source file or switch parsers; do not just increase Top K
OCR number errors	Amounts, dates, IDs	Improve image clarity and manually verify high-risk fields

Configuration notes

Configuration item	Recommended starting point	When to adjust	Action after adjustment
Document processor	Use default processing first	Body text out of order, missing tables, scanned pages empty	Reindex the sample document
OCR	Prefer local or system capabilities for clear scans	Image-based pages have no text or many errors	Reindex and verify key fields
Chunk size and overlap	Keep the knowledge base default settings first	Conditions and conclusions are split apart	Change only one thing per round and reindex
Validation question	3 to 5 real questions	After changing the processor, OCR, or chunking	Use the same question set for comparison

User case

Xiaolin imported a two-column business travel policy PDF. The status was already ready, but the body text had the left and right columns interleaved, and the approval conditions in the retrieval results were incomplete. Instead of increasing Top K first, he switched to a processor better suited for layout analysis, reindexed the same file, and then checked the body text and Chunks.

The acceptance standard is: the approval conditions should be readable in the original order, the amounts and dates should be correct, and fixed questions should retrieve chunks containing the complete conditions.

Common questions

If the body text is correct, do I still need to check Chunks?

Yes. Correct body text only means parsing passed; conditions and conclusions may still be split apart during chunking.

Can increasing Top K fix parsing problems?

No. Top K only controls how many chunks are returned; it cannot recover content that has already been lost or misordered in the body text.

Why did the results not change after switching processors?

The old documents are still using the original index. Run [Reindex] on the relevant items, then retest with the same questions.

Continue reading

Add and organize materials

Select the source and check the processing status.

Check materials and recall

Use fixed questions to validate parsing and chunking.

Data, privacy, and maintenance

Confirm the data boundaries between local and cloud.