Document Parsing and OCR

The knowledge base can only retrieve text that has already been parsed. For scanned PDFs, double-column papers, complex tables, and image-based pages, first make sure the main text is parsed correctly, then adjust the model and retrieval parameters.

To judge whether parsing is acceptable, do not look at the words “Import successful”; check whether the main text order, key tables, amounts, dates, and scanned text can all be read correctly.

First determine the document type

Document type	Recommended starting point	Required checks
Markdown, TXT, HTML	Read by default	Heading levels, encoding, line breaks
PDF, DOCX, PPTX with copyable text	Use default processing first	Paragraph order, headers and footers, tables
Scanned PDFs, screenshots, image-based pages	Local or system OCR	Recognition language, amounts, dates, serial numbers
Multicolumn, formula-heavy, or complex-table PDFs	Dedicated document processor	Reading order, table structure, footnotes

Where parsing sits in the retrieval pipeline

Parsing errors continue downstream to chunking and retrieval; downstream models cannot recover content that has already been lost from the main text.

Configure and validate one sample document

1

1. Select a representative sample

Do not import the entire batch first. Pick one document that best exposes problems, such as a scanned PDF with tables or a double-column manual.

2

2. Configure processing capabilities

Open [Settings] → [Document Processing], and configure the document parsing service and OCR as needed. Cloud services usually require an API key or service address; local capabilities may require downloading a model first.

First configure the services you plan to use so they are available, then return to the knowledge base and select a processor.

3

3. Import and wait until ready

Add the sample document to the knowledge base. After processing is complete, open the main text and check the headings, paragraphs, page numbers, tables, and OCR text.

4

4. Check the chunks

Make sure key conditions and conclusions are not split apart; headers, footers, and the table of contents should not take up too many chunks as repeated noise.

Only check chunking after the main text is correct; parsing errors cannot be fixed by increasing chunk size.

5

5. Retest with real questions

In [Retrieval Test], enter a question whose answer is in this document. The result should include the correct source, complete conditions, and key numbers.

6

6. Fix the setup before batch importing

After the sample passes, batch import documents of the same type. After changing the processor, OCR, or chunking settings, run [Reindex] on old documents and repeat the test.

Switching processors or OCR will not automatically fix old documents that have already been indexed. You must reindex the relevant items to compare the old and new results.

How to choose a processor and OCR

Choice	When to use	Advantages	Notes
Read by default	Common text-based formats	Few settings, fast speed	Complex layouts and scanned pages may lose content
System OCR	Supported by the system and clear images	No extra API key required, fast speed	Accuracy depends on the operating system, language, and image quality
Local PaddleOCR	Offline recognition needed	Documents do not leave the local machine	Local models need to be downloaded before first use
Cloud or self-hosted processor	Double-column, complex tables, many formulas	Usually stronger at layout analysis	Cloud solutions receive the document content for processing

Before using a cloud document processor for sensitive data, first confirm the service terms, data retention policy, and account permissions. Fully offline use requires local capabilities for parsing, OCR, embeddings, reranking, and chat.

How to troubleshoot typical issues

Symptom	Check first	Action to take
Main text is empty or very short	Whether the file is a scanned document	Enable OCR or switch processors
Double-column text is interleaved	Reading order of the main text	Use a processor that is good at layout analysis
Tables become scattered text	Table headers, row-column relationships	Switch processors, or organize key rules into Markdown notes
Headers and footers repeat over and over	Duplicate noise in chunks	Clean up the source file or change the parser; do not just increase Top K
OCR gets numbers wrong	Amounts, dates, serial numbers	Improve image clarity and manually verify high-risk fields

Configuration notes

Configuration item	Recommended starting point	When to adjust	Actions after adjustment
Document processor	Use default processing first	Wrong text order, missing tables, scanned pages are blank	Reindex the sample document
OCR	Clear scanned documents should use local or system capabilities first	Image-based pages have no text or many typos	Reindex and verify key fields
Chunk size and overlap	Keep the knowledge base default settings first	Conditions and conclusions are split apart	Change only one item per round and reindex
Validation question	3–5 real questions	After changing the processor, OCR, or chunking	Use the same question set for comparison

User case

Xiaolin imported a double-column travel policy PDF. The status was already ready, but the main text interleaved the left and right columns, and the approval conditions in the retrieval results were incomplete. Instead of first increasing Top K, he switched to a processor better suited for layout analysis, reindexed the same file, and then checked the main text and chunks.

The completion standard is: the approval conditions are readable in the original order, the amounts and dates are correct, and fixed questions can retrieve chunks containing the complete conditions.

FAQ

If the main text is correct, do we still need to check chunks?

Yes. Correct main text only means parsing passed; conditions and conclusions may still be split apart during chunking.

Can increasing Top K fix parsing issues?

No. Top K only controls how many chunks are returned; it cannot restore content that has already been lost or misordered in the main text.

Why did changing processors not change the result?

The old documents are still using the original index. Run [Reindex] on the relevant items, then retest with the same questions.

Continue reading

Add and organize materials

Select the source and check the processing status.

Check materials and recall

Use fixed questions to validate parsing and chunking.

Data, privacy, and maintenance

Confirm the data boundaries between local and cloud.