emoji detected PictureItem and colons not found

[Choi-YoungHyun]

Question

Thank you for developing the Docling library.
I have two questions.

The test file is a PDF exported from Notion.

When converting this file to Markdown, I encountered two issues and would like some advice.

Issue 1. Colons are not recognized correctly and are instead interpreted as \ue092 and \ue09d.
Some of these cases may be due to text that was intentionally written without spaces for testing.

Issue 2. When emojis are used, there is a tendency for the text to be recognized as an image.

[NOTI]

• This file is exported to notion page

[TEST-FILE]
notion_to.pdf

[TEST CODE]

from docling.datamodel.pipeline_options import PdfPipelineOptions, TableStructureOptions, EasyOcrOptions
from docling.datamodel.accelerator_options import AcceleratorDevice, AcceleratorOptions
from docling.datamodel.base_models import InputFormat
from docling_core.types.doc import TableItem,TextItem,RefItem
from docling_core.types.doc import ImageRefMode

accelerator_options = AcceleratorOptions(
    num_threads=5, 
    device=AcceleratorDevice.CPU
)

pipeline_options = PdfPipelineOptions(
    generate_parsed_pages=True,
    generate_picture_images=True,
    do_table_structure=True,
    do_ocr=True,
    ocr_options=EasyOcrOptions(lang=['ko', 'en']),
    accelerator_options=accelerator_options,
    images_scale=2.0,  # 해상도 높이기
    generate_page_images=True,  # 추가!
    table_structure_options=TableStructureOptions(
        do_cell_matching=True,
        mode="accurate"
    )
)

converter = DocumentConverter(
    format_options={
        InputFormat.PDF: PdfFormatOption(
            pipeline_options=pipeline_options
        )
    }
)

result = converter.convert("이모지테스트_노션_DPF.pdf") #파일명

for item,_  in result.document.iterate_items():
    print(item)

# 4. 이제 마크다운 출력
markdown = result.document.export_to_markdown(
    image_mode=ImageRefMode.EMBEDDED
)

print(markdown)

[RESULT PRINT ISSUE ITEM]

self_ref='#/texts/2' parent=RefItem(cref='#/body') children=[] content_layer=<ContentLayer.BODY: 'body'> meta=None label=<DocItemLabel.TEXT: 'text'> prov=[ProvenanceItem(page_no=1, bbox=BoundingBox(l=72.0, t=675.0450029101562, r=121.107, b=636.5250029101562, coord_origin=<CoordOrigin.BOTTOMLEFT: 'BOTTOMLEFT'>), charspan=(0, 14))] orig='SERVER\ue092 FRONT\ue09d' text='SERVER\ue092 FRONT\ue09d' formatting=None hyperlink=None

[RESULT]

[dosubot]

Both issues you’re seeing are known limitations with PDF extraction and OCR in Docling, especially with PDFs exported from Notion.

For the colon issue: Colons being replaced by Unicode private use characters (like \ue092, \ue09d) usually happens because the PDF’s font encoding is non-standard or subsetted, and the PDF backend or OCR engine can’t map the glyphs correctly. This is common with PDFs from Notion and similar tools. Docling doesn’t have special logic to fix these; the characters are passed through as-is from the PDF or OCR layer. You can sometimes improve results by enabling full-page OCR (do_ocr=True and ocr_options.force_full_page_ocr=True), which forces OCR on all content and may bypass the problematic text layer. Flattening the PDF before processing or switching to a different PDF backend (like PyPdfiumDocumentBackend) can also help in some cases, but results vary by file [source] [source].

For the emoji-as-image issue: Emojis are often detected as images (PictureItem) because the PDF either embeds them as bitmap images or uses glyphs that the OCR/layout classifier can’t recognize as text. When this happens, Docling’s pipeline classifies them as images, and the Markdown export will serialize them as embedded images, not inline text. There’s no special handling for emojis in the Markdown export logic—they’re treated according to how the upstream pipeline classifies them. Improving OCR accuracy (e.g., by rescaling, denoising, or thresholding images before OCR) can sometimes help, but if the PDF encodes the emoji as an image, it will always be treated as such [source] [source].

There’s no universal fix for either issue yet, but you can try: enabling full-page OCR, flattening the PDF, switching PDF backends, or preprocessing images before OCR. If you want to debug further, inspect result.document for annotations and check your pipeline logs for warnings or errors about missing glyphs or OCR failures [source].

Let me know if you want example code for any of these workarounds.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[Ritinikhil]

Hi @Choi-YoungHyun,

I looked into this issue. It seems related to how Notion handles font encoding and emoji embedding in its PDF exports.

Issue 1: Colons as \ue092 / \ue09d

This happens because the PDF has a non-standard font map (common with Notion exports), so the extraction layer reads the Private Use Area codes instead of the actual characters.

To fix this, you should force Full Page OCR. This tells Docling to ignore the potentially corrupted text layer and purely OCR the visual page. You can enable this by adding force_full_page_ocr=True to your ocr_options.

Modified Code:

pipeline_options = PdfPipelineOptions(
    # ... other options ...
    do_ocr=True,
    # Add force_full_page_ocr=True here:
    ocr_options=EasyOcrOptions(lang=['ko', 'en'], force_full_page_ocr=True),
    # ...
)

[Choi-YoungHyun]

@Ritinikhil
Thx for your reply.
But I already force_full_page_ocr=True ! 😂