Feature Request: Support table image output in Markdown export

[ryyhan]

Feature Request: Support table image output in Markdown export

Description

When exporting a document to Markdown using docling, TableItem elements are currently only exported as text-based Markdown tables (or HTML). Even when image_mode is set to ImageRefMode.REFERENCED or ImageRefMode.EMBEDDED and generate_table_images=True is used, the table images are not included in the Markdown output.

This contrasts with PictureItem elements, which correctly respect the image_mode setting and output image links.

This issue was originally reported in docling (docling-project/docling#2820), but the root cause and fix lie within docling-core's serialization logic.

Current Behavior

• MarkdownTableSerializer (in docling_core) strictly exports tables as text/markdown grids.
• It ignores the image_mode parameter for tables.
• It ignores the presence of item.image (if populated).

Expected Behavior

If image_mode is set to REFERENCED or EMBEDDED, and the TableItem has an associated image (item.image), the serializer should:

1. Preferentially output the image (or include it alongside the text, depending on desired behavior, but likely behaving similar to PictureItem).
2. Generate the appropriate Markdown image syntax (e.g., ![Table](image_path.png) or embedded base64).

Proposed Solution

Modify MarkdownTableItemSerializer (or MarkdownTableSerializer) in docling_core/transforms/serializer/markdown.py.

The serialize method should be updated to check self.params.image_mode:

# Pseudo-code logic to be added to serialize()

if self.params.image_mode in [ImageRefMode.REFERENCED, ImageRefMode.EMBEDDED] and item.image:
    # Logic similar to MarkdownPictureSerializer
    image_link = self._get_image_link(item) # Helper to generate path/base64
    return getattr(self, "create_ser_result")(text=f"![{label}]({image_link})", span_source=item)

# Fallback to existing text table generation
...

[erikvullings]

It would be great if this could be implemented - besides the use cases already mentioned upstream, another one is for RAG pipelines. More complicated tables cannot faithfully be represented using markdown format, so it would be great to either have the option to use a VLM before ingestion to describe the table, of afterwards on retrieval to parse the table to extract the requested information.

[romitsenmi]

Hi, just wanted to know if there's any update on this? Thanks a lot!

[erikvullings]

@romitsenmi Until it is integrated into docling, I've taken a different approach: I've converted the whole page to an image and used the docling's JSON to get the bounding box. I've asked AI to extract the code I am using into a script. Hope that helps...

"""Standalone example: Extract tables from PDFs using Docling.

This snippet demonstrates how to use Docling to:
1. Convert PDF documents
2. Extract tables as images (cropped from page images)
3. Extract table structure as JSON (cell grid with row/column info)
4. Export tables as Markdown

Requirements:
    pip install docling docling-core Pillow

Usage:
    python extract_tables_with_docling.py your_document.pdf
"""

import io
import json
from pathlib import Path

from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import PdfPipelineOptions
from docling.document_converter import DocumentConverter as DoclingConverter
from docling.document_converter import PdfFormatOption
from docling_core.types.doc.base import CoordOrigin
from docling_core.types.doc.document import TableItem
from PIL import Image


def crop_bbox_from_page_image(
    page_image: Image.Image,
    bbox,
    page_width: float,
    page_height: float,
    padding: int = 5,
) -> Image.Image:
    """Crop a Docling bbox from a page image.

    Handles coordinate conversion from Docling's document coordinate space
    (PDF points, possibly bottom-left origin) to PIL pixel coordinates.

    Args:
        page_image: The full page image.
        bbox: Docling BoundingBox object with l, t, r, b and coord_origin.
        page_width: Page width in document coordinates (PDF points).
        page_height: Page height in document coordinates (PDF points).
        padding: Pixel padding around the crop area.

    Returns:
        Cropped image region.
    """
    img_width, img_height = page_image.size

    # Convert to top-left origin if bbox uses bottom-left (common for PDFs)
    if bbox.coord_origin == CoordOrigin.BOTTOMLEFT:
        bbox = bbox.to_top_left_origin(page_height)

    # Scale from document coordinates to image pixels
    scale_x = img_width / page_width
    scale_y = img_height / page_height

    left = max(0, int(bbox.l * scale_x) - padding)
    top = max(0, int(bbox.t * scale_y) - padding)
    right = min(img_width, int(bbox.r * scale_x) + padding)
    bottom = min(img_height, int(bbox.b * scale_y) + padding)

    return page_image.crop((left, top, right, bottom))


def extract_table_cells_json(table_data) -> dict | None:
    """Convert Docling TableData.table_cells to a cell-grid JSON structure.

    Args:
        table_data: Docling TableData object with table_cells, num_rows, num_cols.

    Returns:
        Dictionary with cell-grid structure, or None if data is unavailable.
    """
    if not table_data or not getattr(table_data, "table_cells", None):
        return None

    cells = []
    for cell in table_data.table_cells:
        cells.append(
            {
                "text": cell.text,
                "row": cell.start_row_offset_idx,
                "col": cell.start_col_offset_idx,
                "row_span": cell.row_span,
                "col_span": cell.col_span,
                "is_column_header": cell.column_header,
                "is_row_header": cell.row_header,
            }
        )

    return {
        "num_rows": table_data.num_rows,
        "num_cols": table_data.num_cols,
        "cells": cells,
    }


def extract_tables_from_pdf(
    pdf_path: Path, output_dir: Path | None = None
) -> list[dict]:
    """Extract all tables from a PDF document.

    Args:
        pdf_path: Path to the PDF file.
        output_dir: Optional directory to save table images and JSON.
                   If None, returns data without saving to disk.

    Returns:
        List of table data dictionaries containing:
        - table_id: Unique identifier
        - page: Page number (1-indexed)
        - image: PIL Image of the cropped table (or None if extraction failed)
        - markdown: Table as Markdown string
        - cells_json: Structured cell data as dict
        - num_rows: Number of rows
        - num_cols: Number of columns
    """
    # Configure PDF pipeline with table structure extraction
    pdf_options = PdfPipelineOptions()
    pdf_options.do_ocr = True  # Enable OCR for scanned PDFs
    pdf_options.do_table_structure = True  # Enable table structure detection
    pdf_options.images_scale = 2.0  # Higher resolution for better image quality
    pdf_options.generate_page_images = True  # Generate page images for cropping

    # Create converter
    converter = DoclingConverter(
        allowed_formats=[InputFormat.PDF],
        format_options={
            InputFormat.PDF: PdfFormatOption(pipeline_options=pdf_options),
        },
    )

    # Convert document
    result = converter.convert(str(pdf_path))

    # Create output directory if specified
    if output_dir:
        output_dir.mkdir(parents=True, exist_ok=True)

    tables = []
    page_image_cache: dict[int, Image.Image] = {}

    # Iterate through document elements
    for element, _level in result.document.iterate_items():
        if not isinstance(element, TableItem):
            continue

        if not element.prov:
            continue

        prov = element.prov[0]
        if not hasattr(prov, "page_no") or not prov.bbox:
            continue

        page_no = prov.page_no

        # Lazy load page image
        if page_no not in page_image_cache:
            page = result.document.pages.get(page_no)
            if page and page.image and page.image.pil_image:
                page_image_cache[page_no] = page.image.pil_image

        if page_no not in page_image_cache:
            continue

        # Get page dimensions for coordinate conversion
        page_obj = result.document.pages.get(page_no)
        if not page_obj or not page_obj.size:
            continue

        # Crop table image from page
        try:
            cropped_image = crop_bbox_from_page_image(
                page_image_cache[page_no],
                prov.bbox,
                page_obj.size.width,
                page_obj.size.height,
            )
        except Exception as e:
            print(f"Failed to crop table on page {page_no}: {e}")
            cropped_image = None

        # Extract table markdown
        table_markdown = None
        try:
            table_markdown = element.export_to_markdown(doc=result.document)
        except Exception:
            pass

        # Extract structured cell data
        cells_json = None
        num_rows = None
        num_cols = None
        if hasattr(element, "data") and element.data:
            num_rows = getattr(element.data, "num_rows", None)
            num_cols = getattr(element.data, "num_cols", None)
            cells_json = extract_table_cells_json(element.data)

        # Build table data
        table_id = str(element.self_ref) if element.self_ref else f"table_{len(tables)}"
        table_data = {
            "table_id": table_id,
            "page": page_no,
            "image": cropped_image,
            "markdown": table_markdown,
            "cells_json": cells_json,
            "num_rows": num_rows,
            "num_cols": num_cols,
        }

        tables.append(table_data)

        # Save to disk if output directory specified
        if output_dir and cropped_image:
            base_name = f"table_p{page_no}_{len(tables)}"

            # Save image
            image_path = output_dir / f"{base_name}.png"
            cropped_image.save(image_path, format="PNG")

            # Save JSON
            json_data = {
                "table_id": table_id,
                "page": page_no,
                "num_rows": num_rows,
                "num_cols": num_cols,
                "markdown": table_markdown,
                "cells": cells_json,
            }
            json_path = output_dir / f"{base_name}.json"
            json_path.write_text(json.dumps(json_data, indent=2, ensure_ascii=False))

            print(f"Saved: {image_path.name} and {json_path.name}")

    return tables


def main():
    """Example usage."""
    import sys

    if len(sys.argv) < 2:
        print("Usage: python extract_tables_with_docling.py <pdf_path> [output_dir]")
        print("\nExample:")
        print("  python extract_tables_with_docling.py document.pdf ./output/tables")
        sys.exit(1)

    pdf_path = Path(sys.argv[1])
    output_dir = Path(sys.argv[2]) if len(sys.argv) > 2 else None

    if not pdf_path.exists():
        print(f"Error: File not found: {pdf_path}")
        sys.exit(1)

    print(f"Extracting tables from: {pdf_path}")
    tables = extract_tables_from_pdf(pdf_path, output_dir)

    print(f"\nFound {len(tables)} table(s):")
    for i, table in enumerate(tables, 1):
        print(f"\n--- Table {i} (Page {table['page']}) ---")
        print(f"  Dimensions: {table['num_rows']} rows × {table['num_cols']} cols")
        if table["markdown"]:
            print(f"  Markdown preview:")
            for line in table["markdown"].split("\n")[:5]:
                print(f"    {line}")
            if table["markdown"].count("\n") > 5:
                print("    ...")


if __name__ == "__main__":
    main()

[romitsenmi]

@erikvullings Thanks a lot for sharing the code, I will try this out. My initial issue arose from having a rotated table in a pdf which Docling was unable to parse properly, and there's another issue on that which I'm tracking https://github.com/docling-project/docling/issues/1376 for that. I thought maybe converting the table to image would help out so I checked out this open issue. Anyways, really appreciate your help!