在图片描述中使用api方法（PictureDescriptionApiOptions），拿不到返回结果

[Dithob]

Question

在图片描述中使用api方法，PictureDescriptionApiOptions，拿不到返回结果，模型api的token是消耗了的

调用脚本：

from pathlib import Path

from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import PdfPipelineOptions
from docling.document_converter import DocumentConverter, PdfFormatOption, ImageFormatOption
from docling.datamodel.pipeline_options import PictureDescriptionVlmOptions, granite_picture_description, PictureDescriptionApiOptions
from docling_core.types.doc.base import ImageRefMode
from docling_core.types.doc.document import PictureDescriptionData
from IPython import display

DOC_SOURCE = 图片
pipeline_options = PdfPipelineOptions(artifacts_path="/a/domains/docling/docling_models")

pipeline_options.do_picture_description = True

""" 本地加载vlm模型 """
# temp_picture_description = PictureDescriptionVlmOptions(
#     repo_id="Qwen/Qwen3-VL-2B-Instruct",
#     prompt="详细描述一下这张图片",
#     generation_config = dict(max_new_tokens=500, do_sample=False)
# )
#
# pipeline_options.picture_description_options = (
#     temp_picture_description  # <-- the model choice
# )

""" 使用vlm模型api """
pipeline_options.enable_remote_services=True  # 运行远程服务
pipeline_options.picture_description_options = PictureDescriptionApiOptions(
    url="https://ark.cn-beijing.volces.com/api/v3/chat/completions",
    headers={
        "Content-Type": "application/json",
        "Authorization": "Bearer xxx"
    },
    params=dict(
        model="doubao-seed-1-6-flash-250828",
        seed=42,
        max_completion_tokens=500,
    ),
    prompt="详细描述一下这张图片.",
    timeout=60,
)

pipeline_options.images_scale = 2.0
pipeline_options.generate_picture_images = True

converter = DocumentConverter(
    format_options={
        InputFormat.PDF: PdfFormatOption(
            pipeline_options=pipeline_options,
        ),
        InputFormat.IMAGE: ImageFormatOption(
            pipeline_options=pipeline_options
        )
    }
)
extract_result = converter.convert(DOC_SOURCE)
markdown_output = extract_result.document.export_to_markdown()
print("--- markdown result ---")
print(markdown_output)

annotation = extract_result.document.pictures[0].annotations
print("--- annotation result ---")
print(annotation)

output_dir = Path("save_files")
output_dir.mkdir(parents=True, exist_ok=True)
doc_filename = extract_result.input.file.stem
html_filename = output_dir / f"{doc_filename}-with-images.md"
extract_result.document.save_as_markdown(html_filename, image_mode=ImageRefMode.REFERENCED)

输出结果：

--- markdown result ---
员工发起用户来电申信取消，员工审核为盛假，录入异常系统但

不扣工程师推荐分并也不向工程师发送虚假取消消息

<!-- image -->
/a/domains/docling/docling_master/picture_description.py:62: DeprecationWarning: Field `annotations` is deprecated; use `meta` instead.
  annotation = extract_result.document.pictures[0].annotations
--- annotation result ---
[DescriptionAnnotation(kind='description', text='', provenance='not-implemented')]

[dosubot]

Hi @Dithob! I'm Dosu and I’m helping the docling team.

你遇到的问题很常见，原因通常是 API 返回的结果格式和 Docling 预期的不一致。Docling 的 PictureDescriptionApiOptions 默认要求 API 返回的数据结构和 OpenAI 的 chat/completions 接口一致，也就是 response 里要有 choices[0].message.content 字段，里面是图片描述文本。如果你的 API 返回格式不同（比如字段名不一样、嵌套层级不同等），Docling 会静默失败，annotation 结果就是空的，但 token 还是会被消耗（API 实际被调用了）参考。

建议排查和解决方法：

1. 确认 API 返回格式：用 Postman 或 curl 单独请求一次，确认返回内容里有 choices[0].message.content 字段。如果没有，需要让服务端适配，或者你在 Docling 里加一个适配层。
2. 开启 debug 日志：Docling 会把 API 错误和解析异常写到日志里，开启 debug 可以看到原始响应和错误信息，方便定位问题参考。
3. 自定义适配：如果 API 返回格式无法调整，可以 monkey-patch api_image_request，或者继承 PictureDescriptionApiModel，自己解析 API 响应，然后注册到 Docling 的 factory 系统参考。
4. 检查图片尺寸阈值：默认只有大于页面 5% 的图片才会被描述，太小的图片会被跳过。可以设置 picture_area_threshold=0 让所有图片都被处理参考。
5. markdown 没有 annotation：如果 PDF 里有图片 caption，markdown 导出时会优先用 caption，annotation 可能不会显示。可以 patch markdown serializer 或导出时手动合并 annotation参考。

如果你需要自定义适配代码，可以参考下面的 monkey-patch 示例：

import docling.utils.api_image_request as api_mod

def my_api_image_request(*args, **kwargs):
    # 调用你的 API，解析自定义字段
    # 返回 (描述文本, token 数, stop reason)
    ...
api_mod.api_image_request = my_api_image_request

或者继承模型并注册：

from docling.models.picture_description_api_model import PictureDescriptionApiModel
from docling.models.factories import get_picture_description_factory
from docling.datamodel.pipeline_options import PictureDescriptionApiOptions

class MyPictureDescriptionApiModel(PictureDescriptionApiModel):
    @classmethod
    def get_options_type(cls):
        return PictureDescriptionApiOptions

    def _annotate_images(self, images):
        # 用你自己的 HTTP 客户端和解析逻辑
        ...
        return ["你的描述"]

factory = get_picture_description_factory()
factory.register(MyPictureDescriptionApiModel, plugin_name="my_plugin", plugin_module_name=__name__)

核心就是让 API 返回的描述文本能被 Docling 正确提取出来。如果还有问题，可以贴一下 API 的原始响应内容。

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[Dithob]

Question

在图片描述中使用api方法，PictureDescriptionApiOptions，拿不到返回结果，模型api的token是消耗了的

调用脚本：

from pathlib import Path

from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import PdfPipelineOptions
from docling.document_converter import DocumentConverter, PdfFormatOption, ImageFormatOption
from docling.datamodel.pipeline_options import PictureDescriptionVlmOptions, granite_picture_description, PictureDescriptionApiOptions
from docling_core.types.doc.base import ImageRefMode
from docling_core.types.doc.document import PictureDescriptionData
from IPython import display

DOC_SOURCE = 图片
pipeline_options = PdfPipelineOptions(artifacts_path="/a/domains/docling/docling_models")

pipeline_options.do_picture_description = True

""" 使用vlm模型api """
pipeline_options.enable_remote_services=True # 运行远程服务
pipeline_options.picture_description_options = PictureDescriptionApiOptions(
url="https://ark.cn-beijing.volces.com/api/v3/chat/completions",
headers={
"Content-Type": "application/json",
"Authorization": "Bearer xxx"
},
params=dict(
model="doubao-seed-1-6-flash-250828",
seed=42,
max_completion_tokens=500,
),
prompt="详细描述一下这张图片.",
timeout=60,
)

pipeline_options.images_scale = 2.0
pipeline_options.generate_picture_images = True

converter = DocumentConverter(
format_options={
InputFormat.PDF: PdfFormatOption(
pipeline_options=pipeline_options,
),
InputFormat.IMAGE: ImageFormatOption(
pipeline_options=pipeline_options
)
}
)
extract_result = converter.convert(DOC_SOURCE)
markdown_output = extract_result.document.export_to_markdown()
print("--- markdown result ---")
print(markdown_output)

annotation = extract_result.document.pictures[0].annotations
print("--- annotation result ---")
print(annotation)

output_dir = Path("save_files")
output_dir.mkdir(parents=True, exist_ok=True)
doc_filename = extract_result.input.file.stem
html_filename = output_dir / f"{doc_filename}-with-images.md"
extract_result.document.save_as_markdown(html_filename, image_mode=ImageRefMode.REFERENCED)
输出结果：

--- markdown result ---
员工发起用户来电申信取消，员工审核为盛假，录入异常系统但

不扣工程师推荐分并也不向工程师发送虚假取消消息

<!-- image -->
/a/domains/docling/docling_master/picture_description.py:62: DeprecationWarning: Field `annotations` is deprecated; use `meta` instead.
  annotation = extract_result.document.pictures[0].annotations
--- annotation result ---
[DescriptionAnnotation(kind='description', text='', provenance='not-implemented')]

把思考关了好像就行了》

pipeline_options.picture_description_options = PictureDescriptionApiOptions(
    url="https://ark.cn-beijing.volces.com/api/v3/chat/completions",
    headers={
        "Content-Type": "application/json",
        "Authorization": "Bearer xxx"
    },
    params=dict(
        model="doubao-seed-1-6-flash-250828",
        max_completion_tokens=500,
        thinking={"type":"disabled"},
    ),
    prompt="详细描述一下这张图片.",
    timeout=60,
)