Support structured output with Pydantic models in `langchain-huggingface`

[bitsydarel]

The documentation indicates that it does, but the PydanticOutputParser is not being utilised.

langchain/libs/partners/huggingface/langchain_huggingface/chat_models/huggingface.py

Line 882 in 0f39155

Pydantic class is currently supported.

[bitsydarel]

        self.generation_model = generation_model.with_structured_output(
            include_raw=True,
            method=LLMConfig.OUTPUT_MODE,
            schema=QualityAssuranceReview,
        )

Since we do have a retry logic, it cause the parsing to retry the prompt which is burning tokens/credits.

[bitsydarel]

Adding support for Pydantic models is quite straightforward. Therefore, I am curious to know why Pydantic support was not implemented. Is it an issue with the Hugging Face inference APIs or something else?

class PydanticChatHuggingFace(ChatHuggingFace):
    def __init__(self, llm: HuggingFaceEndpoint, **kwargs):
        super().__init__(llm=llm, **kwargs)

    def with_structured_output(
            self,
            schema: Optional[Union[dict, type[BaseModel]]] = None,
            *,
            method: Literal[
                "function_calling", "json_mode", "json_schema"
            ] = "function_calling",
            include_raw: bool = False,
            **kwargs: Any,
    ) -> Runnable[LanguageModelInput, Union[dict, BaseModel]]:
        _ = kwargs.pop("strict", None)
        if kwargs:
            msg = f"Received unsupported arguments {kwargs}"
            raise ValueError(msg)
        is_pydantic_schema = isinstance(schema, type) and is_basemodel_subclass(schema)

        # We always set method type to json_schema.
        if is_pydantic_schema and method == "json_schema":
            if schema is None:
                raise ValueError(
                    "schema must be specified when method is not 'json_mode'. "
                    "Received None."
                )

            schema = cast(TypeBaseModel, schema)

            # We always use pydantic import not pydantic.v1
            response_format = schema.model_json_schema()

            llm = self.bind(
                response_format={"type": "json_object", "schema": response_format},
                ls_structured_output_format={
                    "kwargs": {"method": method},
                    "schema": schema,
                },
            )
            output_parser = PydanticOutputParser(pydantic_object=schema)

            if include_raw:
                parser_assign = RunnablePassthrough.assign(
                    parsed=itemgetter("raw") | output_parser, parsing_error=lambda _: None
                )
                parser_none = RunnablePassthrough.assign(parsed=lambda _: None)
                parser_with_fallback = parser_assign.with_fallbacks(
                    [parser_none], exception_key="parsing_error"
                )
                return RunnableMap(raw=llm) | parser_with_fallback
            else:
                return llm | output_parser
        else:
            return super().with_structured_output(schema=schema, method=method, include_raw=include_raw, **kwargs)

Our agentic RAG component is finally operational.

[rahulmansharamani14]

Hi @mdrxy , is someone working on it? If not, I'd love to investigate further and find the solution to this.

[rahulmansharamani14]

Hi @mdrxy,
I’ve opened a draft PR #33141 for adding Pydantic structured output support in ChatHuggingFace. Could you take a look and share your thoughts on the approach / next steps?

Thanks!