Skip to content

Better BuiltinToolReturnPart content for web search #3044

@sfrey1

Description

@sfrey1

Description

Currently a lot of meaningful content is stripped out of web search tool results for gemini. Our use case relies on the underlying citation information returned in a web search. This could be accomplished with multiple calls but imo a better pattern would be to use the native provider web search result into BuiltinToolReturnPart.content

Example Google

# provider result from generate content api
GenerateContentResponse.candidates[].content.grounding_metadata(
    grounding_chunks=[
        web=GroundingChunkWeb(
            title="example.com",
            uri="https://{vertex_ai_grounding_redirect}",
        ),
    ],
    grounding_supports=[
        GroundingSupport(
            grounding_chunk_indices=[0],
            segment=Segment(
                start_index=10,
                end_index=50,
                text="yada yada yada {the search query} yada yada yada",
            )
        )
    ],
)

# pydantic ai result
AgentRunResult.all_messages()[].parts[
    BuiltinToolReturnPart(
        content=[
            {
                "domain": None,
                "title": "example.com",
                "uri": "https://{vertex_ai_grounding_redirect}",
                # dumping the segments and merging them into the content seems like a quick win
            },
        ],
    ),
]

References

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions