Bing WebSearchEnginePlugin for AzureAssistantAgent

[CristinaStn]

We are using ### AzureAssistantAgent to provide specialised agents and we need to use the Web Search Engine Plugin with BingConnector as proposed https://github.com/microsoft/semantic-kernel/blob/main/python/semantic_kernel/core_plugins/web_search_engine_plugin.py.

However, I see the Bing API is transitioning to Grounding with Bing Search. We are able to create such resource and get the api key.
Adding the WebSearchEnginePlugin as plugin to our agent and asking the agent with something that requires web search, we get: "statusCode": 401, "message": "Unauthorized. Invalid or expired token."

More details:
packages\semantic_kernel\connectors\search_engine\bing_connector.py", line 92, in search
raise ServiceInvalidRequestError("Failed to get search results.") from ex
semantic_kernel.exceptions.service_exceptions.ServiceInvalidRequestError: Failed to get search results.

Q: is it because of using AzureAssistantAgent and not AzureAIAgents? if yes, what other web search plugins do you recommend?

[arafattehsin]

@CristinaStn Unfortunately, Bing Search and Bing Custom Search APIs will be retired on 11th August 2025.

Have you tried to look at https://platform.openai.com/docs/guides/tools-web-search?api-mode=chat?

[CristinaStn]

Hi
Thanks for suggestion!
We took into consideration Azure services only, but I will take a look on OpenAI web search tool to see the implications.

[moonbox3]

Hi @CristinaStn, as @arafattehsin pointed out, there is a move to retire those Bing APIs. AzureAI Agents support the Bing Grounding tool. Their API is built on top of the Assistants v2 API, so although it's a bit different, you'll notice similarities while working it compared to the Assistant agent. Using the AzureAI Agent does require one to set up a project in Foundry. You can review the getting started items here. We do have some Bing Ground tool samples, like this one or one that uses streaming responses.

I know this doesn't help in the near term because it's not available yet, but keep an eye out for:

• Azure OpenAI Responses API to support the web search tool. When it does, you can leverage our AzureResponsesAgent.
• Azure OpenAI's "search" model like the one OpenAI provides (gpt-4o-search-preview). I don't see it listed in the available models yet.

[CristinaStn]

Thank you very much.
Our entire AI application is based on AzureAssistantAgent.
Q: Do you know if this type of agents will be able to use the Bing Grounding tool soon?

Our infrastructure is based solely on Azure services, and for this reason, we would prefer to remain exclusively on Azure.

[moonbox3]

I am not aware of the bing grounding tool coming to the Azure Assistant agent, but I can ask. From an API surface, this is what a change to the AzureAIAgent would look like:

Currently, you have something like:

from semantic_kernel.agents import AssistantAgentThread, AzureAssistantAgent
from semantic_kernel.connectors.ai.open_ai import AzureOpenAISettings

# 1. Create the client using Azure OpenAI resources and configuration
client = AzureAssistantAgent.create_client()

# 2. Configure the code interpreter tool and resources for the Assistant
code_interpreter_tool, code_interpreter_tool_resources = AzureAssistantAgent.configure_code_interpreter_tool()

# 3. Create the assistant on the Azure OpenAI service
definition = await client.beta.assistants.create(
    model=AzureOpenAISettings().chat_deployment_name,
    name="CodeRunner",
    instructions="Run the provided request as code and return the result.",
    tools=code_interpreter_tool,
    tool_resources=code_interpreter_tool_resources,
)

# 4. Create a Semantic Kernel agent for the Azure OpenAI assistant
agent = AzureAssistantAgent(
    client=client,
    definition=definition,
)

# 5. Create a new thread for use with the assistant
# If no thread is provided, a new thread will be
# created and returned with the initial response
thread: AssistantAgentThread = None

An AzureAIAgent would look like:

from azure.ai.agents.models import CodeInterpreterTool
from azure.identity.aio import DefaultAzureCredential

from semantic_kernel.agents import AzureAIAgent, AzureAIAgentSettings, AzureAIAgentThread

async with (
    # Note: AzureAIAgents strictly use Entra auth/RBAC - no API key access
    DefaultAzureCredential() as creds,
    AzureAIAgent.create_client(credential=creds) as client,
):
    # 1. Create a code interpreter tool
    code_interpreter = CodeInterpreterTool()
    agent_definition = await client.agents.create_agent(
        model=AzureAIAgentSettings().model_deployment_name,
        tools=code_interpreter.definitions,
        tool_resources=code_interpreter.resources,
    )

    # 2. Create a Semantic Kernel agent for the Azure AI agent
    agent = AzureAIAgent(
        client=client,
        definition=agent_definition,
    )

    # 3. Create a thread for the agent
    # If no thread is provided, a new thread will be
    # created and returned with the initial response
    thread: AzureAIAgentThread | None = None

Both agents are invoked the same way:

for user_input in user_inputs:
    print(f"# User: '{user_input}'")
    first_chunk = True
    async for response in agent.invoke_stream(
        messages=user_input,
        thread=thread,
    ):
        if first_chunk:
            print(f"# {response.role}: ", end="", flush=True)
            first_chunk = False
        print(response.content, end="", flush=True)
        thread = response.thread
    print()

All of the underlying content types that are returned are the same, ChatMessageContent, StreamingChatMessageContent with types like FunctionCallContent, or FunctionResultContent.

The good news is that SK handles the inner-workings for you - you are solely responsible for the top-most API layer where you define your agent, invoke it, and handle any types returned. You'd enable bing grounding on the AzureAIAgent, as I liked to above:

# Enter your Bing Grounding Connection Name
bing_connection = await client.connections.get(name="<your-bing-grounding-connection-name>")
conn_id = bing_connection.id

# Initialize agent bing tool and add the connection id
bing_grounding = BingGroundingTool(connection_id=conn_id)

agent_definition = await client.agents.create_agent(
    name="BingGroundingAgent",
    instructions="Use the Bing grounding tool to answer the user's question.",
    model=AzureAIAgentSettings().model_deployment_name,
    tools=bing_grounding.definitions,
)

[CristinaStn]

Thank you. It seems that the transition is very straightforward. Our team will review this and will likely plan support for this new type of agents.