Agent Monitoring Continuous Evaluation for Custom Agents Failing with error "No trace data found"

[hante-sonova]

• Package Name: Agent Monitoring Continuous Evaluation
• Package Version:
• Operating System:
• Python Version:

Describe the bug
We have setup continuous monitoring via the UI for our custom agent, however evaluation run fails with this error even though traces exist:

There was an error with this evaluation run.No trace data found where customDimensions.gen_ai.input.messages or customDimensions.gen_ai.output.messages contain data. Please check the agent running status for agent 'ifd-mono' during time range: from 2026-07-02T05:50:08+00:00 to 2026-07-02T06:50:08+00:00

To Reproduce
Set up monitoring for your custom agents

1. Onboard your custom agent to Foundry using the instructions in Register and manage custom agents.
2. Instrument your agent to comply with the semantic conventions for generative AI solutions in the OpenTelemetry standard.
3. Configure your agent to send telemetry to the same Application Insights instance as your Foundry project to enable continuous evaluation features.
4. In Foundry Control Plane, go to the Asset page and select your agent.
5. Select the Monitor tab to view your metrics and charts.
6. Set up continuous evaluations using the methods outlined in Configure settings.

Note: We only implemented the configure settings via the UI and didn't create the python implementation as we understood it achieves the same thing?

Expected behavior
A clear and concise description of what you expected to happen.
Evalution result should appear in the Evaluation page successfully

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
See user_log attached

user_logs (12).txt

[github-actions]

🎯 Agentic Issue Triage — Needs Manual Review

🏷️ Label Decision

• Candidate labels considered: Client + AI Projects — The issue describes a failure in Azure Foundry's continuous evaluation feature for custom agents, referencing the agent monitoring functionality associated with the azure-ai-projects package (/sdk/ai/azure-ai-projects/). Category is Client (data-plane) since this is not a management-plane azure-mgmt-* package.
• Confidence blockers: None — prediction met all criteria.
• Outcome: Applied Client + AI Projects — prediction was confident.

👥 Owner Routing

• CODEOWNERS scan: Scanned bottom-to-top; the first # ServiceLabel: entry whose service label matched "AI Projects" was found at line 51.
• Matched entry: Line 51 — # ServiceLabel: %AI Projects %Service Attention (the %Service Attention token is ignored per matching rules; "AI Projects" matches the predicted service label).
• Owners found: No # AzureSdkOwners: or # ServiceOwners: comment lines are present in this block. The /sdk/ai/azure-ai-projects/ code-path line lists code reviewers only, not issue owners.
• Outcome: Applied needs-team-triage — the matched CODEOWNERS entry has no AzureSdkOwners or ServiceOwners listed; manual assignment is required.

Generated by Agentic Triage for issue #47812 · 40.8 AIC · ⌖ 9.52 AIC · ⊞ 9.5K · ◷

[kashifkhan]

Thank you for the feedback @hante-sonova . Im routing this to the right team that can help you further

[github-actions]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @bobogogo1990 @dargilco @glharper @howieleung @kingernupur @nick863 @trangevi @trrwilson.