Merge pull request #5241 from lgayhardt/githubaction0525update

Github action eval project update

Author: prmerger-automator[bot]

articles/ai-foundry/how-to/evaluation-github-action.md

@@ -5,7 +5,7 @@ description: How to run evaluation in GitHub Action to streamline the evaluation
 manager: scottpolly
 ms.service: azure-ai-foundry
 ms.topic: how-to
-ms.date: 05/19/2025
+ms.date: 06/1/2025
 ms.reviewer: hanch
 ms.author: lagayhar
 author: lgayhardt
@@ -27,7 +27,7 @@ Offline evaluation involves testing AI models and agents using test datasets to
 
 ## Prerequisites
 
-[!INCLUDE [hub-only-prereq](../includes/hub-only-prereq.md)]
+Foundry project or Hubs based project. To learn more, see [Create a project](create-projects.md).
 
 Two GitHub Actions are available for evaluating AI applications: **ai-agent-evals** and **genai-evals**.
 
@@ -45,8 +45,16 @@ The input of ai-agent-evals includes:
 
 **Required:**
 
-- `azure-aiproject-connection-string`: The connection string for the Azure AI project. This is used to connect to Azure OpenAI to simulate conversations with each agent, and to connect to the Azure AI evaluation SDK to perform the evaluation.
-- `deployment-name`: the deployed model name.
+# [Foundry project](#tab/foundry-project)
+
+- `azure-ai-project-endpoint`: The endpoint of the Azure AI project. This is used to connect to your AI project to simulate conversations with each agent, and to connect to the Azure AI evaluation SDK to perform the evaluation.
+
+# [Hub based project](#tab/hub-project)
+
+- `azure-aiproject-connection-string`: The connection string of the Azure AI project. This is used to connect to your AI project to simulate conversations with each agent, and to connect to the Azure AI evaluation SDK to perform the evaluation.
+
+---
+- `deployment-name`: the deployed model name for evaluation judgement.
 - `data-path`: Path to the input data file containing the conversation starters. Each conversation starter is sent to each agent for a pairwise comparison of evaluation results.
   - `evaluators`: built-in evaluator names.
   - `data`: a set of conversation starters/queries.
@@ -55,6 +63,7 @@ The input of ai-agent-evals includes:
   - When only one `agent-id` is specified, the evaluation results include the absolute values for each metric along with the corresponding confidence intervals.
   - When multiple `agent-ids` are specified, the results include absolute values for each agent and a statistical comparison against the designated baseline agent ID.
 
+
 **Optional:**
 
 - `api-version`: the API version of deployed model.
@@ -92,6 +101,48 @@ To use the GitHub Action, add the GitHub Action to your CI/CD workflows and spec
 
 This example illustrates how Azure Agent AI Evaluation can be run when comparing different agents with agent IDs.
 
+# [Foundry project](#tab/foundry-project)
+
+```YAML
+name: "AI Agent Evaluation"
+
+on:
+  workflow_dispatch:
+  push:
+    branches:
+      - main
+
+permissions:
+  id-token: write
+  contents: read
+
+jobs:
+  run-action:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Azure login using Federated Credentials
+        uses: azure/login@v2
+        with:
+          client-id: ${{ vars.AZURE_CLIENT_ID }}
+          tenant-id: ${{ vars.AZURE_TENANT_ID }}
+          subscription-id: ${{ vars.AZURE_SUBSCRIPTION_ID }}
+
+      - name: Run Evaluation
+        uses: microsoft/ai-agent-evals@v2-beta
+        with:
+          # Replace placeholders with values for your Azure AI Project
+          azure-ai-project-endpoint: "<your-ai-project-endpoint>"
+          deployment-name: "<your-deployment-name>"
+          agent-ids: "<your-ai-agent-ids>"
+          data-path: ${{ github.workspace }}/path/to/your/data-file
+
+```
+
+# [Hub based project](#tab/hub-project)
+
 ```YAML
 name: "AI Agent Evaluation"
 
@@ -130,6 +181,8 @@ jobs:
 
 ```
 
+---
+
 ### AI agent evaluations output
 
 Evaluation results are outputted to the summary section for each AI evaluation GitHub Action run under Actions in GitHub.com.