Skip to content
Open
Show file tree
Hide file tree
Changes from 96 commits
Commits
Show all changes
97 commits
Select commit Hold shift + click to select a range
4318329
Prepare evals SDK Release
May 28, 2025
192b980
Fix bug
May 28, 2025
758adb4
Fix for ADV_CONV for FDP projects
May 29, 2025
de09fd1
Update release date
May 29, 2025
ef60fe6
Merge branch 'main' into main
nagkumar91 May 29, 2025
8ca51d0
Merge branch 'Azure:main' into main
nagkumar91 May 30, 2025
98bfc3a
Merge branch 'Azure:main' into main
nagkumar91 Jun 2, 2025
a5f32e8
Merge branch 'Azure:main' into main
nagkumar91 Jun 9, 2025
5fd88b6
Merge branch 'Azure:main' into main
nagkumar91 Jun 10, 2025
51f2b44
Merge branch 'Azure:main' into main
nagkumar91 Jun 10, 2025
a5be8b5
Merge branch 'Azure:main' into main
nagkumar91 Jun 16, 2025
75965b7
Merge branch 'Azure:main' into main
nagkumar91 Jun 25, 2025
d0c5e53
Merge branch 'Azure:main' into main
nagkumar91 Jun 25, 2025
b790276
Merge branch 'Azure:main' into main
nagkumar91 Jun 26, 2025
d5ca243
Merge branch 'Azure:main' into main
nagkumar91 Jun 26, 2025
8d62e36
re-add pyrit to matrix
Jun 26, 2025
59a70f2
Change grader ids
Jun 26, 2025
4d146d7
Merge branch 'Azure:main' into main
nagkumar91 Jun 26, 2025
f7a4c83
Update unit test
Jun 27, 2025
79e3a40
replace all old grader IDs in tests
Jun 27, 2025
588cbec
Merge branch 'main' into main
nagkumar91 Jun 30, 2025
7514472
Update platform-matrix.json
nagkumar91 Jun 30, 2025
28b2513
Update test to ensure everything is mocked
Jul 1, 2025
8603e0e
tox/black fixes
Jul 1, 2025
895f226
Skip that test with issues
Jul 1, 2025
b4b2daf
Merge branch 'Azure:main' into main
nagkumar91 Jul 1, 2025
023f07f
update grader ID according to API View feedback
Jul 1, 2025
45b5f5d
Update test
Jul 2, 2025
1ccb4db
remove string check for grader ID
Jul 2, 2025
6fd9aa5
Merge branch 'Azure:main' into main
nagkumar91 Jul 2, 2025
f871855
Update changelog and officialy start freeze
Jul 2, 2025
59ac230
update the enum according to suggestions
Jul 2, 2025
794a2c4
update the changelog
Jul 2, 2025
b33363c
Finalize logic
Jul 2, 2025
464e2dd
Merge branch 'Azure:main' into main
nagkumar91 Jul 3, 2025
4585b14
Merge branch 'Azure:main' into main
nagkumar91 Jul 7, 2025
89c2988
Initial plan
Copilot Jul 7, 2025
6805018
Fix client request ID headers in azure-ai-evaluation
Copilot Jul 7, 2025
aad48df
Fix client request ID header format in rai_service.py
Copilot Jul 7, 2025
db75552
Merge pull request #5 from nagkumar91/copilot/fix-4
nagkumar91 Jul 10, 2025
b8eebf3
Merge branch 'Azure:main' into main
nagkumar91 Jul 10, 2025
2899ad4
Merge branch 'Azure:main' into main
nagkumar91 Jul 10, 2025
c431563
Merge branch 'Azure:main' into main
nagkumar91 Jul 17, 2025
79ed63c
Merge branch 'Azure:main' into main
nagkumar91 Jul 18, 2025
a3be3fc
Merge branch 'Azure:main' into main
nagkumar91 Jul 21, 2025
056ac4d
Passing threshold in AzureOpenAIScoreModelGrader
Jul 21, 2025
1779059
Add changelog
Jul 21, 2025
43fecff
Adding the self.pass_threshold instead of pass_threshold
Jul 21, 2025
b0c102b
Merge branch 'Azure:main' into main
nagkumar91 Jul 22, 2025
7bf5f1f
Add the python grader
Jul 22, 2025
3248ad0
Remove redundant test
Jul 22, 2025
d76f59b
Add class to exception list and format code
Jul 23, 2025
4d60e43
Merge branch 'main' into feature/python_grader
nagkumar91 Jul 24, 2025
98d1626
Merge branch 'Azure:main' into main
nagkumar91 Jul 24, 2025
9248c38
Add properties to evaluation upload run for FDP
Jul 24, 2025
74b760f
Remove debug
Jul 24, 2025
23dbc85
Merge branch 'feature/python_grader'
Jul 24, 2025
467ccb6
Remove the redundant property
Jul 24, 2025
c2beee8
Merge branch 'Azure:main' into main
nagkumar91 Jul 24, 2025
be9a19a
Fix changelog
Jul 24, 2025
de3a1e1
Fix the multiple features added section
Jul 24, 2025
f9faa61
removed the properties in update
Jul 24, 2025
69e783a
Merge branch 'Azure:main' into main
nagkumar91 Jul 28, 2025
8ebea2a
Merge branch 'Azure:main' into main
nagkumar91 Jul 31, 2025
3f9c818
Merge branch 'Azure:main' into main
nagkumar91 Aug 1, 2025
3b3159c
Merge branch 'Azure:main' into main
nagkumar91 Aug 5, 2025
d78b834
Merge branch 'Azure:main' into main
nagkumar91 Aug 6, 2025
ae3fc52
Merge branch 'Azure:main' into main
nagkumar91 Aug 8, 2025
706c042
Merge branch 'Azure:main' into main
nagkumar91 Aug 11, 2025
f91ee63
Merge branch 'Azure:main' into main
nagkumar91 Aug 12, 2025
7b3b889
Merge branch 'Azure:main' into main
nagkumar91 Aug 20, 2025
21b71cf
Merge branch 'Azure:main' into main
nagkumar91 Sep 9, 2025
823177a
Merge branch 'Azure:main' into main
nagkumar91 Sep 12, 2025
48f630e
Merge branch 'Azure:main' into main
nagkumar91 Sep 15, 2025
33cf2a2
Merge branch 'Azure:main' into main
nagkumar91 Sep 18, 2025
9ab6f9d
Merge remote-tracking branch 'refs/remotes/origin/main'
Sep 19, 2025
fea99ed
Merge branch 'Azure:main' into main
nagkumar91 Sep 22, 2025
d2fbd9f
Merge branch 'Azure:main' into main
nagkumar91 Sep 23, 2025
9015498
Merge branch 'Azure:main' into main
nagkumar91 Sep 24, 2025
6d6343c
Merge branch 'Azure:main' into main
nagkumar91 Oct 1, 2025
fd6bb5b
Merge branch 'Azure:main' into main
nagkumar91 Oct 1, 2025
4b6df68
Merge branch 'Azure:main' into main
nagkumar91 Oct 1, 2025
97074de
Release 1.12.0
nagkumar91 Oct 1, 2025
1d62311
Merge branch 'Azure:main' into main
nagkumar91 Oct 2, 2025
2fc4b30
Merge branch 'Azure:main' into main
nagkumar91 Oct 6, 2025
a995a04
Merge branch 'Azure:main' into main
nagkumar91 Oct 10, 2025
3b8eb36
Merge branch 'Azure:main' into main
nagkumar91 Oct 13, 2025
fc47836
Merge branch 'Azure:main' into main
nagkumar91 Oct 20, 2025
33878fc
Merge branch 'Azure:main' into main
nagkumar91 Oct 27, 2025
d5f759b
Merge branch 'Azure:main' into main
nagkumar91 Oct 28, 2025
1b8e95a
Merge branch 'Azure:main' into main
nagkumar91 Nov 1, 2025
831c664
Merge branch 'Azure:main' into main
nagkumar91 Nov 4, 2025
fd463f3
Initial plan
Copilot Nov 4, 2025
8872c92
Set default value for image_tag parameter to "2025-05-08"
Copilot Nov 4, 2025
b9cdf34
Update docstring to reflect default value for image_tag
Copilot Nov 4, 2025
f9ed449
Merge pull request #6 from nagkumar91/copilot/set-default-value-image…
nagkumar91 Nov 4, 2025
18b37da
Merge branch 'Azure:main' into main
nagkumar91 Nov 6, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,8 @@ class AzureOpenAIPythonGrader(AzureOpenAIGrader):
~azure.ai.evaluation.OpenAIModelConfiguration]
:param name: The name of the grader.
:type name: str
:param image_tag: The image tag for the Python execution environment.
:type image_tag: str
:param image_tag: The image tag for the Python execution environment. Defaults to "2025-05-08".
:type image_tag: Optional[str]
:param pass_threshold: Score threshold for pass/fail classification. Scores >= threshold are considered passing.
:type pass_threshold: float
:param source: Python source code containing the grade function.
Expand Down Expand Up @@ -63,7 +63,7 @@ def __init__(
name: str,
pass_threshold: float,
source: str,
image_tag: Optional[str] = None,
image_tag: Optional[str] = "2025-05-08",
credential: Optional[TokenCredential] = None,
**kwargs: Any,
):
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,3 +52,28 @@ def test_invalid_pass_threshold(self):
pass_threshold=1.5,
source=source_code,
)

def test_default_image_tag(self):
"""Test that image_tag has a default value of '2025-05-08'."""
model_config = AzureOpenAIModelConfiguration(
azure_endpoint="https://test.openai.azure.com",
api_key="test-key",
azure_deployment="test-deployment",
)

source_code = """
def grade(sample: dict, item: dict) -> float:
return 1.0
"""

# Create grader without specifying image_tag
grader = AzureOpenAIPythonGrader(
model_config=model_config,
name="python_test",
pass_threshold=0.5,
source=source_code,
)

# Verify the grader was created successfully
assert grader.pass_threshold == 0.5
assert grader.id == "azureai://built-in/evaluators/azure-openai/python_grader"
Loading