Skip to content

Commit 545af81

Browse files
authored
Update Evaluation SDK README (#37527)
* Update README * update readme * add sample link * updates to readme * Update README.md * Update README.md * add product documentation link
1 parent cefb6e7 commit 545af81

File tree

1 file changed

+73
-8
lines changed

1 file changed

+73
-8
lines changed

sdk/evaluation/azure-ai-evaluation/README.md

Lines changed: 73 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,26 @@
11
# Azure AI Evaluation client library for Python
22

3+
We are excited to introduce the public preview of the Azure AI Evaluation SDK.
4+
5+
[Source code][source_code]
6+
| [Package (PyPI)][evaluation_pypi]
7+
| [API reference documentation][evaluation_ref_docs]
8+
| [Product documentation][product_documentation]
9+
| [Samples][evaluation_samples]
10+
11+
This package has been tested with Python 3.8, 3.9, 3.10, 3.11, and 3.12.
12+
13+
For a more complete set of Azure libraries, see https://aka.ms/azsdk/python/all
14+
315
## Getting started
416

17+
### Prerequisites
18+
19+
- Python 3.8 or later is required to use this package.
20+
521
### Install the package
622

7-
Install the Azure AI Evaluation library for Python with:
23+
Install the Azure AI Evaluation library for Python with [pip][pip_link]::
824

925
```bash
1026
pip install azure-ai-evaluation
@@ -16,6 +32,8 @@ Evaluators are custom or prebuilt classes or functions that are designed to meas
1632

1733
## Examples
1834

35+
### Evaluators
36+
1937
Users can create evaluator runs on the local machine as shown in the example below:
2038

2139
```python
@@ -87,9 +105,13 @@ if __name__ == "__main__":
87105

88106
pprint(result)
89107
```
90-
## Simulator
108+
### Simulator
109+
110+
111+
Simulators allow users to generate synthentic data using their application. Simulator expects the user to have a callback method that invokes
112+
their AI application.
91113

92-
Sample application prompty
114+
#### Simulating with a Prompty
93115

94116
```yaml
95117
---
@@ -213,8 +235,7 @@ if __name__ == "__main__":
213235
print("done!")
214236
```
215237

216-
Simulators allow users to generate synthentic data using their application. Simulator expects the user to have a callback method that invokes
217-
their AI application. Here's a sample of a callback which invokes AsyncAzureOpenAI:
238+
#### Adversarial Simulator
218239

219240
```python
220241
from from azure.ai.evaluation.simulator import AdversarialSimulator, AdversarialScenario
@@ -282,9 +303,9 @@ async def callback(
282303
}
283304

284305
```
285-
## Adversarial Simulator
286306

287-
### Adversarial QA:
307+
#### Adversarial QA
308+
288309
```python
289310
scenario = AdversarialScenario.ADVERSARIAL_QA
290311
simulator = AdversarialSimulator(azure_ai_project=azure_ai_project, credential=DefaultAzureCredential())
@@ -300,7 +321,7 @@ outputs = asyncio.run(
300321

301322
print(outputs.to_eval_qa_json_lines())
302323
```
303-
### Direct Attack Simulator
324+
#### Direct Attack Simulator
304325

305326
```python
306327
scenario = AdversarialScenario.ADVERSARIAL_QA
@@ -319,6 +340,50 @@ print(outputs)
319340
```
320341
## Troubleshooting
321342

343+
### General
344+
345+
Azure ML clients raise exceptions defined in [Azure Core][azure_core_readme].
346+
347+
### Logging
348+
349+
This library uses the standard
350+
[logging][python_logging] library for logging.
351+
Basic information about HTTP sessions (URLs, headers, etc.) is logged at INFO
352+
level.
353+
354+
Detailed DEBUG level logging, including request/response bodies and unredacted
355+
headers, can be enabled on a client with the `logging_enable` argument.
356+
357+
See full SDK logging documentation with examples [here][sdk_logging_docs].
358+
322359
## Next steps
323360

361+
- View our [samples][evaluation_samples].
362+
- View our [documentation][product_documentation]
363+
324364
## Contributing
365+
366+
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit [cla.microsoft.com][cla].
367+
368+
When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
369+
370+
This project has adopted the [Microsoft Open Source Code of Conduct][code_of_conduct]. For more information see the [Code of Conduct FAQ][coc_faq] or contact [[email protected]][coc_contact] with any additional questions or comments.
371+
372+
<!-- LINKS -->
373+
374+
[source_code]: https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/evaluation/azure-ai-evaluation
375+
[evaluation_pypi]: https://pypi.org/project/azure-ai-evaluation/
376+
[evaluation_ref_docs]: https://learn.microsoft.com/python/api/azure-ai-evaluation/azure.ai.evaluation?view=azure-python-preview
377+
[evaluation_samples]: https://github.com/Azure-Samples/azureai-samples/tree/main/scenarios
378+
[product_documentation]: https://learn.microsoft.com/azure/ai-studio/how-to/develop/evaluate-sdk
379+
[python_logging]: https://docs.python.org/3/library/logging.html
380+
[sdk_logging_docs]: https://docs.microsoft.com/azure/developer/python/azure-sdk-logging
381+
[azure_core_readme]: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/core/azure-core/README.md
382+
[pip_link]: https://pypi.org/project/pip/
383+
[azure_core_ref_docs]: https://aka.ms/azsdk-python-core-policies
384+
[azure_core]: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/core/azure-core/README.md
385+
[azure_identity]: https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/identity/azure-identity
386+
[cla]: https://cla.microsoft.com
387+
[code_of_conduct]: https://opensource.microsoft.com/codeofconduct/
388+
[coc_faq]: https://opensource.microsoft.com/codeofconduct/faq/
389+
[coc_contact]: mailto:[email protected]

0 commit comments

Comments
 (0)