You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/ai-studio/how-to/llmops-github-promptflow.md
+12-12Lines changed: 12 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,7 +1,7 @@
1
1
---
2
2
title: LLMOps with prompt flow and GitHub in Azure AI Studio
3
3
titleSuffix: Azure AI Studio
4
-
description: Learn how to set up a LLMOps environment and pipeline on Github for prompt flow project using Azure AI Studio.
4
+
description: Learn how to set up a LLMOps environment and workflows in Github for prompt flow project using Azure AI Studio.
5
5
services: azure-ai-studio
6
6
author: ritesh-modi
7
7
ms.author: rimod
@@ -46,7 +46,7 @@ LLMOps with prompt flow is a "LLMOps template and guidance" to help you build LL
46
46
-**Centralized Code Hosting**: This repo supports hosting code for multiple flows based on prompt flow, providing a single repository for all your flows. Think of this platform as a single repository where all your prompt flow code resides. It's like a library for your flows, making it easy to find, access, and collaborate on different projects.
47
47
48
48
-**Lifecycle Management**: Each flow enjoys its own lifecycle, allowing for smooth transitions from local experimentation to production deployment.
49
-
:::image type="content" source="../media/prompt-flow/llmops/pipeline.png" alt-text="Screenshot of pipeline." lightbox = "../media/prompt-flow/llmops/pipeline.png":::
49
+
:::image type="content" source="../media/prompt-flow/llmops/workflow.png" alt-text="Screenshot of workflow." lightbox = "../media/prompt-flow/llmops/workflow.png":::
50
50
51
51
-**Variant and Hyperparameter Experimentation**: Experiment with multiple variants and hyperparameters, evaluating flow variants with ease. Variants and hyperparameters are like ingredients in a recipe. This platform allows you to experiment with different combinations of variants across multiple nodes in a flow.
52
52
@@ -71,7 +71,7 @@ Other features for customization:
71
71
72
72
- Provides execution of both **prompt experimentation and evaluation** locally as well on cloud.
73
73
74
-
- Endpoint testing within pipeline after deployment to check its availability and readiness.
74
+
- Endpoint testing within workflow after deployment to check its availability and readiness.
75
75
76
76
- Provides optional Human-in-loop to validate prompt metrics before deployment.
77
77
@@ -101,22 +101,22 @@ The repository for this article is available at [LLMOps with Prompt flow templat
101
101
102
102
1. The prompt engineer/data scientist opens a feature branch where they work on the specific task or feature. The prompt engineer/data scientist iterates on the flow using prompt flow for Microsoft Visual Studio Code, periodically committing changes and pushing those changes to the feature branch.
103
103
104
-
2. Once local development and experimentation are completed, the prompt engineer/data scientist opens a pull request from the Feature branch to the Main branch. The pull request (PR) triggers a PR pipeline. This pipeline runs fast quality checks that should include:
104
+
2. Once local development and experimentation are completed, the prompt engineer/data scientist opens a pull request from the Feature branch to the Main branch. The pull request (PR) triggers a PR workflow. This workflow runs fast quality checks that should include:
105
105
106
106
- Execution of experimentation flows
107
107
- Execution of configured unit tests
108
108
- Compilation of the codebase
109
109
- Static code analysis
110
110
111
-
3. The pipeline can contain a step that requires at least one team member to manually approve the PR before merging. The approver can't be the committer and they mush have prompt flow expertise and familiarity with the project requirements. If the PR isn't approved, the merge is blocked. If the PR is approved, or there's no approval step, the feature branch is merged into the Main branch.
111
+
3. The workflow can contain a step that requires at least one team member to manually approve the PR before merging. The approver can't be the committer and they mush have prompt flow expertise and familiarity with the project requirements. If the PR isn't approved, the merge is blocked. If the PR is approved, or there's no approval step, the feature branch is merged into the Main branch.
112
112
113
113
4. The merge to Main triggers the build and release process for the Development environment. Specifically:
114
114
115
-
a. The CI pipeline is triggered from the merge to Main. The CI pipeline performs all the steps done in the PR pipeline, and the following steps:
115
+
a. The CI workflow is triggered from the merge to Main. The CI workflow performs all the steps done in the PR workflow, and the following steps:
116
116
- Experimentation flow
117
117
- Evaluation flow
118
118
- Registers the flows in the AI Studio Registry when changes are detected
119
-
b. The CD pipeline is triggered after the completion of the CI pipeline. This flow performs the following steps:
119
+
b. The CD workflow is triggered after the completion of the CI workflow. This flow performs the following steps:
120
120
- Deploys the flow from the Machine Learning registry to a AI Studio Deployment
121
121
- Runs integration tests that target the online endpoint
122
122
- Runs smoke tests that target the online endpoint
@@ -174,7 +174,7 @@ There are multiple steps that should be undertaken for setting up LLMOps process
174
174
175
175
### Fork and configure the repo
176
176
177
-
Follow the [guidelines](https://github.com/microsoft/llmops-promptflow-template/blob/main/docs/github_workflows_how_to_setup.md#set-up-github-repo) to create a forked repo in your GitHub organization. This repo uses two branches - `main` and `development` for code promotions and execution of pipelines in lieu of changes to code in them.
177
+
Follow the [guidelines](https://github.com/microsoft/llmops-promptflow-template/blob/main/docs/github_workflows_how_to_setup.md#set-up-github-repo) to create a forked repo in your GitHub organization. This repo uses two branches - `main` and `development` for code promotions and execution of workflows in lieu of changes to code in them.
178
178
179
179
### Set up authentication between GitHub and Azure
180
180
@@ -190,14 +190,14 @@ Follow the [guidelines](https://github.com/microsoft/llmops-promptflow-template/
190
190
This helps us create a new feature branch from development branch and incorporate changes.
191
191
192
192
193
-
## Test the pipelines
193
+
## Test the workflows
194
194
195
-
Follow the [guidelines](https://github.com/microsoft/llmops-promptflow-template/blob/main/docs/github_workflows_how_to_setup.md#cloning-the-repos) to test the pipelines. The steps are
195
+
Follow the [guidelines](https://github.com/microsoft/llmops-promptflow-template/blob/main/docs/github_workflows_how_to_setup.md#cloning-the-repos) to test the workflows. The steps are
196
196
197
197
1. Raise a PR(Pull Request) from a feature branch to development branch.
198
-
2. The PR pipeline should execute automatically as result of branch policy configuration.
198
+
2. The PR workflow should execute automatically as result of branch policy configuration.
199
199
3. The PR is then merged to the development branch.
200
-
4. The associated 'dev' pipeline is executed. This results in full CI and CD execution and result in provisioning or updating of existing Azure AI Studio Deployment.
200
+
4. The associated 'dev' workflow is executed. This results in full CI and CD execution and result in provisioning or updating of existing Azure AI Studio Deployment.
201
201
202
202
The test outputs should be similar to ones shown at [here](https://github.com/microsoft/llmops-promptflow-template/blob/main/docs/github_workflows_how_to_setup.md#example-prompt-run-evaluation-and-deployment-scenario).
0 commit comments