You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/machine-learning/prompt-flow/concept-llmops-maturity.md
+16Lines changed: 16 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -20,8 +20,15 @@ Large Language Model Operations, or **LLMOps**, describes the operational practi
20
20
21
21
Use the descriptions below to find your *LLMOps Maturity Model* ranking level. These levels provide a general understanding and practical application level of your organization. The guidelines provide you with helpful links to expand your LLMOps knowledge base.
22
22
23
+
Or use this [LLMOps Maturity Model Assessment](/assessments/e14e1e9f-d339-4d7e-b2bb-24f056cf08b6/) to determine your organization's current LLMOps maturity level. The questionnaire is designed to help you understand your organization's current capabilities and identify areas for improvement.
24
+
25
+
Your results from the assessment corresponds to a *LLMOps Maturity Model* ranking level, providing a general understanding and practical application level of your organization. These guidelines provide you with helpful links to expand your LLMOps knowledge base
26
+
23
27
## <aname="level1"></a>Level 1 - initial
24
28
29
+
> [!TIP]
30
+
> Score from [LLMOps Maturity Model Assessment](/assessments/e14e1e9f-d339-4d7e-b2bb-24f056cf08b6/): initial (0-9).
31
+
25
32
**Description:** Your organization is at the initial foundational stage of LLMOps maturity. You're exploring the capabilities of LLMs but haven't yet developed structured practices or systematic approaches.
26
33
27
34
Begin by familiarizing yourself with different LLM APIs and their capabilities. Next, start experimenting with structured prompt design and basic prompt engineering. Review ***Microsoft Learning*** articles as a starting point. Taking what you’ve learned, discover how to introduce basic metrics for LLM application performance evaluation.
@@ -43,6 +50,9 @@ To better understand LLMOps, consider available MS Learning courses and workshop
43
50
44
51
## <aname="level2"></a> Level 2 - defined
45
52
53
+
> [!TIP]
54
+
> Score from [LLMOps Maturity Model Assessment](/assessments/e14e1e9f-d339-4d7e-b2bb-24f056cf08b6/): maturing (9-14).
55
+
46
56
**Description:** Your organization has started to systematize LLM operations, with a focus on structured development and experimentation. However, there's room for more sophisticated integration and optimization.
47
57
48
58
To improve your capabilities and skills, learn how to develop more complex prompts and begin integrating them effectively into applications. During this journey, you’ll want to implement a systematic approach for LLM application deployment, possibly exploring CI/CD integration. Once you understand the core, you can begin employing more advanced evaluation metrics like groundedness, relevance, and similarity. Ultimately, you’ll want to focus on content safety and ethical considerations in LLM usage.
@@ -61,6 +71,9 @@ To improve your capabilities and skills, learn how to develop more complex promp
61
71
62
72
## <aname="level3"></a> Level 3 - managed
63
73
74
+
> [!TIP]
75
+
> Score from [LLMOps Maturity Model Assessment](/assessments/e14e1e9f-d339-4d7e-b2bb-24f056cf08b6/): maturing (15-19).
76
+
64
77
**Description:** Your organization is managing advanced LLM workflows with proactive monitoring and structured deployment strategies. You're close to achieving operational excellence.
65
78
66
79
To expand your base knowledge, focus on continuous improvement and innovation in your LLM applications. As you progress, you can enhance your monitoring strategies with predictive analytics and comprehensive content safety measures. Learn to optimize and fine-tune your LLM applications for specific requirements. Ultimately, you want to strengthen your asset management strategies through advanced version control and rollback capabilities.
@@ -74,6 +87,9 @@ To expand your base knowledge, focus on continuous improvement and innovation in
74
87
75
88
## <aname="level4"></a> Level 4 - optimized
76
89
90
+
> [!TIP]
91
+
> Score from [LLMOps Maturity Model Assessment](/assessments/e14e1e9f-d339-4d7e-b2bb-24f056cf08b6/): optimized (19-28).
92
+
77
93
**Description:** Your organization demonstrates operational excellence in LLMOps. You have a sophisticated approach to LLM application development, deployment, and monitoring.
78
94
79
95
As LLMs evolve, you’ll want to maintain your cutting-edge position by staying updated with the latest LLM advancements. Continuously evaluate the alignment of your LLM strategies with evolving business objectives. Ensure that you foster a culture of innovation and continuous learning within your team. Last, but not least, share your knowledge and best practices with the wider community to establish thought leadership in the field.
0 commit comments