Skip to content

Commit b91412b

Browse files
authored
Explain Serving options, add headings
1 parent 1120b34 commit b91412b

File tree

1 file changed

+10
-3
lines changed

1 file changed

+10
-3
lines changed

docs-gb/README.md

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,17 @@ Seldon Core 2 is a source-available framework for deploying and managing machine
66

77
## Model Deployment
88

9-
Seldon Core 2 orchestrates and scales machine learning components running as production-grade microservices. These components can be deployed locally or in enterprise-scale kubernetes clusters. The components of your ML system - such as models, processing steps, custom logic, or monitoring methods - are deployed as **Models**, leveraging serving solutions compatible with Core 2 such as MLServer, Alibi, LLM Module, or Triton Inference Server. These serving solutions package the required dependencies and standardize inference using the Open Inference Protocol. This ensures that, regardless of your model types and use-cases, all request and responses follow a unified format. After models are deployed, they can process REST or gRPC requests for real-time inference.
9+
Seldon Core 2 orchestrates and scales machine learning components running as production-grade microservices. These components can be deployed locally or in enterprise-scale kubernetes clusters. The components of your ML system - such as models, processing steps, custom logic, or monitoring methods - are deployed as **Models**. Core 2 is compabitible with various serving solutions to deploy various types of models or ML components depending on the use-case:
10+
* Seldon's [MLServer](https://mlserver.readthedocs.io/en/stable/index.html) or [Triton Inference Server](https://developer.nvidia.com/triton-inference-server) for ML models
11+
* Seldon's Alibi libraries for [drift and outlier detection](https://docs.seldon.io/projects/alibi-detect/en/latest/) and [explainability](https://docs.seldon.io/projects/alibi/en/latest/) methods, or
12+
* Seldon's [LLM Module](https://www.seldon.io/solutions/llm-module) for deploying LLM-powered applications.
13+
14+
These serving solutions package the required dependencies and standardize inference using the Open Inference Protocol. This ensures that, regardless of your model types and use-cases, all request and responses follow a unified format. After models are deployed, they can process REST or gRPC requests for real-time inference.
1015

1116
## Complex Applications & Orchestration
1217

13-
Machine learning applications are increasingly complex. They’ve evolved from individual models deployed as services, to complex applications that can consist of multiple models, processing steps, custom logic, and asynchronous monitoring components. With Core you can build Pipelines that connect any of these components to make data-centric applications. Core 2 handles orchestration and scaling of the underlying components of such an application, and exposes the data streamed through the application in real time using Kafka.
18+
### Pipelines
19+
Machine learning applications are increasingly complex. They’ve evolved from individual models deployed as services, to complex applications that can consist of multiple models, processing steps, custom logic, and asynchronous monitoring components. With Core you can build **Pipelines** that connect any of these components to make data-centric applications. Core 2 handles orchestration and scaling of the underlying components of such an application, and exposes the data streamed through the application in real time using Kafka.
1420

1521
{% hint style="info" %}
1622
Data-centricity is an approach that places the management, integrity, and flow of data at the core of the machine learning deployment framework.
@@ -21,7 +27,8 @@ This approach to MLOps, influenced by our position paper [Desiderata for next ge
2127

2228
![Data-centric pipeline](images/pipeline-intro.png)
2329

24-
Lastly, Core 2 provides Experiments as part of its orchestration capabilities, enabling users to implement routing logic such as A/B tests or Canary deployments to models or pipelines in production. After experiments are run, you can promote new models or pipelines, or launch new experiments, so that you can continuously improve the performance of your ML applications.
30+
### Experiments
31+
Lastly, Core 2 provides **Experiments** as part of its orchestration capabilities, enabling users to implement routing logic such as A/B tests or Canary deployments to models or pipelines in production. After experiments are run, you can promote new models or pipelines, or launch new experiments, so that you can continuously improve the performance of your ML applications.
2532

2633

2734
## Resource Management

0 commit comments

Comments
 (0)