You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
# Core 2: Deploy Modular, Data-centric AI applications at scale
2
2
3
-
Seldon Core 2 provides a state of the art solution for machine learning inference which can be run locally on a laptop as well as on Kubernetes for production.
3
+
## 📖 About
4
+
Seldon Core 2 is an MLOps and LLMOps framework for deploying, managing and scaling AI systems in Kubernetes - from singular models, to modular, data-centric applications. With Core 2 you can deploy across a wide range of model types, on-prem or in any cloud, in a standardized way that is production-ready out of the box.
* A single platform for inference of wide range of standard and custom artifacts.
10
-
* Deploy locally in Docker during development and testing of models.
11
-
* Deploy at scale on Kubernetes for production.
12
-
* Deploy single models to multi-step pipelines.
13
-
* Save infrastructure costs by deploying multiple models transparently in inference servers.
14
-
* Overcommit on resources to deploy more models than available memory.
15
-
* Dynamically extended models with pipelines with a data-centric perspective backed by Kafka.
16
-
* Explain individual models and pipelines with state of the art explanation techniques.
17
-
* Deploy drift and outlier detectors alongside models.
18
-
* Kubernetes Service mesh agnostic - use the service mesh of your choice.
19
-
10
+
***Pipelines**: Deploy composable AI pipelines, leveraging Kafka for realtime data streaming between components
11
+
***Autoscaling** for models and application components based on native or custom logic
12
+
***Multi-Model Serving**: Save infrastructure costs by consolidating multiple models on shared inference servers
13
+
***Overcommit**: Deploy more models than available memory allows, saving infrastructure costs for unused models
14
+
***Experiments**: Route data between candidate models or pipeline, with support for A/B tests and shadow deployments
15
+
***Custom Components**: Implement custom logic, drift & outlier detection, LLMs and more through plug-and-play integrate with the rest of Seldon's ecosytem of ML/AI products!
16
+
20
17
## Publication
21
18
22
19
These features are influenced by our position paper on the next generation of ML model serving frameworks:
@@ -26,45 +23,11 @@ These features are influenced by our position paper on the next generation of ML
26
23
*Workshop*: Challenges in deploying and monitoring ML systems workshop - NeurIPS 2022
0 commit comments