The Open Context Engine for AI
SODA Contexture is an open source project under SODA Foundation (a sub-foundation under Linux Foundation). It is an open context building engine for AI. SODA Contexture provides a platform to build enriched operational contexts to AI Agents for various data sources at scale. It improves the accuracy, efficiency, and speed of data inferences and insights significantly.
The project defines the Open Context Specification(OCS) to describe the data in a structured way. The specification provides the context implementation guidelines. SODA Contexture builds contexts using internal context agents based on OCS and also third party context sources.
There is no standard way of communication to AI to get things done! Hence, the data inference and insights suffer from:
- Low Accuracy
- The accuracy of results varies drastically based on the nature of data and inputs = Mixing guesses and different sources of knowledge confuses AI
- Inconsistency
- Hallucination is key known issue with AI
- High Latency
- Based on the type of query and volume of data, it fails to give ontime results
- Huge Cost
- Iterations to get a close results and verification add costs
- Lack of Scale
- Works for small amount of data or 1 agent, when it comes to scale, it fails
- Low Reliability = Due uncertain results AI is not fully dependable
One of the solutions to these problems is to provide the right context to the AI, for it to understand better to fetch the right pieces of data to derive the right inference. However this is not easy. Because, the data relationships and types can vary. That is why SODA Contexture is trying to solve the issue of “Missing Context” through OCS and building various components connecting to provide enriched and structured context.
SODA Contexture derives enriched context based on the OCS (Open Context Specification) implementation for the specific data sources and fills the issue of “Missing Context”. It builds the best possible context using its context building engine based on OCS for the input queries. Using this enhanced context AI models can understand the context better and fetch the right data (or data sets) to provide accurate inferences and insights.
- SODA Contexture Engine: The core component that processes user requests and orchestrates context generation.
- Open Context Specification: The specification which details the operational context building attributes for various types of data.
- Data Connectors: Logical Connectors to different types of data such as prometheus, sql, s3 and so on to understand the nature of data storage and layout. These connectors provide SODA Contexture a better idea to use the OCS to build the context better. These are logical connectors for specific data source
- Context Providers: Sources that provide enriched context information (e.g., Istio, Kubernetes).
OCS (Open Context Specification) provides the specification for operational data context spec for different kinds of data sources. It provides the key attributes to derive the best possible context to enable AI to provide more accurate results.
OCS Defines the key attributes to build the operational context:
- Identity and Origin (The "Who" and "Where")
- Defines the unique fingerprint of the data source.
- AI needs this to distinguish between similar metrics from different environments
- Dimensionality & Topology (The "Relationship")
- Defines how this metric relates to other components
- This is the most critical part for AI reasoning.
- Metric Semantics (The "What")
- Define what the number actually represents
- This will avoid the AI comparing unrelated mertrics.
- Temporal Context (The "When")
- AI needs to know if it's looking at a "point-in-time" value or a trend.
- Interval, Duration, Time stamp etc
- Operational Constraints (The "How")
- This tells the AI how to interpret the health of the metric.
- Threshold, Polarity, Aggregation
We are actively developing the project. So if you would like to join the design, OCS and other components, please join us!