|
| 1 | +--- |
| 2 | +title: EigenAI Overview |
| 3 | +sidebar_position: 5 |
| 4 | +--- |
| 5 | + |
| 6 | +## Overview |
| 7 | +Build verifiable applications leveraging LLM inference without wondering if the same LLM call might produce different results on different runs, or whether your prompts, models, or responses are modified in any way. EigenAI offers a *deterministic, verifiable* API, compatible with the OpenAI API, so you can simply place the API endpoint in your existing application to start shipping AI-based applications you and your users can trust. |
| 8 | + |
| 9 | +**AI is one of the greatest technological advancements in human history. Our mission is to enable any developer to build with scalable & verifiable AI, so that any developer anywhere in the world can build trusted applications.** |
| 10 | + |
| 11 | +## Use Cases |
| 12 | + |
| 13 | +Builders are leveraging EigenAI to build applications such as: |
| 14 | +- **Prediction Market Agents**: Build agents who can interpret real world events, news, etc and place bets or dispute market settlements. |
| 15 | +- **Trading Agents**: Build agents who can reason through financial data with consistent quality of thinking (no need to worry if models are quantized or not in production) while you ensure they process all of the information they're given (unmodified prompts) and that agents actually use the unmodified responses. You can also ensure they reliably make the same trading decision if prompted about the same data multiple times (via EigenAI's determinism). |
| 16 | +- **Verifiable AI Games**: Build games with AI characters or AI governance, where you can prove to your users that their interactions with the AI aren't being gamed. |
| 17 | +- **Verifiable AI Judges**: Whether it's contests / games, admissions committees, or prediction market settlements, AI can be used to verifiably judge entries / submissions. |
| 18 | + |
| 19 | +<img src="/img/eigenai-use-cases.jpg" alt="EigenAI Use Cases"/> |
| 20 | + |
| 21 | +## Get started |
| 22 | + |
| 23 | +A few key points: |
| 24 | + |
| 25 | +- By OpenAI compliancy we specifically mean the messages-based Chat Completions API: https://platform.openai.com/docs/api-reference/chat/create |
| 26 | +- By “deterministic” we specifically mean that one request (prompt, parameters, etc) provided to the API multiple times will produce the **same output bit-by-bit**, compared to the potentially varying responses one would typically get if calling an OpenAI, Anthropic, etc endpoint as they do not guarantee deterministic behavior. We will be releasing more details shortly on how EigenAI achieves this across the stack. |
| 27 | +- On wanting non-determinism: |
| 28 | + - You can still introduce non-determinism in your application if you want. By setting a different seed for requests but otherwise keeping the request the same, the API will produce a different output. |
| 29 | +- On verification: As part of EigenAI’s mainnet alpha release, the code will be open sourced shortly after. Anyone with access to commodity GPUs will be able to leverage the determinism of EigenAI's software to re-execute any requests and verify the responses given to them by EigenAI. |
| 30 | + - As we go towards general availability, we will stand up another API that can be used for this verification flow. |
| 31 | + |
| 32 | +## Usage |
| 33 | + |
| 34 | +We're starting off with supporting the `gpt-oss-120b-f16` model based on initial demand and expanding from there. To get started or request another model, visit our [onboarding page](https://onboarding.eigencloud.xyz/). |
| 35 | + |
| 36 | +```bash |
| 37 | +$ curl -X POST https://eigenai.eigencloud.xyz/v1/chat/completions \ |
| 38 | +-H "X-API-Key: <api-key>" \ |
| 39 | +-H "Content-Type: application/json" \ |
| 40 | +-d '{ |
| 41 | + "model": "gpt-oss-120b-f16", |
| 42 | + "max_tokens": 120, |
| 43 | + "seed": 42, |
| 44 | + "messages": [{"role": "user", "content": "Write a story about programming"}] |
| 45 | +}' | jq |
| 46 | + |
| 47 | +{ |
| 48 | + "id": "chatcmpl-eigenai-llama-1758835890305965", |
| 49 | + "created": 1758835890, |
| 50 | + "model": "gpt-oss-120b-f16", |
| 51 | + "system_fingerprint": "eigenai-llama@0.1.0+eigenai-llama", |
| 52 | + "object": "chat.completion", |
| 53 | + "usage": { |
| 54 | + "completion_tokens": 120, |
| 55 | + "prompt_tokens": 72, |
| 56 | + "total_tokens": 192 |
| 57 | + }, |
| 58 | + "choices": [ |
| 59 | + { |
| 60 | + "index": 0, |
| 61 | + "message": { |
| 62 | + "role": "assistant", |
| 63 | + "content": "<|channel|>analysis<|message|>The user asks: \"Write a story about programming\". Need to produce a story. Probably creative, about programming, characters, maybe AI, debugging, etc. Should be engaging. No constraints other than story. I will craft a narrative with a protagonist, their journey, maybe metaphorical. Use vivid language. Provide a story. Ensure it's appropriate. Probably around 800-1200 words? Not too long, but enough. I'll deliver.<|end|><|start|>assistant<|channel|>final<|message|>**The Compiler’s Apprentice**\n\nWhen Maya first walked into the cramped, fluorescent‑lit office of LumenTech" |
| 64 | + }, |
| 65 | + "finish_reason": "length" |
| 66 | + } |
| 67 | + ], |
| 68 | + "signature": "2ee2e48ae9f6796c1de67977981d937c886efec3957c9bfc28f7a3f3be8262f04af609f22878b3bc2f0c4858fd04005d76877208881abd473208317df1dcfaf11c" |
| 69 | +} |
| 70 | +``` |
| 71 | + |
| 72 | +### OpenAI Client usage |
| 73 | + |
| 74 | +```python |
| 75 | +from openai import OpenAI |
| 76 | + |
| 77 | +client = OpenAI( |
| 78 | + base_url="https://eigenai.eigencloud.xyz/v1", |
| 79 | + default_headers={"x-api-key": api_key} |
| 80 | +) |
| 81 | + |
| 82 | +tools: List[Dict[str, Any]] = [ |
| 83 | + { |
| 84 | + "type": "function", |
| 85 | + "function": { |
| 86 | + "name": "get_current_weather", |
| 87 | + "description": "Get the current weather in a given location", |
| 88 | + "parameters": { |
| 89 | + "type": "object", |
| 90 | + "properties": { |
| 91 | + "location": { |
| 92 | + "type": "string", |
| 93 | + "description": "The city and state, e.g. San Francisco, CA", |
| 94 | + }, |
| 95 | + "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}, |
| 96 | + }, |
| 97 | + "required": ["location"], |
| 98 | + }, |
| 99 | + }, |
| 100 | + } |
| 101 | +] |
| 102 | + |
| 103 | +step1 = client.chat.completions.create( |
| 104 | + model=model, |
| 105 | + messages=[{"role": "user", "content": "What is the weather like in Boston today?"}], |
| 106 | + tools=tools, |
| 107 | + tool_choice="auto", |
| 108 | +) |
| 109 | + |
| 110 | +""" |
| 111 | +Response: |
| 112 | +{ |
| 113 | + "id": "chatcmpl-eigenai-llama-1758836092182536", |
| 114 | + "object": "chat.completion", |
| 115 | + "created": 1758727565, |
| 116 | + "model": "gpt-oss-120b-f16", |
| 117 | + "choices": [ |
| 118 | + { |
| 119 | + "index": 0, |
| 120 | + "message": { |
| 121 | + "role": "assistant", |
| 122 | + "content": null, |
| 123 | + "tool_calls": [ |
| 124 | + { |
| 125 | + "id": "call_YDzzMHFtp1yuURbiPe09uyHt", |
| 126 | + "type": "function", |
| 127 | + "function": { |
| 128 | + "name": "get_current_weather", |
| 129 | + "arguments": "{\"location\":\"Boston, MA\",\"unit\":\"fahrenheit\"}" |
| 130 | + } |
| 131 | + } |
| 132 | + ], |
| 133 | + "refusal": null, |
| 134 | + "annotations": [] |
| 135 | + }, |
| 136 | + "finish_reason": "tool_calls" |
| 137 | + } |
| 138 | + ], |
| 139 | + "usage": { |
| 140 | + "prompt_tokens": 81, |
| 141 | + "completion_tokens": 223, |
| 142 | + "total_tokens": 304, |
| 143 | + "prompt_tokens_details": { |
| 144 | + "cached_tokens": 0, |
| 145 | + "audio_tokens": 0 |
| 146 | + }, |
| 147 | + "completion_tokens_details": { |
| 148 | + "reasoning_tokens": 192, |
| 149 | + "audio_tokens": 0, |
| 150 | + "accepted_prediction_tokens": 0, |
| 151 | + "rejected_prediction_tokens": 0 |
| 152 | + } |
| 153 | + }, |
| 154 | +""" |
| 155 | + |
| 156 | +messages_step2: List[Dict[str, Any]] = [ |
| 157 | + {"role": "user", "content": "What is the weather like in Boston today?"}, |
| 158 | + { |
| 159 | + "role": "assistant", |
| 160 | + "content": None, |
| 161 | + "tool_calls": [ |
| 162 | + { |
| 163 | + "id": tool_call_id, |
| 164 | + "type": "function", |
| 165 | + "function": { |
| 166 | + "name": "get_current_weather", |
| 167 | + "arguments": json.dumps({"location": "Boston, MA", "unit": "fahrenheit"}), |
| 168 | + }, |
| 169 | + } |
| 170 | + ], |
| 171 | + }, |
| 172 | + {"role": "tool", "tool_call_id": tool_call_id, "content": "58 degrees"}, |
| 173 | + {"role": "user", "content": "Do I need a sweater for this weather?"}, |
| 174 | +] |
| 175 | + |
| 176 | +step2 = client.chat.completions.create(model=model, messages=messages_step2) |
| 177 | + |
| 178 | +""" |
| 179 | +Response |
| 180 | +{ |
| 181 | + "id": "chatcmpl-eigenai-llama-CJOZTszzusoHvAYYrW8PT5lv6vzKo", |
| 182 | + "object": "chat.completion", |
| 183 | + "created": 1758738719, |
| 184 | + "model": "gpt-oss-120b-f16", |
| 185 | + "choices": [ |
| 186 | + { |
| 187 | + "index": 0, |
| 188 | + "message": { |
| 189 | + "role": "assistant", |
| 190 | + "content": "At around 58°F in Boston you’ll feel a noticeable chill—especially if there’s any breeze or you’re out in the morning or evening. I’d recommend throwing on a light sweater or layering a long-sleeve shirt under a casual jacket. If you tend to run cold, go with a medium-weight knit; if you’re just mildly sensitive, a thin cardigan or pullover should be enough.", |
| 191 | + "refusal": null, |
| 192 | + "annotations": [] |
| 193 | + }, |
| 194 | + "finish_reason": "stop" |
| 195 | + } |
| 196 | + ], |
| 197 | + "usage": { |
| 198 | + "prompt_tokens": 67, |
| 199 | + "completion_tokens": 294, |
| 200 | + "total_tokens": 361, |
| 201 | + "prompt_tokens_details": { |
| 202 | + "cached_tokens": 0, |
| 203 | + "audio_tokens": 0 |
| 204 | + }, |
| 205 | + "completion_tokens_details": { |
| 206 | + "reasoning_tokens": 192, |
| 207 | + "audio_tokens": 0, |
| 208 | + "accepted_prediction_tokens": 0, |
| 209 | + "rejected_prediction_tokens": 0 |
| 210 | + } |
| 211 | + }, |
| 212 | +""" |
| 213 | +``` |
| 214 | + |
| 215 | +## Supported parameters |
| 216 | + |
| 217 | +This list will be expanding to cover the full parameter set of the Chat Completions API. |
| 218 | + |
| 219 | +- `messages: array` |
| 220 | + - A list of messages comprising the conversation so far |
| 221 | +- `model: string` |
| 222 | + - Model ID used to generate the response, like `gpt-oss-120b-f16` |
| 223 | +- `max_tokens: (optional) integer` |
| 224 | + - The maximum number of [tokens](https://platform.openai.com/tokenizer) that can be generated in the chat completion. This value can be used to control [costs](https://openai.com/api/pricing/) for text generated via API. |
| 225 | +- `seed: (optional) integer` |
| 226 | + - If specified, our system will run the inference deterministically, such that repeated requests with the same `seed` and parameters should return the same result. |
| 227 | +- `stream: (optional) bool` |
| 228 | + - If set to true, the model response data will be streamed to the client as it is generated using Server-Side Events (SSE). |
| 229 | +- `temperature: (optional) number` |
| 230 | + - What sampling temperature to use, between 0 and 2. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. |
| 231 | +- `top_p: (optional) number` |
| 232 | + - An alternative to sampling with temperature, called nucleus sampling, where the model considers the results of the tokens with top_p probability mass. So 0.1 means only the tokens comprising the top 10% probability mass are considered. |
| 233 | +- `logprobs: (optional) bool` |
| 234 | + - Whether to return log probabilities of the output tokens or not. If true, returns the log probabilities of each output token returned in the `content` of `message` |
| 235 | +- `frequency_penalty: (optional) number` |
| 236 | + - Number between -2.0 and 2.0. Positive values penalize new tokens based on their existing frequency in the text so far, decreasing the model's likelihood to repeat the same line verbatim. |
| 237 | +- `presence_penalty: (optional) number` |
| 238 | + - Number between -2.0 and 2.0. Positive values penalize new tokens based on whether they appear in the text so far, increasing the model's likelihood to talk about new topics. |
| 239 | +- `tools: array` |
| 240 | + - A list of tools ([function tools](https://platform.openai.com/docs/guides/function-calling)) the model may call. |
| 241 | +- `tool_choice: (optional) string` |
| 242 | + - “auto”, “required”, “none” |
| 243 | + - Controls which (if any) tool is called by the model. `none` means the model will not call any tool and instead generates a message. `auto` means the model can pick between generating a message or calling one or more tools. `required` means the model must call one or more tools. Specifying a particular tool via `{"type": "function", "function": {"name": "my_function"}}` forces the model to call that tool. |
| 244 | + - `none` is the default when no tools are present. `auto` is the default if tools are present. |
0 commit comments