diff --git a/docs/build/private-llms/usage.mdx b/docs/build/private-llms/usage.mdx index 4eb0a5d..7192c52 100644 --- a/docs/build/private-llms/usage.mdx +++ b/docs/build/private-llms/usage.mdx @@ -8,6 +8,96 @@ Once you have [nilAI API access](/build/network-api-access), you can start using ## Getting Started with Private LLMs + + ## Getting Started with Private LLMs + + + + + + + ```bash + pip install nilai-py + ``` + + + ```bash + uv pip install nilai-py + ``` + + + + + ```bash + pnpm install @nillion/nilai-ts + ``` + + + + + You can either use: + - `API Key` flow as the sole developer / organization or + - `Delegation Flow` to provide permissions to another user / organization. + + ### API Key Flow + + 1. Use `https://nilai-a779.nillion.network/nuc/v1` as the BASE URL + 2. Check [available models](/build/private-llms/overview#available-models) or query the [`/v1/models`](/api/nilai/get-models-v-1-models-get) endpoint or + 3. Select an available model and use it with the [`/v1/chat/completions`](/api/nilai/chat-completion-v-1-chat-completions-post) nilAI node endpoint + + With OpenAI compatibility, you can use any OpenAI library. Here's an example for querying the `Llama-3.1-8B` model: + + + + ```python reference showGithubLink + https://github.com/NillionNetwork/nilai-py/blob/main/examples/0-api_key_mode.py + ``` + + + ```typescript + import "dotenv/config"; + import { NilaiOpenAIClient, NilAuthInstance } from "@nillion/nilai-ts"; + + // To obtain an API key, navigate to https://subscription.nillion.com + // and create a new subscription. + // The API key will be displayed in the subscription details. + // The NilaiOpenAIClient class automatically handles the NUC token creation and management. + + const API_KEY = process.env.NILLION_API_KEY; + + async function main() { + // Initialize the client in API key mode + // For sandbox, use the following: + const client = new NilaiOpenAIClient({ + baseURL: "https://nilai-a779.nillion.network/v1/", + apiKey: API_KEY, + nilauthInstance: NilAuthInstance.SANDBOX, + // For production, use the following: + // nilauthInstance: NilAuthInstance.PRODUCTION, + }); + + // Make a request to the Nilai API + const response = await client.chat.completions.create({ + model: "google/gemma-3-27b-it", + messages: [ + { role: "user", content: "Hello! Can you help me with something?" } + ], + }); + + console.log(`Response: ${response.choices[0].message.content}`); + } + + // Run the example + main().catch(console.error); + ``` + + + + ### Delegation flow + + To use the delegation flow, you need to create a delegation token server. + + The server then creates the delegation tokens and managing their expiration and usage. Then the delegation token allows you to make requests to the nilAI API.