Skip to content

Commit e03de62

Browse files
fix: format doc;
1 parent 10cbe18 commit e03de62

File tree

1 file changed

+28
-30
lines changed

1 file changed

+28
-30
lines changed

docs/cloud/private-inference.mdx

Lines changed: 28 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -69,36 +69,36 @@ NEAR AI Cloud's private inference provides three core guarantees:
6969

7070
NEAR AI Cloud combines Intel TDX and NVIDIA TEE technologies to create isolated, secure environments for AI computation:
7171

72-
**Intel TDX (Trust Domain Extensions)**
73-
Creates confidential virtual machines (CVMs) that isolate your AI workloads from the host system, preventing unauthorized access to data in memory.
72+
- **Intel TDX (Trust Domain Extensions)**
73+
Creates confidential virtual machines (CVMs) that isolate your AI workloads from the host system, preventing unauthorized access to data in memory.
7474

75-
**NVIDIA TEE**
76-
Provides GPU-level isolation for model inference, ensuring model weights and computations remain completely private during processing.
75+
- **NVIDIA TEE**
76+
Provides GPU-level isolation for model inference, ensuring model weights and computations remain completely private during processing.
7777

78-
**Cryptographic Attestation**
79-
Each TEE environment generates cryptographic proofs of its integrity and configuration, enabling independent verification of the secure execution environment.
78+
- **Cryptographic Attestation**
79+
Each TEE environment generates cryptographic proofs of its integrity and configuration, enabling independent verification of the secure execution environment.
8080

8181
### The Inference Process
8282

8383
When you make a request to NEAR AI Cloud, your data flows through a secure pipeline designed to maintain privacy at every step:
8484

85-
**1. Request Initiation**
86-
You send chat completion requests to the LLM Gateway, which operates within a secure TEE environment and manages authentication.
85+
1. **Request Initiation:**
86+
You send chat completion requests to the LLM Gateway, which operates within a secure TEE environment and manages authentication.
8787

88-
**2. Secure Request Routing**
89-
The LLM Gateway routes your request to the appropriate Private LLM Node based on the requested model, availability, and load balancing requirements.
88+
2. **Secure Request Routing:**
89+
The LLM Gateway routes your request to the appropriate Private LLM Node based on the requested model, availability, and load balancing requirements.
9090

91-
**3. Secure Inference**
92-
AI inference computations execute inside the Private LLM Node's TEE, where all data and model weights are protected by hardware-enforced isolation.
91+
3. **Secure Inference:**
92+
AI inference computations execute inside the Private LLM Node's TEE, where all data and model weights are protected by hardware-enforced isolation.
9393

94-
**4. Attestation Generation**
95-
The TEE generates CPU and GPU attestation reports that provide cryptographic proof of the environment's integrity and configuration.
94+
4. **Attestation Generation**
95+
The TEE generates CPU and GPU attestation reports that provide cryptographic proof of the environment's integrity and configuration.
9696

97-
**5. Cryptographic Signing**
98-
The TEE cryptographically signs both your original request and the inference results to ensure authenticity and prevent tampering.
97+
5. **Cryptographic Signing:**
98+
The TEE cryptographically signs both your original request and the inference results to ensure authenticity and prevent tampering.
9999

100-
**6. Verifiable Response**
101-
You receive the AI response along with cryptographic signatures and attestation data for independent verification.
100+
6. **Verifiable Response:**
101+
You receive the AI response along with cryptographic signatures and attestation data for independent verification.
102102

103103
---
104104

@@ -125,9 +125,7 @@ NEAR AI Cloud operates through a distributed architecture consisting of an LLM G
125125
└─────────────────┘
126126
```
127127

128-
### Key Components
129-
130-
**Private LLM Nodes**
128+
### Private LLM Nodes
131129

132130
Each Private LLM Node provides secure, isolated AI inference capabilities:
133131

@@ -136,7 +134,7 @@ Each Private LLM Node provides secure, isolated AI inference capabilities:
136134
- **Private-ML-SDK**: Manages secure model execution, attestation generation, and cryptographic signing
137135
- **Health Monitoring**: Automated liveness checks and monitoring ensure continuous availability
138136

139-
**LLM Gateway**
137+
### LLM Gateway
140138

141139
The LLM Gateway serves as the central orchestration layer:
142140

@@ -153,29 +151,29 @@ The LLM Gateway serves as the central orchestration layer:
153151

154152
NEAR AI Cloud's private inference implements multiple layers of security to protect your data:
155153

156-
**Hardware-Level Isolation**
157-
TEEs create isolated execution environments enforced at the hardware level, preventing unauthorized access to memory and computation even from privileged system administrators or cloud providers.
154+
- **Hardware-Level Isolation**
155+
TEEs create isolated execution environments enforced at the hardware level, preventing unauthorized access to memory and computation even from privileged system administrators or cloud providers.
158156

159-
**Secure Communication**
157+
- **Secure Communication**
160158
All communication between your applications and the LLM infrastructure uses end-to-end encryption, protecting data in transit from network-level attacks.
161159

162-
**Cryptographic Attestation**
160+
- **Cryptographic Attestation**
163161
Every TEE environment generates cryptographic proofs that verify the integrity of the execution environment, allowing you to independently confirm your computations occurred in a genuine, unmodified TEE.
164162

165-
**Result Authentication**
163+
- **Result Authentication**
166164
All AI outputs are cryptographically signed inside the TEE before leaving the secure environment, ensuring the authenticity and integrity of responses.
167165

168166
### Threat Protection
169167

170168
NEAR AI Cloud's architecture protects against common attack vectors:
171169

172-
**Malicious Infrastructure Providers**
170+
- **Malicious Infrastructure Providers**
173171
Hardware-enforced TEE isolation prevents cloud infrastructure providers from accessing your prompts, model weights, or inference results, even with physical access to servers.
174172

175-
**Network-Based Attacks**
173+
- **Network-Based Attacks**
176174
End-to-end encryption protects your data during transmission, preventing man-in-the-middle attacks and network eavesdropping.
177175

178-
**Model Extraction Attempts**
176+
- **Model Extraction Attempts**
179177
Model weights remain encrypted and isolated within the TEE, making extraction computationally infeasible even for attackers with privileged system access.
180178

181179
**Result Tampering**

0 commit comments

Comments
 (0)