Skip to content

Commit dd2ddad

Browse files
committed
stamp: self-hosting onnxruntime instructions
1 parent 16551d3 commit dd2ddad

File tree

1 file changed

+23
-0
lines changed

1 file changed

+23
-0
lines changed

ext/ai/README.md

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,29 @@ const output = await pipe("This embed will be generated from rust land", {
7373
});
7474
```
7575

76+
### Self-Hosting
77+
78+
**Caching filepath**:
79+
The `EXT_AI_CACHE_DIR` environment variable can be use to set a custom cache path
80+
81+
**Memory clean up**:
82+
For Self-Hosting users an extra method is available for `main/index.ts` scope and should be used to clean up unused sessions, consider adding it into your main entrypoint file:
83+
84+
```typescript
85+
// cleanup unused sessions every 30s
86+
setInterval(async () => {
87+
try {
88+
const cleanupCount = await EdgeRuntime.ai.tryCleanupUnusedSession();
89+
if (cleanupCount == 0) {
90+
return;
91+
}
92+
console.log('EdgeRuntime.ai.tryCleanupUnusedSession', cleanupCount);
93+
} catch (e) {
94+
console.error(e.toString());
95+
}
96+
}, 30 * 1000);
97+
```
98+
7699
</details>
77100

78101
<details>

0 commit comments

Comments
 (0)