Skip to content

Commit 5080e53

Browse files
committed
docs: Add README
1 parent beb9e1b commit 5080e53

File tree

1 file changed

+49
-0
lines changed
  • packages/kernel-language-model-service/src/test-utils/queue

1 file changed

+49
-0
lines changed
Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
# Queue-based Language Model Service (Testing Utility)
2+
3+
[`makeQueueService`](./service.ts) is a testing utility that creates a `LanguageModelService` implementation for use in tests. It provides a queue-based language model where responses are manually queued using the `push()` method and consumed by `sample()` calls.
4+
5+
## Usage
6+
7+
1. Create a service using `makeQueueService()`
8+
2. Create a model instance using `makeInstance()`
9+
3. Queue responses using `push()` on the model instance
10+
4. Consume responses by calling `sample()`
11+
12+
Note that `makeInstance` and `sample` ignore their arguments, but expect them nonetheless.
13+
14+
## Examples
15+
16+
### Basic Example
17+
18+
```typescript
19+
import { makeQueueService } from '@ocap/kernel-language-model-service/test-utils';
20+
21+
const service = makeQueueService();
22+
const model = await service.makeInstance({ model: 'test' });
23+
24+
// Queue a response
25+
model.push('Hello, world!');
26+
27+
// Consume the response
28+
const result = await model.sample({ prompt: 'Say hello' });
29+
for await (const chunk of result.stream) {
30+
console.log(chunk.response); // 'Hello, world!'
31+
}
32+
```
33+
34+
### Multiple Queued Responses
35+
36+
```typescript
37+
const service = makeQueueService();
38+
const model = await service.makeInstance({ model: 'test' });
39+
40+
// Queue multiple responses
41+
model.push('First response');
42+
model.push('Second response');
43+
44+
// Each sample() call consumes the next queued response
45+
const first = await model.sample({ prompt: 'test' });
46+
const second = await model.sample({ prompt: 'test' });
47+
48+
// Process streams...
49+
```

0 commit comments

Comments
 (0)