|
| 1 | +# Queue-based Language Model Service (Testing Utility) |
| 2 | + |
| 3 | +[`makeQueueService`](./service.ts) is a testing utility that creates a `LanguageModelService` implementation for use in tests. It provides a queue-based language model where responses are manually queued using the `push()` method and consumed by `sample()` calls. |
| 4 | + |
| 5 | +## Usage |
| 6 | + |
| 7 | +1. Create a service using `makeQueueService()` |
| 8 | +2. Create a model instance using `makeInstance()` |
| 9 | +3. Queue responses using `push()` on the model instance |
| 10 | +4. Consume responses by calling `sample()` |
| 11 | + |
| 12 | +Note that `makeInstance` and `sample` ignore their arguments, but expect them nonetheless. |
| 13 | + |
| 14 | +## Examples |
| 15 | + |
| 16 | +### Basic Example |
| 17 | + |
| 18 | +```typescript |
| 19 | +import { makeQueueService } from '@ocap/kernel-language-model-service/test-utils'; |
| 20 | + |
| 21 | +const service = makeQueueService(); |
| 22 | +const model = await service.makeInstance({ model: 'test' }); |
| 23 | + |
| 24 | +// Queue a response |
| 25 | +model.push('Hello, world!'); |
| 26 | + |
| 27 | +// Consume the response |
| 28 | +const result = await model.sample({ prompt: 'Say hello' }); |
| 29 | +for await (const chunk of result.stream) { |
| 30 | + console.log(chunk.response); // 'Hello, world!' |
| 31 | +} |
| 32 | +``` |
| 33 | + |
| 34 | +### Multiple Queued Responses |
| 35 | + |
| 36 | +```typescript |
| 37 | +const service = makeQueueService(); |
| 38 | +const model = await service.makeInstance({ model: 'test' }); |
| 39 | + |
| 40 | +// Queue multiple responses |
| 41 | +model.push('First response'); |
| 42 | +model.push('Second response'); |
| 43 | + |
| 44 | +// Each sample() call consumes the next queued response |
| 45 | +const first = await model.sample({ prompt: 'test' }); |
| 46 | +const second = await model.sample({ prompt: 'test' }); |
| 47 | + |
| 48 | +// Process streams... |
| 49 | +``` |
0 commit comments