Skip to content

Commit 927afeb

Browse files
jboldajbolda
committed
readme updates
1 parent eafa7cd commit 927afeb

File tree

1 file changed

+125
-95
lines changed

1 file changed

+125
-95
lines changed

‎packages/server/README.md‎

Lines changed: 125 additions & 95 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,7 @@ node --import tsx ./simulators/service-graph.ts
6060
Secondly, we can use this in tests. It is convenient to place in an `beforeAll()` or a `beforeEach()`. This is built on `effection`, and should handle all shutdown and clean up of the services when the function passes out of lexical scope.
6161

6262
```ts
63+
import { test, beforeEach } from "test-runner";
6364
import { run } from "effection";
6465
import { services } from "./simulators/service-graph.ts";
6566

@@ -79,136 +80,165 @@ test("things", async () => {
7980

8081
## Operation-based service orchestration
8182

82-
`@simulacrum/server` provides operations to start and manage services with lifecycle hooks. The recommended pattern is to create `Operation<void>` instances for each service (typically via `useService`) and pass them to `useServiceGraph` which starts the services respecting a dependency DAG and provides lifecycle hooks for startup and shutdown.
83+
`@simulacrum/server` provides operations to start and manage services with lifecycle hooks. The recommended pattern is to create `Operation<void>` instances for each service (typically via `useService`, `useSimulation`, or `useChildSimulation`) and pass them to `useServiceGraph` which starts the services respecting a dependency DAG and provides lifecycle hooks for startup and shutdown.
8384

84-
Key points:
85+
See the `@simulacrum/foundation-simulator` for a basis to build simulator(s) for your services.
8586

86-
- `useServiceGraph(services: ServicesMap, options?: { globalData?: Record<string, unknown>; watch?: boolean; watchDebounce?: number }): ServiceRunner<ServicesMap>` — returns a _runner function_ which you call to start the graph: `const run = useServiceGraph(services, options); const provided = yield* run(subset?: string[] | string);`. By default services in the same topological layer run concurrently; pass `options.watch = true` and `options.watchDebounce` to enable file watching and restart propagation.
87-
- `ServiceDefinition.operation` (required) — an `Operation<void>` which indicates the service has started. This operation may be long-lived (e.g. `useService`) or may return once the service is ready while a background child keeps the service running. See the example below.
88-
- `dependsOn` — an optional object `{ startup: string[]; restart?: string[] }` listing service names this service depends on. Use `startup` to list services that must start before this service; use `restart` to list services that should trigger a restart of this service when they are restarted (for example, due to a watched file change). Services without dependencies in the same layer are started concurrently by default, or serially when `options.sequential` is true.
87+
## API reference
8988

90-
- `subset` (runner argument) — when calling the runner returned by `useServiceGraph` you may pass a subset (e.g. `yield* run(['serviceA'])` or `yield* run('serviceA')`) to start only a subset of services; any startup dependencies required by that subset are automatically included.
89+
### useServiceGraph(services, options?)
9190

92-
- Watching & restart propagation — pass `{ watch: true }` to `useServiceGraph` and define `watch` paths in each `ServiceDefinition` to enable file watching. The watcher will precompute transitive dependents (based on `dependsOn.restart`) and automatically emit restart updates for dependents when a watched path changes, so restarts propagate efficiently and deterministically.
91+
`useServiceGraph(services: ServicesMap, options?: { globalData?: Record<string, unknown>; watch?: boolean; watchDebounce?: number }): ServiceRunner<ServicesMap>`
9392

94-
### Global data & the simulacrum gateway 🔁
93+
Returns a "runner" function. Call the runner inside an Effection scope to start the graph:
9594

96-
The graph can optionally start a small local HTTP data service (the _simulacrum gateway_) to expose a `globalData` object to child simulations and tests. The gateway registers its listening port on the returned `servicePorts` map under the key `"simulacrum"`, which tests and examples can use to discover it. See the `test/child-simulation-simulacrum.test.ts` and `example/simulation-graph.ts` for examples and coverage of this flow.
95+
```ts
96+
const run = useServiceGraph(services, options);
97+
const services = yield * run(subset); // holds while services run, subset is optional
98+
```
9799

98-
### Lifecycle hooks
100+
File watching: pass `options.watch = true` and `options.watchDebounce` to enable watching and restart propagation across dependents. This is enabled through the CLI helper.
99101

100-
- Lifecycle hooks can be implemented by arranging operations and using try/finally within your service `operation` to perform startup and cleanup logic. You can keep the operation alive with `yield* suspend()` and perform cleanup in the `finally` block when the service is stopped.
102+
#### ServiceDefinition
101103

102-
Example:
104+
The `ServicesMap` passed as the first argument to `useServiceGraph`.
103105

104106
```ts
105-
import { main, spawn, sleep } from "effection";
106-
import { useServiceGraph, useService } from "@simulacrum/server";
107-
108-
main(function* () {
109-
yield* spawn(function* () {
110-
// In many situations, pass `useService` directly: it returns once the
111-
// process is spawned and, if a wellnessCheck is provided, once the
112-
// wellnessCheck passes. The service is automatically shut down by
113-
// effection when the operation goes out of scope.
114-
const run = useServiceGraph({
115-
A: {
116-
operation: useService(
117-
"A",
118-
"node --import tsx ./test/services/service-a.ts"
119-
),
120-
},
121-
B: {
122-
operation: useService(
123-
"B",
124-
"node --import tsx ./test/services/service-b.ts"
125-
),
126-
dependsOn: { startup: ["A"] },
127-
},
128-
});
129-
});
130-
});
107+
const services: ServicesMap = {
108+
serviceKey: {
109+
operation,
110+
dependsOn,
111+
watch,
112+
},
113+
};
131114
```
132115

133-
Notes:
116+
##### `operation`
117+
118+
- Each service must provide an `operation: Operation<void>` which signals that the service has started.
119+
- The operation needs to be long-lived or return once a child process is started while it keeps the service running in the background (e.g., `useService` or `useChildSimulation`).
120+
- If you are defining your own customer operation, use `try { ... yield* suspend(); } finally { ... }` inside an `operation` to run cleanup logic when the service stops. Using `resource()` from `effection` allows the service to stay in scope and continue running. See the `effection` documentation or the helper functions in this library for more information and examples.
121+
122+
##### `dependsOn`
123+
124+
Type: `{ startup?: string[]; restart?: string[] }`
125+
126+
- `startup` lists services that must start before this one.
127+
- `restart` lists services whose restart should trigger a restart of this service (useful when using the watcher).
128+
129+
##### `watch` Watching & restart propagation
134130

135-
- `useServiceGraph` returns a _runner function_; calling the runner (e.g. `yield* run()`) returns an `Operation<void>` that holds while services run and only cancels on parent scope termination. The returned runner has a `.services` property for introspection and can be passed directly to `simulationCLI`.
136-
- If you want to start services sequentially or add more advanced concurrency control, compose operations yourself and use `spawn` to control how operations run.
131+
To enable file-watching: pass `{ watch: true }` to `useServiceGraph` options (second argument) and add `watch` paths to `ServiceDefinition` objects. The watcher computes transitive dependents (using `dependsOn.restart`) and emits restart updates so restarts propagate deterministically.
137132

138-
### Lifecycle hooks
133+
### ServiceRunner & returned values
139134

140-
Each `ServiceDefinition` supports lifecycle hook operations. These hooks run in the parent scope and are useful for performing orchestration tasks, logging, or writing sentinel files for integration tests. Hooks are `Operation<void>` as well.
135+
The runner returned by `useServiceGraph` is itself an operation. This allows it to be portable. Define it in one spot, then import it into any CLI, start scripts or test runners of your choosing at start it there. Optionally, it takes an argument, `subset`, to only start part of the graph.
136+
137+
##### `subset`
138+
139+
When calling the runner you may pass a subset (e.g., `yield* runner(['serviceA'])` or `yield* runner('serviceA')`, the latter being a comma separated list) to start only a subset of services. Any required startup dependencies are included automatically. This is particularly use when focusing on a specific feature / feedback loop, such as in a test. Only start the services you _actually_ need.
140+
141+
##### returns
142+
143+
The "runner" returns and exposes a `services` object when executed. The started graph exposes:
144+
145+
- `servicePorts` — a Map of service name => listening port when a service returns `{ port: number }` from its operation. This is convenient for tests to discover HTTP endpoints. Note that these are only filled in if the `operation` supports this functionality. The `useChildSimulation` and `useSimulation` both support it.
146+
- `services` - the object initially passed, useful for debugging
147+
- `serviceUpdates` and `serviceChanges` - both a `Stream` (see `effection`) of updates from the watcher, useful for debugging
148+
149+
### Simulation & process helpers 🔧
150+
151+
This package provides a few helpers to run simulations and external processes in common patterns:
152+
153+
#### useSimulation(name, factory)
154+
155+
`useSimulation(name: string, createFactory: (initData?: unknown) => FoundationSimulator)`
156+
157+
Run a simulator _in-process_ via a factory that returns a `FoundationSimulator` (or a Promise resolving to one). Useful when you want the simulator instance in the same Node process as the runner. This API _will_ allow watching and restarts, but these restarts will not pick up changes in your code, see `useChildSimulation`.
158+
159+
- If `globalData` is set on the runner, `useSimulation` will fetch it from the simulacrum gateway and pass it as the `initData` argument to your factory.
160+
- The factory should return a `FoundationSimulator` (see below). `useSimulation` calls `await simulator.listen()` to obtain `{ port }` and registers that port on `servicePorts`.
161+
162+
Example:
141163

142164
```ts
143-
const services = {
144-
A: {
145-
operation: (function* () {
146-
// start the service via useService or useChildSimulation
147-
yield* useService("A", "node --import tsx ./test/services/service-a.ts");
148-
// signal that the service is ready
149-
console.log("A has started");
150-
try {
151-
// keep running until cancelled
152-
yield* suspend();
153-
} finally {
154-
// cleanup runs automatically on scope cancellation
155-
console.log("A is stopping");
156-
}
157-
})(),
158-
},
159-
};
165+
// in a service definition
166+
operation: useSimulation("app", (initData) => {
167+
// do something with initData and/or pass it to your simulator through the closure
168+
return createFoundationSimulationServer({ port: 0 });
169+
});
160170
```
161171

162-
Notes:
172+
#### useChildSimulation(name, modulePath)
163173

164-
- Use a try/finally in your `operation` to run cleanup logic when the service is stopped
165-
- This approach leverages Effection scopes and ensures cleanup runs in reverse dependency order when the graph is shut down
166-
- Use `useService` or `useChildSimulation` inside your operation as needed to start underlying processes
174+
`useChildSimulation(name: string, modulePath: string)`
167175

168-
Try it
176+
Run a simulator in a fresh child Node process (isolates module cache and supports restarts). Otherwise this feels the same as using `useSimulation`.
169177

170-
```bash
171-
# Run the server package tests
172-
cd packages/server
173-
npm test
178+
- The child is started using a wrapper, `./bin/run-simulation-child.ts <modulePath>`, and, when present, the `--simulacrum-port` is passed so the child can fetch `globalData`.
179+
- The wrapper prints a JSON line to stdout like `{ "ready": true, "port": 12345 }` as its first ready signal. `useChildSimulation` reads that line to discover the port and registers it on `servicePorts`.
180+
- Non-JSON stdout lines are forwarded to logs; if the child exits before emitting the ready JSON, `useChildSimulation` rejects.
181+
- If using this with a simulator created from `@simulacrum/foundation-simulator`, all this wiring will be handled for you.
182+
183+
Example:
184+
185+
```ts
186+
operation: useChildSimulation(
187+
"service-key-for-logs",
188+
"./simulator/my-simulator.js"
189+
);
174190
```
175191

176-
## Examples
192+
> [!WARNING]
193+
> This does rely on having `tsx` installed which will handle the TypeScript types when running. It will allow for a simulator defined through a `.js` file or a `.ts`, so your choosing.
177194
178-
The `example` folder contains runnable examples demonstrating `useServiceGraph` and `useService`.
195+
#### About `@simulacrum/foundation-simulator`
179196

180-
Run the simulation-based example (starts simulators via child simulations):
197+
- A `FoundationSimulator` is a small helper that provides two key primitives you should expect from your factory:
198+
- `simulator.listen(): Promise<{ port: number }>` — starts the server and resolves when it is listening (the object is registered in `servicePorts`).
199+
- `simulator.ensureClose(): Promise<void>` — used by the runner to cleanly shut down the simulator when its containing scope is cancelled.
200+
- Use `createFoundationSimulationServer()` to create a server that listens on an ephemeral port and returns an object compatible with `useSimulation` and `useChildSimulation`.
181201

182-
```bash
183-
cd packages/server
184-
npm run example:sim
185-
```
202+
#### useService(name, cmd, options?)
186203

187-
Run the process-based example (spawns processes via `useService`):
204+
Spawn an external process (via the configured command) and optionally run a wellness check. `useService` forwards stdout/stderr to the package logging and keeps the operation alive until it goes out of scope.
188205

189-
```bash
190-
cd packages/server
191-
npm run example:process
192-
```
206+
- `options`:
207+
- `wellnessCheck.operation(stdio)` — an operation, `Operation<>` that needs to return a `Result` (both from `effection`) to consider the service successfully started. It is passed the stdio from the process. You may use any `effection` semantics, and inspect the stdio or http calls, etc, to decide when your service is "ready".
208+
- `wellnessCheck.timeout` and `wellnessCheck.frequency` can be provided to control checking behavior, most useful in repeatedly `fetch`ing a `/status` or `/healthcheck` response.
193209

194-
Run the concurrency example:
210+
#### simulationCLI(serviceGraph)
195211

196-
```bash
197-
cd packages/server
198-
npm run example:concurrency
199-
```
212+
- `simulationCLI` wraps the runner in a small CLI loop and provides convenience flags: `--services`, `--watch`, and `--watch-debounce`.
213+
- Use the CLI helper for local development workflows where you want to run your graph directly from a file (see `service-graph.ts` examples above).
200214

201-
Run examples directly (each example module can be executed with `tsx`):
215+
## Global data & the simulacrum gateway 🔁
202216

203-
```bash
204-
cd packages/server
205-
node --import tsx ./example/simulation-graph.ts
206-
node --import tsx ./example/process-graph.ts
207-
node --import tsx ./example/concurrency-layers.ts
217+
When you call `useServiceGraph(...)` you may pass an optional `globalData` object in the options. The runner starts a tiny local HTTP data service (the **simulacrum gateway**) that serves that object so tests and child simulations can discover configuration or shared fixtures.
218+
219+
- Endpoints: `GET /data` (returns the full `globalData` JSON) and `GET /data/<key>` (returns a single key, or a 404/400 as appropriate).
220+
- Discovery: the gateway registers its listening port on the runner's `servicePorts` map under the key `"simulacrum"`. You can read the port from your test or harness with `const port = services.servicePorts!.get("simulacrum");` and then `fetch` `http://127.0.0.1:${port}/data`.
221+
- Service integration: when starting child simulations via `useChildSimulation` / `simulationCLI` we pass the gateway port (if present) to the child. The child will fetch `/data` on startup and receive the `globalData` object. The simulator function you define may expect to receive that global object as the first argument to the function. Useful for passing "world-level" data to all of your simulators.
222+
223+
```ts
224+
const runner = useServiceGraph(
225+
{
226+
child: { operation: useChildSimulation("child", "./child-main.ts") },
227+
},
228+
{ globalData: { featureFlag: true } }
229+
);
230+
231+
const services = yield * runner();
232+
const simulacrumPort = services.servicePorts!.get("simulacrum");
233+
// fetch global data in a test or helper
234+
const res = await fetch(`http://127.0.0.1:${simulacrumPort}/data`);
235+
const data = await res.json();
208236
```
209237

210-
### Sharing exported values between services (note)
238+
Notes:
239+
240+
The gateway is intended for local development and tests only (it is not a production data layer). Future work around this layer may include improved logging and observability. Conceptually, it provides an "orchestration status" service.
211241

212-
Previously services could expose their return value via a public `exportsOperation` that consumers could await. That mechanism has been removed in this branch as we move to a child-process-focused runner model. Provider-returned values are still delivered to dependent service factories internally, but no longer exposed as an operation on the public `services` map.
242+
## Development
213243

214-
For convenience tests may use the `servicePorts` map exposed by the running graph to discover HTTP ports that services registered when they start. The `servicePorts` map is available on the object returned by the runner and contains service name => port when a service's `operation` returns an object with a `{ port: number }` property.
244+
The `example` folder contains runnable examples demonstrating `useServiceGraph`. The `test` folder includes tests based on the Node test runner which pull from the `example` folder or create their own fixtures to test the APIs.

0 commit comments

Comments
 (0)