Migrate from src/lib/fsp.ts to @wollybeard/kit's fs and fs-loc modules

[jasonkuhrt]

Summary

Migrate from custom src/lib/fsp.ts to @wollybeard/kit's Effect-based fs and fs-loc modules. This is a major architectural change requiring conversion from Promise-based to Effect-based filesystem operations.

Motivation

• Reduce internal code to maintain (70 lines → delete)
• Gain type-safe filesystem paths (FsLoc branded types)
• Better error handling with Effect's error channel
• Auto-creates parent directories (already built-in)
• Standardization with kit usage across projects

Current State

fsp.ts Exports (8 functions)

• Fs (type)
• statMaybeExists(fs, path)
• fileExists(fs, path)
• isPathToADirectory(fs, path)
• toAbsolutePath(cwd, path)
• toFilePath(fileName, path)
• readJsonFile<T>(fs, path)
• writeFileAndCreateDir(fs, filePath, content)

Files Affected (5)

1. src/generator/configFile/loader.ts - Effect migration
2. src/generator/config/configInit.ts - Type change only
3. src/generator/config/config.ts - Major Effect migration
4. src/generator/config/config.test.ts - Test framework change
5. src/cli/index.ts - Simple path replacement

Function Mapping

Current fsp.ts	Kit Equivalent	Complexity
fileExists(fs, path)	Fs.exists(fsLoc)	Simple
toAbsolutePath(cwd, path)	FsLoc.toAbs(rel, base)	Simple
writeFileAndCreateDir	Fs.write(fsLoc, content)	Simple
isPathToADirectory	Fs.stat(loc).pipe(Effect.map(s => s.type === 'Directory'))	Medium
readJsonFile	Fs.readString(loc).pipe(Effect.map(JSON.parse), Effect.option)	Medium
toFilePath	Custom logic + FsLoc.join	Medium
statMaybeExists	Fs.stat(loc).pipe(Effect.option)	Medium

Breaking Changes

1. ConfigInit.fs Type Change

// Before
export interface ConfigInit {
  fs?: Fs | undefined  // Node.js fs/promises
}

// After
export interface ConfigInit {
  fs?: FileSystem.FileSystem | undefined  // Effect service
}

2. Functions Return Effect

// Before
export const createConfig = async (config: ConfigInit): Promise<Config> => { ... }

// After
export const createConfig = (config: ConfigInit): Effect.Effect<Config, Error, FileSystem.FileSystem> =>
  Effect.gen(function*() { ... })

3. Call Sites Must Use Effect.runPromise

// Before
const config = await createConfig(configInit)

// After
import { NodeFileSystem } from '@effect/platform-node'
const config = await Effect.runPromise(
  createConfig(configInit).pipe(Effect.provide(NodeFileSystem.layer))
)

Example Transformation

Before (Promise-based)

import { isPathToADirectory, toAbsolutePath } from '#src/lib/fsp.js'

export const load = async (input?: Input): Promise<...> => {
  const fs = await import(`node:fs/promises`)
  const absolutePath = toAbsolutePath(process.cwd(), input)
  if (await isPathToADirectory(fs, absolutePath)) {
    // ...
  }
}

After (Effect-based)

import { Fs } from '@wollybeard/kit/fs'
import { FsLoc } from '@wollybeard/kit/fs-loc'
import { Effect } from 'effect'
import { FileSystem } from '@effect/platform'

export const load = (input?: Input): Effect.Effect<..., Error, FileSystem.FileSystem> =>
  Effect.gen(function*() {
    const cwd = FsLoc.AbsDir.decodeStringSync(process.cwd() + '/')
    const absolutePath = FsLoc.toAbs(FsLoc.RelFile.decodeStringSync(`./${input}`), cwd)
    const statInfo = yield* Fs.stat(absolutePath)
    if (statInfo.type === 'Directory') {
      // ...
    }
  })

// Call site
const result = await Effect.runPromise(
  load(input).pipe(Effect.provide(NodeFileSystem.layer))
)

Implementation Plan

Phase 1: Setup (1h)

• Create src/lib/fs-helpers.ts with string ↔ FsLoc conversion helpers
• Create src/test/effect-helpers.ts with test utilities
• Add equivalence tests proving Kit APIs match current behavior

Phase 2: Migrate Tests (2h)

• Update config.test.ts to use @wollybeard/kit/fs-memory instead of memfs
• Replace writeFileAndCreateDir with Fs.write
• Add Effect.runPromise wrappers

Phase 3: Simple Files (1h)

• Migrate cli/index.ts - only uses toAbsolutePath
• Migrate configInit.ts - type-only change

Phase 4: Complex Files (4h)

• Migrate config.ts - largest migration
  • Convert createConfig to Effect.gen
  • Convert createConfigSchema to Effect.gen
  • Replace all fsp functions with Kit equivalents
• Migrate loader.ts - depends on config.ts changes

Phase 5: Call Sites (2h)

• Add Effect.runPromise at all boundaries
• Provide NodeFileSystem.layer
• Update error handling

Phase 6: Integration & Cleanup (3h)

• Run generator against test schemas
• Compare outputs before/after
• Test CLI with various flags
• Delete src/lib/fsp.ts
• Remove #lib/fsp import alias from package.json

Phase 7: Documentation (1h)

• Update API docs for breaking changes
• Add migration guide for external users
• Document Effect usage patterns

Risks

High Risk

• Effect runtime layer propagation - Must be correct or runtime errors
• Path type conversions - FsLoc is strict, strings are loose
• Breaking API changes - External consumers will break

Medium Risk

• Test migration - memfs to fs-memory transition
• Error handling changes - Different patterns
• Performance - Effect overhead (likely negligible)

Low Risk

• Pure functions - toAbsolutePath etc. are straightforward
• Type safety - FsLoc improves type safety

Timeline Estimate

• Setup: 1h
• Migrate tests: 2h
• Simple files: 1h
• Complex files: 4h
• Call sites: 2h
• Integration & cleanup: 3h
• Documentation: 1h
• Buffer: 2h

Total: ~16 hours

Success Criteria

• All tests pass
• Generator produces identical output for test schemas
• CLI works with all flag combinations
• No runtime errors related to filesystem operations
• Type checker passes
• Performance within 10% of baseline
• Documentation updated
• No fsp.ts imports remaining in codebase

Rollback Plan

• Keep commits atomic per file
• Revert individually if issues arise
• Alternative: Keep fsp.ts alongside Kit temporarily for gradual migration

[lnmp4000]

Not sure if this is related to this change, but my tests have just started to fail following updating to @latest with the following error:
It may be a coincidence and on Monday I'll take the time to rollback versions, I'm making the link between @wollybeard/kit and this work item.

 Cannot find module '@vltpkg/semver' from '../node_modules/@wollybeard/kit/build/domains/semver/official-release.js'

    Require stack:
      /xxx/node_modules/@wollybeard/kit/build/domains/semver/official-release.js
      /xxx/node_modules/@wollybeard/kit/build/domains/semver/$$.js
      /xxx/node_modules/@wollybeard/kit/build/domains/semver/$.js
      /xxx/node_modules/@wollybeard/kit/build/exports/index.js
      /xxx/node_modules/graffle/build/lib/grafaid/typed-document/TypedDocument.js
      /xxx/node_modules/graffle/build/lib/grafaid/typed-document/$.js
      /xxx/node_modules/graffle/build/exports/index.js
      /xxx/pact-utils/dist/lib/pact-utils.js
      /xxx/pact-utils/dist/index.js
      __tests__/pact/me-test.js

I have manually installed @vltpkg/semver but they continue to fail.

[jasonkuhrt]

Thanks @lnmp4000 I'll check this out!