Skip to content

Add file-sync example project foundation#580

Draft
schickling wants to merge 3 commits intodevfrom
feature/file-sync-foundation
Draft

Add file-sync example project foundation#580
schickling wants to merge 3 commits intodevfrom
feature/file-sync-foundation

Conversation

@schickling
Copy link
Collaborator

Summary

Add comprehensive file synchronization CLI project using LiveStore and Effect as a new example in examples/file-sync/.

  • Bidirectional sync: Changes in either folder sync to the other
  • One-time or continuous: Support both sync and watch modes
  • Conflict resolution: Graceful handling of concurrent modifications
  • User vs app change detection: Differentiate user changes from sync operations
  • Full audit trail: Event sourcing provides complete change history
  • Type-safe: Built with Effect for comprehensive error handling

What's Included

📋 Comprehensive Design Plan

  • PLAN.md: 200+ line technical specification covering complete system design
  • Event schema design with file lifecycle, sync operations, and conflict handling
  • SQLite state schema for files, conflicts, and sync metadata
  • Materializers for transforming events to queryable state
  • Service architecture breakdown
  • Implementation phases and success criteria

🏗️ Project Foundation

  • Complete TypeScript project structure with proper configuration
  • Service-oriented architecture: FileWatcher, SyncEngine, ConflictResolver services
  • CLI framework using Effect CLI with subcommands (sync, watch, status, conflicts)
  • Comprehensive placeholder implementations with detailed TODO comments
  • Proper package.json with all necessary dependencies and build scripts

🎯 LiveStore Integration

  • Event sourcing patterns following project conventions
  • SQLite materializers for state management
  • Cross-platform file system monitoring (chokidar)
  • Effect-based error handling and concurrency
  • Type-safe schema definitions throughout

Architecture Highlights

FileWatcherService    ─┐
                      ├─► SyncEngineService ─► ConflictResolverService
File System Changes  ─┘                     │
                                            ▼
                      LiveStore Events ─► SQLite State ─► CLI Output

Event Sourcing Design

  • File lifecycle events: FileDetected, FileModified, FileDeleted, FileMoved
  • Sync operation events: SyncIntent, SyncCompleted, SyncFailed
  • Conflict handling events: ConflictDetected, ConflictResolved

Key Features Planned

  • User vs app change detection via sync intent tracking and timestamp windows
  • Multiple conflict resolution strategies (newest, largest, interactive)
  • Cross-platform compatibility with proper path normalization
  • Performance optimizations for large directories and files

Implementation Status

  • Phase 1: Project foundation and comprehensive design (This PR)
  • Phase 2: Event Schema & State implementation
  • Phase 3: File System Integration with chokidar
  • Phase 4: Core Sync Engine logic
  • Phase 5: Conflict Detection & Resolution
  • Phase 6: CLI Commands implementation
  • Phase 7: Advanced features and optimizations
  • Phase 8: Testing & Documentation

Files Added

examples/file-sync/
├── README.md              # Quick start and overview
├── PLAN.md                # Comprehensive technical design (200+ lines)  
├── package.json           # Dependencies and scripts
├── tsconfig.json          # TypeScript configuration
└── src/
    ├── main.ts            # CLI entry point with Effect CLI
    ├── commands/          # CLI command implementations
    │   ├── sync.ts        # One-time sync command
    │   ├── watch.ts       # Continuous watch mode
    │   ├── status.ts      # Status display
    │   └── conflicts.ts   # Conflict management
    ├── services/          # Core business logic services
    │   ├── file-watcher.ts    # File system monitoring
    │   ├── sync-engine.ts     # Bidirectional sync logic
    │   └── conflict-resolver.ts # Conflict detection/resolution
    ├── livestore/         # LiveStore integration
    │   ├── schema.ts      # Events, state, materializers
    │   └── queries.ts     # Database queries
    └── utils/            # Utility functions
        ├── file-utils.ts     # File system operations
        ├── path-utils.ts     # Path normalization  
        └── cli-utils.ts      # CLI formatting helpers

Test Plan

This PR establishes the foundation. Future PRs will add:

  1. Unit tests for each service and utility module
  2. Integration tests with real file systems and edge cases
  3. Performance benchmarks for large directories
  4. Cross-platform testing (macOS, Linux, Windows)

Learning Value

This example demonstrates:

  • Advanced LiveStore patterns for real-world applications
  • Effect service architecture with proper dependency injection
  • Complex event sourcing with conflict resolution
  • Cross-platform Node.js development best practices
  • CLI application design with comprehensive user experience

The project serves as both a useful file sync tool and a comprehensive reference for building complex LiveStore applications.


📚 Next Steps: Review the PLAN.md for technical details, then begin Phase 2 implementation of the event schema and state management.

schickling and others added 3 commits August 27, 2025 19:14
Add comprehensive file synchronization CLI project using LiveStore and Effect.

Features planned:
- Bidirectional file sync between directories
- One-time sync and continuous watch modes
- Conflict detection and resolution
- Event sourcing with full audit trail
- User vs app change differentiation
- Cross-platform file system monitoring

Project structure:
- Complete design documentation in PLAN.md
- TypeScript CLI with Effect and LiveStore integration
- Service architecture for FileWatcher, SyncEngine, ConflictResolver
- Placeholder implementations with detailed TODOs
- Proper package.json with dependencies and build scripts

This establishes the foundation for implementing a sophisticated file sync
system that demonstrates advanced LiveStore patterns and Effect usage.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Add file-sync project dependencies to pnpm lockfile to resolve CI issues.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Add file-sync project to tsconfig.examples.json references
- Fix TypeScript module configuration (ESNext -> NodeNext)
- Fix string arithmetic error in status command
- Remove invalid summary property from CLI configuration

This ensures the file-sync project is included in the overall TypeScript build
process and compiles without errors.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@alvinometric
Copy link

This is fantastic, I hope it gets finished 🙏

@schickling
Copy link
Collaborator Author

schickling commented Dec 5, 2025

Heads-up that this PR overlaps with the roadmap item in #352, which tracks supporting multiple state targets beyond SQLite. Instead of hand-rolling file synchronization logic here, we could treat the file system itself as another state target: LiveStore would materialize directly into the local tree just like it already does with SQLite. Once that adapter exists, the example could be drastically cleaner because we would just compose a new 'State.Filesystem' target rather than wedging file I/O into the example project.\n\n_Comment added via Conductor assistant on behalf of @schickling._

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants