Skip to content

module‐structure

Jump to bottom
Andrei G edited this page Aug 12, 2025 · 2 revisions

Module Structure

This diagram shows the detailed module dependencies and boundaries within the PJS codebase.

graph TB
    subgraph "Workspace Root"
        subgraph "pjs-core (Main Library)"
            subgraph "Domain Layer"
                DomainEntities[entities/<br/>- stream_session.rs<br/>- stream.rs<br/>- frame.rs]
                DomainValues[value_objects/<br/>- priority.rs ⭐<br/>- json_path.rs<br/>- session_id.rs<br/>- stream_id.rs]
                DomainEvents[events/<br/>- session_events.rs<br/>- stream_events.rs<br/>- priority_distribution.rs ⭐]
                DomainServices[services/<br/>- priority_calculator.rs<br/>- frame_generator.rs]
                DomainErrors[errors/<br/>- domain_error.rs<br/>- validation_error.rs]
            end

            subgraph "Application Layer"
                AppCommands[commands/<br/>- create_session.rs<br/>- add_stream.rs<br/>- complete_session.rs]
                AppQueries[queries/<br/>- session_query.rs<br/>- stream_query.rs<br/>- metrics_query.rs]
                AppServices[services/<br/>- session_service.rs<br/>- streaming_service.rs<br/>- metrics_service.rs]
                AppShared[shared.rs ⭐<br/>- AdjustmentUrgency<br/>- SharedTypes]
                AppDTOs[dtos/<br/>- session_dto.rs<br/>- stream_dto.rs<br/>- frame_dto.rs]
            end

            subgraph "Infrastructure Layer"
                InfraHTTP[http/<br/>- axum_handlers.rs<br/>- streaming_routes.rs<br/>- middleware.rs]
                InfraWS[websocket/<br/>- connection_manager.rs<br/>- message_handler.rs]
                InfraRepo[repositories/<br/>- memory_session_repo.rs<br/>- memory_stream_repo.rs]
                InfraMetrics[metrics/<br/>- prometheus_exporter.rs<br/>- performance_tracker.rs]
            end

            subgraph "Core Processing"
                StreamCore[stream/<br/>- priority.rs → domain/value_objects<br/>- frame_processor.rs<br/>- json_reconstructor.rs ⭐<br/>- stream_orchestrator.rs]
                ParsingCore[parsing/<br/>- sonic_parser.rs<br/>- simd_accelerated.rs<br/>- semantic_analyzer.rs]
                CompressionCore[compression/<br/>- schema_aware.rs<br/>- dictionary_encoder.rs<br/>- adaptive_compressor.rs]
            end

            subgraph "Universal Framework Integration"
                FrameworkCore[framework/<br/>- streaming_adapter.rs<br/>- universal_adapter.rs<br/>- integration_macros.rs]
                FrameworkExamples[examples/<br/>- axum_integration.rs<br/>- actix_integration.rs<br/>- warp_integration.rs]
            end
        end

        subgraph "pjs-bench (Benchmarking)"
            BenchLib[lib.rs<br/>- Re-exports from pjs-core ⭐<br/>- BenchmarkSuite]
            BenchTests[benches/<br/>- parsing_benchmarks.rs<br/>- streaming_benchmarks.rs<br/>- memory_benchmarks.rs]
            BenchUtils[utils/<br/>- data_generators.rs<br/>- performance_helpers.rs]
        end

        subgraph "pjs-demo (Examples)"
            DemoLib[lib.rs<br/>- DemoJsonReconstructor ⭐<br/>- Demo utilities]
            DemoExamples[examples/<br/>- basic_streaming.rs<br/>- priority_demo.rs<br/>- performance_comparison.rs]
            DemoData[data/<br/>- sample_large.json<br/>- sample_nested.json<br/>- sample_arrays.json]
        end

        subgraph "pjs-js-client (TypeScript)"
            JSCore[src/core/<br/>- client.ts<br/>- frame-processor.ts<br/>- json-reconstructor.ts]
            JSTransport[src/transport/<br/>- http.ts<br/>- websocket.ts<br/>- sse.ts]
            JSTypes[src/types/<br/>- priority.ts<br/>- frame.ts<br/>- client-config.ts]
            JSUtils[src/utils/<br/>- validation.ts<br/>- json-path.ts]
        end

    end

    %% Dependencies (Clean Architecture)
    InfraHTTP -.-> AppServices
    InfraWS -.-> AppServices
    InfraRepo -.-> DomainEntities
    InfraMetrics -.-> DomainEvents

    AppCommands -.-> DomainServices
    AppQueries -.-> DomainEntities
    AppServices -.-> DomainEntities
    AppDTOs -.-> DomainValues

    StreamCore -.-> DomainValues
    ParsingCore -.-> DomainValues
    CompressionCore -.-> DomainEvents

    FrameworkCore -.-> AppServices
    FrameworkExamples -.-> FrameworkCore

    %% Cross-crate dependencies
    BenchLib -.-> DomainValues
    BenchLib -.-> StreamCore
    BenchTests -.-> BenchLib

    DemoLib -.-> DomainValues
    DemoExamples -.-> DemoLib

    %% External dependencies
    ParsingCore -.-> SonicRS[sonic-rs<br/>SIMD JSON]
    InfraHTTP -.-> Axum[axum<br/>HTTP framework]
    InfraWS -.-> Tokio[tokio-tungstenite<br/>WebSocket]
    InfraMetrics -.-> Prometheus[prometheus<br/>Metrics]

    %% Shared type dependencies (showing deduplication results)
    StreamCore -.-> DomainValues
    BenchLib -.-> DomainValues
    AppShared -.-> DomainValues
    DemoLib -.-> DomainValues

    %% Key architectural boundaries
    classDef domainLayer fill:#e1f5fe,stroke:#01579b,stroke-width:2px
    classDef applicationLayer fill:#f3e5f5,stroke:#4a148c,stroke-width:2px
    classDef infrastructureLayer fill:#e8f5e8,stroke:#1b5e20,stroke-width:2px
    classDef processingLayer fill:#fce4ec,stroke:#880e4f,stroke-width:2px
    classDef clientLayer fill:#fff3e0,stroke:#e65100,stroke-width:2px
    classDef externalLayer fill:#f5f5f5,stroke:#424242,stroke-width:2px
    classDef deduplicatedType fill:#fff9c4,stroke:#f57f17,stroke-width:3px

    class DomainEntities,DomainValues,DomainEvents,DomainServices,DomainErrors domainLayer
    class AppCommands,AppQueries,AppServices,AppShared,AppDTOs applicationLayer
    class InfraHTTP,InfraWS,InfraRepo,InfraMetrics infrastructureLayer
    class StreamCore,ParsingCore,CompressionCore,FrameworkCore processingLayer
    class JSCore,JSTransport,JSTypes,JSUtils,BenchLib,DemoLib clientLayer
    class SonicRS,Axum,Tokio,Prometheus externalLayer
    class DomainValues,DomainEvents,AppShared,StreamCore deduplicatedType
Loading

Module Responsibilities

Domain Layer (Core Business Logic)

  • entities/: Aggregate roots and entities with identity and lifecycle
  • value_objects/: Immutable objects with equality semantics ⭐ Deduplicated
  • events/: Domain events for decoupled communication ⭐ Deduplicated
  • services/: Complex business logic that doesn't belong to entities
  • errors/: Domain-specific error types

Application Layer (Use Cases)

  • commands/: Write operations following CQRS pattern
  • queries/: Read operations with optimized data models
  • services/: Use case orchestration and workflows
  • shared.rs: Shared application types ⭐ Deduplicated
  • dtos/: Data transfer objects for API boundaries

Infrastructure Layer (External Concerns)

  • http/: HTTP transport with Axum framework integration
  • websocket/: WebSocket real-time communication
  • repositories/: Data persistence abstractions and implementations
  • metrics/: Performance monitoring and observability

Core Processing (Performance-Critical)

  • stream/: Priority streaming and frame processing ⭐ Deduplicated
  • parsing/: SIMD-accelerated JSON parsing with sonic-rs
  • compression/: Schema-aware compression algorithms

Framework Integration (Universal Adapter)

  • framework/: Universal adapter for any web framework
  • examples/: Integration examples for popular frameworks

Key Architectural Principles

1. Dependency Inversion

All dependencies point inward toward the domain layer. Infrastructure depends on application, application depends on domain.

2. Single Source of Truth

⭐ Deduplication Results:

  • Priority enum: Unified in domain/value_objects/priority.rs
  • PriorityDistribution: Canonical in domain/events/
  • AdjustmentUrgency: Moved to application/shared.rs
  • JsonReconstructor: Main version exported, others import or renamed

3. Clean Boundaries

Each layer has well-defined interfaces and responsibilities. Cross-layer communication happens through defined contracts.

4. Framework Agnostic

Universal adapter pattern allows integration with any web framework without tight coupling.

5. Performance Isolation

Performance-critical code (parsing, compression) is isolated in dedicated modules with optimized implementations.

External Dependencies

Core Performance

  • sonic-rs: SIMD-accelerated JSON parsing
  • rayon: Data parallelism for multi-threaded processing
  • crossbeam: Lock-free data structures

Web Framework Integration

  • axum: Modern async web framework
  • tokio: Async runtime
  • tower: Middleware and service abstractions

Observability

  • prometheus: Metrics collection and export
  • tracing: Structured logging and spans

Module Interaction Patterns

1. Command Flow

HTTP Handler → Application Service → Domain Service → Entity → Repository

2. Query Flow

HTTP Handler → Query Service → DTO Mapper → Repository → Response

3. Event Flow

Domain Entity → Domain Event → Application Handler → Infrastructure Action

4. Processing Pipeline

Input → Parser → Analyzer → Prioritizer → Compressor → Transport

Development Guidelines

Adding New Features

  1. Start with domain modeling (entities, value objects)
  2. Define application services and DTOs
  3. Implement infrastructure adapters
  4. Add framework integration if needed

Performance Considerations

  • Keep hot paths in the processing layer
  • Use zero-copy operations where possible
  • Leverage SIMD acceleration for data-intensive tasks
  • Profile regularly with the benchmarking suite

Testing Strategy

  • Unit tests for domain logic
  • Integration tests for application services
  • Performance tests in the bench crate
  • End-to-end tests in the demo crate
Clone this wiki locally