Release v0.12.0: Size Validation & Warnings by schovi · Pull Request #58 · schovi/baked_file_system

schovi · 2025-11-03T16:45:07Z

Problem

BakedFileSystem had no mechanism to prevent accidentally embedding huge files that bloat binary size. Users could unknowingly create multi-hundred-megabyte binaries without any warnings or feedback during compilation.

Solution

Implemented comprehensive compile-time size validation with configurable limits (TASK.6):

Automatic size tracking: Monitors total and per-file compressed sizes during compilation
Default limits: 50MB max total size (configurable), 10MB warning threshold
Environment variables: BAKED_FILE_SYSTEM_MAX_SIZE and BAKED_FILE_SYSTEM_WARN_THRESHOLD
Per-folder control: max_size parameter for granular limits
Clear reporting: Displays file count, compression ratio, and size statistics
Actionable errors: Helpful messages guide users to solutions

Breaking Change

⚠️ This is a breaking change - builds will now fail at compile-time if embedded files exceed 50MB compressed (default).

Migration options:

# Option 1: Environment variable (temporary)
BAKED_FILE_SYSTEM_MAX_SIZE=104857600 crystal build app.cr

# Option 2: Per-folder parameter (recommended)
bake_folder "./assets", max_size: 104_857_600  # 100MB

Changes

New Files

src/loader/stats.cr - Size tracking and validation logic
src/loader/byte_counter.cr - IO wrapper for accurate byte counting
examples/ - Working demonstrations of validation features
CHANGELOG.md - Comprehensive changelog with migration guide

Modified Files

src/baked_file_system.cr - Added max_size parameter to bake_folder macro
src/loader.cr - Parse max_size from command-line arguments
src/loader/loader.cr - Integrated size tracking and reporting
spec/loader_spec.cr - 174 new lines of comprehensive test coverage
README.md - Added "Size Management & Limits" documentation
shard.yml - Version bump to 0.12.0
src/baked_file_system/version.cr - Version bump to 0.12.0

Test Coverage

Stats calculation (file count, sizes, compression ratio)
Large file detection and warnings
Environment variable configuration
Max size limit enforcement
ByteCounter functionality
All 41 specs passing

Other

Version Strategy

Following SemVer for pre-1.0 releases where breaking changes bump MINOR version:

Previous: v0.11.0 (June 2025)
This release: v0.12.0 (November 2025)
Consistent with historical pattern (v0.9.6 → v0.10.0 removed mime_type)

Release Assets

Git tag: v0.12.0 (pushed)
CHANGELOG.md with detailed migration guide
Working examples demonstrating new features

Why This Change?

Safety: Prevents accidental huge binaries
Feedback: Early detection during compilation vs runtime surprises
Flexibility: Configurable per-project or per-folder
Developer experience: Clear error messages with solutions

See CHANGELOG.md for complete release notes and migration guide.

…t, and documentation This commit implements several code quality improvements: **Error Handling:** - Add ReadOnlyError exception for write attempts with descriptive messages - Add DuplicatePathError exception to prevent duplicate file paths - Replace generic string errors with properly typed exceptions **Resource Management:** - Implement close() and closed?() methods following IO conventions - Add block form to get/get? for automatic resource cleanup - Add finalize for garbage collection safety - Validate file state before operations **Thread Safety:** - Create new BakedFile instances on each get() call - Ensures independent state per caller, preventing conflicts **Code Quality:** - Refactor StringEncoder to use readable character literals - Replace magic numbers with self-documenting code **Documentation:** - Add comprehensive architecture documentation to BakedFile class - Document rewind recreation logic with performance implications - Explain design decisions and alternatives considered - Add detailed method documentation **Testing:** - Add tests for write protection and error messages - Add tests for close functionality and block forms - Add tests for duplicate path detection - All tests passing These improvements enhance maintainability, thread safety, and developer experience while maintaining backward compatibility. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

Add baseline and baked benchmark applications to measure BakedFileSystem performance compared to traditional File I/O. - Baseline app: simple Kemal server with File I/O (baseline.cr, 8 lines) - Baked app: Kemal server with BakedFileSystem (baked.cr, 16 lines) - Shared test assets: small.txt (1.1KB), medium.json (170KB), large.dat (1MB) - Add .gitignore to exclude binaries, debug files, and dependencies Both apps are minimal single-file implementations serving files on /files/:name endpoint for accurate benchmarking.

Implements complete benchmarking framework for BakedFileSystem: TASK.38.3: Compile Time Benchmarking - compile_time.cr: Measures compilation overhead - Runs 5 iterations for statistical validity - Outputs results/compile_time.json TASK.38.4: Binary Size Analysis - binary_size.cr: Analyzes binary size and compression - Calculates overhead factor and compression ratio - Outputs results/binary_size.json TASK.38.5: Memory Usage Benchmarking - memory.cr: Profiles RSS at various stages - Measures startup, file access, and post-GC memory - Outputs results/memory.json TASK.38.6: Performance Benchmarking - performance.cr: Measures latency and throughput - Tests small/medium/large files - 1000 requests with 10 concurrent clients - Outputs results/performance.json TASK.38.7: Report Generator - report_generator.cr: Aggregates all results - Generates comprehensive markdown report - Includes tables, ASCII charts, and conclusions - Outputs results/REPORT.md Supporting Files: - run_all.sh: Master script to run all benchmarks - README.md: Complete usage documentation - .gitignore: Excludes JSON/MD results from git All scripts properly handle: - Process error capture - Server lifecycle management - Port cleanup - Statistical analysis - Cross-platform compatibility 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

Process#pid returns Int64, so get_rss_mb must accept Int64 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

Replace all numeric sleep calls with Time::Span: - sleep 0.1 → sleep 0.1.seconds - sleep 1 → sleep 1.second - sleep 2 → sleep 2.seconds - sleep WARMUP_DELAY → sleep WARMUP_DELAY.seconds Fixes Crystal 1.17+ deprecation warnings. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

Add comprehensive Benchmarks section with: - Link to benchmarks/README.md for details - System specifications - Compile time: +3.7s (30% overhead) - Binary size: 0.88x compression ratio - Memory: minimal overhead with lazy decompression - Performance: comparable latency/throughput - Clear recommendations on when to use BakedFileSystem Provides users with objective performance data to make informed decisions about using BakedFileSystem. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

Add comprehensive compile-time size validation with configurable limits: - New Stats class tracks total and per-file compressed sizes - ByteCounter IO wrapper for accurate size tracking during encoding - Environment variables for global configuration: - BAKED_FILE_SYSTEM_MAX_SIZE (default: 50MB) - BAKED_FILE_SYSTEM_WARN_THRESHOLD (default: 10MB) - Per-folder max_size parameter for granular control - Large file warnings (>1MB compressed) - Clear error messages with actionable guidance - Comprehensive test coverage for all validation scenarios - Examples demonstrating warnings and size limit enforcement Breaking change: Builds may fail if embedded files exceed default 50MB limit. Configure via environment variable or max_size parameter to override.

BREAKING CHANGE: Compile-time size validation now enforced This release introduces automatic size validation for embedded files with a default maximum of 50MB compressed. Builds will fail at compile-time if this limit is exceeded. Migration options: - Environment variable: BAKED_FILE_SYSTEM_MAX_SIZE=<bytes> - Per-folder parameter: bake_folder "./assets", max_size: <bytes> See CHANGELOG.md for detailed migration guide. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

Sija · 2025-11-08T00:16:02Z

src/baked_file_system.cr

+    def self.add_baked_file(file : BakedFileSystem::BakedFile)
+      if @@paths.includes?(file.path)
+        raise BakedFileSystem::DuplicatePathError.new("Duplicate file path: #{file.path}. File already baked.")
+      end
+      @@paths << file.path
+      @@files << file
    end

    def self.add_baked_file(file : BakedFileSystem::BakedFile)
      if @@paths.includes?(file.path)
        raise BakedFileSystem::DuplicatePathError.new("Duplicate file path: #{file.path}. File already baked.")
      end
      @@paths << file.path
      @@files << file
    end
  end


Duplicated method definition.

Sija · 2025-11-08T00:18:46Z

src/loader/stats.cr

+      private def human_size(bytes : Int64) : String
+        return "#{bytes} B" if bytes < 1024
+        return "#{(bytes / 1024.0).round(1)} KB" if bytes < 1024 * 1024
+        return "#{(bytes / (1024.0 * 1024)).round(1)} MB" if bytes < 1024 * 1024 * 1024
+        "#{(bytes / (1024.0 * 1024 * 1024)).round(1)} GB"
+      end


There's already Number#humanize for this use case.

Sija · 2025-11-08T00:20:21Z

src/loader/stats.cr

+        value = ENV[var_name]?
+        return nil unless value
+
+        value.to_i64?


Can be shortened to:

Suggested change

value = ENV[var_name]?

return nil unless value

value.to_i64?

ENV[var_name]?.try(&.to_i64?)

schovi and others added 9 commits November 2, 2025 20:49

fix: correct pid type in memory benchmark (Int64 not Int32)

27a1f9c

Process#pid returns Int64, so get_rss_mb must accept Int64 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

Merge branch 'master' into size-limits

1eea012

schovi merged commit 0a434bd into master Nov 4, 2025
12 checks passed

schovi deleted the size-limits branch November 4, 2025 09:27

Sija reviewed Nov 8, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Release v0.12.0: Size Validation & Warnings#58

Release v0.12.0: Size Validation & Warnings#58
schovi merged 9 commits intomasterfrom
size-limits

schovi commented Nov 3, 2025 •

edited

Loading

Uh oh!

Uh oh!

Sija Nov 8, 2025

Uh oh!

Sija Nov 8, 2025

Uh oh!

Sija Nov 8, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

schovi commented Nov 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Solution

Breaking Change

Changes

New Files

Modified Files

Test Coverage

Other

Version Strategy

Release Assets

Why This Change?

Uh oh!

Uh oh!

Sija Nov 8, 2025

Choose a reason for hiding this comment

Uh oh!

Sija Nov 8, 2025

Choose a reason for hiding this comment

Uh oh!

Sija Nov 8, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

schovi commented Nov 3, 2025 •

edited

Loading