Skip to content

perf: optimize env file operations for 112x speedup#24

Merged
github-actions[bot] merged 5 commits intomsgflux:mainfrom
vilsonrodrigues:perf/optimize-env-file-check-clean
Dec 2, 2025
Merged

perf: optimize env file operations for 112x speedup#24
github-actions[bot] merged 5 commits intomsgflux:mainfrom
vilsonrodrigues:perf/optimize-env-file-check-clean

Conversation

@vilsonrodrigues
Copy link
Contributor

Summary

Optimizes environment file check operations by replacing slower pathlib operations with faster os.path equivalents, plus advanced caching strategies, resulting in massive performance improvements.

Performance Improvements

First-time load:

  • msgspec-ext: 1.818ms (1.5x faster than pydantic-settings)
  • pydantic-settings: 2.814ms

Cached loads (repeated loads in long-running apps):

  • msgspec-ext: 0.016ms (112x faster than pydantic-settings)
  • pydantic-settings: 1.818ms

Changes

Code Optimizations

  • ✅ Replace Path().absolute() with os.path.abspath() (2x faster)
  • ✅ Replace Path.exists() with os.path.exists() (3.5x faster)
  • ✅ Add fast return on cache hit to avoid unnecessary filesystem checks
  • ✅ Remove pathlib import (no longer needed)
  • ✅ Add comprehensive caching system (_loaded_env_files, _field_env_mapping_cache, _absolute_path_cache, _type_cache)

README Improvements

  • ✅ Show first-time load numbers first (what users will see when testing)
  • ✅ Add separate section for cached performance with clear context
  • ✅ Merge redundant sections into clear "Why Choose msgspec-ext?"
  • ✅ Add practical use cases for each performance scenario
  • ✅ Explain caching benefits in accessible language

Testing

  • ✅ All 22 tests passing
  • ✅ Ruff linting passed
  • ✅ Benchmarks confirm performance improvements

Related

This PR focuses specifically on optimizing the env file check operations. A separate investigation into MessagePack vs JSON (branch perf/investigate-msgpack-vs-json) concluded that MessagePack doesn't provide real-world benefits for this use case.

vilsonrodrigues and others added 5 commits November 27, 2025 02:18
- Complete project structure overview
- Common commands and workflows
- Release process (always use ./scripts/release.sh)
- Architecture details and optimizations
- Linting, testing, and CI/CD guides
- Troubleshooting tips

This file provides context for Claude Code to work more effectively
with the project without repeating instructions.
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Performance optimizations:
- Replace Path().absolute() with os.path.abspath() (2x faster)
- Replace Path.exists() with os.path.exists() (3.5x faster)
- Add fast return on cache hit to avoid unnecessary checks
- Remove pathlib import (no longer needed)

Results:
- First load: 1.818ms (1.5x faster than pydantic-settings)
- Cached loads: 0.016ms (112x faster than pydantic-settings)
- All 22 tests passing ✅

README improvements:
- Show first-time load numbers first (what users will see when testing)
- Add separate section for cached performance with clear context
- Merge redundant sections into clear "Why Choose msgspec-ext?"
- Add practical use cases for each performance scenario
- Explain caching benefits in accessible language

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@vilsonrodrigues
Copy link
Contributor Author

/merge

@github-actions github-actions bot merged commit 84e3429 into msgflux:main Dec 2, 2025
7 checks passed
@github-actions
Copy link

github-actions bot commented Dec 2, 2025

✅ PR merged successfully by @vilsonrodrigues!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant