Optimise CI to reduce runners usage, caching NPM modules when needed

[mdelapenya]

What is this PR doing?

This PR implements two major workflow optimizations:

1. Intelligent Module Detection and Selective Testing

• Adds a changed-modules.sh script that analyzes which modules have been modified in a PR
• Only runs tests for affected modules instead of testing everything
• Automatically detects if core changes require testing all modules
• Excludes certain paths (like docs and .github templates) from triggering unnecessary builds
• Creates different stages that optimise the runner utilisation, not using all of them at the same time.

2. Optimized NPM Dependency Management

• Implements a reusable npm-setup composite action for consistent dependency installation
• Adds smart caching of node_modules at multiple levels:
  • Root level dependencies
  • Core testcontainers module
  • Individual module-specific dependencies
• Cache keys are based on runner, Node.js version, workspace, and package-lock.json hashes

Why is it important?

1. Improved CI Performance

• Reduces unnecessary test runs by only testing affected modules
• Significantly speeds up builds through intelligent NPM caching
• Prevents redundant dependency installations across parallel jobs
• Do not block other testcontainers projects when a PR used all the GH runners

2. Better Resource Utilization

• Avoids running full test suites when only a single module is modified
• Optimizes CI minutes by skipping unaffected modules
• Reduces load on CI runners by caching heavy node_modules directories

3. Developer Experience

• Faster feedback cycles for PRs that only touch specific modules
• More efficient use of GitHub Actions minutes
• Consistent dependency installation behavior across different environments

4. Maintainability

• Centralizes NPM setup logic in a reusable composite action
• Makes caching behavior consistent across all jobs
• Provides clear documentation and test cases for the module detection logic

The combination of selective testing and optimized dependency management makes the CI pipeline both faster and more resource-efficient, while maintaining reliability by ensuring comprehensive testing when core changes are made.

How to test this

The shell script to detect changes comes with some heading comments with the different examples. Please read and execute them in your shell to understand which modules will be added to the build.

Effective measures

Before these changes, the build took ~60minutes, now it takes ~15 minutes for the full blown project (the testcontainers package adds 12 minutes to the build (6 for docker and 6 for podman), each module takes around 1-2 minutes in running all the tests). We expect that the build time for a PR targeting just one module takes around 2-3 minutes.

[netlify]

✅ Deploy Preview for testcontainers-node ready!

Name	Link
🔨 Latest commit	39b904d
🔍 Latest deploy log	https://app.netlify.com/sites/testcontainers-node/deploys/67cf18d94cc9cf000841504a
😎 Deploy Preview	https://deploy-preview-918--testcontainers-node.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.