refactor: Enhance checkpoint file exclusion system

[hannesrudolph]

Comprehensive update to the checkpoints exclusion module:

• Modularized exclusion patterns into category-specific functions
• Added detailed documentation for file filtering strategy
• Introduced new utility functions for Git LFS pattern handling
• Expanded coverage of excluded file types and patterns
• Improved maintainability and extensibility of exclusion rules

Context

Comprehensive update to the checkpoints exclusion module: The changes refactor the checkpoint file exclusion system, improving its organization, extensibility, and coverage of files that should be excluded when creating checkpoints. This reduces checkpoint size and improves performance by avoiding unnecessary tracking of irrelevant files.

Implementation

The changes modularize the exclusion logic in src/services/checkpoints/constants.ts. Previously, all exclusion patterns were defined in a single large array (GIT_EXCLUDES). The refactoring introduces several key improvements:

1. Category-Specific Functions: Exclusion patterns are now grouped into category-specific functions, such as getBuildArtifactPatterns(), getMediaFilePatterns(), getCacheFilePatterns(), etc. This improves readability and makes it easier to manage and update specific types of exclusions.
2. Detailed Documentation: A JSDoc comment block has been added at the top of the file, explaining the purpose and functionality of the CheckpointExclusions module. This documentation clarifies the file filtering strategy and how the different parts of the module work together.
3. New Utility Functions: New utility functions, such as getDefaultExclusions(), writeExcludesFile(), and getLfsPatterns(), have been added to handle Git LFS pattern integration and writing the combined exclusion patterns to Git's exclude file.
4. Expanded Coverage: The list of excluded file types and patterns has been significantly expanded to include more comprehensive coverage of build artifacts, media files, cache files, environment files, large data files, database files, geospatial datasets, and log files.
5. getDefaultExclusions() function: This function now combines all category-specific exclusions and also incorporates Git LFS patterns. The old GIT_EXCLUDES is deprecated and now just calls getDefaultExclusions().
6. ExclusionResult Interface: This interface is added to define the return of an exclusion check.

Screenshots

before	after
N/A	N/A

---

Important

Refactor constants.ts to modularize checkpoint exclusion patterns, add utility functions for Git LFS, and expand excluded file types.

• Modularization:
  • Refactor src/services/checkpoints/constants.ts to use category-specific functions for exclusion patterns, such as getBuildArtifactPatterns(), getMediaFilePatterns(), etc.
  • Deprecate GIT_EXCLUDES in favor of getDefaultExclusions().
• Utility Functions:
  • Add getDefaultExclusions(), writeExcludesFile(), and getLfsPatterns() for handling Git LFS patterns and writing exclusion patterns.
• Documentation:
  • Add JSDoc comments to explain the CheckpointExclusions module and its file filtering strategy.
• Expanded Coverage:
  • Expand excluded file types to include more comprehensive patterns for build artifacts, media files, cache files, environment files, large data files, database files, geospatial datasets, and log files.
• Interface:
  • Add ExclusionResult interface to define the return of an exclusion check.

^{This description was created by for ed83d53. It will automatically update as commits are pushed.}

THANK YOU CLINE

[changeset-bot]

⚠️ No Changeset found

Latest commit: ed83d53

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[hannesrudolph]

@mrubens shred me.. what did I miss?

[samhvw8]

coool

[cte]

Closing in favor of #1420.

[hannesrudolph]

@cte much better job. Thank you.