Skip to content

Conversation

@adammcdonagh
Copy link
Owner

@adammcdonagh adammcdonagh commented Sep 20, 2025

This pull request introduces several improvements and new features, focusing on enhanced logging for the sftp and local protocols, the addition of a rename option for the email destination protocol, schema validation improvements, and performance optimizations in config loading. The changes also include a new release version bump to v25.37.0.

Key changes:

Logging Enhancements

  • Added more detailed logging to the sftp and local protocols, including file copy, rename, and deletion operations, as well as file download/upload actions. This should make debugging and monitoring file operations easier. [1] [2] [3] [4] [5] [6] [7]

Email Protocol Improvements

  • Introduced a new rename option for the email destination protocol, allowing file attachments to be renamed according to a regex pattern and substitution. This includes updates to the schema and logic for handling renamed attachments. [1] [2] [3] [4]

Config Loader Performance

  • Improved performance of config loading by preloading all JSON and Jinja2 files in the config directory into memory, reducing file I/O during runtime. The loader now uses a file cache for faster access. [1] [2] [3] [4]
  • Ensures that overrides via env vars match the original type if they are integers. Allows overriding integer variables properly, e.g. port numbers that are stored as int rather than strings

Schema Validation and Error Reporting

  • Enhanced schema validation for batch, execution, and transfer tasks by logging validation errors, making it easier to identify issues with task definitions. [1] [2] [3] [4] [5] [6] [7]

Properly implements #118

adammcdonagh and others added 5 commits September 20, 2025 14:06
* Tidy GPG logic and move gpg home to a temp dir

* Forgot to keep exceptions
* Add email filename renaming

* Add email rename tests
@codecov
Copy link

codecov bot commented Sep 20, 2025

Codecov Report

❌ Patch coverage is 87.61905% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.83%. Comparing base (7cfb1dc) to head (9df751a).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/opentaskpy/cli/batch_validator.py 52.63% 9 Missing ⚠️
src/opentaskpy/taskhandlers/transfer.py 85.71% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #143      +/-   ##
==========================================
- Coverage   89.20%   88.83%   -0.38%     
==========================================
  Files          22       22              
  Lines        2584     2650      +66     
==========================================
+ Hits         2305     2354      +49     
- Misses        279      296      +17     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@adammcdonagh adammcdonagh merged commit 586bb04 into main Sep 20, 2025
21 checks passed
@adammcdonagh adammcdonagh deleted the feat/v25.37.0 branch September 20, 2025 21:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants