Skip to content

feat(serialization): implement truncation logic for rendered values#61878

Draft
imrichardwu wants to merge 13 commits intoapache:mainfrom
imrichardwu:truncation
Draft

feat(serialization): implement truncation logic for rendered values#61878
imrichardwu wants to merge 13 commits intoapache:mainfrom
imrichardwu:truncation

Conversation

@imrichardwu
Copy link
Contributor

@imrichardwu imrichardwu commented Feb 13, 2026

This pull request introduces a shared helper for truncating long rendered template fields, so truncation is handled consistently everywhere. The goal is to ensure users always get a clear truncation message and, when there is enough space, see the truncated content wrapped in quotes. The new logic is applied across both the Airflow core and the task SDK, and is backed by thorough unit tests.

Truncation logic changes

  • Added a new _truncate_rendered_value helper in both airflow/serialization/helpers.py and airflow/sdk/execution_time/task_runner.py to centralize truncation behavior. The helper prioritizes a clear truncation message and preserves quoted content when possible.
  • Updated existing serialization code to use _safe_truncate_rendered_value instead of the previous inline truncation logic, removing duplication and keeping behavior consistent between the two codebases.

Tests

  • Added a new test_helpers.py unit test module to validate the truncation helper.
  • The tests cover edge cases such as very small max_length values, handling of quoted strings, and exact expected output for truncated values.

related: #59877


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)
  • Used ChatGPT to help create and edit my test cases that were mentioned by a reviewer in a closed pr.

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

…n template fields

Added a new function to truncate rendered values based on a specified maximum length, ensuring that truncation messages are prioritized. This functionality is integrated into the serialization of template fields, enhancing the handling of long strings in the system.
@imrichardwu imrichardwu marked this pull request as draft February 13, 2026 20:05
Modified the test for runtime task instances to dynamically retrieve the rendered fields from the mock supervisor communications, ensuring accurate assertions for the SetRenderedFields message type. This change enhances the robustness of the test by adapting to varying truncation formats based on configuration.
…ialization

Replaced direct truncation messages in test assertions with calls to the new serialize_template_field function. This change ensures consistency in how large strings and objects are handled in the rendered task instance fields tests, leveraging the updated serialization logic for better clarity and maintainability.
@imrichardwu imrichardwu marked this pull request as ready for review February 13, 2026 22:37
@imrichardwu imrichardwu marked this pull request as draft February 15, 2026 21:24
@imrichardwu imrichardwu marked this pull request as ready for review February 15, 2026 21:24
Copy link
Contributor

@SameerMesiah97 SameerMesiah97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this needs some refactoring. The current implementation looks functional but it is not very clear. I think the algorithm should be as follows:

  1. If max_length <= 0, return ""

  2. Build the truncation message once and, if max_length is smaller than its length, return it immediately.

  3. Determine the quoting strategy and compute the formatting overhead (prefix + quotes + suffix) and calculate available space.

  4. If available space is less than MIN_CONTENT_LENGTH, return the truncation message only.

  5. Otherwise slice the content to fit and construct the final string and ensure the result does not exceed max_length, trimming it if necessary.

Currently, I think you are mixing 3, 4 and 5 when they should each be in disparate blocks of code. And this is resulting in needless duplication.

Also, I was just wondering why this has to be duplicated across 2 files? Perhaps, a maintainer/committer could weigh in on this but is it possible to have this in airflow.utils.helpers instead?

@imrichardwu imrichardwu marked this pull request as draft February 17, 2026 00:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants