feat(serialization): implement truncation logic for rendered values#61878
feat(serialization): implement truncation logic for rendered values#61878imrichardwu wants to merge 13 commits intoapache:mainfrom
Conversation
…n template fields Added a new function to truncate rendered values based on a specified maximum length, ensuring that truncation messages are prioritized. This functionality is integrated into the serialization of template fields, enhancing the handling of long strings in the system.
Modified the test for runtime task instances to dynamically retrieve the rendered fields from the mock supervisor communications, ensuring accurate assertions for the SetRenderedFields message type. This change enhances the robustness of the test by adapting to varying truncation formats based on configuration.
…ialization Replaced direct truncation messages in test assertions with calls to the new serialize_template_field function. This change ensures consistency in how large strings and objects are handled in the rendered task instance fields tests, leveraging the updated serialization logic for better clarity and maintainability.
SameerMesiah97
left a comment
There was a problem hiding this comment.
I think this needs some refactoring. The current implementation looks functional but it is not very clear. I think the algorithm should be as follows:
-
If
max_length <= 0,return "" -
Build the truncation message once and, if
max_lengthis smaller than its length, return it immediately. -
Determine the quoting strategy and compute the formatting overhead (prefix + quotes + suffix) and calculate available space.
-
If available space is less than
MIN_CONTENT_LENGTH, return the truncation message only. -
Otherwise slice the content to fit and construct the final string and ensure the result does not exceed
max_length, trimming it if necessary.
Currently, I think you are mixing 3, 4 and 5 when they should each be in disparate blocks of code. And this is resulting in needless duplication.
Also, I was just wondering why this has to be duplicated across 2 files? Perhaps, a maintainer/committer could weigh in on this but is it possible to have this in airflow.utils.helpers instead?
…dant safe wrapper
b130b81 to
37ca60e
Compare
…or accurate truncation
This pull request introduces a shared helper for truncating long rendered template fields, so truncation is handled consistently everywhere. The goal is to ensure users always get a clear truncation message and, when there is enough space, see the truncated content wrapped in quotes. The new logic is applied across both the Airflow core and the task SDK, and is backed by thorough unit tests.
Truncation logic changes
Tests
related: #59877
Was generative AI tooling used to co-author this PR?
{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.