Description
In the aml_ohsu_2022 study, clinical sample string values appear to be silently truncated at 255 characters. For example, the sample ID aml_ohsu_2022_2010_BA2304, in the SURFACE_ANTIGENS_IMMUNOHISTOCHEMICAL_STAINS column, contains the following original value:
"The monocytic cells (percentages shown) are atypical in that they express partial CD56 (about 50% positive), but otherwise demonstrate phenotypically normal expression of CD13, bright CD33, bright CD64, CD15, HLA-DR, and partial myeloperoxidase (Flow cytometric analysis of the bone marrow aspirate specimen indicates that 4% of the CD45 positive leukocytes are lymphocytes, 89% are monocytic cells, and 7% are myeloid cells.) | CD10, CD11b, CD13, CD16, CD33, and CD64 (Myeloid (phenotypically normal))"
However, the value stored in the system is truncated to:
"The monocytic cells (percentages shown) are atypical in that they express partial CD56 (about 50% positive), but otherwise demonstrate phenotypically normal expression of CD13, bright CD33, bright CD64, CD15, HLA-DR, and partial myeloperoxidase (Flow"
This indicates a hard character limit is being applied—likely 255 characters—but neither the validator nor the loader provides any feedback about the truncation.
Expected Behavior
- The validator should raise an error if any clinical attribute value exceeds the allowed maximum length.
- The loader should at minimum emit a warning, but ideally raise an error if truncation occurs.
Why This Matters
Silent truncation results in significant loss of clinical detail and may misrepresent key phenotypic information. Users need to be explicitly informed to ensure data integrity and correct downstream usage.
Description
In the
aml_ohsu_2022study, clinical sample string values appear to be silently truncated at 255 characters. For example, the sample IDaml_ohsu_2022_2010_BA2304, in theSURFACE_ANTIGENS_IMMUNOHISTOCHEMICAL_STAINScolumn, contains the following original value:However, the value stored in the system is truncated to:
This indicates a hard character limit is being applied—likely 255 characters—but neither the validator nor the loader provides any feedback about the truncation.
Expected Behavior
Why This Matters
Silent truncation results in significant loss of clinical detail and may misrepresent key phenotypic information. Users need to be explicitly informed to ensure data integrity and correct downstream usage.