Using validate_iods in code

[urbenand]

Hello developers

I'm currently prototyping a tool to bulk-validate a big load of DICOM files. My goal is to save the information from validate_iods within a small SQLite database to check, which DICOM tags are missing or unexpected to get a big picture of what to optimize in the configuration of our systems.

I tried a few different ways, but I can't make it work, that I can save the information generated bi DicomValidateFile into an object.

Is there a way I can achieve this?

Thanks

[mrbean-bremen]

Currently, the validation result is mostly meant to be used for console display (and for some testing), but the format is far from ideal for post-processing.
You could of course serialize the resulting error dictionary to json and save that in the database, but this is probably not what you want.
Also, a description of the returned dictionary is missing - it may be a good idea to add this, even if the format may change in the future.

I'm not sure about your workflow. Assuming you have a root path where all DICOM files reside that you want to validate, you could do something like this (note: this is out of my head and not tested):

    # download and process the needed revision of the DICOM standard - needs only to be done once for a given standard
    dicom_info_path = str(Path.home() / "dicom-validator"  # this is the default
    reader = EditionReader(dicom_info_path)
    reader.get_revision("2025b")  # could also use "current" for the current standard

    # validate all DICOM files in the path (this is recursive)
    # alternatively, you could iterate over the files yourself and validate each separately
    # - probably the better way for a large number of files
    json_path = Path(dicom_info_path, "json")
    dicom_info = EditionReader.load_dicom_info(json_path)
    validator = DicomFileValidator(dicom_info)
    result = validator.validate(dicom_file_path)

    for file_path, errors in result.items():
        for module, module_errors in errors.items():
            # 'module' is the name of the module in the standard where the violation occurs
            for error_msg, tags in module_errors:
                # error_msg is the message that is logged, e.g. 'Tag (0018,1000) (Device Serial Number) is missing'
                # tags is a list of related tags (mostly only one)
                # this has to be added to the database in some form, depending on your needs
                ...

You could structure your database in a similar way, e.g. a file_path table, a modules table (1:N from file_path) and an errors table (1:N from modules).
Unfortunately, there is currently nothing like an error code in this dictionary, so you can either add the message itself to the database, or parse it beforehand to get the kind of error.

I actually had in mind to structure the result in a way that is better machine-readable (e.g. at least use error codes instead of error messages, and construct the messages from these), but until now this has not been needed. Maybe it's time for this now - I'm open to suggestions or feature requests (see #205).

[mrbean-bremen]

If you want to check that, please check out the main branch - as I wrote above, I have refactored the result generation. While this is not final yet, it is definitely the direction I want to go, so it would be helpful if you can check if you can work with that, and what do you want to be changed.

[urbenand]

If you want to check that, please check out the main branch - as I wrote above, I have refactored the result generation. While this is not final yet, it is definitely the direction I want to go, so it would be helpful if you can check if you can work with that, and what do you want to be changed.

I'm so sorry - I thought the package in my .venv got automatically updated. I'm not very experienced in Python yet but I managed to update it by hand.

Currently trying out a few things and it's already a big difference!

I read through files and errors like this, to fit it to my database structure.

for file_path, errors in result.items():
    ds = dcmread(file_path)
    accession_number = ds.get("AccessionNumber", None)
    print("-"*50)

    for module in errors.module_errors:
        for error in errors.module_errors.values():
            for key, value in error.items():
                  print(f"{module}, {key.tag}, {key.parents}, {value.type}, {value.code}, {value.scope}, {value.context}")

When I use my testfiles, I get a lot of values back like this:

{module}	{key.tag}	{key.parents}	{value.type}	{value.code}	{value.scope}	{value.context}
SR Document Content	4235344	[4237104]	1	ErrorCode.TagMissing	ErrorScope.General	{}
DX Image	(0032,1032)	None	TagType.Undefined	ErrorCode.TagUnexpected	ErrorScope.General	None
General	542113824	None	1	ErrorCode.TagMissing	ErrorScope.General	{}
General	(0032,1032)	None	TagType.Undefined	ErrorCode.TagUnexpected	ErrorScope.General	None

which is indeed machine readable, but as I'm new to this, I'm not sure, why there are a lot of TagType.Undefined or what the definition of ErrorScope means.

Also, is there a way to connect the DICOM Tag to the human readable value, e.g. (0040,0320) --> (Billing Procedure Step Sequence)? I will have a look at the pydicom documentation for sure, but maybe you'll have a shortcut for me. :-)

[mrbean-bremen]

First (as this is the first entry in your table), there is an open issue that recursive definitions are ignored - I'm currently working on this, but it will currently cause incorrect entries specifically for SR Document Content. (EDIT: now fixed in main branch)

As for the values in your table:

• module shows the module name as defined in the standard, except for "General", which is a placeholder for tags that do not belong to a supported module
• the tag (key.tag) is now saved as an int, as you have noticed; there is a small helper function tag_tools.tag_id_string() that returns the standard string representation for an int tag value
  EDIT: With the current version, a pydicom BaseTag is always used for key.tag, which will automatically be displayed correctly, while still being an int (e.g. derived from int)
• tag.parents is a list of parent sequence tags (also now of type BaseTag), if the tag itself is nested in a sequence item
• the value type is only set where it matters, you probably won't need it; same with the value scope - that just says if it's an error specific to Functional Groups
• value.code should be self-explaining, you can check the comments in the definition (I haven't yet added documentation)
• value.context is an optional dictionary with additional information for some types of errors (such as the value and allowed value for invalid enum values)

Let me know if you need more information.

[urbenand]

That looks good to me. I made it work the way I need to.

One more question: Is there a way to disable the command line output? When importing the files and validation info into my database, there's no need for me to print out all the validation results, as they are stored into the DB.

[mrbean-bremen]

You can set the log level in the IODValidator to CRITICAL, so nothing would be logged at any lower level (e.g. nothing would be logged at all), or you could use your own error handler that does nothing (or whatever you want to do).