Skip to content

Handle race conditions in apply_registration_to_image table copying #918

@jluethi

Description

@jluethi

At the end of the apply_registration_to_image task, we copy the tables from the reference cycle. This can lead to race conditions when the overwrite_input option is on, because every cycle copies tables from the reference cycle.

I thought I had this handled with:

            while current_round < max_retries:
                try:
                    old_table_group = zarr.open_group(
                        table_dict[table], mode="r"
                    )
                    current_round = max_retries
                except zarr.errors.GroupNotFoundError:
                    logger.debug(
                        f"Table {table} not found in attempt {current_round}. "
                        f"Waiting {sleep_time} seconds before trying again."
                    )
                    current_round += 1
                    time.sleep(sleep_time)
            # Write the Zarr table
            curr_table = ad.read_zarr(table_dict[table])
            write_table(
                new_image_group,
                table,
                curr_table,
                table_attrs=old_table_group.attrs.asdict(),
                overwrite=True,
            )

But actually, the ad.read_zarr sometimes hits the race condition now. I'll move this into the try/except part, that should hopefully fix this.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions