-
Notifications
You must be signed in to change notification settings - Fork 7
Closed
Description
At the end of the apply_registration_to_image task, we copy the tables from the reference cycle. This can lead to race conditions when the overwrite_input option is on, because every cycle copies tables from the reference cycle.
I thought I had this handled with:
while current_round < max_retries:
try:
old_table_group = zarr.open_group(
table_dict[table], mode="r"
)
current_round = max_retries
except zarr.errors.GroupNotFoundError:
logger.debug(
f"Table {table} not found in attempt {current_round}. "
f"Waiting {sleep_time} seconds before trying again."
)
current_round += 1
time.sleep(sleep_time)
# Write the Zarr table
curr_table = ad.read_zarr(table_dict[table])
write_table(
new_image_group,
table,
curr_table,
table_attrs=old_table_group.attrs.asdict(),
overwrite=True,
)
But actually, the ad.read_zarr sometimes hits the race condition now. I'll move this into the try/except part, that should hopefully fix this.
Metadata
Metadata
Assignees
Labels
No labels
Type
Projects
Status
Done