When another user re-saves a file and changes ownership, does the hash change and invalidate a file target? [help]

[monicathieu]

Help

• I understand and agree to https://books.ropensci.org/targets/help.html.

Description

I have a targets pipeline for a scientific data analysis project. The underlying source data are being used in another research project as well that has a targets pipeline tracking some of the same files as targets. (It was originally just one targets pipeline, but we wanted to freeze some analyses for a manuscript while others can make changes, so duplicating and separating the targets project folders seemed to make the most sense.)

I have noticed something a bit annoying--sometimes the other user will tar_make()s something on their pipeline, and they overwrite a shared target file with them as the owner. I'm quite sure that we are running the same functions and that the shared target file's contents should not have changed, only the ownership and the modified date. However, my file target in my pipeline will read as invalidated and attempt to re-run. Does this mean that the hash of a file changes when its ownership and/or modified date change?

I am currently experimenting with changing the cue argument to use tar_cue(mode="never") or maybe with file=FALSE to make the up-to-date checking in my pipeline a little laxer, but I would rather keep doing the thorough checking, because I do want to know when some of the shared files might legitimately be out of date!

Unfortunately because it's a file ownership thing, it's very difficult to provide a reprex, sorry! Thank you for your assistance nonetheless.

[wlandau]

Does this mean that the hash of a file changes when its ownership and/or modified date change?

Possibly. Is either pipeline trying to overwrite the file (regardless of what actually changes) or are both pipelines just tracking the file as input? In the latter case, not even ownership should change. In the former, I would recommend aligning on a common set of inputs that neither side tries to modify.

[monicathieu]

Both pipelines are currently trying to overwrite the files in question. (The initial sole pipeline actively wrote these files out, so we didn't change the target code when we cloned pipelines. Now both pipelines try to overwrite the same file)

Would you suggest treating one pipeline as the primary pipeline which writes the files, and the other pipeline as secondary and only tracks the files and doesn't also try to modify them?

[wlandau]

Yeah, I would suggest to have only one pipeline, at most, writing to files that both pipelines access.

[monicathieu]

Gotcha! It's a little space-inefficient, but in that case I might modify my version of the pipeline to write copies of the files in question elsewhere to prioritize fully reproducing every intermediate target.