FileNotFoundError on anaconda, but code runs locally

[hjforbes]

Hi, I'm doing a test run on dummy data via anaconda. The action 07b_an_multivariable_cox_models_FULL_non_covid_death produces a file not found error, though this do file is running fine on STATA locally. Output on anaconda is:

File "c:\users\qc18278\anaconda3\lib\site-packages\opensafely_vendor\jobrunner\manage_jobs.py", line 258, in finalise_job
with open(log_dir / "metadata.json", "w") as f:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\qc18278\OneDrive - University of Bristol\Documents\GitHub\school-age-children-and-covid2-opensafely\metadata\.logs\2021-07\job-school-age-children-and-covid2-opensafely-07b-an-multivariable-cox-models-full-non-covid-death-r2un2u5x5jew73s3\metadata.json'

[StevenMaude]

My guess is that this is because your path:

C:\Users\qc18278\OneDrive - University of Bristol\Documents\GitHub\school-age-children-and-covid2-opensafely\metadata\.logs\2021-07\job-school-age-children-and-covid2-opensafely-07b-an-multivariable-cox-models-full-non-covid-death-r2un2u5x5jew73s3\metadata.json

is more than 260 characters long and that's a Windows limit:

In editions of Windows before Windows 10 version 1607, the maximum length for a path is MAX_PATH, which is defined as 260 characters. In later versions of Windows, changing a registry key or using the Group Policy tool is required to remove the limit. See Maximum Path Length Limitation for full details.

https://docs.microsoft.com/en-us/windows/win32/fileio/naming-a-file#maximum-path-length-limitation

In this case, it's only just over the limit by a character(!). If you shorten the directory name that holds the repository to e.g. covid2-os or something similar, e.g. C:\Users\qc18278\OneDrive - University of Bristol\Documents\GitHub\covid2-os\… it should fix this.

---

Alternatively, if you are an administrator and on non-Home edition Windows with access to the Group Policy editor (gpedit.msc), you can make Windows allow longer paths. See https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation

That Microsoft link also has registry edits that a Home edition Windows user could configure, if, again your user account is a computer administrator and you are brave enough to do so. (Windows Home users don't have access to the Group Policy editor without hacky and unapproved workarounds.)

[evansd]

Beat me to it! Thanks @StevenMaude

[StevenMaude]

(I had off-by-one error in number of characters which I just read your reply and fixed @evansd 😁)

[StevenMaude]

I should add that if you rename the directory, and are using GitHub Desktop, you will get prompted to fix up the now-missing path in GitHub Desktop. See here for more information on this.

[evansd]

Ah, I know exactly what's happening here. A fun "feature" of Windows is that without some trickery it can't handle file paths that are more than 260 characters long. And C:\Users\qc18278\OneDrive - University of Bristol\ ... \metadata.json is 261 characters long!

The very quickest thing you could do for now is to rename the school-age-children-and-covid2-opensafely folder to something slightly shorter. (Git doesn't care whether it actually matches the repository name or not.)

We can also work on making the filenames we generate internally shorter. They make sense in production where lots of different projects are running, but there's no need for them to be quite then long when running locally.

[hjforbes]

fixed! thank you