You can try using the continue-on-error directive at the job level. Maybe base it off labels (then apply that label to this PR) since using pull_request.number would be redundant; e.g. say we name the label 'ignore-atime-failure':
continue-on-error: ${{ contains(github.event.pull_request.labels.*.name, 'ignore-atime-failure') }}
Originally posted by @Anirban166 in #7361 (comment)