Skip to content

Comments

[CI] Cache test_data for translate tests#41

Merged
FlorianDeconinck merged 1 commit intoNOAA-GFDL:developfrom
romanc:romanc/cache-test-data
Feb 12, 2025
Merged

[CI] Cache test_data for translate tests#41
FlorianDeconinck merged 1 commit intoNOAA-GFDL:developfrom
romanc:romanc/cache-test-data

Conversation

@romanc
Copy link
Collaborator

@romanc romanc commented Feb 10, 2025

Description

Currently, translate tests go and fetch test_data/ from scratch every run. We had issues with data fetching in the past so I put in a cache under the assumption that GH cache servers are very reliable. They are also closer (in a network sense) to the executing machines, leading to slightly faster transfer times (not that this would matter in the 30min runtime that we are looking at).

How Has This Been Tested?

  • Changed CI workflow is still passing.

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation: N/A
  • My changes generate no new warnings
  • Any dependent changes have been merged and published in downstream modules: N/A
  • New check tests, if applicable, are included
  • Targeted model if this changed was triggered by a model need/shortcoming: N/A

@romanc romanc force-pushed the romanc/cache-test-data branch from d6c686e to cc7fb8e Compare February 10, 2025 16:39
@romanc romanc marked this pull request as ready for review February 10, 2025 17:29
Currently, translate tests go and fetch test_data/ from scratch every
run. Every now and then, the server isn't available. This PR caches
test_data/ as part of the translate test workflow.
@romanc romanc force-pushed the romanc/cache-test-data branch from 58e2510 to 8455f45 Compare February 10, 2025 17:33
@romanc
Copy link
Collaborator Author

romanc commented Feb 10, 2025

@FlorianDeconinck Another quick PR to tie up loose ends before we start the new dev cycle.

I might put in something similar in the DaCe workflow once we re-enable these tests on DaCe's mainline branch. For now, I think we don't run enough DaCe tests that this would be justified.

Copy link
Collaborator

@FlorianDeconinck FlorianDeconinck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Hold for NOAA review

@FlorianDeconinck FlorianDeconinck merged commit 856437f into NOAA-GFDL:develop Feb 12, 2025
4 checks passed
@romanc romanc deleted the romanc/cache-test-data branch February 12, 2025 17:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants