Skip to content

Fix nightly rename of finial directory path#4716

Merged
DavidHuber-NOAA merged 20 commits intoNOAA-EMC:developfrom
TerrenceMcGuinness-NOAA:fix_nightly_rename
Mar 31, 2026
Merged

Fix nightly rename of finial directory path#4716
DavidHuber-NOAA merged 20 commits intoNOAA-EMC:developfrom
TerrenceMcGuinness-NOAA:fix_nightly_rename

Conversation

@TerrenceMcGuinness-NOAA
Copy link
Copy Markdown
Collaborator

PR: Remove Nightly Workspace Directory Rename

Description

The GitLab CI pipeline uses WORKSPACE_ID (containing CI_PIPELINE_ID) as the
workspace directory name throughout all stages — build, setup, run, and finalize.
For nightly scheduled pipelines, the finalize stage renames this directory from
the WORKSPACE_ID format to a WORKSPACE_DATE format using a bash date call:

WORKSPACE_ID:   nightly_0_affeae_12345678    (used during build/test)
WORKSPACE_DATE: nightly_affeae_032626        (renamed at finalize)

This post-hoc rename causes path issues in the true paths that are in the Rocoto XML files.

Resolves #4715

Solution

Remove the WORKSPACE_DATE concept entirely and use WORKSPACE_ID as the
single, stable directory name throughout the pipeline lifecycle. The stable
symlink (already in use) provides the "latest successful nightly" pointer that
downstream consumers need.

Changes

.gitlab-ci.yml (before_script):

  • Removed DATE and WORKSPACE_DATE variable computation
  • Removed associated comments describing the date-based naming scheme

dev/ci/gitlab-ci-cases.yml (finalize_success_template):

  • Replaced the rename logic (mv WORKSPACE_ID → WORKSPACE_DATE) with a direct
    symlink: ln -sf WORKSPACE_ID stable
  • Removed the conditional branching for rename vs. already-matching names
  • Kept the cleanup of old nightly directories (unchanged)
  • Removed stale ${DATE} reference from run_experiments_template echo

Before (finalize for nightly)

mv "${WORKSPACE_ID}" "${WORKSPACE_DATE}"
ln -sf "${WORKSPACE_DATE}" stable

After (finalize for nightly)

ln -sf "${WORKSPACE_ID}" stable

Directory Layout

After a successful nightly run:

CI_BUILDS_DIR/
  ├── nightly_0_affeae_12345678/       # WORKSPACE_ID (unchanged)
  │   ├── global-workflow/             # GIT_CLONE_PATH
  │   └── RUNTESTS/                    # Test output
  └── stable -> nightly_0_affeae_12345678/  # Symlink to latest

Impact

  • No path breakage from directory renames mid-pipeline
  • stable symlink behavior is preserved for downstream access
  • Pipeline names and IDs remain unique per CI_PIPELINE_ID
  • Date information is still available via CI_PIPELINE_CREATED_AT if needed
  • Simplifies the codebase by removing unnecessary complexity

-->

Type of change

  • Bug fix (fixes something broken)
  • New feature (adds functionality)
  • Maintenance (code refactor, clean-up, new CI test, etc.)

Remove WORKSPACE_DATE and DATE variables from the CI pipeline.
The post-hoc rename of WORKSPACE_ID to WORKSPACE_DATE during
finalize caused path issues since GIT_CLONE_PATH is set at YAML
level before any shell runs.

Now the stable symlink points directly to WORKSPACE_ID, eliminating
the rename step while preserving the latest-nightly pointer.

Resolves #368
Remove CI/CD documentation files that came from the gitlab_documentation
branch. These will be handled separately.

Reverted files:
- docs/source/index.rst (restored to develop)
- docs/source/ci_cd_pipeline.rst (removed)
- docs/source/_static/ci_cd_architecture.svg (removed)
Remove CI/CD documentation files that came from the gitlab_documentation
branch. These will be handled separately.

Reverted files:
- docs/source/index.rst (restored to develop)
- docs/source/ci_cd_pipeline.rst (removed)
- docs/source/_static/ci_cd_architecture.svg (removed)
Copilot AI review requested due to automatic review settings March 26, 2026 19:52
@TerrenceMcGuinness-NOAA TerrenceMcGuinness-NOAA added bug Something isn't working CI/CD Issue related to CI/CD labels Mar 26, 2026
@TerrenceMcGuinness-NOAA TerrenceMcGuinness-NOAA changed the title Fix nightly rename Fix nightly rename of finial directory path Mar 26, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Removes the nightly “post-hoc workspace rename” behavior in GitLab CI to keep workspace paths stable across all pipeline stages, preventing Rocoto XML path breakage (issue #4715).

Changes:

  • Eliminated DATE / WORKSPACE_DATE computation and logging from .gitlab-ci.yml.
  • Simplified nightly finalize logic to update stable to point directly at WORKSPACE_ID (no mv rename).
  • Cleaned up a stale ${DATE} reference in the run-experiments logging.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
.gitlab-ci.yml Removes date-based workspace naming (WORKSPACE_DATE) so the pipeline uses WORKSPACE_ID consistently.
dev/ci/gitlab-ci-cases.yml Updates nightly finalize to only refresh the stable symlink and removes outdated ${DATE} logging.

@DavidHuber-NOAA DavidHuber-NOAA merged commit 5351a53 into NOAA-EMC:develop Mar 31, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working CI/CD Issue related to CI/CD

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI: Remove nightly workspace directory rename to fix path issue

3 participants