Skip to content

Commit 0da8901

Browse files
committed
remove REV from integration test task definitions
This might be the most hacky thing we're doing here so far, but it's necessary for translations integration tests, where cached ancestor tasks will have a `REV` that doesn't match the `REF`, leading to errors such as: ``` [vcs 2024-12-20T16:35:08.992Z] executing ['git', 'config', '--global', '--add', 'safe.directory', '/builds/worker/checkouts/vcs'] [vcs 2024-12-20T16:35:08.994Z] executing ['git', 'clone', 'https://github.com/mozilla-releng/staging-firefox-translations-training', '/builds/worker/checkouts/vcs'] [vcs 2024-12-20T16:35:08.995Z] Cloning into '/builds/worker/checkouts/vcs'... [vcs 2024-12-20T16:35:10.747Z] executing ['git', 'fetch', '--tags', '--force', 'https://github.com/mozilla-releng/staging-firefox-translations-training', 'refs/heads/main'] [vcs 2024-12-20T16:35:10.949Z] From https://github.com/mozilla-releng/staging-firefox-translations-training [vcs 2024-12-20T16:35:10.949Z] * branch main -> FETCH_HEAD [vcs 2024-12-20T16:35:10.955Z] executing ['git', 'fetch', '--no-tags', 'https://github.com/mozilla-releng/staging-firefox-translations-training', 'refs/heads/main'] [vcs 2024-12-20T16:35:11.128Z] From https://github.com/mozilla-releng/staging-firefox-translations-training [vcs 2024-12-20T16:35:11.128Z] * branch main -> FETCH_HEAD [vcs 2024-12-20T16:35:11.133Z] executing ['git', 'checkout', '-f', '-B', 'refs/heads/main', '876ad1a2de046271a45c979551c8650e5e1b75a2'] [vcs 2024-12-20T16:35:11.135Z] fatal: reference is not a tree: 876ad1a2de046271a45c979551c8650e5e1b75a2 ``` Removing the `REV` will cause the `REF` to be used without issues: ``` [vcs 2024-12-20T18:15:48.936Z] executing ['git', 'config', '--global', '--add', 'safe.directory', '/builds/worker/checkouts/vcs'] [vcs 2024-12-20T18:15:48.938Z] executing ['git', 'clone', 'https://github.com/mozilla-releng/staging-firefox-translations-training', '/builds/worker/checkouts/vcs'] [vcs 2024-12-20T18:15:48.939Z] Cloning into '/builds/worker/checkouts/vcs'... [vcs 2024-12-20T18:15:50.544Z] executing ['git', 'fetch', '--tags', '--force', 'https://github.com/mozilla-releng/staging-firefox-translations-training', 'refs/heads/main'] [vcs 2024-12-20T18:15:50.758Z] From https://github.com/mozilla-releng/staging-firefox-translations-training [vcs 2024-12-20T18:15:50.758Z] * branch main -> FETCH_HEAD [vcs 2024-12-20T18:15:50.763Z] executing ['git', 'fetch', '--no-tags', 'https://github.com/mozilla-releng/staging-firefox-translations-training', 'refs/heads/main'] [vcs 2024-12-20T18:15:50.926Z] From https://github.com/mozilla-releng/staging-firefox-translations-training [vcs 2024-12-20T18:15:50.926Z] * branch main -> FETCH_HEAD [vcs 2024-12-20T18:15:50.931Z] executing ['git', 'checkout', '-f', '-B', 'refs/heads/main', 'FETCH_HEAD'] [vcs 2024-12-20T18:15:51.929Z] Switched to a new branch 'refs/heads/main' ```
1 parent 9b3ecdd commit 0da8901

File tree

1 file changed

+20
-1
lines changed

1 file changed

+20
-1
lines changed

taskcluster/fxci_config_taskgraph/util/integration.py

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -101,6 +101,20 @@ def _rewrite_task_datestamps(task_def):
101101
return task_def
102102

103103

104+
def _remove_task_revisions(task_def):
105+
"""Rewrite revisions in task payloads to ensure that tasks do not refer to
106+
out of date revisions."""
107+
to_remove = set()
108+
for k in task_def.get("payload", {}).get("env", {}):
109+
if k.endswith("_REV"):
110+
to_remove.add(k)
111+
112+
for k in to_remove:
113+
del task_def["payload"]["env"][k]
114+
115+
return task_def
116+
117+
104118
def find_tasks(
105119
decision_index_path: str,
106120
include_attrs: dict[str, list[str]],
@@ -153,6 +167,11 @@ def find_tasks(
153167
# All datestamps come in as absolute ones, many of which
154168
# will be in the past. We need to rewrite these to relative
155169
# ones to make the task reschedulable.
156-
tasks[upstream_task_id] = _rewrite_task_datestamps(task_def)
170+
# We also need to remove absolute revisions from payloads
171+
# to avoid issues with revisions not matching the refs
172+
# that are given.
173+
tasks[upstream_task_id] = _remove_task_revisions(
174+
_rewrite_task_datestamps(task_def)
175+
)
157176

158177
return tasks

0 commit comments

Comments
 (0)