Skip to content

Conversation

@lym953
Copy link
Contributor

@lym953 lym953 commented Nov 25, 2025

Problem

The bottlecap GitLab job usually takes < 10 minutes, but it sometimes gets stuck and times out at 1 hour. This failure signal causes confusion in PRs and blocks PR merge. As a result, engineers often need to rerun this job.

This PR

For this job, add:

  1. timeout of 10 minutes
  2. a retry in case of stuck_or_timeout_failure or runner_system_failure

Testing

Will merge and see if these errors become less frequent.

@lym953 lym953 force-pushed the yiming.luo/fix-gitlab-timeout branch from 04c93df to bdcd6f2 Compare November 25, 2025 20:51
@lym953 lym953 marked this pull request as ready for review November 25, 2025 21:19
@lym953 lym953 requested a review from a team as a code owner November 25, 2025 21:19
Copy link
Contributor

@astuyve astuyve left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

omg ty

image: registry.ddbuild.io/images/docker:20.10
tags: ["arch:{{ $flavor.arch }}"]
needs: []
# This job sometimes times out on GitLab runner 17227436 for unclear reason.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we report this to the internal CI team?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I was wrong. It's not just this runner. It can happen for both amd64 (which uses 17227436) and arm64 (which uses 17335539), so it might still be our problem.

@lym953 lym953 merged commit 39a0f5e into main Nov 25, 2025
13 of 14 checks passed
@lym953 lym953 deleted the yiming.luo/fix-gitlab-timeout branch November 25, 2025 21:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants