Unable to kick-off a build in CodeMagic

[Sush-HD]

If you are looking for help, please confirm the following...

• I confirm I have searched the Docs, Codemagic Sample Projects, and GitHub Discussions.
• I confirm the software versions on my local computer match the ones in CI environment (e.g. Xcode version, Gradle version). See here
• I confirm I am able to do a fresh clone of the repository and build the project on my local computer.

Which mobile framework are you using?

Flutter (Dart)

Steps to reproduce

1. Select the project
2. Start a new build with the desired workflow

Expected results

Build should kick-off

Actual results

I do not see a build being kicked-off

Build id (optional)

No response

[CaleyD]

we're running into the same issue - is there an alternative way to initiate builds in the web site?

[CaleyD]

More information for debugging - the "Select File Workflow" UI never loads after I select a branch to build

[dmytro-stepanov-hd]

Experiencing the same issue.
I checked the pipeline and it actually doesn't show any build started for 8 hours at least.

Also UI popup for starting new builds often doesn't allow to click "Start new build" button, showing it as grey.
No errors, no builds.
Please fix ASAP, we are blocked.

[cliffom]

Experiencing the same issue. I am unable to select a workflow file as the selector is stuck in a codemagic.yaml is loading... state.

[cliffom]

It's been this way for about 2 hours now :(

[cbaradero]

Also stuck on the same issue.

[oashrafouad]

I'm also having problems adding a new project, after I choose my repository and platform, nothing happens after pressing "Add Application".

[ogoldberg]

I'm unable to choose a github branch while in the flow to create a build.

[gregg062]

I'm experiencing a similar problem, I am able to select the branch but unable to select the workflow when trying to kick off a build.

[cliffom]

Over 3 hours now 👎

[gferreirakalshi]

I'm unable to trigger the build, please fix this

[ogoldberg]

One of the really frustrating things about this situation is that since this discussion forum is the only form of communication with the team, and nobody has replied, there isn't even a way to know if they are aware of the situation--and since the slack group is gone it feels safe to assume that since there's no response after multiple hours that nobody on the codemagic team is getting these notifications.

[cliffom]

Extremely frustrating. I'm at a standstill right now and have been for hours. Would be nice to get some acknowledgment and know we aren't raising issues into a void.

[ogoldberg]

I just went to their linkedin page and sent messages to the CEO and the CTO. Unfortunately it's like 5am in Estonia, so it will probably be 3-4 more hours before anyone over there notices what is going on.

[naman-modi]

Similar issue, not able to start a build.

[btbishop93]

Same here, seems like its now been down for hours...Not a good look. May have to rethink our build pipeline.

[himesh-cm]

Hi Everyone,
I believe everything is working properly now. Please proceed to trigger your builds. we apologize for any inconvenience caused 🙏 .

[dmytro-stepanov-hd]

I observe random connection issues, like this:

curl: (28) Operation timed out after 45001 milliseconds with 338377513 out of 918476002 bytes received
Downloading Flutter release timed out, try again...

[himesh-cm]

Please share the build id @dmytro-stepanov-hd

[dmytro-stepanov-hd]

There was a build once..

[himesh-cm]

Please trigger it again after some time; it's probably a temp network issue while downloading.

[cliffom]

Thank you for the update and the resolution. Any chance we can get a post-mortem and what is being done to prevent this from happening again?

[helinanever]

Dear users,

We want to provide an update on the incident that temporarily impacted our services and share some insights into the situation.

Approximately at 23:00 UTC (November 16), our team received an alert signaling potential issues with our service. Unfortunately, due to incomplete actions, the alert was not handled as effectively as it should have been. Regrettably, this led to the escalation of the situation, resulting in a temporary service disruption.

As a result, users with repositories connected via SSH keys were not able to start builds. Repositories using other authentication methods, such as GitHub App or OAuth, were not affected.

The issue was fully resolved around 04:00 UTC (November 17).

We understand the frustration and inconvenience this may have caused, and we sincerely apologize for any disruption to your experience with Codemagic. We want to be transparent about the issue and assure you that we are actively addressing the root cause to prevent similar incidents in the future.

One of the key challenges we faced during this incident was a lack of immediate availability to handle the escalated alert. We acknowledge the importance of timely response and are implementing measures to ensure that there is always a designated team ready to address potential issues promptly.

Here are the steps we are taking to enhance our incident response and prevent future disruptions:

1. Immediate Action Plan: We are revising and strengthening our procedures to ensure that alerts are handled with the urgency they demand. This includes comprehensive training for our team members on recognizing, prioritizing, and responding to alerts promptly.
2. 24/7 Monitoring: We are implementing 24/7 monitoring to ensure that potential issues are identified and addressed around the clock. This proactive approach will enable us to mitigate risks before they impact our services.
3. Enhanced Communication Protocols: Improving communication during incidents is crucial. We are refining our communication protocols to provide timely updates and transparent information to our users during any service disruptions.

We understand the trust you place in our services, and we are committed to learning from this incident to better serve you in the future. Your feedback is invaluable, and we encourage you to reach out to us with any concerns or suggestions.

Thank you for your understanding, and we appreciate your continued support as we work to strengthen our systems and processes.

Sincerely,
Team Codemagic

[martynbowis]

@helinanever This issue is happening again today - cannot get Codemagic to build - stalls at submitted status

[cliffom]

@martynbowis yup, unfortunately it cropped up again https://github.com/orgs/codemagic-ci-cd/discussions/2228