Skip to content

Conversation

dennisameling
Copy link
Contributor

We're seeing intermittent OSProvisioningTimedOut errors on VM creation, which typically happens within 20-40 mins of the deployment hanging. Most creation jobs seem to work fine, though.

Let's try and find out what's going wrong, so we can see if we need to fix things on our end.

@dennisameling dennisameling requested a review from dscho September 25, 2024 09:00
Copy link
Member

@dscho dscho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great stuff! Let's do this.


- name: Show some more information on failure
if: failure()
uses: azure/cli@v2
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Elsewhere, we moved away from Azure/CLI, maybe we can do that here, too?

@dennisameling dennisameling force-pushed the logging-on-vm-creation-failure branch 2 times, most recently from 91041db to d1e7797 Compare October 2, 2024 20:41
We currently depend on the azure/login GitHub Action in various places. It simply runs an "az login" and "az account set" behind the scenes, and has support for plenty of other scenarios.

As we're moving away from azure/cli in more and more places, we could also replace the login script with our own. Let's create a Composite Workflow to achieve this goal.

Signed-off-by: Dennis Ameling <[email protected]>
We now have our own simple workflow to log into Azure, using "az login". As we're moving way from things like the Azure CLI, let's also start using our own login script instead.

Signed-off-by: Dennis Ameling <[email protected]>
We're seeing intermittent `OSProvisioningTimedOut` errors on VM creation, which typically happens within 20-40 mins of the deployment hanging.

Let's try and find out what's going wrong, so we can see if we need to fix things on our end.

Ref: https://github.com/git-for-windows/git-for-windows-automation/actions/runs/11025772033/job/30621189230
Signed-off-by: Dennis Ameling <[email protected]>
Even if the deployment of a runner failed, we'd like to see the post-deployment script output. Maybe something failed there.

This is under the assumption that the azure/arm-deploy action will output the information, even if the deployment failed. Let's see if it works.

Signed-off-by: Dennis Ameling <[email protected]>
The azure/cli action runs in a Docker container that only has a very limited subset of tools compared to the GitHub Actions runners. Luckily, the Azure CLI comes pre-installed with Hosted Runners. Let's use use it directly, so we can enjoy all the other tools that it offers.

Signed-off-by: Dennis Ameling <[email protected]>
We now have our own simple workflow to log into Azure, using "az login". As we're moving way from things like the Azure CLI, let's also start using our own login script instead.

Signed-off-by: Dennis Ameling <[email protected]>
@dennisameling dennisameling force-pushed the logging-on-vm-creation-failure branch from 65c2212 to ab3fad1 Compare October 3, 2024 07:06
@dennisameling dennisameling marked this pull request as ready for review October 3, 2024 07:09
@dennisameling
Copy link
Contributor Author

@dscho I've updated the PR to move away from the @azure/cli action. I've also created a Composite Action for the login part, which we can reuse. However, I just learned that this may not add much benefit compared to the @azure/login action. I was under the assumption that said action only worked in combination with Azure-provided actions, but turns out it simply runs az login and az account set on the OS behind the scenes.

So I'd be OK to keep using @azure/login if we want (since we can also run our own az commands after that step), otherwise we could use our own Composite Action instead. Any preference from your side?

Also, here's a successful workflow run with the new setup.

@dennisameling dennisameling requested a review from dscho October 3, 2024 07:23
Copy link
Member

@dscho dscho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the Composite Action!

@dscho dscho merged commit df83428 into main Oct 3, 2024
2 checks passed
@dscho dscho deleted the logging-on-vm-creation-failure branch October 3, 2024 07:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants