Link checker throttled on GitHub links

[itowlson]

We are getting PRs failing because the backlinks to the GitHub sources are getting 429.

@karthik2804 Does your shiny new link checker handle this better? Could we plumb that in here?

[karthik2804]

@itowlson, I would think so because it does also allow us to pass in authorization header which should remove rate limits. I can look into it.

[itowlson]

@karthik2804 Looks like the test errors are blocking deployments so it would be great to solve this sooner rather than later - sorry, I know you have other things to be looking at as well!

[karthik2804]

Ah didn't realise it blocked deployments, I'll prioritize this.

[itowlson]

chuckle Nor did I, or I would have given you more of a heads up - sorry about that and thanks!

[vdice]

We/I could also short-circuit the link checks (npm test) in the build workflow if we're calling the workflow from deploy (eg merge to main). Could either be temporary until Karthik has a chance to swap link checkers -- or permanent if we decide we're good with broken links just being gated at PR time and not bothering running them again pre-deploy.

[karthik2804]

@vdice that sounds like a reasonable to do for now. I'll likely get around to testing things only in the morning for me.

It is also reasonable to me to skip link checking on publish if we have made a conscious decision to override it on PR merges.

[karthik2804]

I started updating the link checker, but wanted to check how we wanted to prioritize this since now the deployments are not blocked anymore.

[vdice]

@karthik2804 I'd say if your efforts here can also be easily duplicated (without say, 2x the work) in fermyon/developer, where this is also needed, priority could be on the higher side, just because it affects all PRs in both repos. WDYT? Happy to pair if it would help (maybe I could run with some portion of the work?)