Retry status-0 URL check failures with curl (#3342)

* Retry status-0 URL check failures with curl before failing

The cr.yp.to links consistently fail with status 0 (connection
failure) from GitHub Actions runners despite being valid URLs.

- Increase markdown-link-check timeout from default 10s to 30s
- Add a retry step that re-checks status-0 failures using curl
  with retries, so genuine dead links still fail but servers that
  don't respond to the Node.js HTTP client get a second chance

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fix pipefail: tee was swallowing markdown-link-check exit code

Adding `| tee` to capture output caused the pipeline exit code to
always be 0 (from tee), so link check failures were silently ignored.
Adding `set -o pipefail` ensures the pipeline fails when
markdown-link-check fails.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Use continue-on-error so curl retry can rescue the job

The `if: failure()` approach ran the retry step but couldn't change
the job outcome. Using `continue-on-error: true` with step ID lets
the retry step determine the final pass/fail result.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

Author: tghosth

.github/workflows/config/url-checker-config.json

@@ -20,6 +20,7 @@
     ],
     "retryOn429": true,
     "aliveStatusCodes": [200, 403],
+    "timeout": "30s",
     "fallbackRetryDelay": "30s",
     "see": "https://github.com/tcort/markdown-link-check#config-file-format"
 }

.github/workflows/url-checker.yml

@@ -37,11 +37,14 @@ jobs:
           cat /tmp/files_to_check.txt
 
       - name: Check URLs
+        id: linkcheck
+        continue-on-error: true
         run: |
+          set -o pipefail
           ERROR_FOUND=0
           while IFS= read -r file; do
             if [ -n "$file" ]; then
-              markdown-link-check "$file" --config .github/workflows/config/url-checker-config.json -q || ERROR_FOUND=1
+              markdown-link-check "$file" --config .github/workflows/config/url-checker-config.json -q 2>&1 | tee -a /tmp/link-check-output.txt || ERROR_FOUND=1
             fi
           done < /tmp/files_to_check.txt
 
@@ -53,3 +56,49 @@ jobs:
             echo ""
             echo "All links are good!"
           fi
+
+      - name: Retry status-0 failures with curl
+        if: steps.linkcheck.outcome == 'failure'
+        run: |
+          echo "Retrying status-0 (connection failure) URLs with curl..."
+          DEAD_LINKS=0
+
+          # Extract URLs that failed with status 0
+          grep -oP '\[✖\] \Khttps?://\S+(?= → Status: 0)' /tmp/link-check-output.txt > /tmp/status0-urls.txt || true
+
+          if [ ! -s /tmp/status0-urls.txt ]; then
+            echo "No status-0 failures found to retry - all failures are genuine dead links."
+            exit 1
+          fi
+
+          # Check if there were non-status-0 failures too
+          NON_ZERO_FAILURES=$(grep '\[✖\]' /tmp/link-check-output.txt | grep -v 'Status: 0' | wc -l)
+          if [ "$NON_ZERO_FAILURES" -gt 0 ]; then
+            echo "Found $NON_ZERO_FAILURES non-connection failures (these are genuine dead links):"
+            grep '\[✖\]' /tmp/link-check-output.txt | grep -v 'Status: 0'
+            DEAD_LINKS=1
+          fi
+
+          echo ""
+          echo "Retrying $(wc -l < /tmp/status0-urls.txt) status-0 URL(s) with curl..."
+          echo ""
+
+          while IFS= read -r url; do
+            if [ -n "$url" ]; then
+              HTTP_CODE=$(curl -sS -o /dev/null -w "%{http_code}" --max-time 30 --retry 2 --retry-delay 5 -L -A "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:64.0) Gecko/20100101 Firefox/64.0" "$url" 2>/dev/null || echo "000")
+              if [ "$HTTP_CODE" -ge 200 ] && [ "$HTTP_CODE" -lt 400 ]; then
+                echo "[✓] $url → curl status: $HTTP_CODE (OK)"
+              else
+                echo "[✖] $url → curl status: $HTTP_CODE (FAILED)"
+                DEAD_LINKS=1
+              fi
+            fi
+          done < /tmp/status0-urls.txt
+
+          echo ""
+          if [ "$DEAD_LINKS" -eq 1 ]; then
+            echo "ERROR: Dead links confirmed after retry!"
+            exit 1
+          else
+            echo "All status-0 URLs passed curl retry - links are alive!"
+          fi