Copilot coding agent breaks commit syncs

[colby-swandale]

👋🏻 Hello from rubygems.org

We recently had our first PR generated from the new Copilot coding agent, which, when merged, caused our Shipit Stacks to immediately break and be marked as inaccessible from:

shipit-engine/app/jobs/shipit/github_sync_job.rb

Lines 55 to 61 in af87232

	def handle_github_errors
	yield
	rescue Octokit::NotFound
	stack.mark_as_inaccessible!
	else
	stack.mark_as_accessible!
	end

The GitHub API makes it seem like the GitHub Copilot user is a real author, but it seems as though the API to get the Copilot user will just return 404.

$ gh api repos/rubygems/rubygems.org/pulls/5929 | jq .user
{
  "login": "Copilot",
  "id": 198982749,
  "node_id": "BOT_kgDOC9w8XQ",
  "avatar_url": "https://avatars.githubusercontent.com/in/1143301?v=4",
  "gravatar_id": "",
  "url": "https://api.github.com/users/Copilot",
  "html_url": "https://github.com/apps/copilot-swe-agent",
  "followers_url": "https://api.github.com/users/Copilot/followers",
  "following_url": "https://api.github.com/users/Copilot/following{/other_user}",
  "gists_url": "https://api.github.com/users/Copilot/gists{/gist_id}",
  "starred_url": "https://api.github.com/users/Copilot/starred{/owner}{/repo}",
  "subscriptions_url": "https://api.github.com/users/Copilot/subscriptions",
  "organizations_url": "https://api.github.com/users/Copilot/orgs",
  "repos_url": "https://api.github.com/users/Copilot/repos",
  "events_url": "https://api.github.com/users/Copilot/events{/privacy}",
  "received_events_url": "https://api.github.com/users/Copilot/received_events",
  "type": "Bot",
  "user_view_type": "public",
  "site_admin": false
}

❯ gh api users/Copilot
{
  "message": "Not Found",
  "documentation_url": "https://docs.github.com/rest",
  "status": "404"
}
gh: Not Found (HTTP 404)

I'm not particularly sure what we could do to handle this particular scenario, and we've since disabled the Copilot agent on rubygems.org. I figured the best first step would be to get your reaction and see what happens from there.

[simi]

I have reported to GitHub support also.

[simi]

Meanwhile I do battle GitHub support to explain on the inconsistency, it should be possible to load the user by ID instead of handle.

[retro@retro2  ~]❤ gh api user/198982749
{
  "login": "Copilot",
  "id": 198982749,
  "node_id": "BOT_kgDOC9w8XQ",
  "avatar_url": "https://avatars.githubusercontent.com/in/1143301?v=4",
  "gravatar_id": "",
  "url": "https://api.github.com/users/Copilot",
  "html_url": "https://github.com/apps/copilot-swe-agent",
  "followers_url": "https://api.github.com/users/Copilot/followers",
  "following_url": "https://api.github.com/users/Copilot/following{/other_user}",
  "gists_url": "https://api.github.com/users/Copilot/gists{/gist_id}",
  "starred_url": "https://api.github.com/users/Copilot/starred{/owner}{/repo}",
  "subscriptions_url": "https://api.github.com/users/Copilot/subscriptions",
  "organizations_url": "https://api.github.com/users/Copilot/orgs",
  "repos_url": "https://api.github.com/users/Copilot/repos",
  "events_url": "https://api.github.com/users/Copilot/events{/privacy}",
  "received_events_url": "https://api.github.com/users/Copilot/received_events",
  "type": "Bot",
  "user_view_type": "public",
  "site_admin": false,
  "name": null,
  "company": null,
  "blog": "",
  "location": null,
  "email": null,
  "hireable": null,
  "bio": null,
  "twitter_username": null,
  "public_repos": 0,
  "public_gists": 0,
  "followers": 0,
  "following": 0,
  "created_at": "2025-02-12T20:26:08Z",
  "updated_at": "2025-02-13T21:14:06Z"
}

[indiebrain]

FWIW, the author payload of a commit made by Copilot lacks the name field. This causes the author user creation bit in Shipit::User to attempt fetching the self ref from the hypermedia links in the author payload here. This is whence the error arises.

[sndrgrdn]

Hey, I want to chime in for whom encounter the same issue.
We manually created a Copilot user as temporary bandaid until there is some kind of fix.

Shipit::User.create(
  github_id: 198982749,
  name: "Copilot",
  login: "Copilot",
  avatar_url: "https://avatars.githubusercontent.com/in/1143301?v=4",
  api_url: "https://api.github.com/users/Copilot"
)

[htrungngx]

manually create copilot user works! probably root issue comes from github api. Sent email to GH about this, hopefully they will fix it soon