Skip to content

[INC-300] baton-github: include the error in the returned message #98

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Bencheng21 merged 6 commits into from
Jul 24, 2025
Merged

[INC-300] baton-github: include the error in the returned message #98

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
ca19315
[INC-300] baton-github: include the error in the returned message
Bencheng21 Jul 24, 2025
b9eed40
now
Bencheng21 Jul 24, 2025
cdc712e
add isRatelimited to All list methods
Bencheng21 Jul 24, 2025
4239b2b
more
Bencheng21 Jul 24, 2025
0eaff5b
address comments
Bencheng21 Jul 24, 2025
2c8e33f
cursor
Bencheng21 Jul 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions pkg/connector/api_token.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@ import (
"github.com/conductorone/baton-sdk/pkg/annotations"
"github.com/conductorone/baton-sdk/pkg/pagination"
resourceSdk "github.com/conductorone/baton-sdk/pkg/types/resource"
"github.com/conductorone/baton-sdk/pkg/uhttp"
"github.com/google/go-github/v69/github"
"google.golang.org/grpc/codes"
)

func apiTokenResource(ctx context.Context, token *github.PersonalAccessToken) (*v2.Resource, error) {
Expand Down Expand Up @@ -92,6 +94,9 @@ func (o *apiTokenResourceType) List(
},
})
if err != nil {
if isRatelimited(resp) {
Copy link
Contributor Author

@Bencheng21 Bencheng21 Jul 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I add isRatelimited to Every API calls, including List and Grants

return nil, "", nil, uhttp.WrapErrors(codes.Unavailable, "too many requests", err)
}
return nil, "", nil, err
}

Expand Down
5 changes: 4 additions & 1 deletion pkg/connector/connector.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,7 @@ func (gh *GitHub) Validate(ctx context.Context) (annotations.Annotations, error)
membership, _, err := gh.client.Organizations.GetOrgMembership(ctx, "", o)
if err != nil {
if filterOrgs {
return nil, fmt.Errorf("access token must be an admin on the %s organization", o)
return nil, fmt.Errorf("access token must be an admin on the %s organization: %w", o, err)
}
continue
}
Expand Down Expand Up @@ -450,6 +450,9 @@ func getOrgs(ctx context.Context, client *github.Client, orgs []string) ([]strin
for {
orgs, resp, err := client.Organizations.List(ctx, "", &github.ListOptions{Page: page, PerPage: maxPageSize})
if err != nil {
if isRatelimited(resp) {
Copy link
Contributor Author

@Bencheng21 Bencheng21 Jul 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's Validate function, baton-sdk doesn't do the retry for Validate function, I don't know why we have to handle with this error.

Copy link
Contributor

@laurenleach laurenleach Jul 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are adding retry

Copy link
Contributor

@laurenleach laurenleach Jul 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ConductorOne/baton-sdk#411

Copy link
Contributor

@laurenleach laurenleach Jul 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and also the error message is not correct

can you return an error instead of nil?

Copy link
Contributor Author

@Bencheng21 Bencheng21 Jul 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, that makes sense. TIL

return nil, uhttp.WrapErrors(codes.Unavailable, "too many requests", err)
}
return nil, fmt.Errorf("github-connector: failed to retrieve org: %w", err)
}
if resp.StatusCode == http.StatusUnauthorized {
Expand Down
7 changes: 6 additions & 1 deletion pkg/connector/enterprise_role.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,9 @@ import (
"github.com/conductorone/baton-sdk/pkg/types/entitlement"
"github.com/conductorone/baton-sdk/pkg/types/grant"
resourceSdk "github.com/conductorone/baton-sdk/pkg/types/resource"
"github.com/conductorone/baton-sdk/pkg/uhttp"
"github.com/google/go-github/v69/github"
"google.golang.org/grpc/codes"
)

type enterpriseRoleResourceType struct {
Expand Down Expand Up @@ -137,8 +139,11 @@ func (o *enterpriseRoleResourceType) Grants(

ret := []*v2.Grant{}
for _, userLogin := range cache[resource.Id.Resource] {
user, _, err := o.client.Users.Get(ctx, userLogin)
user, resp, err := o.client.Users.Get(ctx, userLogin)
if err != nil {
if isRatelimited(resp) {
return nil, "", nil, uhttp.WrapErrors(codes.Unavailable, "too many requests", err)
}
return nil, "", nil, fmt.Errorf("baton-github: error getting user %s: %w", userLogin, err)
}

Expand Down
7 changes: 7 additions & 0 deletions pkg/connector/org.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,9 @@ func (o *orgResourceType) List(

orgs, resp, err := o.client.Organizations.List(ctx, "", opts)
if err != nil {
if isRatelimited(resp) {
return nil, "", nil, uhttp.WrapErrors(codes.Unavailable, "too many requests", err)
Copy link
Contributor Author

@Bencheng21 Bencheng21 Jul 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here. Add isRatelimited to List()

}
return nil, "", nil, fmt.Errorf("github-connector: failed to fetch org: %w", err)
}

Expand All @@ -126,6 +129,10 @@ func (o *orgResourceType) List(
l.Warn("insufficient access to list org membership, skipping org", zap.String("org", org.GetLogin()))
continue
}

if isRatelimited(resp) {
return nil, "", nil, uhttp.WrapErrors(codes.Unavailable, "too many requests", err)
}
return nil, "", nil, err
}

Expand Down
8 changes: 8 additions & 0 deletions pkg/connector/org_role.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,11 @@ import (
"github.com/conductorone/baton-sdk/pkg/types/entitlement"
"github.com/conductorone/baton-sdk/pkg/types/grant"
"github.com/conductorone/baton-sdk/pkg/types/resource"
"github.com/conductorone/baton-sdk/pkg/uhttp"
"github.com/google/go-github/v69/github"
"github.com/grpc-ecosystem/go-grpc-middleware/logging/zap/ctxzap"
"go.uber.org/zap"
"google.golang.org/grpc/codes"
)

type OrganizationRole struct {
Expand Down Expand Up @@ -88,6 +90,9 @@ func (o *orgRoleResourceType) List(
// Return empty list with no error to indicate we skipped this resource
return nil, "", nil, nil
}
if isRatelimited(resp) {
return nil, "", nil, uhttp.WrapErrors(codes.Unavailable, "too many requests", err)
}
return nil, "", nil, fmt.Errorf("failed to list organization roles: %w", err)
}

Expand Down Expand Up @@ -222,6 +227,9 @@ func (o *orgRoleResourceType) Grants(
}
return nil, pageToken, nil, nil
}
if isRatelimited(resp) {
return nil, "", nil, uhttp.WrapErrors(codes.Unavailable, "too many requests", err)
}
return nil, "", nil, fmt.Errorf("failed to list role teams: %w", err)
}

Expand Down
7 changes: 7 additions & 0 deletions pkg/connector/repository.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,9 @@ func (o *repositoryResourceType) Grants(
if isNotFoundError(resp) {
return nil, "", nil, uhttp.WrapErrors(codes.NotFound, fmt.Sprintf("repo: %s not found", resource.DisplayName))
}
if isRatelimited(resp) {
return nil, "", nil, uhttp.WrapErrors(codes.Unavailable, "too many requests", err)
}
return nil, "", nil, fmt.Errorf("github-connector: failed to list repos: %w", err)
}

Expand Down Expand Up @@ -233,6 +236,10 @@ func (o *repositoryResourceType) Grants(
if isNotFoundError(resp) {
return nil, "", nil, uhttp.WrapErrors(codes.NotFound, fmt.Sprintf("repo: %s not found", resource.DisplayName))
}

if isRatelimited(resp) {
return nil, "", nil, uhttp.WrapErrors(codes.Unavailable, "too many requests", err)
}
return nil, "", nil, fmt.Errorf("github-connector: failed to list repos: %w", err)
}

Expand Down
13 changes: 11 additions & 2 deletions pkg/connector/team.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,9 @@ func (o *teamResourceType) List(ctx context.Context, parentID *v2.ResourceId, pt

teams, resp, err := o.client.Teams.ListTeams(ctx, orgName, opts)
if err != nil {
if isRatelimited(resp) {
return nil, "", nil, uhttp.WrapErrors(codes.Unavailable, "too many requests", err)
}
return nil, "", nil, fmt.Errorf("github-connector: failed to list teams: %w", err)
}

Expand All @@ -104,8 +107,11 @@ func (o *teamResourceType) List(ctx context.Context, parentID *v2.ResourceId, pt
}

for _, team := range teams {
fullTeam, _, err := o.client.Teams.GetTeamByID(ctx, orgID, team.GetID()) //nolint:staticcheck // TODO: migrate to GetTeamBySlug
fullTeam, resp, err := o.client.Teams.GetTeamByID(ctx, orgID, team.GetID()) //nolint:staticcheck // TODO: migrate to GetTeamBySlug
if err != nil {
if isRatelimited(resp) {
return nil, "", nil, uhttp.WrapErrors(codes.Unavailable, "too many requests", err)
}
return nil, "", nil, err
}

Expand Down Expand Up @@ -164,8 +170,11 @@ func (o *teamResourceType) Grants(ctx context.Context, resource *v2.Resource, pT
return nil, "", nil, fmt.Errorf("error fetching orgID from team profile")
}

org, _, err := o.client.Organizations.GetByID(ctx, orgID)
org, resp, err := o.client.Organizations.GetByID(ctx, orgID)
if err != nil {
if isRatelimited(resp) {
return nil, "", nil, uhttp.WrapErrors(codes.Unavailable, "too many requests", err)
}
return nil, "", nil, err
}

Expand Down
8 changes: 8 additions & 0 deletions pkg/connector/user.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,12 @@ import (
"github.com/conductorone/baton-sdk/pkg/annotations"
"github.com/conductorone/baton-sdk/pkg/pagination"
"github.com/conductorone/baton-sdk/pkg/types/resource"
"github.com/conductorone/baton-sdk/pkg/uhttp"
"github.com/google/go-github/v69/github"
"github.com/grpc-ecosystem/go-grpc-middleware/logging/zap/ctxzap"
"github.com/shurcooL/githubv4"
"go.uber.org/zap"
"google.golang.org/grpc/codes"
"google.golang.org/protobuf/types/known/timestamppb"
)

Expand Down Expand Up @@ -131,6 +133,9 @@ func (o *userResourceType) List(ctx context.Context, parentID *v2.ResourceId, pt

users, resp, err := o.client.Organizations.ListMembers(ctx, orgName, &opts)
if err != nil {
if isRatelimited(resp) {
return nil, "", nil, uhttp.WrapErrors(codes.Unavailable, "too many requests", err)
}
return nil, "", nil, fmt.Errorf("github-connector: ListMembers failed: %w", err)
}

Expand All @@ -154,6 +159,9 @@ func (o *userResourceType) List(ctx context.Context, parentID *v2.ResourceId, pt
for _, user := range users {
u, res, err := o.client.Users.GetByID(ctx, user.GetID())
if err != nil {
if isRatelimited(res) {
return nil, "", nil, uhttp.WrapErrors(codes.Unavailable, "too many requests", err)
}
Copy link

@cursor cursor bot Jul 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: GitHub API Client Nil Response Handling

The isRatelimited function is called on GitHub API responses without a preceding nil check. GitHub API client methods can return a nil response object alongside an error (e.g., due to network issues or specific undocumented API behaviors like Users.GetByID), leading to nil pointer dereferences.

Locations (2)

Fix in Cursor • Fix in Web

Copy link
Contributor Author

@Bencheng21 Bencheng21 Jul 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isRatelimited has checked nil pointer.

// This undocumented API can return 404 for some users. If this fails it means we won't get some of their details like email
if res == nil || res.StatusCode != http.StatusNotFound {
return nil, "", nil, err
Expand Down
Loading