Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 24 additions & 34 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

[![Go Report Card](https://goreportcard.com/badge/github.com/gofri/go-github-ratelimit)](https://goreportcard.com/report/github.com/gofri/go-github-ratelimit)

Package `go-github-ratelimit` providesa middleware (http.RoundTripper) that handles both [Primary Rate Limit](https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?#about-primary-rate-limits) and [Secondary Rate Limit](https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?#about-secondary-rate-limits) for the GitHub API.
Package `go-github-ratelimit` provides a middleware (http.RoundTripper) that handles both [Primary Rate Limit](https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?#about-primary-rate-limits) and [Secondary Rate Limit](https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api?#about-secondary-rate-limits) for the GitHub API.

* Primary rate limits are handled by returning a detailed error.
* Secondary rate limits are handled by waiting in blocking mode (sleep) and then issuing/retrying requests.
Expand All @@ -24,22 +24,21 @@ It is best to stack the pagination round-tripper on top of the rate limit round-

## Usage Example (with [go-github](https://github.com/google/go-github))

see [example/basic.go](example/basic.go) for a runnable example.
```go
import "github.com/google/go-github/v69/github"
import "github.com/gofri/go-github-ratelimit/v2/github_ratelimit"
rateLimiter := github_ratelimit.NewClient(nil)
client := github.NewClient(rateLimiter) // .WithAuthToken("your personal access token")

func main() {
// use the plain ratelimiter, without options / callbacks / underlying http.RoundTripper.
rateLimiter, err := github_ratelimit.New(nil)
if err != nil {
panic(err)
}
client := github.NewClient(rateLimiter).WithAuthToken("your personal access token")
// disable go-github's built-in rate limiting
ctx := context.WithValue(context.Background(), github.BypassRateLimitCheck, true)

// disable go-github's built-in rate limiting
ctx := context.WithValue(context.Background(), github.BypassRateLimitCheck)
tags, _, err := client.Repositories.ListTags(ctx, "gofri", "go-github-ratelimit", nil)
if err != nil {
panic(err)
}

// now use the client as you please
for _, tag := range tags {
fmt.Printf("- %v\n", *tag.Name)
}
```

Expand All @@ -48,7 +47,7 @@ func main() {
Both RoundTrippers support a set of options to configure their behavior and set callbacks.
nil callbacks are treated as no-op.

### Primary Rate Limit Options:
### Primary Rate Limit Options (see [options.go](github_ratelimit/github_primary_ratelimit/options.go)):

- `WithLimitDetectedCallback(callback)`: the callback is triggered when any primary rate limit is detected.
- `WithRequestPreventedCallback(callback)`: the callback is triggered when a request is prevented due to an active rate limit.
Expand All @@ -57,7 +56,7 @@ nil callbacks are treated as no-op.
- `WithSharedState(state)`: share state between multiple clients (e.g., for a single user running concurrently).
- `WithBypassLimit()`: bypass the rate limit mechanism, i.e., do not prevent requests when a rate limit is active.

### Secondary Rate Limit Options:
### Secondary Rate Limit Options (see [options.go](github_ratelimit/github_secondary_ratelimit/options.go)):

- `WithLimitDetectedCallback(callback)`: the callback is triggered before a sleep.
- `WithSingleSleepLimit(duration, callback)`: limit the sleep duration for a single secondary rate limit & trigger a callback when the limit is exceeded.
Expand All @@ -72,18 +71,8 @@ as well as fine-grained policy control (e.g., for a sophisticated pagination mec

## Advanced Example

See [example/advanced.go](example/advanced.go) for a runnable example.
```go
import "github.com/google/go-github/v69/github"
import "github.com/gofri/go-github-ratelimit/v2/github_ratelimit"
import "github.com/gofri/go-github-ratelimit/v2/github_ratelimit/github_primary_ratelimit"
import "github.com/gofri/go-github-ratelimit/v2/github_ratelimit/github_secondary_ratelimit"
import "github.com/gofri/go-github-pagination/githubpagination"

func main() {
var username string
fmt.Print("Enter GitHub username: ")
fmt.Scanf("%s", &username)

rateLimiter := github_ratelimit.New(nil,
github_primary_ratelimit.WithLimitDetectedCallback(func(ctx *github_primary_ratelimit.CallbackContext) {
fmt.Printf("Primary rate limit detected: category %s, reset time: %v\n", ctx.Category, ctx.ResetTime)
Expand All @@ -96,19 +85,20 @@ func main() {
paginator := githubpagination.NewClient(rateLimiter,
githubpagination.WithPerPage(100), // default to 100 results per page
)
client := github.NewClient(paginator)
client := github.NewClient(paginator) // .WithAuthToken("your personal access token")

// arbitrary usage of the client
repos, _, err := client.Repositories.ListByUser(context.Background(), username, nil)
// disable go-github's built-in rate limiting
ctx := context.WithValue(context.Background(), github.BypassRateLimitCheck, true)

// list repository tags
tags, _, err := client.Repositories.ListTags(ctx, "gofri", "go-github-ratelimit", nil)
if err != nil {
fmt.Printf("Error: %v\n", err)
return
panic(err)
}

for i, repo := range repos {
fmt.Printf("%v. %v\n", i+1, repo.GetName())
for _, tag := range tags {
fmt.Printf("- %v\n", *tag.Name)
}
}
```

## Migration (V1 => V2)
Expand Down
41 changes: 41 additions & 0 deletions example/advanced.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
package main

import (
"context"
"fmt"

"github.com/gofri/go-github-pagination/githubpagination"
"github.com/gofri/go-github-ratelimit/v2/github_ratelimit"
"github.com/gofri/go-github-ratelimit/v2/github_ratelimit/github_primary_ratelimit"
"github.com/gofri/go-github-ratelimit/v2/github_ratelimit/github_secondary_ratelimit"
"github.com/google/go-github/v69/github"
)

func main() {
rateLimiter := github_ratelimit.New(nil,
github_primary_ratelimit.WithLimitDetectedCallback(func(ctx *github_primary_ratelimit.CallbackContext) {
fmt.Printf("Primary rate limit detected: category %s, reset time: %v\n", ctx.Category, ctx.ResetTime)
}),
github_secondary_ratelimit.WithLimitDetectedCallback(func(ctx *github_secondary_ratelimit.CallbackContext) {
fmt.Printf("Secondary rate limit detected: reset time: %v, total sleep time: %v\n", ctx.ResetTime, ctx.TotalSleepTime)
}),
)

paginator := githubpagination.NewClient(rateLimiter,
githubpagination.WithPerPage(100), // default to 100 results per page
)
client := github.NewClient(paginator) // .WithAuthToken("your personal access token")

// disable go-github's built-in rate limiting
ctx := context.WithValue(context.Background(), github.BypassRateLimitCheck, true)

// list repository tags
tags, _, err := client.Repositories.ListTags(ctx, "gofri", "go-github-ratelimit", nil)
if err != nil {
panic(err)
}

for _, tag := range tags {
fmt.Printf("- %v\n", *tag.Name)
}
}
27 changes: 27 additions & 0 deletions example/basic.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
package main

import (
"context"
"fmt"

"github.com/gofri/go-github-ratelimit/v2/github_ratelimit"
"github.com/google/go-github/v69/github"
)

func main() {
// use the plain ratelimiter, without options / callbacks / underlying http.RoundTripper.
rateLimiter := github_ratelimit.NewClient(nil)
client := github.NewClient(rateLimiter) // .WithAuthToken("your personal access token")

// disable go-github's built-in rate limiting
ctx := context.WithValue(context.Background(), github.BypassRateLimitCheck, true)

tags, _, err := client.Repositories.ListTags(ctx, "gofri", "go-github-ratelimit", nil)
if err != nil {
panic(err)
}

for _, tag := range tags {
fmt.Printf("- %v\n", *tag.Name)
}
}
13 changes: 13 additions & 0 deletions example/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
module github.com/gofri/go-github-ratelimit/v2/example

replace github.com/gofri/go-github-ratelimit/v2 => ../

go 1.23.1

require (
github.com/gofri/go-github-pagination v1.0.0
github.com/gofri/go-github-ratelimit/v2 v2.0.1
github.com/google/go-github/v69 v69.2.0
)

require github.com/google/go-querystring v1.1.0 // indirect
10 changes: 10 additions & 0 deletions example/go.sum
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
github.com/gofri/go-github-pagination v1.0.0 h1:nnCi+1xT5ybqY/plctISgiQPWZOtfSciVQlbx/hM/Yw=
github.com/gofri/go-github-pagination v1.0.0/go.mod h1:Qij55Fb4fNPjam3SB+8cLnqp4pgR8RGMyIspYXcyHX0=
github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-github/v69 v69.2.0 h1:wR+Wi/fN2zdUx9YxSmYE0ktiX9IAR/BeePzeaUUbEHE=
github.com/google/go-github/v69 v69.2.0/go.mod h1:xne4jymxLR6Uj9b7J7PyTpkMYstEMMwGZa0Aehh1azM=
github.com/google/go-querystring v1.1.0 h1:AnCroh3fv4ZBgVIf1Iwtovgjaw/GiKJo8M8yD/fhyJ8=
github.com/google/go-querystring v1.1.0/go.mod h1:Kcdr2DB4koayq7X8pmAG4sNG59So17icRSOU623lUBU=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
8 changes: 8 additions & 0 deletions github_ratelimit/github_ratelimiter.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ func NewSecondaryLimiter(base http.RoundTripper, opts ...SecondaryRateLimiterOpt

// New creates a combined limiter by stacking a SecondaryRateLimiter on top of a PrimaryRateLimiterOption.
// It accepts options of both types and creates the RoundTrippers.
// Check out options.go @ github_primary_ratelimit / github_secondary_ratelimit for available options.
func New(base http.RoundTripper, opts ...any) http.RoundTripper {
primaryOpts, secondaryOpts := gatherOptions(opts...)
primary := NewPrimaryLimiter(base, primaryOpts...)
Expand All @@ -37,6 +38,13 @@ func New(base http.RoundTripper, opts ...any) http.RoundTripper {
return secondary
}

// NewClient creates a new HTTP client with the combined rate limiter.
func NewClient(base http.RoundTripper, opts ...any) *http.Client {
return &http.Client{
Transport: New(base, opts...),
}
}

// WithOverrideConfig adds config overrides to the context.
// The overrides are applied on top of the existing config.
// Allows for request-specific overrides.
Expand Down
4 changes: 2 additions & 2 deletions github_ratelimit/github_secondary_ratelimit/callback.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ type CallbackContext struct {
Response *http.Response
}

// OnLimitDetected is a callback to be called when a new rate limit is detected (before the sleep)
// The totalSleepTime includes the sleep duration for the upcoming sleep
// OnLimitDetected is a callback to be called when a new rate limit is detected (before the sleep).
// The totalSleepTime includes the sleep duration for the upcoming sleep.
// Note: called while holding the lock.
type OnLimitDetected func(*CallbackContext)

Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
module github.com/gofri/go-github-ratelimit/v2

go 1.23.1
go 1.23.0