Skip to content

Conversation

@sunny-dubey
Copy link

⏺ PR Title

perf: reuse HTTP client in task service to enable connection pooling

PR Description

Summary

Moved HTTP client creation outside the task execution loop in ExecutePendingTasks to
enable connection pooling and reuse.

Problem

A new http.Client was created inside the loop for each task:

for _, task := range tasks {
    go func(t *domain.Task) {
        httpClient := &http.Client{...}  // New client per task
        httpClient.Do(req)
    }(task)
}

According to the official Go documentation:

"Clients and Transports are safe for concurrent use by multiple goroutines and for
efficiency should only be created once and re-used."
— https://pkg.go.dev/net/http#Client

Creating a new client per request causes:
- No connection reuse (each task opens new TCP/TLS connection)
- Potential file descriptor exhaustion under load
- Unnecessary memory allocation for separate connection pools
- Repeated TLS handshakes (expensive operation)

Solution

Create a single HTTP client before the loop with proper pool configuration:

httpClient := &http.Client{
    Timeout: 53 * time.Second,
    Transport: &http.Transport{
        MaxIdleConns:        100,
        MaxIdleConnsPerHost: 100,
        IdleConnTimeout:     90 * time.Second,
        ...
    },
}

for _, task := range tasks {
    go func(t *domain.Task) {
        httpClient.Do(req)  // Reuses connections from pool
    }(task)
}

References

- Go http.Client documentation: https://pkg.go.dev/net/http#Client
- Go http.Transport documentation: https://pkg.go.dev/net/http#Transport
- "Don't create a new HTTP client for every request" -
https://www.gopherguides.com/articles/how-to-use-the-http-client-in-go

Test plan

- Existing unit tests pass (go test ./internal/service/...)
- Build succeeds (go build ./...)

@codecov
Copy link

codecov bot commented Dec 10, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.66%. Comparing base (2d8a04c) to head (2c2ca90).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #155   +/-   ##
=======================================
  Coverage   82.66%   82.66%           
=======================================
  Files         201      201           
  Lines       40945    40948    +3     
=======================================
+ Hits        33848    33851    +3     
  Misses       5378     5378           
  Partials     1719     1719           
Flag Coverage Δ
unit 82.66% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@pierre-b
Copy link
Contributor

Thanks for the info I will handle it in upcoming release!

@pierre-b pierre-b closed this Dec 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants