Skip to content
Open
Show file tree
Hide file tree
Changes from 42 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
9888798
initial commit
Umang01-hash Jul 28, 2025
0a8c265
add traces metrics and logs
Umang01-hash Jul 28, 2025
a1799e9
Merge remote-tracking branch 'origin/development' into en/db_resolver
Umang01-hash Jul 29, 2025
c75b89e
Merge remote-tracking branch 'origin/development' into en/db_resolver
Umang01-hash Jul 29, 2025
75956df
resolve linters
Umang01-hash Jul 29, 2025
2d4c32d
Merge remote-tracking branch 'origin' into en/db_resolver
Umang01-hash Aug 4, 2025
7baeb33
add mocks and tests
Umang01-hash Aug 4, 2025
547602c
improve test coverage
Umang01-hash Aug 5, 2025
686ada4
build(deps): bump go.opentelemetry.io/otel/exporters/prometheus
dependabot[bot] Aug 5, 2025
847ecf5
build(deps): bump google.golang.org/api from 0.243.0 to 0.244.0
dependabot[bot] Aug 5, 2025
af78d89
build(deps): bump gofr.dev in /examples/using-add-filestore
dependabot[bot] Aug 5, 2025
006ad33
fix connection issues of replica's
Umang01-hash Aug 5, 2025
2a3c0c6
optimize implementation for performance
Umang01-hash Aug 6, 2025
2aa7e57
fix performance bottlenecks
Umang01-hash Aug 7, 2025
552f580
add documentation
Umang01-hash Aug 7, 2025
cd6bed2
Merge remote-tracking branch 'origin' into en/db_resolver
Umang01-hash Aug 7, 2025
d3a342e
fix linters
Umang01-hash Aug 7, 2025
c4f1a23
resolve small issue in external_db.go
Umang01-hash Aug 7, 2025
f38c2ef
fix import in datasources.go
Umang01-hash Aug 7, 2025
6f22477
fix import issue in datasources.go
Umang01-hash Aug 7, 2025
a264cd5
retry import fixing
Umang01-hash Aug 7, 2025
c448892
Merge branch 'development' into en/db_resolver
Umang01-hash Aug 8, 2025
6b66f6b
resolve reveiw comments
Umang01-hash Aug 8, 2025
2b9c180
enforce dbresovler to use go 1.24.0
Umang01-hash Aug 12, 2025
11a3a07
enforce dbresovler to use go 1.24.0
Umang01-hash Aug 12, 2025
64236a9
Merge remote-tracking branch 'origin' into en/db_resolver
Umang01-hash Aug 13, 2025
ee93394
Merge branch 'development' into en/db_resolver
Umang01-hash Aug 14, 2025
eae293b
Merge branch 'development' into en/db_resolver
Umang01-hash Aug 14, 2025
6419c98
Merge branch 'development' into en/db_resolver
Umang01-hash Aug 14, 2025
da15a7d
Merge branch 'development' into en/db_resolver
Umang01-hash Aug 19, 2025
17a33c4
Merge remote-tracking branch 'origin' into en/db_resolver
Umang01-hash Aug 20, 2025
f5d0101
sort go.work dependencies
Umang01-hash Aug 20, 2025
02cc07a
Merge remote-tracking branch 'origin' into en/db_resolver
Umang01-hash Aug 25, 2025
50ed2aa
resolve review comments
Umang01-hash Aug 25, 2025
1340259
Merge branch 'development' into en/db_resolver
Umang01-hash Aug 26, 2025
0a08490
Merge remote-tracking branch 'origin' into en/db_resolver
Umang01-hash Aug 29, 2025
b379e4e
resolve PR review comments
Umang01-hash Aug 29, 2025
33d8e50
add new configs for db_replica_users and passwords
Umang01-hash Aug 29, 2025
b53ab3b
Merge branch 'development' into en/db_resolver
coolwednesday Aug 29, 2025
7ef7fc5
add new configs in docs
Umang01-hash Aug 29, 2025
a8440e2
Merge remote-tracking branch 'origin' into en/db_resolver
Umang01-hash Aug 29, 2025
2adf3f0
Merge remote-tracking branch 'origin/en/db_resolver' into en/db_resolver
Umang01-hash Aug 29, 2025
ae3b0d5
resolve linters
Umang01-hash Aug 29, 2025
64cf385
Merge branch 'development' into en/db_resolver
Umang01-hash Aug 30, 2025
de83572
Merge branch 'development' into en/db_resolver
Umang01-hash Sep 1, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions docs/quick-start/connecting-mysql/page.md
Original file line number Diff line number Diff line change
Expand Up @@ -239,3 +239,73 @@ Now when we access {% new-tab-link title="http://localhost:9000/customer" href="
```

**Note:** When using PostgreSQL or Supabase, you may need to use `$1` instead of `?` in SQL queries, depending on your driver configuration.

### Enabling Read/Write Splitting in MySQL (DBResolver)
GoFr provides built-in support for read/write splitting using its `DBRESOLVER` module for **MySQL**.
This feature allows applications to route write queries (e.g., `INSERT`, `UPDATE`) to the **primary database**, and
distribute read queries (`SELECT`) across **one or more replicas**, boosting performance, scalability, and reliability.

Import the GoFr's dbresolver for MySQL:

```shell
go get gofr.dev/pkg/gofr/datasource/dbresolver@latest
```

After importing the package, you can configure the DBResolver in your GoFr application using the `AddDBResolver` method.
You can choose the load balancing strategy and enable fallback to primary:

```go
// Add DB resolver with round-robin strategy and fallback enabled
resolver := dbresolver.NewProvider("round-robin", true)
a.AddDBResolver(resolver)
```

- The first argument specifies the **load balancing strategy** for read queries. Supported values:
- `round-robin`: Distributes reads evenly across replicas.
- `random`: Selects a replica at random for each read.
- The second argument enables **fallback** to the primary if all replicas are unavailable.

### Configuration

#### 1. Replica Hosts
Add replica hosts, ports,users, passwords to your `.env` file using the following configs:

```env
DB_REPLICA_HOSTS=localhost,replica1,replica2
DB_REPLICA_PORTS=3307,3308,3309
DB_REPLICA_USERS=readonly1,readonly2,readonly3
DB_REPLICA_PASSWORDS=pass1,pass2,pass3

```

These hosts will be treated as **read replicas**.

#### 2. Replica Connection Pool Tuning
By default, GoFr automatically scales connection pools for replicas based on your primary database settings:

`DB_MAX_IDLE_CONNECTION` → multiplied by 4
`DB_MAX_OPEN_CONNECTION` → multiplied by 2

This ensures replicas can handle higher read concurrency without impacting the primary.
GoFr also applies min/max caps to keep values safe. These can be customized:

```go
# Primary DB pool settings
DB_MAX_IDLE_CONNECTION=2
DB_MAX_OPEN_CONNECTION=20

# Replica pool overrides (optional)
DB_REPLICA_MAX_IDLE_CAP=100
DB_REPLICA_MIN_IDLE=5
DB_REPLICA_DEFAULT_IDLE=15

DB_REPLICA_MAX_OPEN_CAP=500
DB_REPLICA_MIN_OPEN=20
DB_REPLICA_DEFAULT_OPEN=150

```

**Benefits**
- Performance: Offloads read traffic from the primary, reducing latency.
- Scalability: Easily scale reads by adding more replicas.
- Resilience: Ensures high availability through automatic fallback.
61 changes: 61 additions & 0 deletions docs/references/configs/page.md
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,67 @@ This document lists all the configuration options supported by the GoFr framewor

---

- DB_REPLICA_HOSTS
- Comma-separated list of replica database hosts. Used for read replicas.
- None

---

- DB_REPLICA_PORTS
- Comma-separated list of replica database ports. Used for read replicas.
- None

---

- DB_REPLICA_USERS
- Comma-separated list of replica database users. Used for read replicas.
- None

---

- DB_REPLICA_PASSWORDS_
- Comma-separated list of replica database passwords. Used for read replicas.
- None

---

- DB_REPLICA_MAX_IDLE_CONNECTIONS
- Maximum idle connections allowed for a replica
- 50

---

- DB_REPLICA_MIN_IDLE_CONNECTIONS
- Minimum idle connections for a replica
- 10

---

- DB_REPLICA_DEFAULT_IDLE_CONNECTIONS
- Idle connections used if no primary setting is provided
- 10

---

- DB_REPLICA_MAX_OPEN_CONNECTIONS
- Maximum open connections allowed for a replica
- 200

---

- DB_REPLICA_MIN_OPEN_CONNECTIONS
- Minimum open connections for a replica
- 50

---

- DB_REPLICA_DEFAULT_OPEN_CONNECTIONS
- Open connections used if no primary setting is provided
- 100

---


- DB_CHARSET
- The character set for database connection
- utf8
Expand Down
1 change: 1 addition & 0 deletions go.work
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ use (
./pkg/gofr/datasource/cassandra
./pkg/gofr/datasource/clickhouse
./pkg/gofr/datasource/couchbase
./pkg/gofr/datasource/dbresolver
./pkg/gofr/datasource/dgraph
./pkg/gofr/datasource/elasticsearch
./pkg/gofr/datasource/file/ftp
Expand Down
54 changes: 52 additions & 2 deletions go.work.sum

Large diffs are not rendered by default.

8 changes: 8 additions & 0 deletions pkg/gofr/container/datasources.go
Original file line number Diff line number Diff line change
Expand Up @@ -776,3 +776,11 @@ type CouchbaseProvider interface {

provider
}

// DBResolverProvider defines an interface for SQL read/write splitting providers.
type DBResolverProvider interface {
// Build creates a resolver with the given primary and replicas.
Build(primary DB, replicas []DB) (DB, error)

provider
}
87 changes: 87 additions & 0 deletions pkg/gofr/container/mock_datasources.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

55 changes: 55 additions & 0 deletions pkg/gofr/datasource/dbresolver/circuit_breaker.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
package dbresolver

import (
"sync/atomic"
"time"
)

// Circuit breaker for replica health with atomic operations.
type circuitBreaker struct {
failures atomic.Int32
lastFailure atomic.Int64
state atomic.Int32 // 0=closed, 1=open, 2=half-open.
maxFailures int32
timeout time.Duration
}

func newCircuitBreaker(maxFailures int32, timeout time.Duration) *circuitBreaker {
return &circuitBreaker{
maxFailures: maxFailures,
timeout: timeout,
}
}

func (cb *circuitBreaker) allowRequest() bool {
state := cb.state.Load()

switch state {
case circuitStateClosed:
return true
case circuitStateOpen:
if time.Since(time.Unix(0, cb.lastFailure.Load())) > cb.timeout {
return cb.state.CompareAndSwap(circuitStateOpen, circuitStateHalfOpen)
}

return false
case circuitStateHalfOpen:
return true
default:
return true
}
}

func (cb *circuitBreaker) recordSuccess() {
cb.failures.Store(0)
cb.state.Store(0)
}

func (cb *circuitBreaker) recordFailure() {
failures := cb.failures.Add(1)
cb.lastFailure.Store(time.Now().UnixNano())

if failures >= cb.maxFailures {
cb.state.Store(1)
}
}
78 changes: 78 additions & 0 deletions pkg/gofr/datasource/dbresolver/dbresolver_wrapper.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
package dbresolver

import (
"errors"

"go.opentelemetry.io/otel/trace"
"gofr.dev/pkg/gofr/container"
)

var (
errPrimaryNil = errors.New("primary database cannot be nil")
errReplicaFailedNoFallback = errors.New("replica query failed and fallback disabled")
)

// ResolverWrapper implements container.DBResolverProvider interface
// It acts as a factory that creates a Resolver with given config.
type ResolverWrapper struct {
logger Logger
metrics Metrics
tracer trace.Tracer
strategy Strategy
readFallback bool
}

// NewProvider creates a new ResolverWrapper with strategy and fallback config.
func NewProvider(strategy Strategy, readFallback bool) *ResolverWrapper {
return &ResolverWrapper{
strategy: strategy,
readFallback: readFallback,
}
}

// UseLogger sets the logger instance.
func (r *ResolverWrapper) UseLogger(logger any) {
if l, ok := logger.(Logger); ok {
r.logger = l
}
}

// UseMetrics sets the metrics instance.
func (r *ResolverWrapper) UseMetrics(metrics any) {
if m, ok := metrics.(Metrics); ok {
r.metrics = m
}
}

// UseTracer sets the tracer instance.
func (r *ResolverWrapper) UseTracer(tracer any) {
if t, ok := tracer.(trace.Tracer); ok {
r.tracer = t
}
}

// Connect is no-op for wrapper as connections are created externally.
func (*ResolverWrapper) Connect() {
// no-op
}

// Build creates a Resolver instance with primary and replica DBs.
func (r *ResolverWrapper) Build(primary container.DB, replicas []container.DB) (container.DB, error) {
if primary == nil {
return nil, errPrimaryNil
}

// Update RoundRobinStrategy count if needed
if rr, ok := r.strategy.(*RoundRobinStrategy); ok {
rr.count = len(replicas)
}

// Create options slice
var opts []Option

// Add options.
opts = append(opts, WithStrategy(r.strategy), WithFallback(r.readFallback))

// Create and return the resolver.
return NewResolver(primary, replicas, r.logger, r.metrics, opts...), nil
}
Loading
Loading