Skip to content

Commit 6c58521

Browse files
NeurostepNeurostep
authored
Merge pull request #104 from scribd/maksimt/MLD-6274/redis-client
[SERF-3255] Add Redis client configuration and instrumentation
2 parents 1c0939a + 07f0a59 commit 6c58521

File tree

11 files changed

+980
-159
lines changed

11 files changed

+980
-159
lines changed

‎README.md

Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,8 @@ SDK, the Go version.
3030
- [Usage of ORM](#usage-of-orm)
3131
- [PubSub](#pubsub)
3232
- [Kafka specific configuration](#kafka-specific-configuration)
33+
- [Cache](#cache)
34+
- [Redis](#redis-specific-configuration)
3335
- [APM & Instrumentation](#apm---instrumentation)
3436
- [Request ID middleware](#request-id-middleware)
3537
- [HTTP server Request ID middleware](#http-server-request-id-middleware)
@@ -42,6 +44,8 @@ SDK, the Go version.
4244
- [AWS Session instrumentation](#aws-session-instrumentation)
4345
- [PubSub instrumentation and logging](#pubsub-instrumentation-and-logging)
4446
- [Kafka](#kafka)
47+
- [Cache instrumentation and logging](#cache-instrumentation-and-logging)
48+
- [Redis](#redis)
4549
- [Profiling](#profiling)
4650
- [Custom Metrics](#custom-metrics)
4751
- [Using the `go-sdk` in isolation](#using-the--go-sdk--in-isolation)
@@ -909,6 +913,76 @@ To authenticate the requests to Kafka, Go SDK provides a configuration set for T
909913
| Assumable role | This role will be used to establish connection to AWS MSK ignoring the static credentials | `role` | `APP_PUBSUB_KAFKA_SASL_AWS_MSK_IAM_ROLE` | string | AWS ARN string |
910914
| Session name | Will be passed to AWS STS when assuming the role | `session_name` | `APP_PUBSUB_KAFKA_SASL_AWS_MSK_IAM_SESSION_NAME` | string | application |
911915

916+
### Cache
917+
918+
`go-sdk` provides a convenient way to create an application Cache configuration.
919+
920+
Top-level configuration is split into sections specific for the vendor. Let's look into the sample `cache.yml` file:
921+
922+
```yaml
923+
common: &common
924+
store: redis
925+
redis:
926+
url: ""
927+
# Set by APP_CACHE_REDIS_ADDRS env variable
928+
addrs:
929+
- "localhost:6379"
930+
# Set by APP_CACHE_REDIS_USERNAME env variable
931+
username: "username"
932+
# Set by APP_CACHE_REDIS_PASSWORD env variable
933+
password: "password"
934+
```
935+
936+
#### Redis specific configuration
937+
938+
Redis top-level configuration is based on the official [go-redis UniversalOptions](https://pkg.go.dev/github.com/redis/go-redis/v9#UniversalOptions) library options. The following table describes the configuration options:
939+
940+
| Setting | Description | YAML variable | Environment variable (ENV) | Type | Possible Values |
941+
|-----------------------|-------------------------------------------------------------------------------------------------------------------------------------|---------------------------|-------------------------------------------|---------------|------------------------------------|
942+
| URL | URL into Redis ClusterOptions that can be used to connect to Redis | `url` | `APP_CACHE_REDIS_URL` | string | redis://:password@localhost:6379/0 |
943+
| Addrs | Either a single address or a seed list of host:port addresses of cluster/sentinel nodes. | `addrs` | `APP_CACHE_REDIS_ADDRS` | list(string) | localhost:6379 |
944+
| ClientName | ClientName will execute the `CLIENT SETNAME ClientName` command for each conn. | `client_name` | `APP_CACHE_REDIS_CLIENT_NAME` | string | client-name |
945+
| DB | Database to be selected after connecting to the server. Only single-node and failover clients. | `db` | `APP_CACHE_REDIS_DB` | int | 0 |
946+
| Protocol | Protocol 2 or 3. Use the version to negotiate RESP version with redis-server. | `protocol` | `APP_CACHE_REDIS_PROTOCOL` | int | 2, 3 |
947+
| Username | Username for Redis authentication. | `username` | `APP_CACHE_REDIS_USERNAME` | string | username |
948+
| Password | Password for Redis authentication. | `password` | `APP_CACHE_REDIS_PASSWORD` | string | password |
949+
| SentinelUsername | SentinelUsername is the username for Sentinel authentication. | `sentinel_username` | `APP_CACHE_REDIS_SENTINEL_USERNAME` | string | sentinel-username |
950+
| SentinelPassword | SentinelPassword is the password for Sentinel authentication. | `sentinel_password` | `APP_CACHE_REDIS_SENTINEL_PASSWORD` | string | sentinel-password |
951+
| MaxRetries | Maximum number of retries before giving up. | `max_retries` | `APP_CACHE_REDIS_MAX_RETRIES` | int | 0 |
952+
| MinRetryBackoff | Minimum backoff between each retry. | `min_retry_backoff` | `APP_CACHE_REDIS_MIN_RETRY_BACKOFF` | time.Duration | 8ms |
953+
| MaxRetryBackoff | Maximum backoff between each retry. | `max_retry_backoff` | `APP_CACHE_REDIS_MAX_RETRY_BACKOFF` | time.Duration | 512ms |
954+
| DialTimeout | The timeout for establishing a connection. | `dial_timeout` | `APP_CACHE_REDIS_DIAL_TIMEOUT` | time.Duration | 5s |
955+
| ReadTimeout | The timeout for socket reads. | `read_timeout` | `APP_CACHE_REDIS_READ_TIMEOUT` | time.Duration | 3s |
956+
| WriteTimeout | The timeout for socket writes. | `write_timeout` | `APP_CACHE_REDIS_WRITE_TIMEOUT` | time.Duration | 3s |
957+
| ContextTimeoutEnabled | Controls whether the client respects context timeouts and deadlines. | `context_timeout_enabled` | `APP_CACHE_REDIS_CONTEXT_TIMEOUT_ENABLED` | bool | true, false |
958+
| PoolSize | Base number of socket connections. | `pool_size` | `APP_CACHE_REDIS_POOL_SIZE` | int | 10 |
959+
| PoolTimeout | Amount of time client waits for connection if all connections are busy before returning an error. | `pool_timeout` | `APP_CACHE_REDIS_POOL_TIMEOUT` | time.Duration | 4s |
960+
| MaxIdleConns | Maximum number of idle connections. | `max_idle_conns` | `APP_CACHE_REDIS_MAX_IDLE_CONNS` | int | 10 |
961+
| MinIdleConns | Minimum number of idle connections. | `min_idle_conns` | `APP_CACHE_REDIS_MIN_IDLE_CONNS` | int | 5 |
962+
| MaxActiveConns | Maximum number of connections allocated by the pool at a given time. | `max_active_conns` | `APP_CACHE_REDIS_MAX_ACTIVE_CONNS` | int | 100 |
963+
| ConnMaxIdleTime | Maximum amount of time a connection may be idle. Should be less than server's timeout. | `conn_max_idle_time` | `APP_CACHE_REDIS_CONN_MAX_IDLE_TIME` | time.Duration | 5m |
964+
| ConnMaxLifetime | Maximum amount of time a connection may be reused. | `conn_max_lifetime` | `APP_CACHE_REDIS_CONN_MAX_LIFETIME` | time.Duration | 5m |
965+
| MaxRedirects | The maximum number of retries before giving up. Command is retried on network errors and MOVED/ASK redirects. Only cluster clients. | `max_redirects` | `APP_CACHE_REDIS_MAX_REDIRECTS` | int | 8 |
966+
| ReadOnly | Enable read-only commands on slave nodes. Only cluster clients. | `read_only` | `APP_CACHE_REDIS_READ_ONLY` | bool | true, false |
967+
| RouteByLatency | Route read-only commands to the closest master or slave node. Only cluster clients. | `route_by_latency` | `APP_CACHE_REDIS_ROUTE_BY_LATENCY` | bool | true, false |
968+
| RouteRandomly | Route read-only commands to a random master or slave node. Only cluster clients. | `route_randomly` | `APP_CACHE_REDIS_ROUTE_RANDOMLY` | bool | true, false |
969+
| MasterName | Name of the master to use for Sentinel failover. | `master_name` | `APP_CACHE_REDIS_MASTER_NAME` | string | master-name |
970+
| DisableIndentity | Disable set-lib on connect. | `disable_indentity` | `APP_CACHE_REDIS_DISABLE_INDENTITY` | bool | true, false |
971+
| IdentitySuffix | Add suffix to client name. | `identity_suffix` | `APP_CACHE_REDIS_IDENTITY_SUFFIX` | string | suffix |
972+
973+
To secure the requests to Redis, Go SDK provides a configuration set for TLS:
974+
975+
**TLS**:
976+
977+
| Setting | Description | YAML variable | Environment variable (ENV) | Type | Possible Values |
978+
|-----------------------|------------------------------------------------------------------|------------------------|--------------------------------------------|--------|---------------------|
979+
| Enabled | Whether TLS is enabled or not | `enabled` | `APP_CACHE_REDIS_TLS_ENABLED` | bool | true, false |
980+
| Root certificate | Ca Root CA certificate | `ca` | `APP_CACHE_REDIS_TLS_CA` | string | Root CA |
981+
| Cert PEM | Client's public key string (PEM format) used for authentication | `cert_pem` | `APP_CACHE_REDIS_TLS_CERT_PEM` | string | long PEM string |
982+
| Cert PEM Key | Client's private key string (PEM format) used for authentication | `cert_pem_key` | `APP_CACHE_REDIS_TLS_CERT_PEM_KEY` | string | long PEM key string |
983+
| Passphrase | Passphrase is used in case the private key needs to be decrypted | `passphrase` | `APP_CACHE_REDIS_TLS_PASSPHRASE` | string | pass phrase |
984+
| Skip TLS verification | Turn on / off TLS verification | `insecure_skip_verify` | `APP_CACHE_REDIS_TLS_INSECURE_SKIP_VERIFY` | bool | true, false |
985+
912986
## APM & Instrumentation
913987

914988
The `go-sdk` provides an easy way to add application performance monitoring
@@ -1250,6 +1324,49 @@ func main() {
12501324
}
12511325
```
12521326

1327+
### Cache instrumentation and logging
1328+
1329+
#### Redis
1330+
1331+
`go-sdk` provides a flexible way to instrument the Redis client with logging and tracing capabilities.
1332+
1333+
The [go-redis](https://github.com/redis/go-redis) Redis client library is wrapped by the `go-sdk` using [dd-trace-go](https://github.com/DataDog/dd-trace-go/tree/fda4cc5e15b744b10d378a62e026dc61d35f364a/contrib/redis/go-redis.v9) library and traces calls to Redis.
1334+
1335+
For logging, `go-sdk` sets the [go-redis](https://pkg.go.dev/github.com/redis/go-redis/v9#SetLogger) logger to the SDK logger. This logger will print `go-redis` log stubs as error log entries.
1336+
1337+
```go
1338+
import (
1339+
sdklogger "github.com/scribd/go-sdk/pkg/logger"
1340+
1341+
instrumentation "github.com/scribd/go-sdk/pkg/instrumentation"
1342+
1343+
sdkredis "github.com/scribd/go-sdk/pkg/cache/redis"
1344+
)
1345+
1346+
func main() {
1347+
config, err := sdkconfig.NewConfig()
1348+
if err != nil {
1349+
log.Fatalf("Failed to load SDK config: %s", err)
1350+
}
1351+
1352+
logger, err := sdklogger.NewBuilder(config.Logger).SetTracking(config.Tracking).Build()
1353+
if err != nil {
1354+
log.Fatalf("Failed to load SDK logger: %s", err)
1355+
}
1356+
1357+
redisClient, err := sdkredis.New(&config.Cache.Redis)
1358+
if err != nil {
1359+
logger.WithError(err).Fatalf("Failed to create Redis client: %s", err)
1360+
}
1361+
1362+
// instrument redis client
1363+
instrumentation.InstrumentRedis(redisClient, applicationName)
1364+
1365+
// set logger for redis client
1366+
sdklogger.SetRedisLogger(logger)
1367+
}
1368+
```
1369+
12531370
### Profiling
12541371

12551372
You can send `pprof` samples to DataDog by enabling the profiler.

0 commit comments

Comments
 (0)