CLI and README updates (#76)

Cleaned up CLI options (Astra-specific flags are prefixed with `--astra`). Updated the `README` to favor the use of Astra tokens over using the bundle because it's much easier for users. 

Also,
* Logging wasn't working in `Session` and `Cluster` because it wasn't being set
* Added new flag for `--num-conns` to increase the number of connection per pool. It's possible that it could increase performance in some scenarios, but users should be wary of increasing this value too much.
* Added addition logging to the pool and cluster to aid in debugging outage testing

Co-authored-by: Lorina Poland <[email protected]>

Author: mpenick

README.md

@@ -19,9 +19,6 @@
 
 `cql-proxy` is designed to forward your application's CQL traffic to an appropriate database service. It listens on a local address and securely forwards that traffic.
 
-**Warning**: `cql-proxy` in its current state works well, and you should give it a try. However, it is still under development, so things might break or change. 
-
-Please give it a try and let us know what you think!
 ## When to use `cql-proxy`
 
 The `cql-proxy` sidecar enables unsupported CQL drivers to work with [DataStax Astra][astra]. These drivers include both legacy DataStax [drivers] and community-maintained CQL drivers, such as the [gocql] driver and the [rust-driver].
@@ -39,32 +36,43 @@ $ ./cql-proxy -h
 Usage: cql-proxy
 
 Flags:
-  -h, --help               Show context-sensitive help.
-  -b, --bundle=STRING      Path to secure connect bundle ($BUNDLE)
-  -u, --username=STRING    Username to use for authentication ($USERNAME)
-  -p, --password=STRING    Password to use for authentication ($PASSWORD)
-  -c, --contact-points=CONTACT-POINTS,...
-                           Contact points for cluster. Ignored if using the bundle path
-                           option ($CONTACT_POINTS).
-  -a, --bind=STRING        Address to use to bind serve ($BIND)
-      --debug              Show debug logging ($DEBUG)
-      --profiling          Enable profiling ($PROFILING)
+  -h, --help                                 Show context-sensitive help.
+  -b, --astra-bundle=STRING                  Path to secure connect bundle for an Astra database. Requires '--username' and '--password'. Ignored if using the token or contact points option
+                                             ($ASTRA_BUNDLE).
+  -t, --astra-token=STRING                   Token used to authenticate to an Astra database. Requires '--astra-database-id'. Ignored if using the bundle path or contact points option
+                                             ($ASTRA_TOKEN).
+  -i, --astra-database-id=STRING             Database ID of the Astra database. Requires '--astra-token' ($ASTRA_DATABASE_ID)
+  -c, --contact-points=CONTACT-POINTS,...    Contact points for cluster. Ignored if using the bundle path or token option ($CONTACT_POINTS).
+  -u, --username=STRING                      Username to use for authentication ($USERNAME)
+  -p, --password=STRING                      Password to use for authentication ($PASSWORD)
+  -r, --port=9042                            Default port to use when connecting to cluster ($PORT)
+  -n, --protocol-version="v4"                Initial protocol version to use when connecting to the backend cluster (default: v4, options: v3, v4, v5, DSEv1, DSEv2) ($PROTOCOL_VERSION)
+  -m, --max-protocol-version="v4"            Max protocol version supported by the backend cluster (default: v4, options: v3, v4, v5, DSEv1, DSEv2) ($MAX_PROTOCOL_VERSION)
+  -a, --bind=":9042"                         Address to use to bind server ($BIND)
+      --debug                                Show debug logging ($DEBUG)
+      --health-check                         Enable liveness and readiness checks ($HEALTH_CHECK)
+      --http-bind=":8000"                    Address to use to bind HTTP server used for health checks ($HTTP_BIND)
+      --heartbeat-interval=30s               Interval between performing heartbeats to the cluster ($HEARTBEAT_INTERVAL)
+      --idle-timeout=60s                     Duration between successful heartbeats before a connection to the cluster is considered unresponsive and closed ($IDLE_TIMEOUT)
+      --readiness-timeout=30s                Duration the proxy is unable to connect to the backend cluster before it is considered not ready ($READINESS_TIMEOUT)
+      --num-conns=1                          Number of connection to create to each node of the backend cluster ($NUM_CONNS)
 ```
 
-To pass configuration to `cql-proxy`, either command-line flags or environment variables can be used. Using the `docker` method as an example, the follwing samples show how username, password and bundle are defined with each method.
+To pass configuration to `cql-proxy`, either command-line flags or environment variables can be used. Using the `docker` method as an example, the following samples show how the token and database ID are defined with each method.
 ### Using flags
 
 ```sh
-docker run -v <your-secure-connect-bundle.zip>:/tmp/scb.zip -p 9042:9042 \
+docker run -p 9042:9042 \
   --rm datastax/cql-proxy:v0.0.4 \
-  --bundle /tmp/scb.zip --username <astra-client-id> --password <astra-client-secret>
+  --astra-token <astra-token> --astra-database-id <astra-datbase-id>
 ```
+
 ### Using environment variables
 
 ```sh
-docker run -v <your-secure-connect-bundle.zip>:/tmp/scb.zip -p 9042:9042  \
+docker run -p 9042:9042  \
   --rm datastax/cql-proxy:v0.0.4 \
-  -e BUNDLE=/tmp/scb.zip -e USERNAME=<astra-client-id> -e PASSWORD=<astra-client-secret>
+  -e ASTRA_TOKEN=<astra-token> -e ASTRA_DATABASE_ID=<astra-datbase-id>
 ```
 ## Getting started
 
@@ -86,13 +94,20 @@ There are three methods for using `cql-proxy`:
    - [DataStax Astra][astra] cluster:
 
       ```sh
+      ./cql-proxy --astra-token <astra-token> --astra-database-id <astra-database-id>
+      ```
+
+      The `<astra-token>` can be generated using these [instructions]. The proxy also supports using the [Astra Secure Connect Bundle][bundle] along with a client ID and secret generated using these [instructions]:
+
+      ```
       ./cql-proxy --bundle <your-secure-connect-zip> \
       --username <astra-client-id> --password <astra-client-secret>
       ```
+
    - [Apache Cassandra][cassandra] cluster:
 
       ```sh
-      ./cql-proxy --contact-points <cluster node IPs or DNS names>
+      ./cql-proxy --contact-points <cluster node IPs or DNS names> [--username <username>] [--password <password>]
       ```
 ### Run a `cql-proxy` docker image
 
@@ -101,18 +116,24 @@ There are three methods for using `cql-proxy`:
    - [DataStax Astra][astra] cluster:
 
       ```sh
-      docker run -v <your-secure-connect-bundle.zip>:/tmp/scb.zip -p 9042:9042 \
+      docker run -p 9042:9042 \
         datastax/cql-proxy:v0.0.4 \
-        --bundle /tmp/scb.zip --username <astra-client-id> --password <astra-client-secret>
+        --astra-token <astra-token> --astra-database-id <astra-database-id>
       ```
-      The `<astra-client-id>` and `<astra-client-secret>` can be generated using these [instructions].
 
+      The `<astra-token>` can be generated using these [instructions]. The proxy also supports using the [Astra Secure Connect Bundle][bundle], but it requires mounting the bundle to a volume in the container:
+
+      ```sh
+      docker run -v <your-secure-connect-bundle.zip>:/tmp/scb.zip -p 9042:9042 \
+      --rm datastax/cql-proxy:v0.0.4 \
+      --astra-bundle /tmp/scb.zip --username <astra-client-id> --password <astra-client-secret>
+      ```
    - [Apache Cassandra][cassandra] cluster:
 
       ```sh
       docker run -p 9042:9042 \
         datastax/cql-proxy:v0.0.4 \
-        --contact-points <cluster node IPs or DNS names>
+        --contact-points <cluster node IPs or DNS names> [--username <username>] [--password <password>]
       ```
   If you wish to have the docker image removed after you are done with it, add `--rm` before the image name `datastax/cql-proxy:v0.0.4 `.
 
@@ -128,7 +149,7 @@ Using Kubernetes with `cql-proxy` requires a number of steps:
 
       ```
       command: ["./cql-proxy"]
-      args: ["--bundle=/tmp/scb.zip","--username=Client ID","--password=Client Secret"]
+      args: ["--astra-bundle=/tmp/scb.zip","--username=Client ID","--password=Client Secret"]
       ```
 
 - Volume mounts: Modify `/tmp/` as a volume mount as required.
@@ -188,6 +209,6 @@ Using Kubernetes with `cql-proxy` requires a number of steps:
 [cassandra]: https://cassandra.apache.org/
 [dse]: https://www.datastax.com/products/datastax-enterprise
 [instructions]: https://docs.datastax.com/en/astra/docs/manage-application-tokens.html
-
+[bundle]: https://docs.datastax.com/en/astra/docs/obtaining-database-credentials.html#_getting_your_secure_connect_bundle
 
 

astra/bundle.go

@@ -32,8 +32,6 @@ import (
 	"github.com/datastax/astra-client-go/v2/astra"
 )
 
-const URL = "https://api.astra.datastax.com"
-
 type Bundle struct {
 	tlsConfig *tls.Config
 	host      string

proxy/proxy.go

@@ -140,6 +140,7 @@ func (p *Proxy) Listen(address string) error {
 		ReconnectPolicy:   p.config.ReconnectPolicy,
 		HeartBeatInterval: p.config.HeartBeatInterval,
 		IdleTimeout:       p.config.IdleTimeout,
+		Logger:            p.logger,
 	})
 
 	if err != nil {
@@ -167,6 +168,7 @@ func (p *Proxy) Listen(address string) error {
 		HeartBeatInterval: p.config.HeartBeatInterval,
 		IdleTimeout:       p.config.IdleTimeout,
 		PreparedCache:     p.preparedCache,
+		Logger:            p.logger,
 	})
 
 	if err != nil {
@@ -245,6 +247,7 @@ func (p *Proxy) maybeCreateSession(version primitive.ProtocolVersion, keyspace s
 			Keyspace:          keyspace,
 			HeartBeatInterval: p.config.HeartBeatInterval,
 			IdleTimeout:       p.config.IdleTimeout,
+			Logger:            p.logger,
 		})
 		if err != nil {
 			return nil, err

proxy/run.go

@@ -35,13 +35,14 @@ const livenessPath = "/liveness"
 const readinessPath = "/readiness"
 
 var cli struct {
-	Bundle             string        `help:"Path to secure connect bundle" short:"b" env:"BUNDLE"`
+	AstraBundle        string        `help:"Path to secure connect bundle for an Astra database. Requires '--username' and '--password'. Ignored if using the token or contact points option." short:"b" env:"ASTRA_BUNDLE"`
+	AstraToken         string        `help:"Token used to authenticate to an Astra database. Requires '--astra-database-id'. Ignored if using the bundle path or contact points option." short:"t" env:"ASTRA_TOKEN"`
+	AstraDatabaseID    string        `help:"Database ID of the Astra database. Requires '--astra-token'" short:"i" env:"ASTRA_DATABASE_ID"`
+	AstraApiURL        string        `help:"URL for the Astra API" default:"https://api.astra.datastax.com" env:"ASTRA_API_URL"`
+	ContactPoints      []string      `help:"Contact points for cluster. Ignored if using the bundle path or token option." short:"c" env:"CONTACT_POINTS"`
 	Username           string        `help:"Username to use for authentication" short:"u" env:"USERNAME"`
 	Password           string        `help:"Password to use for authentication" short:"p" env:"PASSWORD"`
-	Token              string        `help:"Token" short:"t" env:"TOKEN"`
-	DatabaseID         string        `help:"Database ID" short:"i" env:"DATABASE_ID"`
-	ContactPoints      []string      `help:"Contact points for cluster. Ignored if using the bundle path option." short:"c" env:"CONTACT_POINTS"`
-	Port               int           `help:"Default port to use when connecting to cluster" default:"9042" short:"t" env:"PORT"`
+	Port               int           `help:"Default port to use when connecting to cluster" default:"9042" short:"r" env:"PORT"`
 	ProtocolVersion    string        `help:"Initial protocol version to use when connecting to the backend cluster (default: v4, options: v3, v4, v5, DSEv1, DSEv2)" default:"v4" short:"n" env:"PROTOCOL_VERSION"`
 	MaxProtocolVersion string        `help:"Max protocol version supported by the backend cluster (default: v4, options: v3, v4, v5, DSEv1, DSEv2)" default:"v4" short:"m" env:"MAX_PROTOCOL_VERSION"`
 	Bind               string        `help:"Address to use to bind server" short:"a" default:":9042" env:"BIND"`
@@ -51,6 +52,7 @@ var cli struct {
 	HeartbeatInterval  time.Duration `help:"Interval between performing heartbeats to the cluster" default:"30s" env:"HEARTBEAT_INTERVAL"`
 	IdleTimeout        time.Duration `help:"Duration between successful heartbeats before a connection to the cluster is considered unresponsive and closed" default:"60s" env:"IDLE_TIMEOUT"`
 	ReadinessTimeout   time.Duration `help:"Duration the proxy is unable to connect to the backend cluster before it is considered not ready" default:"30s" env:"READINESS_TIMEOUT"`
+	NumConns           int           `help:"Number of connection to create to each node of the backend cluster" default:"1" env:"NUM_CONNS"`
 }
 
 // Run starts the proxy command. 'args' shouldn't include the executable (i.e. os.Args[1:]). It returns the exit code
@@ -70,33 +72,39 @@ func Run(ctx context.Context, args []string) int {
 	}
 
 	var resolver proxycore.EndpointResolver
-	if len(cli.Bundle) > 0 {
-		if bundle, err := astra.LoadBundleZipFromPath(cli.Bundle); err != nil {
-			cliCtx.Errorf("unable to open bundle %s from file: %v", cli.Bundle, err)
+	if len(cli.AstraBundle) > 0 {
+		if bundle, err := astra.LoadBundleZipFromPath(cli.AstraBundle); err != nil {
+			cliCtx.Errorf("unable to open bundle %s from file: %v", cli.AstraBundle, err)
 			return 1
 		} else {
 			resolver = astra.NewResolver(bundle)
 		}
-	} else if len(cli.ContactPoints) > 0 {
-		resolver = proxycore.NewResolverWithDefaultPort(cli.ContactPoints, cli.Port)
-	} else if len(cli.Token) > 0 {
-		if len(cli.DatabaseID) == 0 {
+	} else if len(cli.AstraToken) > 0 {
+		if len(cli.AstraDatabaseID) == 0 {
 			cliCtx.Fatalf("database ID is required when using a token")
 		}
-		bundle, err := astra.LoadBundleZipFromURL(astra.URL, cli.DatabaseID, cli.Token, 10*time.Second)
+		bundle, err := astra.LoadBundleZipFromURL(cli.AstraApiURL, cli.AstraDatabaseID, cli.AstraToken, 10*time.Second)
 		if err != nil {
-			cliCtx.Fatalf("unable to load bundle %s from astra: %v", cli.Bundle, err)
+			cliCtx.Fatalf("unable to load bundle %s from astra: %v", cli.AstraBundle, err)
 		}
 		resolver = astra.NewResolver(bundle)
 		cli.Username = "token"
-		cli.Password = cli.Token
+		cli.Password = cli.AstraToken
+	} else if len(cli.ContactPoints) > 0 {
+		resolver = proxycore.NewResolverWithDefaultPort(cli.ContactPoints, cli.Port)
 	} else {
-		cliCtx.Errorf("must provide either bundle path or contact points")
+		cliCtx.Errorf("must provide either bundle path, token, or contact points")
 		return 1
 	}
 
 	if cli.HeartbeatInterval >= cli.IdleTimeout {
-		cliCtx.Errorf("idle-timeout must be greater than heartbeat-interval")
+		cliCtx.Errorf("idle-timeout must be greater than heartbeat-interval (heartbeat interval: %s, idle timeout: %s)",
+			cli.HeartbeatInterval, cli.IdleTimeout)
+		return 1
+	}
+
+	if cli.NumConns < 1 {
+		cliCtx.Errorf("invalid number of connections, must be greater than 0 (provided: %d)", cli.NumConns)
 		return 1
 	}
 
@@ -140,7 +148,7 @@ func Run(ctx context.Context, args []string) int {
 		MaxVersion:        maxVersion,
 		Resolver:          resolver,
 		ReconnectPolicy:   proxycore.NewReconnectPolicy(),
-		NumConns:          1,
+		NumConns:          cli.NumConns,
 		Auth:              auth,
 		Logger:            logger,
 		HeartBeatInterval: cli.HeartbeatInterval,

proxycore/cluster.go

@@ -259,11 +259,13 @@ func (c *Cluster) mergeHosts(hosts []*Host) error {
 		if _, ok := existing[key]; ok {
 			delete(existing, key)
 		} else {
+			c.logger.Info("adding host to the cluster", zap.Stringer("host", host))
 			c.sendEvent(&AddEvent{host})
 		}
 	}
 
 	for _, host := range existing {
+		c.logger.Info("removing host from the cluster", zap.Stringer("host", host))
 		c.sendEvent(&RemoveEvent{host})
 	}
 

proxycore/connpool.go

@@ -183,7 +183,8 @@ func (p *connPool) stayConnected(idx int) {
 		if conn == nil {
 			if !pendingConnect {
 				delay := reconnectPolicy.NextDelay()
-				p.logger.Debug("pool connection attempting to reconnect after delay", zap.Duration("delay", delay))
+				p.logger.Info("pool connection attempting to reconnect after delay",
+					zap.Stringer("host", p.config.Endpoint), zap.Duration("delay", delay))
 				connectTimer = time.NewTimer(reconnectPolicy.NextDelay())
 				pendingConnect = true
 			} else {
@@ -193,7 +194,8 @@ func (p *connPool) stayConnected(idx int) {
 				case <-connectTimer.C:
 					c, err := p.connect()
 					if err != nil {
-						p.logger.Error("pool failed to connect", zap.Stringer("host", p.config.Endpoint), zap.Error(err))
+						p.logger.Error("pool connection failed to connect",
+							zap.Stringer("host", p.config.Endpoint), zap.Error(err))
 					} else {
 						p.connsMu.Lock()
 						conn, p.conns[idx] = c, c
@@ -209,6 +211,7 @@ func (p *connPool) stayConnected(idx int) {
 				done = true
 				_ = conn.Close()
 			case <-conn.IsClosed():
+				p.logger.Info("pool connection closed", zap.Stringer("host", p.config.Endpoint))
 				p.connsMu.Lock()
 				conn, p.conns[idx] = nil, nil
 				p.connsMu.Unlock()