Support for hostname verification

[dimakr]

This issue is more of a feature request.

Currently the driver supports peer verification, which can be enabled by setting openssl::ssl::SslVerifyMode of SslContext to PEER. This allows to ensure that certificate that the server presents is signed by the trusted authority, but it doesn't guarantee that the certificate is issued exactly for the server/domain in question.
This could potentially be a cause of a MITM threat.

It should be possible to enable hostname verification in the driver, for encrypted communication, so that during TLS handshake we can ensure that the certificate was issued for server in question, i.e. that the certificate that servers presents contains its name/IP in the certificate's SAN extension (I'm making this assumption from example of how scylla-go-driver behaves during TLS handshake, when host/peer verification is enabled - it checks that both, certificate is signed by the correct authority and certificate IP/name in the certificate's SAN extension corresponds to the IP/name of the server).

Seems like the openssl::ssl:SslConnector has set_verify_hostname method out of the box, but is probably not a suitable solution for the driver, as the driver uses already openssl::ssl::SslContext for more flexible configuration of SSL/TLS contexts. But openssl::ssl::SslContext doesn't expose any 'verify_hostname'- kind of API.

Example scenario

I'm not familiar with scylla-rust-driver itself, so latte stress tool (https://github.com/scylladb/latte) is used as an example application that uses the driver to communicate with Scylla server, with TLS encryption enabled:

• a certificate config was prepared for server with IP 52.51.61.135, but some other IPs/Names were intentionally set in SAN extension of the config, so that the certificate would fail hostname verification

[req]
default_bits = 4096
prompt = no
default_md = sha256
distinguished_name = dn
req_extensions = req_ext

[dn]
CN = PR-provision-test-dmitriy-db-node-c048f7e9-111
O = ScyllaDB
L = Herzelia
ST = Tel Aviv
C = IL

[req_ext]
subjectAltName = @alt_names
basicConstraints = critical,CA:FALSE
subjectKeyIdentifier = hash

[alt_names]
DNS.1 = PR-provision-test-dmitriy-db-node-c048f7e9-111
DNS.2 = ec2-52-51-61-111.eu-west-1.compute.amazonaws.com
DNS.3 = ip-10-4-3-111.eu-west-1.compute.internal
IP.1 = 10.4.3.111
IP.2 = 52.51.61.111

• the certificate was generated from the config, signed by the CA and distributed to the server
• the TLS handshake using openssl command is OK, when executed without hostname verification

❯ openssl s_client -connect 52.51.61.135:9042 -CAfile ~/Downloads/ssl_conf/ca.pem
CONNECTED(00000003)
...
---
SSL handshake has read 2457 bytes and written 373 bytes
Verification: OK

• the TLS handshake using openssl command is failing (which is expected), when executed with hostname verification

❯ openssl s_client -connect 52.51.61.135:9042 -CAfile ~/Downloads/ssl_conf/ca.pem -verify_hostname 52.51.61.135 -verify_return_error
CONNECTED(00000003)
...
---
SSL handshake has read 1841 bytes and written 300 bytes
Verification error: hostname mismatch

• latte builds the SSL context as:

use crate::config::ConnectionConf;
use crate::scripting::cass_error::{CassError, CassErrorKind};
use crate::scripting::context::Context;
use openssl::ssl::{SslContext, SslContextBuilder, SslFiletype, SslMethod, SslVerifyMode};
use scylla::load_balancing::DefaultPolicy;
use scylla::transport::session::PoolSize;
use scylla::{ExecutionProfile, SessionBuilder};

fn ssl_context(conf: &&ConnectionConf) -> Result<Option<SslContext>, CassError> {
    if conf.ssl {
        let mut ssl = SslContextBuilder::new(SslMethod::tls())?;
        if let Some(path) = &conf.ssl_ca_cert_file {
            ssl.set_ca_file(path)?;
        }
        if let Some(path) = &conf.ssl_cert_file {
            ssl.set_certificate_file(path, SslFiletype::PEM)?;
        }
        if let Some(path) = &conf.ssl_key_file {
            ssl.set_private_key_file(path, SslFiletype::PEM)?;
        }
        if conf.ssl_peer_verification {
            ssl.set_verify(SslVerifyMode::PEER);
        }
        Ok(Some(ssl.build()))
    } else {
        Ok(None)
    }
}

/// Configures connection to Cassandra.
pub async fn connect(conf: &ConnectionConf) -> Result<Context, CassError> {
    let mut policy_builder = DefaultPolicy::builder().token_aware(true);
    let mut datacenter: String = "".to_string();
    if let Some(dc) = &conf.datacenter {
        policy_builder = policy_builder
            .prefer_datacenter(dc.to_owned())
            .permit_dc_failover(true);
        datacenter = dc.clone();
    }
    let profile = ExecutionProfile::builder()
        .consistency(conf.consistency.scylla_consistency())
        .load_balancing_policy(policy_builder.build())
        .request_timeout(Some(conf.request_timeout))
        .build();

    let scylla_session = SessionBuilder::new()
        .known_nodes(&conf.addresses)
        .pool_size(PoolSize::PerShard(conf.count))
        .user(&conf.user, &conf.password)
        .ssl_context(ssl_context(&conf)?)
        .default_execution_profile_handle(profile.into_handle())
        .build()
        .await
        .map_err(|e| CassError(CassErrorKind::FailedToConnect(conf.addresses.clone(), e)))?;
    Ok(Context::new(
        Some(scylla_session),
        conf.page_size.get() as u64,
        datacenter,
        conf.retry_number,
        conf.retry_interval,
    ))
}

• when executing latte with enabled encryption and peer verification, it is executed with no errors, but it would be expected that the driver fails on TLS handshake due to certificate that the server presents is for another hostname

❯ ./target/debug/latte run -d 3s ../scylla-cluster-tests/docker/latte/workloads/workload.rn 52.51.61.135 --ssl --ssl-peer-verification --ssl-ca ~/Downloads/ssl_conf/ca.pem
info: Loading workload script /home/dmitriy/Work/Scylla/scylla-cluster-tests/docker/latte/workloads/workload.rn...
info: Connecting to ["52.51.61.135"]... 
info: Connected to PR-provision-test-dmitriy-db-cluster-c048f7e9 running Cassandra version 3.0.8
info: Preparing...
info: Warming up...
info: Running benchmark...
CONFIG ═════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
            Date            Wed, 06 Nov 2024                                                          
            Time            20:48:49 +0100                                                            
         Cluster            PR-provision-test-dmitriy-db-cluster-c048f7e9                                            
      Datacenter                                                                                      
   Cass. version            3.0.8                                                                     
        Workload            workload.rn                                                               
     Function(s)            run:1                                                                     
     Consistency            LocalQuorum                                                               
            Tags                                                                                      
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
         Threads                    1                                                                 
     Connections                    1                                                                 
     Concurrency     [req]        128                                                                 
        Max rate    [op/s]                                                                            
          Warmup       [s]                                                                            
              └─      [op]          1                                                                 
        Run time       [s]        3.0                                                                 
              └─      [op]                                                                            
        Sampling       [s]        1.0                                                                 
              └─      [op]                                                                            
 Request timeout       [s]          5                                                                 
         Retries                                                                                      
  ┌──────┴number                   10                                                                 
  ├─min interval      [ms]        100                                                                 
  └─max interval      [ms]       5000                                                                 

LOG ════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
    Time    Cycles    Errors    Thrpt.     ────────────────────────────────── Latency [ms/op] ──────────────────────────────
     [s]      [op]      [op]    [op/s]             Min        50        75        90        95        99      99.9       Max
   1.002      2226         0      2223          38.076    49.971    56.820    66.814    79.626    86.770    95.420    98.238
   2.008      2587         0      2571          36.798    47.350    51.053    55.902    67.961    91.554   102.040   102.367
   3.001      2232         0      2247          37.421    49.775    58.163    73.925    79.167   117.506   127.599   128.451
   3.075       115         0      1568          46.137    61.735    81.396    83.558    85.000    88.670    91.161    91.161

SUMMARY STATS ══════════════════════════════════════════════════════════════════════════════════════════════════════════════
    Elapsed time       [s]      3.090                                                                 
        CPU time       [s]      1.369                                                                 
 CPU utilisation       [%]        2.2                                                                 
          Cycles      [op]       7160                                                                 
          Errors      [op]          0                                                                 
              └─       [%]        0.0                   
...              

[Lorak-mmk]

Could you link scylla-go-driver snippet that performs this check?

[dimakr]

Could you link scylla-go-driver snippet that performs this check?

@Lorak-mmk
Similar approach to the above - scylla-bench stress tool is used as an app that uses go-driver. In the scylla-bench host verification is enabled as:

	if tlsEncryption {
		sslOpts := &gocql.SslOptions{
			EnableHostVerification: hostVerification,
		}
        ...
        cluster.SslOpts = sslOpts

And from what I see in the driver, EnableHostVerification sets the crypto/tls package Config.InsecureSkipVerify flag, which controls the certificate and host name verification.

The actual failed hostname verification then:

❯ scylla-bench -workload=sequential -mode=write -max-rate=300 -replication-factor=3 -partition-count=10 -clustering-row-count=100 -clustering-row-size=5120 -concurrency=7 -rows-per-request=10 -error-at-row-limit 1000 -nodes 52.51.61.135 -tls -tls-ca-cert-file ~/Downloads/ssl_conf/ca.pem -tls-host-verification
2024/11/05 14:40:09 gocql: unable to create session: unable to discover protocol version: x509: certificate is valid for 10.4.3.111, 52.51.61.111, not 52.51.61.135
exit status 1

[Lorak-mmk]

@muzarski Could you and @dimakr verify that hostname verification works properly when using rustls? Preferably by making cql-stress use rustls (which is a reasonable change anyway).