Driver reported "[Errno 9] Bad file descriptor"

[timtimb0t]

Argus

Scylla version: 2026.1.0~dev-20251205.866c96f536b0 with build-id 2c38506085b888e1baa43f81d05dab12df5132c1

During latest master runs driver reported following error:

< t:2025-12-06 04:21:39,017 f:cluster.py      l:3723 c:cassandra.cluster    p:WARNING > [control connection] Error connecting to 10.12.33.86:9042: < t:2025-12-06 04:21:39,017 f:cluster.py      l:3723 c:cassandra.cluster    p:WARNING > [control connection] Error connecting to 10.12.33.86:9042:
< t:2025-12-06 04:21:39,017 f:cluster.py      l:3723 c:cassandra.cluster    p:WARNING > Traceback (most recent call last):
< t:2025-12-06 04:21:39,017 f:cluster.py      l:3723 c:cassandra.cluster    p:WARNING >   File "cassandra/cluster.py", line 3546, in cassandra.cluster.ControlConnection._connect_host_in_lbp
< t:2025-12-06 04:21:39,017 f:cluster.py      l:3723 c:cassandra.cluster    p:WARNING >   File "cassandra/cluster.py", line 3662, in cassandra.cluster.ControlConnection._try_connect
< t:2025-12-06 04:21:39,017 f:cluster.py      l:3723 c:cassandra.cluster    p:WARNING >   File "cassandra/cluster.py", line 3646, in cassandra.cluster.ControlConnection._try_connect
< t:2025-12-06 04:21:39,017 f:cluster.py      l:3723 c:cassandra.cluster    p:WARNING > cassandra.connection.ConnectionShutdown: [Errno 9] Bad file descriptor
< t:2025-12-06 04:21:39,017 f:cluster.py      l:3723 c:cassandra.cluster    p:WARNING > Host 10.12.33.86:9042 has been marked down

It seems that such errors appeared each time while one of nodes been down

Also been spotted there:
https://argus.scylladb.com/tests/scylla-cluster-tests/a8cd6873-19c1-49c1-ab5a-dca25655ed6c

Kernel Version: 6.14.0-1017-aws

Extra information

Installation details

Cluster size: 6 nodes (i7i.4xlarge)

Scylla Nodes used in this run:

- longevity-tls-50gb-3d-master-db-node-38f90182-1 (3.228.203.95 | 10.12.35.225) (shards: 14)


- longevity-tls-50gb-3d-master-db-node-38f90182-2 (44.213.201.240 | 10.12.32.118) (shards: 14)


- longevity-tls-50gb-3d-master-db-node-38f90182-3 (100.30.78.169 | 10.12.33.86) (shards: 14)


- longevity-tls-50gb-3d-master-db-node-38f90182-4 (44.207.141.103 | 10.12.35.59) (shards: -1)


- longevity-tls-50gb-3d-master-db-node-38f90182-5 (3.219.68.68 | 10.12.35.232) (shards: 14)


- longevity-tls-50gb-3d-master-db-node-38f90182-6 (34.199.164.159 | 10.12.33.159) (shards: 14)


- longevity-tls-50gb-3d-master-db-node-38f90182-7 (98.82.213.102 | 10.12.33.203) (shards: 14)


- longevity-tls-50gb-3d-master-db-node-38f90182-8 (98.83.182.28 | 10.12.34.200) (shards: 14)

OS / Image: ami-0810c73586fe68036 (aws: N/A)

Test: longevity-50gb-3days-test
Test id: 38f90182-547d-4b60-973c-7e826b926708
Test name: scylla-master/tier1/longevity-50gb-3days-test

Test method: longevity_test.LongevityTest.test_custom_time

Test config file(s):

• longevity-50GB-3days-authorization-and-tls-ssl.yaml

Logs:

• longevity-tls-50gb-3d-master-db-node-38f90182-7

• longevity-tls-50gb-3d-master-db-node-38f90182-3

• core.scylla-longevity-tls-50gb-3d-master-db-node-38f90182-6-2025-12-06_07-36-03.zst

• core.scylla-longevity-tls-50gb-3d-master-db-node-38f90182-6-2025-12-06_07-39-47.zst

• core.scylla-longevity-tls-50gb-3d-master-db-node-38f90182-6-2025-12-06_07-43-23.zst

• db-cluster-38f90182.tar.zst

• schema-logs-38f90182.tar.zst

• sct-runner-events-38f90182.tar.zst

• sct-38f90182.log.tar.zst

• loader-set-38f90182.tar.zst

• monitor-set-38f90182.tar.zst

• ssl-conf-38f90182.tar.zst

• builder-38f90182.log.tar.gz

Jenkins job URL

[timtimb0t]

Hi @dkropachev , could you please take a look at this issue?

[timtimb0t]

reproduced again:

Argus

Scylla version: 2026.1.0~dev-20251211.f7ffa395a8fd with build-id 6ed9dbb170d6894329ed88a93e118dd68cbd62a9

Kernel Version: 6.14.0-1018-aws

Extra information

Installation details

Cluster size: 6 nodes (i7i.2xlarge)

Scylla Nodes used in this run:

- longevity-50gb-12h-master-db-node-70283809-1 (13.218.127.161 | 10.12.9.87) (shards: 4)


- longevity-50gb-12h-master-db-node-70283809-2 (54.91.187.43 | 10.12.8.66) (shards: 4)


- longevity-50gb-12h-master-db-node-70283809-3 (18.212.86.250 | 10.12.8.228) (shards: 6)


- longevity-50gb-12h-master-db-node-70283809-4 (54.92.211.214 | 10.12.10.198) (shards: 6)


- longevity-50gb-12h-master-db-node-70283809-5 (98.84.134.241 | 10.12.8.104) (shards: 5)


- longevity-50gb-12h-master-db-node-70283809-6 (54.160.211.56 | 10.12.9.171) (shards: 4)

OS / Image: ami-02ad235f4c4336f6c (aws: N/A)

Test: longevity-150gb-asymmetric-cluster-12h-test
Test id: 70283809-37aa-4be5-9ebc-d891e1a2d6aa
Test name: scylla-master/tier1/longevity-150gb-asymmetric-cluster-12h-test

Test method: longevity_test.LongevityTest.test_custom_time

Test config file(s):

• longevity-150GB-12h-autorization-LimitedMonkey-cql-stress.yaml

Logs:

• db-cluster-70283809.tar.zst

• sct-runner-events-70283809.tar.zst

• sct-70283809.log.tar.zst

• loader-set-70283809.tar.zst

• monitor-set-70283809.tar.zst

• ssl-conf-70283809.tar.zst

• builder-70283809.log.tar.gz

Jenkins job URL

[timtimb0t]

Argus

Scylla version: 2026.1.0~dev-20251219.f65db4e8eba5 with build-id 683ff5b7a4a313ea6094e72fd639c906693ece37

Kernel Version: 6.14.0-1018-aws

Extra information

Installation details

Cluster size: 6 nodes (i7i.4xlarge)

Scylla Nodes used in this run:

- longevity-tls-50gb-3d-master-db-node-c6beb17a-1 (98.87.193.30 | 10.12.35.220) (shards: 14)


- longevity-tls-50gb-3d-master-db-node-c6beb17a-2 (52.6.69.201 | 10.12.34.173) (shards: 14)


- longevity-tls-50gb-3d-master-db-node-c6beb17a-3 (100.49.143.61 | 10.12.32.22) (shards: 14)


- longevity-tls-50gb-3d-master-db-node-c6beb17a-4 (52.203.20.179 | 10.12.34.56) (shards: 14)


- longevity-tls-50gb-3d-master-db-node-c6beb17a-5 (44.193.182.223 | 10.12.32.49) (shards: 14)


- longevity-tls-50gb-3d-master-db-node-c6beb17a-6 (50.17.245.62 | 10.12.34.86) (shards: 14)


- longevity-tls-50gb-3d-master-db-node-c6beb17a-7 (44.209.62.120 | 10.12.32.166) (shards: 14)


- longevity-tls-50gb-3d-master-db-node-c6beb17a-8 (54.152.201.38 | 10.12.33.224) (shards: 14)


- longevity-tls-50gb-3d-master-db-node-c6beb17a-9 (98.95.22.145 | 10.12.33.230) (shards: 14)


- longevity-tls-50gb-3d-master-db-node-c6beb17a-10 (3.231.75.179 | 10.12.32.60) (shards: 14)


- longevity-tls-50gb-3d-master-db-node-c6beb17a-11 (100.49.20.125 | 10.12.33.136) (shards: -1)


- longevity-tls-50gb-3d-master-db-node-c6beb17a-12 (3.215.138.198 | 10.12.34.73) (shards: 14)

OS / Image: ami-048249cf3c5bfc84f (aws: N/A)

Test: longevity-50gb-3days-test
Test id: c6beb17a-d0b9-43b6-ad05-2fbd45c4201d
Test name: scylla-master/tier1/longevity-50gb-3days-test

Test method: longevity_test.LongevityTest.test_custom_time

Test config file(s):

• longevity-50GB-3days-authorization-and-tls-ssl.yaml

Logs:

• longevity-tls-50gb-3d-master-db-node-c6beb17a-4

• longevity-tls-50gb-3d-master-db-node-c6beb17a-2

• longevity-tls-50gb-3d-master-db-node-c6beb17a-3

• longevity-tls-50gb-3d-master-db-node-c6beb17a-9

• longevity-tls-50gb-3d-master-db-node-c6beb17a-1

• db-cluster-c6beb17a.tar.zst

• sct-runner-events-c6beb17a.tar.zst

• 2025_12_20__04_12_21_688.sct-c6beb17a.log.zst

• 2025_12_20__09_12_55_277.sct-c6beb17a.log.zst

• loader-set-c6beb17a.tar.zst

• monitor-set-c6beb17a.tar.zst

• ssl-conf-c6beb17a.tar.zst

• builder-c6beb17a.log.tar.gz

Jenkins job URL

[timtimb0t]

Argus

Scylla version: 2026.1.0~dev-20251219.f65db4e8eba5 with build-id 683ff5b7a4a313ea6094e72fd639c906693ece37

Kernel Version: 6.14.0-1018-aws

Extra information

Installation details

Cluster size: 6 nodes (i7i.2xlarge)

Scylla Nodes used in this run:

- longevity-50gb-12h-master-db-node-e2d8a05c-1 (34.201.94.66 | 10.12.8.210) (shards: 6)


- longevity-50gb-12h-master-db-node-e2d8a05c-2 (18.234.51.11 | 10.12.8.248) (shards: 6)


- longevity-50gb-12h-master-db-node-e2d8a05c-3 (52.54.112.48 | 10.12.10.136) (shards: 5)


- longevity-50gb-12h-master-db-node-e2d8a05c-4 (13.222.190.127 | 10.12.11.124) (shards: 7)


- longevity-50gb-12h-master-db-node-e2d8a05c-5 (54.145.225.121 | 10.12.10.222) (shards: 7)


- longevity-50gb-12h-master-db-node-e2d8a05c-6 (18.208.221.26 | 10.12.8.44) (shards: 4)

OS / Image: ami-048249cf3c5bfc84f (aws: N/A)

Test: longevity-150gb-asymmetric-cluster-12h-test
Test id: e2d8a05c-55b0-4025-b3bb-00712401b844
Test name: scylla-master/tier1/longevity-150gb-asymmetric-cluster-12h-test

Test method: longevity_test.LongevityTest.test_custom_time

Test config file(s):

• longevity-150GB-12h-autorization-LimitedMonkey-cql-stress.yaml

Logs:

• db-cluster-e2d8a05c.tar.zst

• sct-runner-events-e2d8a05c.tar.zst

• sct-e2d8a05c.log.tar.zst

• loader-set-e2d8a05c.tar.zst

• monitor-set-e2d8a05c.tar.zst

• ssl-conf-e2d8a05c.tar.zst

• builder-e2d8a05c.log.tar.gz

Jenkins job URL