Skip to content

Conversation

fruch
Copy link
Contributor

@fruch fruch commented Jun 12, 2023

seems like commit 4d61471 has changed the output of the bundles and they are default to port 9142 that the nodes are using, and not to the port used by the sni_proxy basiclly bypassing the sni_proxy, which broke some of the java-driver tests (python-driver tests aren't broke, since it's not using the command line)

Fixes: scylladb/java-driver#221

@fruch fruch requested review from Gor027 and Bouncheck June 12, 2023 19:24
@fruch fruch force-pushed the fix_sni_port_setup branch from 7067a04 to 4d000b7 Compare June 12, 2023 19:24
seems like commit 4d61471 has changed
the output of the bundles and they are default to port 9142 that
the nodes are using, and not to the port used by the sni_proxy
basiclly bypassing the sni_proxy, which broke some of the java-driver
tests (python-driver tests aren't broke, since it's not using the
command line)

Ref: scylladb/java-driver#221
@fruch fruch force-pushed the fix_sni_port_setup branch from 4d000b7 to 201f106 Compare June 12, 2023 21:14
fruch added 3 commits June 13, 2023 10:38
this would help fix some issue with the selection
of data to diffrent parts of the sni_proxy impemention
with out need to parse it again and again by positions
those seemed to be breaking every time it was changed
and we don't have any test covering them until now.
since we can't pass a cloudbundle and adddress/port to cqlsh
we should add those if the user is passing `--cloudconf` in
`cqlsh_options`.

this would enable use to write tests, and also write dtest covering
this.
@fruch
Copy link
Contributor Author

fruch commented Jun 13, 2023

@Gor027 @Bouncheck can you review, and give it a try on the drivers tests ? to see it doesn't break ? I'll test it on the python-driver

@Gor027
Copy link
Contributor

Gor027 commented Jun 13, 2023

I have run GitHub Actions test workflows of Java driver scylla-4.x and scylla-3.x versions with updated CCM and tests fail with errors related to ccm as well:

@fruch
Copy link
Contributor Author

fruch commented Jun 13, 2023

I have run GitHub Actions test workflows of Java driver scylla-4.x and scylla-3.x versions with updated CCM and tests fail with errors related to ccm as well:

@Gor027
both are failing cause of something unrelated to this PR:

023-06-13T10:56:25.7675451Z 10:56:25.766 [main] INFO  c.d.o.d.api.testinfra.ccm.CcmBridge - Executing: [ccm, create, ccm_1, -i, 127.0.0., -n, 1:0, -v, release:2022.2.7, --scylla, --config-dir=/tmp/ccm763320022917022317]
2023-06-13T10:56:26.3703808Z 10:56:26.369 [Exec Stream Pumper] ERROR c.d.o.d.api.testinfra.ccm.CcmBridge - ccmerr> Traceback (most recent call last):
2023-06-13T10:56:26.3704811Z 10:56:26.369 [Exec Stream Pumper] ERROR c.d.o.d.api.testinfra.ccm.CcmBridge - ccmerr>   File "/opt/hostedtoolcache/Python/3.11.3/x64/bin/ccm", line 74, in <module>
2023-06-13T10:56:26.3706478Z 10:56:26.370 [Exec Stream Pumper] ERROR c.d.o.d.api.testinfra.ccm.CcmBridge - ccmerr>     cmd.run()
2023-06-13T10:56:26.3707700Z 10:56:26.370 [Exec Stream Pumper] ERROR c.d.o.d.api.testinfra.ccm.CcmBridge - ccmerr>   File "/opt/hostedtoolcache/Python/3.11.3/x64/lib/python3.11/site-packages/ccmlib/cmds/cluster_cmds.py", line 221, in run
2023-06-13T10:56:26.3714529Z 10:56:26.371 [Exec Stream Pumper] ERROR c.d.o.d.api.testinfra.ccm.CcmBridge - ccmerr>     cluster = ScyllaCluster(self.path, self.name, install_dir=self.options.install_dir,
2023-06-13T10:56:26.3715957Z 10:56:26.371 [Exec Stream Pumper] ERROR c.d.o.d.api.testinfra.ccm.CcmBridge - ccmerr>               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-06-13T10:56:26.3717589Z 10:56:26.371 [Exec Stream Pumper] ERROR c.d.o.d.api.testinfra.ccm.CcmBridge - ccmerr>   File "/opt/hostedtoolcache/Python/3.11.3/x64/lib/python3.11/site-packages/ccmlib/scylla_cluster.py", line 49, in __init__
2023-06-13T10:56:26.3727573Z 10:56:26.372 [Exec Stream Pumper] ERROR c.d.o.d.api.testinfra.ccm.CcmBridge - ccmerr>     super(ScyllaCluster, self).__init__(path, name, partitioner,
2023-06-13T10:56:26.3732871Z 10:56:26.373 [Exec Stream Pumper] ERROR c.d.o.d.api.testinfra.ccm.CcmBridge - ccmerr>   File "/opt/hostedtoolcache/Python/3.11.3/x64/lib/python3.11/site-packages/ccmlib/cluster.py", line 69, in __init__
2023-06-13T10:56:26.3738829Z 10:56:26.373 [Exec Stream Pumper] ERROR c.d.o.d.api.testinfra.ccm.CcmBridge - ccmerr>     dir, v = self.load_from_repository(version, verbose)
2023-06-13T10:56:26.3744553Z 10:56:26.374 [Exec Stream Pumper] ERROR c.d.o.d.api.testinfra.ccm.CcmBridge - ccmerr>              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-06-13T10:56:26.3749401Z 10:56:26.374 [Exec Stream Pumper] ERROR c.d.o.d.api.testinfra.ccm.CcmBridge - ccmerr>   File "/opt/hostedtoolcache/Python/3.11.3/x64/lib/python3.11/site-packages/ccmlib/scylla_cluster.py", line 70, in load_from_repository
2023-06-13T10:56:26.3755314Z 10:56:26.375 [Exec Stream Pumper] ERROR c.d.o.d.api.testinfra.ccm.CcmBridge - ccmerr>     install_dir, version = scylla_repository.setup(version, verbose)
2023-06-13T10:56:26.3761294Z 10:56:26.375 [Exec Stream Pumper] ERROR c.d.o.d.api.testinfra.ccm.CcmBridge - ccmerr>                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-06-13T10:56:26.3767097Z 10:56:26.376 [Exec Stream Pumper] ERROR c.d.o.d.api.testinfra.ccm.CcmBridge - ccmerr>   File "/opt/hostedtoolcache/Python/3.11.3/x64/lib/python3.11/site-packages/ccmlib/scylla_repository.py", line 266, in setup
2023-06-13T10:56:26.3774288Z 10:56:26.377 [Exec Stream Pumper] ERROR c.d.o.d.api.testinfra.ccm.CcmBridge - ccmerr>     packages, type_n_version[1] = release_packages(s3_url=s3_url, version=s3_version, arch=scylla_arch, scylla_product=scylla_product)
2023-06-13T10:56:26.3780872Z 10:56:26.377 [Exec Stream Pumper] ERROR c.d.o.d.api.testinfra.ccm.CcmBridge - ccmerr>                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2023-06-13T10:56:26.3793618Z 10:56:26.378 [Exec Stream Pumper] ERROR c.d.o.d.api.testinfra.ccm.CcmBridge - ccmerr>   File "/opt/hostedtoolcache/Python/3.11.3/x64/lib/python3.11/site-packages/ccmlib/scylla_repository.py", line 168, in release_packages
2023-06-13T10:56:26.3795137Z 10:56:26.378 [Exec Stream Pumper] ERROR c.d.o.d.api.testinfra.ccm.CcmBridge - ccmerr>     raise ValueError(
2023-06-13T10:56:26.3795870Z 10:56:26.378 [Exec Stream Pumper] ERROR c.d.o.d.api.testinfra.ccm.CcmBridge - ccmerr> ValueError: Release packages have not been found.
2023-06-13T10:56:26.3817362Z 10:56:26.378 [Exec Stream Pumper] ERROR c.d.o.d.api.testinfra.ccm.CcmBridge - ccmerr> Debug info: all packages: ['scylla-enterprise-jmx-package-2022.2.0-rc0.0.20220814.eb67e6127d6b.tar.gz', 'scylla-enterprise-jmx-package-2022.2.0-rc1.0.20220902.a9bc6d191031.tar.gz', 'scylla-enterprise-jmx-package-2022.2.0-rc2.0.20220919.75d087a2b75a.tar.gz', 'scylla-enterprise-jmx-package-2022.2.0-rc3.0.20221009.994a5f0fbb4c.tar.gz', 'scylla-enterprise-jmx-package-2022.2.0-rc4.0.20221106.f5714e0db12f.tar.gz', 'scylla-enterprise-jmx-package-2022.2.0-rc5.0.20221121.feb292600fc4.tar.gz', 'scylla-enterprise-jmx-package-2022.2.0-rc6.0.20221130.81c9ffb0b459.tar.gz', 'scylla-enterprise-jmx-package-2022.2.0-rc7.0.20230109.32c1fe6c8788.tar.gz', 'scylla-enterprise-jmx-package-2022.2.0.0.20230112.4f0f82ff2e1d.tar.gz', 'scylla-enterprise-jmx-package-2022.2.1-0.20230213.87d6af74da28.tar.gz', 'scylla-enterprise-jmx-package-2022.2.2-0.20230228.e39a4ea914c9.tar.gz', 'scylla-enterprise-jmx-package-2022.2.3-0.20230312.af0a90c86e0c.tar.gz', 'scylla-enterprise-jmx-package-2022.2.4-0.20230403.338de6864d3f.tar.gz', 'scylla-enterprise-jmx-package-2022.2.5-0.20230416.52584372ff6e.tar.gz', 'scylla-enterprise-jmx-package-2022.2.6-0.20230423.b9a1478dba07.tar.gz', 'scylla-enterprise-tools-package-2022.2.0-rc0.0.20220814.eb67e6127d6b.tar.gz', 'scylla-enterprise-tools-package-2022.2.0-rc1.0.20220902.a9bc6d191031.tar.gz', 'scylla-enterprise-tools-package-2022.2.0-rc2.0.20220919.75d087a2b75a.tar.gz', 'scylla-enterprise-tools-package-2022.2.0-rc3.0.20221009.994a5f0fbb4c.tar.gz', 'scylla-enterprise-tools-package-2022.2.0-rc4.0.20221106.f5714e0db12f.tar.gz', 'scylla-enterprise-tools-package-2022.2.0-rc5.0.20221121.feb292600fc4.tar.gz', 'scylla-enterprise-tools-package-2022.2.0-rc6.0.20221130.81c9ffb0b459.tar.gz', 'scylla-enterprise-tools-package-2022.2.0-rc7.0.20230109.32c1fe6c8788.tar.gz', 'scylla-enterprise-tools-package-2022.2.0.0.20230112.4f0f82ff2e1d.tar.gz', 'scylla-enterprise-tools-package-2022.2.1-0.20230213.87d6af74da28.tar.gz', 'scylla-enterprise-tools-package-2022.2.2-0.20230228.e39a4ea914c9.tar.gz', 'scylla-enterprise-tools-package-2022.2.3-0.20230312.af0a90c86e0c.tar.gz', 'scylla-enterprise-tools-package-2022.2.4-0.20230403.338de6864d3f.tar.gz', 'scylla-enterprise-tools-package-2022.2.5-0.20230416.52584372ff6e.tar.gz', 'scylla-enterprise-tools-package-2022.2.6-0.20230423.b9a1478dba07.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.0-rc0.0.20220814.eb67e6127d6b.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.0-rc1.0.20220902.a9bc6d191031.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.0-rc2.0.20220919.75d087a2b75a.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.0-rc3.0.20221009.994a5f0fbb4c.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.0-rc4.0.20221106.f5714e0db12f.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.0-rc5.0.20221121.feb292600fc4.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.0-rc6.0.20221130.81c9ffb0b459.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.0-rc7.0.20230109.32c1fe6c8788.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.0.0.20230112.4f0f82ff2e1d.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.1-0.20230213.87d6af74da28.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.2-0.20230228.e39a4ea914c9.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.3-0.20230312.af0a90c86e0c.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.4-0.20230403.338de6864d3f.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.5-0.20230416.52584372ff6e.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.6-0.20230423.b9a1478dba07.tar.gz']; candidates packages: ['scylla-enterprise-jmx-package-2022.2.0-rc0.0.20220814.eb67e6127d6b.tar.gz', 'scylla-enterprise-jmx-package-2022.2.0-rc1.0.20220902.a9bc6d191031.tar.gz', 'scylla-enterprise-jmx-package-2022.2.0-rc2.0.20220919.75d087a2b75a.tar.gz', 'scylla-enterprise-jmx-package-2022.2.0-rc3.0.20221009.994a5f0fbb4c.tar.gz', 'scylla-enterprise-jmx-package-2022.2.0-rc4.0.20221106.f5714e0db12f.tar.gz', 'scylla-enterprise-jmx-package-2022.2.0-rc5.0.20221121.feb292600fc4.tar.gz', 'scylla-enterprise-jmx-package-2022.2.0-rc6.0.20221130.81c9ffb0b459.tar.gz', 'scylla-enterprise-jmx-package-2022.2.0-rc7.0.20230109.32c1fe6c8788.tar.gz', 'scylla-enterprise-jmx-package-2022.2.0.0.20230112.4f0f82ff2e1d.tar.gz', 'scylla-enterprise-jmx-package-2022.2.1-0.20230213.87d6af74da28.tar.gz', 'scylla-enterprise-jmx-package-2022.2.2-0.20230228.e39a4ea914c9.tar.gz', 'scylla-enterprise-jmx-package-2022.2.3-0.20230312.af0a90c86e0c.tar.gz', 'scylla-enterprise-jmx-package-2022.2.4-0.20230403.338de6864d3f.tar.gz', 'scylla-enterprise-jmx-package-2022.2.5-0.20230416.52584372ff6e.tar.gz', 'scylla-enterprise-jmx-package-2022.2.6-0.20230423.b9a1478dba07.tar.gz', 'scylla-enterprise-tools-package-2022.2.0-rc0.0.20220814.eb67e6127d6b.tar.gz', 'scylla-enterprise-tools-package-2022.2.0-rc1.0.20220902.a9bc6d191031.tar.gz', 'scylla-enterprise-tools-package-2022.2.0-rc2.0.20220919.75d087a2b75a.tar.gz', 'scylla-enterprise-tools-package-2022.2.0-rc3.0.20221009.994a5f0fbb4c.tar.gz', 'scylla-enterprise-tools-package-2022.2.0-rc4.0.20221106.f5714e0db12f.tar.gz', 'scylla-enterprise-tools-package-2022.2.0-rc5.0.20221121.feb292600fc4.tar.gz', 'scylla-enterprise-tools-package-2022.2.0-rc6.0.20221130.81c9ffb0b459.tar.gz', 'scylla-enterprise-tools-package-2022.2.0-rc7.0.20230109.32c1fe6c8788.tar.gz', 'scylla-enterprise-tools-package-2022.2.0.0.20230112.4f0f82ff2e1d.tar.gz', 'scylla-enterprise-tools-package-2022.2.1-0.20230213.87d6af74da28.tar.gz', 'scylla-enterprise-tools-package-2022.2.2-0.20230228.e39a4ea914c9.tar.gz', 'scylla-enterprise-tools-package-2022.2.3-0.20230312.af0a90c86e0c.tar.gz', 'scylla-enterprise-tools-package-2022.2.4-0.20230403.338de6864d3f.tar.gz', 'scylla-enterprise-tools-package-2022.2.5-0.20230416.52584372ff6e.tar.gz', 'scylla-enterprise-tools-package-2022.2.6-0.20230423.b9a1478dba07.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.0-rc0.0.20220814.eb67e6127d6b.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.0-rc1.0.20220902.a9bc6d191031.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.0-rc2.0.20220919.75d087a2b75a.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.0-rc3.0.20221009.994a5f0fbb4c.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.0-rc4.0.20221106.f5714e0db12f.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.0-rc5.0.20221121.feb292600fc4.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.0-rc6.0.20221130.81c9ffb0b459.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.0-rc7.0.20230109.32c1fe6c8788.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.0.0.20230112.4f0f82ff2e1d.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.1-0.20230213.87d6af74da28.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.2-0.20230228.e39a4ea914c9.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.3-0.20230312.af0a90c86e0c.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.4-0.20230403.338de6864d3f.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.5-0.20230416.52584372ff6e.tar.gz', 'scylla-enterprise-x86_64-package-2022.2.6-0.20230423.b9a1478dba07.tar.gz']; last version: 2022.2.7

version 2022.2.7 doesn't exists, I think it was retracted, use 2022.2.6, until 2022.2.8 would be released

Copy link
Contributor

@Gor027 Gor027 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested it with the Rust driver, too. If it is also tested with Python driver, I think it is good to be merged. Also, we should update the CI matrix Scylla and Cassandra versions in Java driver as some versions might not be available anymore. @avelanarius

@fruch fruch merged commit e352883 into scylladb:next Jun 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[3.x] ScyllaSniProxyTest tests are failing
2 participants