Use python 3.12 in the nodejs 22 container for native extensions. #474
Conversation
A nodejs tool "node-gyp" used for compiling native extensions from npm from source requires python version >= 3.8. Install the high-enough python version 3.12 for RHEL 8 container. RHEL 8 base version is 3.6, it is not enough for node-gyp. Python 3.12 in RHEL 8 has support until 2029, the currently projected RHEL 8 EOL, similarly to python 3.6 nodejs-22-minimal container does not do anything with python, leave that Containerfile as-is.
Pull Request validationFailed🔴 Failed or pending statuses - |
|
[test] |
Testing Farm results
|
|
RHEL10 failed here: Fedora 22 passed, but the result is FAILED. Error in common module. The same for CentOS Stream 9 |
phracek
left a comment
phracek
left a comment
There was a problem hiding this comment.
RHEL10 is still failing on rdkafka. See comment.
Not sure what can be done about that currently from our side as that is node-rdkafka bundled library's compilation problem, I'd guess confluentinc/librdkafka#4732 fixes that, but it is still open. But the test got through python in node-gyp in other streams so there it is a betterment on RHEL 8 and fix of no longer using incorrect python with nodejs:22. |
|
Actually this has better description of what's going on, which is OpenSSL incompatibility confluentinc/librdkafka#4911 and confluentinc/librdkafka#3535 should have fixed it, but the npm package node-rdkafka has the library as a git submodule and it's compiled as part of |
|
@khardix was this all that was needed to fixup the issue with respect to RHEL 8 the python version and the problems with node-rdkafka or was there something else? @pacostas confirmed today that the issue on RHEL 9 has been resolved, there was an update 2 days ago that pulled in the new version of librdkafka that the needed fix with respect to OpenSSL. |
This will satisfy node-gyp's requirement for python >= 3.8 (that I last looked at)
One problem fixed, another added. Now, we have too old compiler, default RHEL 8 gcc does not understand $ rpm -qa python3.12 npm nodejs gcc-c++
npm-10.9.2-1.22.13.1.1.module+el8.10.0+22759+46b58560.x86_64
gcc-c++-8.5.0-23.el8_10.x86_64
python3.12-3.12.8-1.el8_10.x86_64
nodejs-22.13.1-1.module+el8.10.0+22759+46b58560.x86_64
$ npm install node-rdkafka
<snip>
npm error make[1]: Leaving directory '/root/node_modules/node-rdkafka/deps/librdkafka'
npm error TOUCH Release/obj.target/deps/librdkafka.stamp
npm error CXX(target) Release/obj.target/node-librdkafka/src/binding.o
npm error rm 11a9e3388a67e1ca5c31c1d8da49cb6d2714eb41.intermediate
npm error make: Leaving directory '/root/node_modules/node-rdkafka/build'
npm error gyp info it worked if it ends with ok
npm error gyp info using [email protected]
npm error gyp info using [email protected] | linux | x64
npm error gyp info find Python using Python version 3.12.8 found at "/usr/bin/python3"
npm error gyp info spawn /usr/bin/python3
npm error gyp info spawn args [
npm error gyp info spawn args '/usr/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
npm error gyp info spawn args 'binding.gyp',
npm error gyp info spawn args '-f',
npm error gyp info spawn args 'make',
npm error gyp info spawn args '-I',
npm error gyp info spawn args '/root/node_modules/node-rdkafka/build/config.gypi',
npm error gyp info spawn args '-I',
npm error gyp info spawn args '/usr/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
npm error gyp info spawn args '-I',
npm error gyp info spawn args '/root/.cache/node-gyp/22.13.1/include/node/common.gypi',
npm error gyp info spawn args '-Dlibrary=shared_library',
npm error gyp info spawn args '-Dvisibility=default',
npm error gyp info spawn args '-Dnode_root_dir=/root/.cache/node-gyp/22.13.1',
npm error gyp info spawn args '-Dnode_gyp_dir=/usr/lib/node_modules/npm/node_modules/node-gyp',
npm error gyp info spawn args '-Dnode_lib_file=/root/.cache/node-gyp/22.13.1/<(target_arch)/node.lib',
npm error gyp info spawn args '-Dmodule_root_dir=/root/node_modules/node-rdkafka',
npm error gyp info spawn args '-Dnode_engine=v8',
npm error gyp info spawn args '--depth=.',
npm error gyp info spawn args '--no-parallel',
npm error gyp info spawn args '--generator-output',
npm error gyp info spawn args 'build',
npm error gyp info spawn args '-Goutput_dir=.'
npm error gyp info spawn args ]
npm error gyp info spawn make
npm error gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
npm error g++: error: unrecognized command line option ‘-std=c++20’; did you mean ‘-std=c++2a’?
npm error make: *** [node-librdkafka.target.mk:128: Release/obj.target/node-librdkafka/src/binding.o] Error 1
npm error gyp ERR! build error
npm error gyp ERR! stack Error: `make` failed with exit code: 2
npm error gyp ERR! stack at ChildProcess.<anonymous> (/usr/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:216:23)
npm error gyp ERR! System Linux 6.12.11-200.fc41.x86_64
npm error gyp ERR! command "/usr/bin/node-22" "/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
npm error gyp ERR! cwd /root/node_modules/node-rdkafka
npm error gyp ERR! node -v v22.13.1
npm error gyp ERR! node-gyp -v v11.0.0
npm error gyp ERR! not okNotice line: |
Well, Node 22 cannot itself be compiled with the default gcc; we use |
That would also mean introducing it for the container, but I think it is a reasonable assumption from npm package developers that a capable enough compiler that was used for node build is used for npm compilation as well. Let's try it out here. |
Node.js 22's minimal GCC requirement is higher than the default GCC and GCC-C++ available on RHEL 8. It is reasonable from npm module developers to expect at least the minimal version node.js supports for compilation will be available to also compile the native extension from source. Therefore use and enable the gcc-toolset-14 collection inside container.
|
@phracek could you please try to rerun CI, I have also, additionally, added gcc-toolset-14 into the container instead of the regular gcc, we don't use the base version for Node.js build itself, and I think npm native extension authors could in theory expect compiler capable of compiling node.js to also compile their extensions. This should make RHEL 8 pass for the binary application test (again). |
|
let's re-run tests [test] |
|
Those related nodejs 22 tests look green, now that there is also the newer gcc-toolset-14 |
|
This loooks fine. Let's get merge it so we do not block #475 |
4 participants
A nodejs tool "node-gyp" used for compiling native extensions from npm from source requires python version >= 3.8.
Install the high-enough python version 3.12 for RHEL 8 container. RHEL 8 base version is 3.6, it is not enough for node-gyp. Python 3.12 in RHEL 8 has support until 2029,
the currently projected RHEL 8 EOL, similarly to python 3.6
nodejs-22-minimal container does not do anything with python, leave that Containerfile as-is.