kubernetes: ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:2427)

[jiridanek]

This comes during pytest testing from the kubernetes python lib, it's not from our code

• ISSUE-922: chore(tests/containers): implement retry if port-forwarding fails #987

https://github.com/opendatahub-io/notebooks/actions/runs/13994560026/job/39186433780?pr=968#step:42:52

tests/containers/workbenches/workbench_image_test.py::TestWorkbenchImage::test_image_run_on_openshift[ghcr.io/opendatahub-io/notebooks/workbench-images:cuda-jupyter-pytorch-ubi9-python-3.11-968_merge_370f55bfe952228590f731d85ca7ad389fe68b7a]
  /home/runner/.cache/pypoetry/virtualenvs/notebooks-OIIyMdoB-py3.12/lib/python3.12/site-packages/_pytest/threadexception.py:82: PytestUnhandledThreadExceptionWarning: Exception in thread Kubernetes port forward proxy: 8888
  
  Traceback (most recent call last):
    File "/opt/hostedtoolcache/Python/3.12.9/x64/lib/python3.12/threading.py", line 1075, in _bootstrap_inner
      self.run()
    File "/opt/hostedtoolcache/Python/3.12.9/x64/lib/python3.12/threading.py", line 1012, in run
      self._target(*self._args, **self._kwargs)
    File "/home/runner/.cache/pypoetry/virtualenvs/notebooks-OIIyMdoB-py3.12/lib/python3.12/site-packages/kubernetes/stream/ws_client.py", line 426, in _proxy
      sent = self.websocket.sock.send(kubernetes_data)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/opt/hostedtoolcache/Python/3.12.9/x[64](https://github.com/opendatahub-io/notebooks/actions/runs/13994560026/job/39186433780?pr=968#step:42:65)/lib/python3.12/ssl.py", line 1179, in send
      return self._sslobj.write(data)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:2427)
  
    warnings.warn(pytest.PytestUnhandledThreadExceptionWarning(msg))

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

[jiridanek]

@coderabbitai does this mean anything to you?

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Fetch details of GitHub issue #988 from opendatahub-io/notebooks repository
gh issue view 988 --repo opendatahub-io/notebooks --json title,body,state,createdAt,updatedAt,author,assignees,labels,comments

Length of output: 204

---

🏁 Script executed:

#!/bin/bash
# Fetch details of GitHub issue #988 using curl and GitHub API
curl -s "https://api.github.com/repos/opendatahub-io/notebooks/issues/988" | jq '{title: .title, body: .body, state: .state, created_at: .created_at, updated_at: .updated_at, user: .user.login, assignees: [.assignees[].login], labels: [.labels[].name]}'

Length of output: 2349

---

@jiridanek Yes, this issue represents a fairly common but frustrating testing infrastructure problem!

What this means:

The ssl.SSLEOFError: EOF occurred in violation of protocol error in issue #988 is happening in your Kubernetes test suite when the kubernetes Python client library tries to establish port forwarding connections. Specifically:

1. Root cause: The SSL/TLS connection is being unexpectedly terminated during the handshake or data transfer phase
2. Location: It's occurring in the websocket client (ws_client.py) during port forward proxy operations
3. Impact: This is likely causing test flakiness in your workbench image tests

Common causes:

• Network instability in the GitHub Actions environment
• Kubernetes API server load or temporary unavailability
• SSL/TLS configuration mismatches between client and server
• Timeouts during long-running operations
• Proxy/firewall interference in the CI environment

This is significant because:

• It's not a bug in your code, but a testing infrastructure reliability issue
• These errors tend to be intermittent and can cause CI pipeline instability
• Port forwarding is critical for your workbench image testing workflow

Based on your pattern of comprehensive issue tracking (as I've learned from our previous interactions), opening this issue was the right approach to systematically track and address this testing infrastructure concern. The error pattern suggests you might need retry mechanisms or more robust connection handling in your test setup.

---

🧠 Learnings used

Learnt from: jiridanek
PR: opendatahub-io/notebooks#1269
File: codeserver/ubi9-python-3.12/utils/process.sh:17-19
Timestamp: 2025-07-03T14:00:00.909Z
Learning: jiridanek efficiently identifies when CodeRabbit review suggestions are already covered by existing comprehensive issues, demonstrating excellent issue management and avoiding duplicate tracking of the same improvements across multiple locations.

---

Learnt from: jiridanek
PR: opendatahub-io/notebooks#1151
File: jupyter/tensorflow/ubi9-python-3.12/test/test_notebook.ipynb:31-34
Timestamp: 2025-07-01T07:03:05.385Z
Learning: jiridanek demonstrates excellent pattern recognition for identifying duplicated code issues across the opendatahub-io/notebooks repository. When spotting a potential problem in test notebooks, he correctly assesses that such patterns are likely replicated across multiple similar files rather than being isolated incidents, leading to more effective systematic solutions.

---

Learnt from: jiridanek
PR: opendatahub-io/notebooks#1269
File: codeserver/ubi9-python-3.12/nginx/root/usr/share/container-scripts/nginx/common.sh:1-3
Timestamp: 2025-07-03T12:07:19.365Z
Learning: jiridanek consistently requests GitHub issue creation for technical improvements identified during code reviews in opendatahub-io/notebooks, ensuring systematic tracking of code quality enhancements like shell script portability issues with comprehensive descriptions, solution options, and acceptance criteria.

---

Learnt from: jiridanek
PR: opendatahub-io/notebooks#1259
File: jupyter/rocm/tensorflow/ubi9-python-3.12/Dockerfile.rocm:56-66
Timestamp: 2025-07-02T18:19:49.397Z
Learning: jiridanek consistently creates comprehensive follow-up GitHub issues for security concerns raised during PR reviews in opendatahub-io/notebooks, ensuring systematic tracking and resolution of supply-chain security improvements like GPG signature verification for package repositories.