test_shelve.py crashes on WSL?

[JasonMendoza2008]

To reproduce, clone the repo, and run ./configure --with-pydebug && make -j $(nproc).

I'm on WSL (Ubuntu 24.04.1 LTS).

I get:

0:04:10 load avg: 1.81 [362/493/1] test_shelve failed (1 error)
test test_shelve failed -- Traceback (most recent call last):
  File "/home/lhott/cpython/Lib/test/test_shelve.py", line 205, in test_custom_serializer_and_deserializer
    s["bytearray_data"] = bytearray_data
    ~^^^^^^^^^^^^^^^^^^
  File "/home/lhott/cpython/Lib/shelve.py", line 138, in __setitem__
    self.dict[key.encode(self.keyencoding)] = serialized_value
    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: gdbm mappings have bytes or string indices only

I had a few optional modules that were not found when I compiled Python, but if that was a problem, shouldn't test_shelve be skipped?

I assume _dbm is the problem but I'm not familiar with dbm modules.

Linked PRs

• gh-137829: Fix shelve tests for backend compatibility #137879

[JasonMendoza2008]

I had a few optional modules that were not found when I compiled Python, but if that was a problem, shouldn't test_shelve be skipped?

I assume `_dbm` is the problem but I'm not familiar with `dbm` modules.

I installed _dbm then I ran ./configure --with-pydebug and make -j"$(nproc)" and I i still have the test that fails (no more _dbm missing when I compile Python though).

[StanFromIreland]

I had a few optional modules that were not found when I compiled Python, but if that was a problem, shouldn't test_shelve be skipped?

It most likely fell back to the Python implementation.

[JasonMendoza2008]

I had a few optional modules that were not found when I compiled Python, but if that was a problem, shouldn't test_shelve be skipped?

It most likely fell back to the Python implementation.

Yeah, reading the error message, I think the problem is this:

cpython/Lib/test/test_shelve.py

Lines 174 to 176 in 4e08a9f

	if isinstance(obj, (bytes, bytearray, str)):
	if protocol == 5:
	return obj

[JasonMendoza2008]

Changing it to:

                 if protocol == 5:
+                    if isinstance(obj, bytearray):
+                        return bytes(obj)
                     return obj
                 return type(obj).__name__

Makes the test case pass.

[furkanonder]

@JasonMendoza2008

Could you please run the following diagnostic script to gather additional information?

import dbm
import os
import glob
import shelve
import sys

print("=== Python Environment ===")
print(f"Python version: {sys.version}")
print(f"Platform: {sys.platform}")

print("\n=== Shelve Default Backend ===")
with shelve.open("myshelf", "c") as shelf:
    shelf["test"] = "data"
backend = dbm.whichdb("myshelf")
print(f"Shelve DBM backend: {backend}")
for file in glob.glob("myshelf*"):
    os.remove(file)

print("\n=== Individual Backend Functionality ===")
for backend_name in ["sqlite3", "gnu", "ndbm", "dumb"]:
    try:
        backend_module = __import__(f"dbm.{backend_name}", fromlist=[""])
        test_db = backend_module.open(f"test_{backend_name}", "c")
        test_db["test"] = b"data"
        test_db.close()
        print(f"dbm.{backend_name}: Functional")
    except Exception as e:
        print(f"dbm.{backend_name}: {e}")

for backend_name in ["sqlite3", "gnu", "ndbm", "dumb"]:
    for file in glob.glob(f"test_{backend_name}*"):
        os.remove(file)

[JasonMendoza2008]

=== Python Environment ===
Python version: 3.15.0a0 (heads/fix-issue-137759:207cccd106, Aug 15 2025, 19:34:35) [GCC 13.3.0]
Platform: linux

=== Shelve Default Backend ===
Shelve DBM backend: dbm.gnu

=== Individual Backend Functionality ===
dbm.sqlite3: No module named '_sqlite3'
dbm.gnu: Functional
dbm.ndbm: Functional
dbm.dumb: Functional

PS: It says heads/fix-issue-137759 but I had the problem before changing anything (and the CI tests run fine in the corresponding PR opened).

[furkanonder]

@JasonMendoza2008

Actually two test are failing;

======================================================================
ERROR: test_custom_incomplete_serializer_and_deserializer (test.test_shelve.TestCase.test_custom_incomplete_serializer_and_deserializer)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/arf/Desktop/source-codes/cpython/Lib/test/test_shelve.py", line 239, in test_custom_incomplete_serializer_and_deserializer
    s["foo"] = "bar"
    ~^^^^^^^
  File "/home/arf/Desktop/source-codes/cpython/Lib/shelve.py", line 138, in __setitem__
    self.dict[key.encode(self.keyencoding)] = serialized_value
    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: gdbm mappings have bytes or string indices only

======================================================================
ERROR: test_custom_serializer_and_deserializer (test.test_shelve.TestCase.test_custom_serializer_and_deserializer) (proto=5)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/arf/Desktop/source-codes/cpython/Lib/test/test_shelve.py", line 205, in test_custom_serializer_and_deserializer
    s["bytearray_data"] = bytearray_data
    ~^^^^^^^^^^^^^^^^^^
  File "/home/arf/Desktop/source-codes/cpython/Lib/shelve.py", line 138, in __setitem__
    self.dict[key.encode(self.keyencoding)] = serialized_value
    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: gdbm mappings have bytes or string indices only

----------------------------------------------------------------------
Ran 356 tests in 1.130s

FAILED (errors=2, skipped=2)
test test_shelve failed
0:00:01 load avg: 1.36 [1/1/1] test_shelve failed (2 errors)

== Tests result: FAILURE ==

1 test failed:
    test_shelve

Total duration: 1.2 sec
Total tests: run=356 skipped=2
Total test files: run=1/1 failed=1
Result: FAILURE

I've implemented a fix for the issue. Thanks for the bug report.

[furkanonder]

Actually, the shelf automatically selects the dbm backend. I think it would be more useful if we had a feature that would specify the dbm backend when the shelf opens.

[picnixz]

I'm confused. The TypeError appears to be because of the key being a string and then we try to encode it somehow and then use it. However the fix seems to be in the serializer, which acts on the value. Am I missing something? (I'm on mobile so hard to review)