setdefault() with a single argument doesn't work for dbm.gnu and dmb.dumb objects

[serhiy-storchaka]

BPO	33385
Nosy	@rhettinger, @ned-deily, @serhiy-storchaka

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2018-04-29.14:00:31.917>
labels = ['3.8', 'type-feature', 'library']
title = "setdefault() with a single argument doesn't work for dbm.gnu and dmb.dumb objects"
updated_at = <Date 2018-04-30.09:11:07.896>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2018-04-30.09:11:07.896>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2018-04-29.14:00:31.917>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 33385
keywords = []
message_count = 4.0
messages = ['315898', '315925', '315931', '315943']
nosy_count = 3.0
nosy_names = ['rhettinger', 'ned.deily', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue33385'
versions = ['Python 3.8']

Linked PRs

• gh-77566: Clarify setdefault() behavior for dbm objects #137911
• gh-77566: Make setdefault methods require explicit default argument for dbm backends #138358

[serhiy-storchaka]

setdefault() is not implemented directly in dbm.gdbm and dmb.ndbm database classes. It is inherited from MutableMapping:

    def setdefault(self, key, default=None):
        try:
            return self[key]
        except KeyError:
            self[key] = default
        return default

But since assigning is supported only for bytes and str, setdefault(key) fails if the key was not set before. It works only if the key was set or with the second argument. d.setdefault(key) is equivalent to d[key] except that it raises a weird TypeError instead of KeyError.

There are two ways of solving this problem:

1. Reimplement setdefault() for dbm.gdbm and dmb.ndbm database classes with default=b'' by default.

2. Make the second argument mandatory.

In both cases this violates the MutableMapping interface.

[rhettinger]

Option 2 seems perfectly reasonable here as a way of preventing bugs (working, correct code wouldn't be using the existing default of None).

Option 1 is problematic because of the variance from Mutable Mapping, because breaking symmetry with get(), and because the change in behavior is implicit.

[serhiy-storchaka]

Thank you Raymond. I hesitated to make a choose.

Ned, is it good to add a deprecation warning in setdefault() methods of the dbm classes when they called with a single argument in 3.7? Their current behavior (no-op or error) is not useful in any case.

[serhiy-storchaka]

Actually dbm.ndbm already implements option 1. Only dbm.gnu and dbm.dumb have non-working with a single argument setdefault().

[furkanonder]

Actually dbm.ndbm already implements option 1. Only dbm.gnu and dbm.dumb have non-working with a single argument setdefault().

We are currently experiencing this issue on all DBM backends.

import tempfile
import os

backends = ["dbm.sqlite3", "dbm.gnu", "dbm.ndbm", "dbm.dumb"]

for backend_name in backends:
    print(f"\n=== Testing {backend_name} ===")

    if backend_name == "dbm.sqlite3":
        import dbm.sqlite3 as backend
    elif backend_name == "dbm.gnu":
        import dbm.gnu as backend
    elif backend_name == "dbm.ndbm":
        import dbm.ndbm as backend
    elif backend_name == "dbm.dumb":
        import dbm.dumb as backend

    with tempfile.TemporaryDirectory() as tmpdir:
        db_path = os.path.join(tmpdir, f'test_{backend_name.split(".")[-1]}.db')
        with backend.open(db_path, "c") as db:
            db["existing"] = "value"
            try:
                result = db.setdefault("missing")
                print(f"setdefault('missing'): {result}")
            except Exception as e:
                print(f"setdefault(missing) failed: {e}")

Output:

=== Testing dbm.sqlite3 ===
setdefault(missing) failed: NOT NULL constraint failed: Dict.value

=== Testing dbm.gnu ===
setdefault(missing) failed: gdbm mappings have bytes or string indices only

=== Testing dbm.ndbm ===
setdefault(missing) failed: cannot add item to database

=== Testing dbm.dumb ===
setdefault(missing) failed: values must be bytes or strings

In my opinion, the first step in fixing this issue would be to specify that the default argument is mandatory in the DBM documentation.