Instances of UserString cannot be used with str.join

[ricardobranco777]

Bug report

Bug description:

from collections import UserString

class Str(UserString):
    def __init__(self, s):
        self.data = s

s = Str("abc")
s2 = Str("def")
print(s)
print(" ".join([s, s2]))

Output:

abc
Traceback (most recent call last):
  File "/tmp/s.py", line 11, in <module>
    print(" ".join([s, s2]))
          ^^^^^^^^^^^^^^^^^
TypeError: sequence item 0: expected str instance, Str found

CPython versions tested on:

3.12

Operating systems tested on:

Linux

[picnixz]

I'd say it depends on the usage. Having ''.join() raise when it's not a strict string is correct IMO because we don't want the arguments to be coerced automatically. On the other hand, for UserString('').join([UserString('')]) to fail, I agree that it's somewhat an inconsistency because:

>>> type(UserString('').lower())
<class 'collections.UserString'>

So it means that lower() is has a covariant return. The same should be done for join(), at least for consistency, or it should be documented (UserString.join returns a str and not a UserString instance).

@rhettinger As the maintainer of collections, what do you think about it? I'm -1 on changing the behaviour of str.join, but I think it's worth changing the behaviour of UserString.join, at least to match the variance of other methods of UserString.

[ericvsmith]

I don't think this request is about coercing the arguments, it's about optimizing str.join(). Although we do occasionally see requests to call str() on all parameters to str.join().

It's a different topic, but you could implement UserString.join() to have the behavior you want.

[rhettinger]

While would I support having the regular str.join() coerce its inputs to str, not everyone agrees (so a PEP would be needed). Mostly the community thinks it that it is only a minimal burden to manually coerce the inputs: ''.join(map(str, data)). That would suffice for the OP's case.

The decision to make UserString.join() return an actual str is very old (dating back to the module's creation back in 2000). Switching the return type now would be a breaking change. In general, it is difficult to guess in advance what a user would actually want here.

FWIW, this class is not much used and when it is, people tend to be doing exotic things that they can't do with a plain str (such as multiple inheritance with otherwise incompatible base types, mutating the underlying data, or making major alternations to the semantics of some of the methods). That makes it even more difficult to anticipate what a user might want instead of what they already have.

[picnixz]

Mostly the community thinks it that it is only a minimal burden to manually coerce the inputs:

I think as well because sometimes you don't need to call str if you know in advance that you are already using a str.

The decision to make UserString.join() return an actual str is very old (dating back to the module's creation back in 2000).

Ah yes, I see the history. Then I prefer not changing anything.

It's a different topic, but you could implement UserString.join() to have the behavior you want.

It would solve my point but not the OP's :( but yes, it's probably better than changing the current behaviour as Raymond said.

FWIW, this class is not much used

Actually, a quick search on Github shows a moderate number of results but I don't know how many are relevant or are CPython (I don't know how to parametrize my search for that and GH search is really bad IMO). Personally, I never used that class but I knew of its existence (if I need to subclass the str class, it's for magical tricks that I know people shouldn't do in general...).

---

I feel that the issue requires more discussion and a wider audience. I suggest to close this issue, and ask on discourse (https://discuss.python.org/c/ideas/6) whether str.join() should automatically coerce its arguments to str() or not (but first check whether the topic does not already exist and that the proposal was not rejected by the community lately).

What do you think @ericvsmith @rhettinger? (Sorry Ezio if you look here, I wrongly pinged you...)

[rhettinger]

I think as well because sometimes you don't need to call str if you know in advance that you are already using a str.

The actual implementation would only call __str__ if the input was not already a str. It is a fallback that would not add overhead.

[picnixz]

Well.. I think we should close this issue for now and open a discussion on discourse + possibly write a PEP to change the behaviour of str.join(), even if the overhead is minimal (I'm just wondering whether you want to keep this specific issue opened or not since you rejected the change on UserString.join() and since changing str.join would require a wider audience, not just the 4(?) of us).

[jccardonar]

This is completely broken, you should either let join interact with UserSring, or just remove UserString entirely.