Shared boxing and boxed self

[hydroper]

Summary of issue:

Reference counting and boxed self.

Details:

Name

What naming convention are you going to use? E.g., shared_ptr and atomic_ptr map to...?

Self as any Boxed Type

fn C.F[self: Atomic(Self)]() {}

Any other information that you want to share?

No response

[hydroper]

This thing of boxed self was something I was thinking of using in Rust after several thoughts with a node pattern, but the Bevy Engine's ECS pattern already solves everything correctly, so it's not an use-case anymore for me.

I asked something related to this for C++ in StackOverflow, but since I deleted my previous account, I'm not finding the question...

[geoffromer]

See this accepted proposal for the overall naming conventions we're following. I don't think we've discussed issues like boxed Self and reference-counting in much depth yet.

[Kubiyak]

What are the thoughts on supporting read/copy/update/ RCU as a key pattern in the language? Eager deletion is a source of many errors in a number of contexts in c++.

Just make the machinery for that part of the language or provide the pieces so that a moderately skilled programmer can glue it together.

[geoffromer]

Can you point to any languages that you think address this problem well?

And yes, cursors can help with this problem in a few cases (and the current plan is that Carbon for loops will be based on cursors), but many others where they can't.

[Kubiyak]

I am not a huge PL expert as most of my practical experience is with C++ and Python. But...

Python addresses iterator invalidation well. I can reason about it and rely on an exception rather than undefined behavior when invalidation occurs. Much of the success of Python is that it is easy to build on the work of other developers in Python and the data structure traversal model in Python does not get in the way of that.

C++ on the other hand is particularly difficult when building on the work of other developers. A lot of event driven code becomes complex quickly and ascertaining the author's intent from the source code itself becomes impractical. It takes a lot of effort for example to understand what is allowed in boost::asio and to work out what can and cannot be done in boost::asio callbacks.

At the least, I would like some additional methods on library types such as sets, maps, vectors, etc which would allow those types to be placed into a context manager which either disallows updates while the context manager is active or possibly applies updates when the destructor of the context manager is called. Context managers can be user defined (and should be) but the underlying methods they can call on sets/maps/ etc have to be provided by the implementation.

And by context manager I mean a very specific type of RAII object which only allows creation on the stack and disallows pointers/references/passing as a parameter/returning from a function.

^^^ A simple stop light function is good enough.

if (map.in_context_manager()) {
// oops cannot update. Take appropriate action
} else {
map.emplace(args);
}

Yes I can wrap a STL map and kinda sorta get to the above but this takes a lot of effort on the part of the user. It would help if carbon took care of the heavy lifting and provided a true context manager type which is a restricted RAII type as discussed.

I'm very optimistic about Carbon but I need carbon to fill in some of the void left by the c++ standard's treatment of UB to really make profitable use of it. I can accept that some of the UB that c++ allows isn't going away but iterators are too important to let C++ have the last word on UB. Please provide some better answers than the c++ standard for iterator invalidation and please do highlight how writing event driven code based on callbacks is simplfied by carbon over and above what is possible in c++.

[geoffromer]

According to the Python documentation, modifying a dict during iteration doesn't necessarily raise an exception-- it can also just silently omit items from the iteration. I see that as a kind of constrained UB- they don't fully define what will happen, but they do put boundaries around it -- it can omit items, but it seemingly can't make up items that aren't there, or crash the program, etc.

We might be able to do something similar in Carbon. For example, my guess is the default map type will be an open-addressed hash table, which I think would pretty easily let us guarantee that mutating during iteration can cause items to be omitted or duplicated, but not cause other misbehavior. However, that only applies if the mutation happens between iterator operations -- in the event of a concurrent mutation, I don't think we can avoid full-blown UB.

Your context manager idea is interesting, but my hunch is that in almost all cases where it would be useful, the mutation during iteration is a bug. If so, rather than trying to recover from those bugs when they happen, I think we should focus on detecting and diagnosing them so that they can be fixed. The advantage of that is that we can do that without complicating the APIs of the containers, and without making the user remember to install a context manager before iterating.

[Kubiyak]

Detection and diagnosing is good enough. This would in effect bound the UB. The problem atm is that when multiple people work on callback based systems it is possible for UB to creep into the system w/o any warning. A seemingly innocuous change can introduce UB.

A simple example is where an event has triggered callbacks and the mechanism for doing so involves iterating some structure to call registered functions and then one of those functions attempts to add another callback.

This specific issue is very difficult to contain in a large system. Usually the developers closest to the underlying library know of this issue and defend against it. But they themselves have to create APIs and developers who use those APIs can often use those APIs in unexpected ways leading eventually after many changes in the system to UB.

Some detection and diagnosis system would be very helpful whatever the means of achieving it. I really think turning classes into context managers by restricting them as described is still a useful concept in its own right and worth further investigation but I will take any solution for bounding UB w/ iterators.

[Kubiyak]

^^ I have just accepted employment w/ a company that restricts my contributions to github etc. so I cannot continue to engage in this discussion. But thank you for reading my posts and I wish you much success in your endeavor and I hope to be a Carbon user in the future.