Reworked the docs for contexts

[FroMage]

Let me try to sum up what I think we can do to improve from the context docs:

Terminology

The terminology is confusing to me. This is more about the API names, because the names chosen attachContext and withContext are either ambiguous or do the opposite of what people expect.

Normally, attaching is the operation of taking something from outside and tacking it on to something. In this scenario, it should mean taking a context and binding it to a pipeline. So, attaching it to the pipeline.

It does not, in most people's mind, imply that this is how you access a previously attached/bound context from a pipeline.

So, attachContext should be renamed something like accessContext or reifyContext (although nobody knows that correct term), or even materializeContext (as you use that term sometimes, which is probably correct, but just like reify I suppose most people won't understand that it means, for all intends and purposes here: access a context). It could be getContext, readContext, accessContext, obtainContext…

Now, withContext is also problematic because it's ambiguous: there's one interpretation of it that does exactly what it currently does, which is "pair/bundle/wrap the item with the context". But there's also a much more commonly used interpretation where it means exactly the opposite: "attach a context to this pipeline / run this pipeline with this context".

In fact, I was convinced this was how to attach a context. Case in point, subscribe().with uses the same verb.

So the withContext operation also allows people to access the context. The difference with attachContext is that one is called during subscription and the other during item production.

I am not sure how it should be named, but I am 100% sure that the current names are confusing. I guess we should discuss this to see if a better name emerges.

What contexts do and are used for

I think there's a blob of text missing that explains that the contexts flow from subscription to the entire Uni pipeline and that this is useful to propagate contexts from one end of the pipeline to another.

There's probably a section missing about how people should use this, as evidenced by our Zulip discussion, about either modifying the context, or creating subcontexs, and how these contexts work with Uni.combine.

I suppose the question ultimately is about branches (Uni.combine) but also scopes: how do you deal with contexts that span from one part of the pipeline until another part?

I suppose one example would help. I bet this is a common use-case.

Missing operation?

I think we should add an API for switching the context as part of the pipeline, like the one I documented:

        Uni<String> uniWithContext = Uni.createFrom().emitter(emitter -> {
            uni.subscribe().with(newContext, emitter::complete, emitter::fail);
        });

I suppose this could be Uni<T> Uni.switchContext(Context) or overrideContext or withContext depending on what we do with the "access context on subscription" operation that is currently using that name.

This might depend on the discussion above about scopes of contexts and how to modify or override them.

What this PR already contains

• Added a section on how to build the context: if you define the operations and how to access it, it made sense to me to also define how to create contexts.
• Added a section on how to attach a context: given my intuition for what is "attaching a context", I've added a section explicitly about attaching contexts, and…
• Removed context attachment from docs about accessing the context: Given the new attachment section, removed those operations from the docs on how to access a context because that confused both operations.
• Simplified the Multi docs with Uni: the transformToUniAndMerge operations were adding complexity that I felt were not relevant to the docs. Unis are just simpler to use as examples.
• Reworked indentation so it fits: hidden long lines of code that require scrolling leads to people not reading the docs.
• Removed "pipeline" names in favour of uni/multi: I had the hardest time guessing the type of the object that was documented, because its definition was not shown in the docs. Using uni makes it simpler and obvious what we're talking about.

This PR is not complete, I figured it's a good start and we can start discussing it next year and I can continue working on it after we hit some consensus :)

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.19%. Comparing base (a1998df) to head (ed6a73c).
⚠️ Report is 15 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #2037      +/-   ##
============================================
+ Coverage     89.18%   89.19%   +0.01%     
- Complexity     3120     3124       +4     
============================================
  Files           412      412              
  Lines         13295    13295              
  Branches       1688     1688              
============================================
+ Hits          11857    11859       +2     
+ Misses          812      808       -4     
- Partials        626      628       +2     

see 10 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[gavinking]

suppose the question ultimately is about branches

so ... don't leave me hanging like this ... what are the semantics of context propagation when you fork the stream (and what are the semantics when you later join the branches).

[FroMage]

suppose the question ultimately is about branches

so ... don't leave me hanging like this ... what are the semantics of context propagation when you fork the stream (and what are the semantics when you later join the branches).

Very good question. This appears to be undefined, and not even mentioned.

I suppose we could make guesses as to how each contextual data works with fork/joins:

• I suppose the request context spans the entire request, and so it propagates to each fork (which does not help very much since CDI is not thread-safe, but at least it's consistent with the MP-CP definition)
• I suppose it's safe that each fork can define its own TX context, and reject inheriting a TX context, and not leak back the TX context to the forker/joiner
• And I suppose that tracing will have its own behaviour

In short, I think it depends on the context.

But in the docs, it's not even mentioned and so most people won't be aware of the issue and its problems and solutions.

[jponge]

Hey @FroMage sorry for taking a while to get back to this.

Thanks for proposing documentation updates on Mutiny contexts / what's in the draft here is already a solid improvement.

On terminology:

• reify is a good name. attachContext (likely) was named due to my understanding of items flowing in a stream, and you'd attach the context to each element as a convenience. If you're in the Uni / 1-shot async operation case, then perhaps it makes less sense indeed.
• speaking of withContext: the name comes from the point-of-view of building a pipeline (aka "the rest from here will have access to it"). Perhaps bindContext? (especially because the method takes a lambda)

[jponge]

A note of how this API was designed, and thinking out loud

You'll find similar constructs in Reactor (see https://projectreactor.io/docs/core/release/reference/advancedFeatures/context.html), but the Mutiny interpretation of what a context API should be differs (basically Reactor has immutability and a mutation operator, etc).

My idea was that each operator in a pipeline should have some context to avoid lambda captures of relevant input information (e.g., you get an item, and you want to use transformToUni to make a HTTP request, and you need some correlation identifier to propagate as a HTTP header, etc).

Such reactive APIs are subscription-based:

• you get to design pipelines that describe what to do as asynchronous events flow,
• you trigger such operations by subscribing to a pipeline, and the subscription gives you flow control (in the Multi case) and cancellation (both Uni and Multi)
• you can subscribe as many times as you want, and every time this triggers a new invocation of the operations in the pipeline.

It hence made sense to tie contexts to subscriptions, because this is what defines when some asynchronous processing starts. This is why the context data gets passed to the subscription.

Operators get passed the context from their subscribers, so effectively all operators in a pipeline share the same subscription-bound context. This is very fine in a model where contexts are essentially meant to convey input data that's relevant for the whole pipeline logic (e.g., correlation identifiers) while the output results leverage the reactive flow (Uni result or Multi stream items).

In short: it was more designed to be read than to be written.

Now if writing becomes important there are indeed a few shortcomings in all operators sharing the same context reference.

1. Using String as keys is a bit simplistic, we might want specialized context key references.
2. Contexts could come with "fork" and "join" callbacks, so that some operators like Uni.combine() or Uni.join() could forward derived contexts to their upstreams, then have a callback to reconcile contexts when they have "joined" all upstream items. This is a vague idea, but perhaps it could help some use-cases like detecting when some forbidden concurrent operations have happened.