Memory model and its impact on refactoring

[EdmondDantes]

Memory model and its impact on refactoring

At this stage of the discussion, I propose focusing on one of the three main questions.

🔑 Key Questions

1. Choosing the Coroutine Model

How coroutines should behave internally:

• stackful vs stackless coroutines
• symmetric vs asymmetric control flow
• suspension and resumption semantics

At the moment, the stackful coroutine model has been chosen, since it allows interaction with C functions and enables mixing C and PHP function calls.
The likelihood of using a different model is low, so this question can be considered resolved.

2. Choosing the Memory Model

How state is managed in asynchronous execution:

• handling of globals and statics
• coroutine-local or thread-local data
• isolation rules vs shared-state trade-offs

The current RFC proposes using the least protected memory model, where coroutines freely share global and static variables.
This question is not resolved and requires further consideration.
The choice of memory model will have a significant impact on the evolution of the language for years to come.

3. Choosing the Runtime Strategy

Defining how async integrates into PHP:

• should PHP’s existing I/O functions become coroutine-compatible?
• or should a new async API be introduced for clarity and safety?

This question will not be discussed at this stage, but it remains important for the consideration of question 2.

🎯 Purpose of Stage 1

The goal of the first discussion stage is to explore the memory model for async execution in PHP.
The memory model has a significant impact on how coroutines will interact with existing PHP code. This is of primary importance. In addition, the memory model determines the direction of the language’s evolution for many years, defining whether a multithreaded model becomes possible or not.
Without resolving this question, we cannot move forward.

Important elements of the discussion can be found here:
https://discourse.thephp.foundation/t/php-dev-vote-true-async-rfc-1-6/4777

Please note the message from
Rowan_Tommins_IMSoP:
https://discourse.thephp.foundation/t/php-dev-vote-true-async-rfc-1-6/4777/75

---

Possible Memory Models for Coroutines

There are three potential memory models for coroutine execution in PHP:

---

1. No Memory Separation (current behavior)

Coroutines freely share:

• global variables
• static variables

Characteristics:

• The programmer is responsible for ensuring safe concurrent access.
• Highest risk of unpredictable interactions.
• for multithreading, the developer is required to explicitly write synchronization code!

---

2. Separated Globals/Statics (objects still shared)

Each coroutine receives:

• its own copy of globals
• its own copy of static variables
• for multithreading, the developer is required to explicitly write synchronization code

But:

• Objects remain shared between coroutines.

Status:

• This model requires deeper investigation.
• Provides partial isolation but still allows shared mutable state.

---

3. Full Shared-Nothing Model (maximum isolation)

This model extends option 2 with strict memory isolation:

Rules:

• PHP forbids passing an object into a coroutine if RefCount > 1.
• A coroutine must own the object it receives.
• Objects are not shared across coroutine boundaries.
• multithreading supportfor multithreading, the developer most likely does not need to change anything, because the memory model already accounts for shared access

Advantages:

• Guarantees memory isolation.
• Strongly resembles models used by Erlang.
• Thread safe by default, unless shared objects or other escape hatches are used.

Disadvantages:

• Requires developers to take special care when working with shared state.
• Requires introducing ownership semantics (own) into the language.
• May require defining a concept of shared objects.
• Requires designing a mechanism for cases where API services must be safely shared between coroutines.

P.S. Models one and two are used in many languages, including Go. In Go example, developers themselves are responsible for the incorrect use of shared memory.

---

[danog]

Hi all.

Senior software artisan here with 15 years of PHP experience, 8 years of async PHP experience (amphp v2, then v3), 5 years of go experience, 3 years of Rust experience, and much more.

My recommendation for async PHP is:

• Stackful coroutines
• Colorless functions
• Existing IO functions should be async
• Some memory isolation: shared statics maybe with a separate attribute (i.e. allow fiber-local statics which have uses, but also allow normal statics as is already the case to allow for caching).
  Getting rid of globals is absolutely not needed to get async into PHP, as mutexes can guard them just fine: they can also be very easily autoreplaced with amphp-like fiber local lookups in legacy code like WordPress, without forcing a shared-nothing model on the rest of the language.
  Race conditions are not an issue neither with statics nor with globals, like they are not an issue in the vast majority of languages, as there is a variety of tools and models to work with them: channels (go), mutexes (go, amphp, rust, C, C++, etc), actor model (any language), thread-locals (amphp fiber-locals, C/C++/Rust thread-local storage), etc.

This model matches the status quo in golang and amphp v3: from my 8 years of experience in writing async both business logic and abstraction logic in multiple languages, this is the best concurrency model.

Some evidence:

I maintain MadelineProto, the biggest PHP MTProto client.

MTProto is an async binary protocol over TCP (no HTTP involved), requires heavy caching in order for clients to function correctly, and a lot of race-heavy abstractions.

MadelineProto is a framework which contains a fully async MTProto client.

I migrated MadelineProto to async PHP with amphp v2 (colored, stackless async) in 2018, and then to amphp v3 (colorless, stackful async) in 2022.

• The biggest problem while using v2 was the colored approach, requiring the use of await (yield, using await for clarity from here on) every time a function may become async: this is the status quo of some languages like JS, but in reality it becomes a giant pain when writing a lot of business logic: not only that, there were constant breaking changes every time a method that previously wasn't async (i.e. had no network/IO logic) suddenly becomes async (thus requiring the use of await).

  In MadelineProto, I worked around this with a custom coroutine runtime (replacing that of amphp v2), which allowed users to await even non-async functions: by then forcing users to always await all methods of the framework, I managed to avoid breaking changes every time an abstraction became async.

  Clearly, this was a crutch, made to workaround a big usability issue for end users, caused by colored functions.

  Another issue was with the stackless approach, which essentially forced to use of call to spawn a new coroutine (creating a new generator) every time when invoking an async function: I again worked around this in my custom coroutine runtime, allowing to spawn a new coroutine simply by using await (yield).

  This in turn led to the creation of a new generator object every time await was used: I was also able to work around this, transforming every yield to a yield from: this reduced the overhead, but it was clearly yet another crutch, made to transform a stackless approach into a more usable stackful approach.

• The switch to amphp v3 finally got rid of the colored approach, switching to a stackful, colorless approach.

  This was a huge improvement in terms of developer UX: no more worrying about when to use await, no more custom coroutine runtimes to implement stackfulness to avoid using call() and await every time when calling an async function, all without impacting safety.

  Amphp v3's model is pretty close to golang's model, and mirrors go's ease of use.

• When migrating the MadelineProto framework to amphp, right away, it was clear to me that with concurrency, race conditions would be an issue.
  Amphp v2 already offered synchronization primitives in the form of mutexes, which are more than enough to guarantee safety.

  However, I wanted an easier approach for me as the library developer, which is why I, right away, chose to communicate instead of sharing memory, by adopting the actor model for MadelineProto's internal core modules (IO, update handling, any moderately large logic requiring synchronization).

  However, mutexes are still used for isolated, smaller logic, as they provide equivalent safety with fewer boilerplate compared to channels/actors.

  A forced shared-nothing approach only adds needless complexity and overhead, when the alternative is simply to isolate logic needing synchronization in a separate actor, or adding a few mutexes in key places, like is already done in many languages like Go, Rust (multithreaded with inner mutability), Java, C, C++, etc

  In other words: it is absolutely okay to share-nothing, actors (which implement the shared-nothing approach) are an excellent tool that perfectly and very easily adapts to a lot of existing logic, but they shouldn't be the only choice.

  Often, a simple mutex is the only thing you need, and is the option that adds the least amount of overhead, and guarantees the best performance.

  Go's approach encourages the use of actors (channels, aka shared-nothing), but also provides mutexes for simpler, high-performance usecases with less boilerplate.

• An issue that is still present to this day in amphp is the requirement to switch to their own IO API in order to write async logic (admittedly a lot better than PHP's stdlib): this is a huge issue especially for PHP developers that have only ever used PHP's own stdlib (curl, etc), or common libraries like guzzle.

  For example: MadelineProto exposes MTProto events through an event handler, which is a class with appropriately decorated user-defined methods which handle chosen events concurrently (each event is handled a new coroutine).

  If users use native PHP functions within those methods, they block execution of:

  • All other events
  • Most importantly, the library itself, which requires the periodic execution of time-sensitive operations in order to maintain connection with the MTProto servers (if MTProto updates aren't acked within a specific time frame, they are either resent or the connection is terminated by the server, plus ping_delay_disconnect must be emitted periodically to signal liveliness of the connection itself)

  To work around this, I run static analysis on code users write, warning them if they use non-async stdlib functions and libraries in the event handler, suggesting async alternatives (and also offering simple to use synchronization primitives).

  To this day, warnings emitted by the static analysis are the single biggest hurdle to thousands of new users of my framework, which simply do not understand why can't they just use file_get_contents or curl instead of amphp/http-client, amphp/file, etc.
  The same thing will happen to PHP, if a split approach is chosen (the old stdlib remains blocking, and a new async stdlib is made)

  To those who think making all of the PHP stdlib async will break stuff, I say: it is a misconception that making all PHP IO functions async will break existing applications.
  Existing legacy frameworks like wordpress will simply keep working as before, using existing application servers like php-fpm or apache.

  New application servers based on spawn will only be able to use modern, maintained frameworks which use proper synchronization primitives (mutexes, channels, etc), and currently-legacy frameworks (again, like Wordpress) can also be adapted with a little bit of effort (switching from globals to thread-locals, etc), just like I adapted the large codebase of MadelineProto over the course of a few months (including both v2 and v3 migrations).

  There are also ways to signal the async-safety of libraries: just like in other languages (go, java, C, C++, etc):

  • Via a note in the documentation (this class is safe to use concurrently), an example from the go prometheus library: https://pkg.go.dev/github.com/prometheus/client_golang/prometheus, All exported functions and methods are safe to be used concurrently unless specified otherwise.
  • Via a class attribute like Concurrent (closer to Rust's Sync attribute, though Rust's Sync doesn't directly indicate thread-safety, as race conditions on interior mutability are still possible on Sync structs; in fact, Rust's Sync attribute is more of a negative attribute, where its absence signal non-thread-safety, but its presence does not guarantee thread safety, and non-thread-safe structs should be marked !Sync, whereas a PHP Concurrent attribute could be a positive attribute, explicitly guaranteeing thread safety)

Outside in PHP: in 2020, I started writing heavily async business logic in Go, and was very positively surprised by the superior developer experience, especially when using async: I was also very pleased to learn that Go encourages (not forces) the use of the actor model through channels.

In my professional Go experience, I wrote heavily concurrent and massively parallel, high-load services, and golang's stackful, colorless approach.

I also used Rust to write async business logic, and suffered from the same issues I suffered with amphp v2: large amounts of boilerplate await keywords, heavily impacting UX.

To this day, I consider Go's approach superior to that of any other language, confirmed by extensive development experience.

[EdmondDantes]

From the PHP Engine side, we can provide developers with additional tools to help detect errors. For example:

1. Validate that an object has the concurrently attribute when a coroutine is created.
2. Track access to variables from multiple coroutines (only in debug mode, for instance).

The concurrently attribute can also be analyzed by static analysis tools.

In addition, code safety can be reinforced through a special directive at the package / Composer level.
If a project does not support the asynchronous style and its composer.json does not specify a special flag, it should not be able to accidentally load a package that contains asynchronous logic.
During the early stages of adoption, this will help prevent subtle and unpleasant mistakes.

There was also an idea to use a flag in php.ini. This gives us several mechanisms at once to avoid unintended side effects.

[bukka]

Treating WordPress and other code using globals heavily as not target for this proposal is likely path to rejection. It was already expressed by couple of voters that such approach is a no-go (it was in FPM context but I would guess it will be the same here) and if you consider that there are probably some "no" votes just based on the complexity and opinion that this is not needed at all, then I think there is not much chance this could pass.

So I think the most likely model to pass is shared nothing.

[bukka]

I don't think it would resolve much because you can often have framework calls using the globals. This is quite usual in WP for example but likely similar in legacy frameworks as well.

In other words I'm not sure the opt in mechanism is enough to get us over those objections.

[bukka]

Well if others think that we should at least try this approach, I wouldn't mind to try it in the next version of the RFC. Basically it would be something like giving the tools to separate globals / static members and clearly describe the potential pitfalls of sharing globals (which I think was even a bigger problem in the last version of the RFC). So something along the lines to improve the text and see what the voting result is if all other problems are ironed out (like the errors which resulted in quite a few other no's).

If that fails, we could look to other models.

[EdmondDantes]

I think we will vote on the memory model separately.
But by that time we need to research the code regarding the feasibility of implementing model 2.
There’s no way around digging into the code, and that’s fine.
It’s important to understand whether such research will actually be useful.

What if there are strong opponents to the idea of global and static being unique per coroutine.

[bukka]

Yeah that's the question. I thnk it might make sense to start 1 that we are sure is feasable and try to tune the RFC so it properly explains everything and see if it can pass in that form. If it doesn't pass, then it would make more sense to investigate more the other approaches.

[EdmondDantes]

Yeah that's the question. I thnk it might make sense to start 1 that we are sure is feasable and try to tune the RFC so it properly explains everything and see if it can pass in that form. If it doesn't pass, then it would make more sense to investigate more the other approaches.

Proposing model 1 only because the code is already written is tempting, but not very responsible.
Before that, I want to obtain proof that models 2 and 3 require a very large amount of effort.
I know that many features in other programming languages were also rejected because of the high implementation complexity.
So it’s normal to abandon something that is too hard to implement.

Then we will have an argument to abandon them.

[bukka]

INI approach is also path to rejection - that will give quite a few "no" votes straight away.

[bartvanhoutte]

This is something that could be a separate vote in the same RFC.

@bukka I don't think there's been a vote on no longer creating new ini settings? It seems to be more of a best practice?

[bukka]

Features could be enabled for example by a declare() in the entry point script (applied to the entire language for the duration of the execution)

Yeah I suggested declare as I think that would be more likely to pass IMO.

I don't think there's been a vote on no longer creating new ini settings? It seems to be more of a best practice?

It's not prohibited but there will be likely people voting against it just because it tries to use INI. This is just based on my experiance but I might be wrong ofc.

[danog]

It might be nice to in general introduce the concept of experimental language features (not covered by a BC promise, unstable, etc) that can be enabled via declare

[bukka]

That would require special policy RFC as we don't have concept of experimental feature in our BC gaurantees. It could be possibly grouped with the actual implementation approach (e.g. using declare).

[EdmondDantes]

That would require special policy RFC as we don't have concept of experimental feature in our BC gaurantees. It could be possibly grouped with the actual implementation approach (e.g. using declare).

I think we will hardly be able to avoid a policy if we seriously want to introduce a new feature into the language in a way that is 100% safe. But that’s part of a separate discussion.

[EdmondDantes]

I want to share a few thoughts regarding models 2 and 3.

It seems that the shared-nothing model is an absolute winner in terms of safety.

I want to point out that this does not come for free.
Erlang/Elixir users warn that this approach works extremely well for some cases and poorly for others. As expected, the strictest memory model increases code size. Unfortunately, it’s hard to estimate how much without reviewing specific cases.

The second memory model (as well as the first, of course) requires manual synchronization from the programmer.
Minus: the programmer may simply not do it.
Plus: maximum flexibility. Specialized objects such as mutexes, semaphores, and atomic counters become a mandatory part of the language.

The third memory model requires the same as the second. The difference is that shared memory must always be declared explicitly in the language.

• Most likely, memory model 1 requires the fewest code changes. However, it won’t allow an easy transition to multithreading in the future if that ever becomes necessary. (Is this a problem? Maybe, maybe not.)
• Memory model 2 is a middle ground between 1 and 3. It solves one of the long-standing issues and allows PHP to execute code in different threads without copying opcodes.
• Memory model 3 requires more effort, but it’s hard to estimate how much.

I have a classic PHP task that must be solved efficiently.
The task is:

• A REST API application with services.
• A service exists in memory only once (usually).

Each request is handled by a separate coroutine.
Question: how do we easily make services shared between coroutines?

Memory model 2 answers:
Just pass the service array into the new coroutine and use it!
I will make all static variables context-dependent per coroutine.

Memory model 3 answers:
I don’t know how to make all services shared across different coroutines.

What are the possible solutions for model 3?

Readonly classes.
We can create a list of services that are guaranteed to be readonly in shared access.
For memory model 3, passing readonly objects is also allowed as long as they are truly 100% readonly.

Using actors.

An actor is a class whose methods are guaranteed to be called only within a single coroutine (at the same time).
All methods of the class are always invoked asynchronously, and their result is always a Future.
This is essentially an analogue of having an async keyword next to a function.

Actors significantly simplify writing asynchronous code, are thread-safe, and can be freely passed between coroutines.

[danog]

An actor is a class whose methods are guaranteed to be called only within a single coroutine (at the same time).
All methods of the class are always invoked asynchronously, and their result is always a Future.
This is essentially an analogue of having an async keyword next to a function.

This is not correct, the actor model does not automatically imply colored functions.

The actor model simply relies exclusively on communication via message passing: each message received by an actor is handled in order, one at a time (the message handler runs in a single thread/fiber, no concurrency, even if the callers can invoke concurrently), and can ONLY:

• Send other messages to other actors (such as a response message to the method call)
• Alter inner object state
• Create new actors

In other words, this is an actor (pseudo-code, not strictly adherent to the rfc, but can be implemented right now using fibers):

class a {
    private int $a;

    private Channel $chan;
    private Fiber $handle;
    public function __construct() {
        $this->handle = spawn(function () {
            foreach ($this->channel as [$reply, $type, $args]) {
                switch ($type) {
                    case 'setA':
                        $this->a = $args[0];
                        $reply();
                        break;
                    case 'getA':
                        $reply($this->a);
                        break;
                    // And so on...
                }
            }
        });
    }

    // Methods CANNOT mutate state, only send messages to $chan

    public function setA(int $a): void {
        $this->channel->send(__METHOD__, $a);
    }
    public function getA(): int {
        return $this->channel->send(__METHOD__);
    }
}

No function coloring needed here.

Actors significantly simplify writing asynchronous code, are thread-safe, and can be freely passed between coroutines.

This is correct.

[danog]

At the same time, as mentioned above by Edmond,

Erlang/Elixir users warn that this approach works extremely well for some cases and poorly for others.

A simple example would be a high-loaded, read-heavy service like the following, where an RWMutex (like in go, multiple read locks allowed, only one write lock allowed) would provide much higher throughput than an actor, simply because multiple coroutines can read the same value at the same time, and the actor isn't a bottleneck:

class a {
    private int $a;

    private RWMutex $mutex;

    public function __construct() {
        $this->mutex = new RWMutex;
    }

    // Methods can mutate state, but must acquire the mutex (higher performance than an actor)

    // Gets 1 RPS in production
    public function setA(int $a): void {
        // Only one coroutine/thread can acquire
        $lock = $this->mutex->acquireW();

        $this->a = $a;

        $lock->release(); // Or implicitly via __destruct
    }

    // Gets 1000 RPS in production
    public function getA(): int {
        // Any number of coroutines/threads can acquire, much higher performance
        $lock = $this->mutex->acquireR();

        $value = $this->a;

        $lock->release();

        return $value;
    }
}

The above is a clear example in favor of allowing a shared memory model that allows, but doesn't force usage of actors, allowing the use of higher performance alternatives based on mutexes, in some conditions.

[danog]

Then again, one could also reach the same performance of RWMutex (with added overhead only on writes) by using the actor model by making sure all possible invokers of getA are reactively sent changes of a, but we already have examples where this (full functional purity, with each function reactively receiving all and only the state they need) has gone off the rails, i.e. see useState in react.

[EdmondDantes]

This is not correct, the actor model does not automatically imply colored functions.

My apologies, that was my inaccuracy.
Colored functions are not needed.
The engine must hide implementation details from the programmer and allow writing sequential code.

[deleugpn]

Although the memory isolation idea is quite clever and brilliant as it fits very well PHP’s execution model, I’d like to bring additional attention here:

Each request is handled by a separate coroutine.
Question: how do we easily make services shared between coroutines?

this already implies that the developer has chosen to switch from PHP-FPM style to an Async-first Webserver. As such, I would argue that in this scenario we can make assumptions that there was an explicit and conscious opt-in choice made by the developer of that project. What this implies is that the developer has taken the risk of dealing with the consequences of working on an asynchronous-first environment and will need to develop his “mental checklist” knowledge.

What I want to point out with this is that here the decision about which model PHP uses is or may be more flexible and primarily focused on existing knowledge and experience of working with async applications. Even if the only option given is option 1 (share everything), it’s fine for the developer to take responsibility of making sure his/her application can cope. This means that adoption might be harder.

Now, the elephant in the room (pun intended) still remains the “hybrid” model: an application that is still running on FPM and wants to make some use of async code at an isolated / partial area of the code.

• Should this have the same memory model as the async-first one?
• If yes, does that hinder how good async-first would be?
• If no, could that become confusing for developers to know how code is executed depending on the server choice?
• Are there any approaches that maximizes developer experience for async-first while allowing FPM hybrid to coexist in a way that doesn’t make async bugs show up in very convoluted ways (the conversation about updating a package and suddenly having async code behaving differently).

To summarize my personal perception: some projects will stay sync forever, some projects will be FPM hybrid and some projects will jump in head first on async-first webserver. The first one just needs a way to have everything running sync. The 2nd one will take efforts in making sure that the async parts don’t horribly affect the sync parts and it’s manageable as viable as possible. The 3rd one is where the developer has given explicit and unambiguous signal that they want maximum Async benefits with maximum async consequences.

———————-

One additional note on memory sharing: Laravel, differently than Wordpress, has first-party thought/care/offering for async (Octane) but also makes heavy use of Service locators (Laravel Facades). A native PHP async solution would likely benefit its own adoption capabilities if it could allow Laravel Octane execution model to work. Would any/all these 3 memory options be compatible with Octane? How easy or hard would they be? Should we even care about that? (I’m assuming we should because we want TrueAsync to be adopted by as many projects as possible).

[derickr]

I believe this "working group" already starts on the wrong foot with even the definition of "Stage 1".

Its first purpose ought to be: "What is the most single basic thing for most of the PHP users."

I believe that would be a simple "Run these three things in parallel and then wait until they're done/exceptioned and get the return values".

Launching straight into the intricacies of implementation isn't a constructive way forward, IMO.

[EdmondDantes]

@bukka No. You start with "Why would we even want to have something like this? Are there any actual (not perceived) use cases?" Nobody has ever given a reasonable answer to the latter question.

The main use case for PHP is an asynchronous API service with minimal latency that efficiently distributes load to job workers. I have written about this several times on internals.

This implies an event-driven pattern that allows the load to be distributed over time and across servers.

This scenario is crucial for the commercial success of PHP as a backend language.

API and microservices are another important pattern that complements this picture.

The efficiency of the asynchronous approach has been reliably proven for many years in languages like JavaScript and Python. Note that these languages are not faster than PHP, but they achieved significant performance boosts through async and thus became strong competitors.

[NHZEX]

@bukka No. You start with "Why would we even want to have something like this? Are there any actual (not perceived) use cases?" Nobody has ever given a reasonable answer to the latter question.

Real-world scenario:
Converting 20+ EMF images to SVG using Inkscape in an admin dashboard:

• Sequential: ~8 seconds (≈300–400ms × 20)
• Concurrent (4 parallel processes): ~1–2s

Current implementation:

$processes = [];
foreach ($images as $image) {
    $process = new Process(['inkscape', /* ... */]);
    $process->start();
    $processes[] = $process;
}

foreach ($processes as $process) {
    $process->wait();
    /* ... */
}

Why this matters:

• Same pattern applies to: parallel API calls, concurrent DB queries, batch file operations
• We're already doing concurrency through workarounds (Process spawning, curl_multi, ReactPHP)
• With native async: simpler code, less overhead, better performance

The demand exists — people are just implementing it the hard way.

[ioweb-gr]

@bukka No. You start with "Why would we even want to have something like this? Are there any actual (not perceived) use cases?" Nobody has ever given a reasonable answer to the latter question.

The need is straightforward: PHP today forces parallelism to be faked through multi-process hacks. Anything that benefits from true concurrency suffers.

A simple example: exporting a product feed for 4M SKUs. Sequential execution is orders of magnitude slower than parallelizing the workload. In real-world systems (e-commerce, ERP sync, large imports/exports, web crawlers, ML preprocessing, etc.), parallelism isn't a “nice to have”—it’s the only way to finish within acceptable time.

Right now we work around PHP’s limits with:

message queues (RabbitMQ, SQS, Redis streams),

spawning multiple workers,

pcntl_fork(),

shell_exec/background jobs,

supervisor process orchestration.

All of these are simulations of concurrency, with the cost of extra processes, memory overhead, IPC complexity, and fragile orchestration.

Native async would remove most of that friction. It’s not about perceived use cases. We already do asynchronous work every day but we torture ourselves with the current way

[EdmondDantes]

The idea of collecting different use cases is very good. So I think we can put them in this separate thread and analyze them afterward:
Async Use Cases

[bukka]

I removed the Stage 1 part from this issue to make clear that it is just one of the things to discuss. I don't think we need to care about numbering of stages.

[Crell]

PHP's reputation has long been tarnished because it's so "easy" to use that PHP gets used a lot by people who don't know what they're doing, and so produce crap code that happens to be in PHP. And for a long time, PHP let them do it. Hell, it still does. 😃

Introducing an "easy" async toolset that means people who don't have solid async experience will shoot themselves in the foot is actively harmful to the language and ecosystem. We must prioritize not allowing mixed-package applications (which is nearly all of them) to develop heisenbugs because some random library did something async. Even if that makes it harder to use, even if that means it's not as powerful as it could be, that must be priority 1.

And I still hold that structured-only nursery-style is the best way to achieve that. If that means certain fringe use cases cannot be easily handled... that's an acceptable tradeoff if it protects users from sloppy other users. (Where "sloppy" means "anyone who doesn't really know how to think about async", which is 99% of PHP devs today.)

[EdmondDantes]

PHP's reputation has long been tarnished because it's so "easy" to use that PHP gets used a lot by people who don't know what they're doing, and so produce crap code that happens to be in PHP. And for a long time, PHP let them do it. Hell, it still does. 😃

This discussion is happening within a clearly defined question. Please do not deviate from it. You may create a separate thread and discuss your topic there. Thank you!

[bukka]

Introducing an "easy" async toolset that means people who don't have solid async experience will shoot themselves in the foot is actively harmful to the language and ecosystem. We must prioritize not allowing mixed-package applications (which is nearly all of them) to develop heisenbugs because some random library did something async. Even if that makes it harder to use, even if that means it's not as powerful as it could be, that must be priority 1.

And I still hold that structured-only nursery-style is the best way to achieve that.

Memory models 2 and 3 can eliminate this problem (2 to some extend and 3 completely) as well and don't require this sort colouring.

It seems quite clear to me that there are lots of disagreements in model 1 as there are various options. It's clear that colouring is not exactly popular between people that went through hell of those migration (myself included). Not doing anything raises immediate concerns about issue with globals. I'm not sure if opt in things like fiber_globals will be successful because well they are opt in so the "footguns" would still remain (which means likely rejection of RFC). So then we have this idea of partial enabling (e.g. declare or some top level blocks or whatever) that would keep the sync code blocking. This kind of suck because it won't be "true" async as the async code can get blocked. I would probably vote against it just on basis that it would make PHP look like language that came up with crappy async...

So more I think about it, I just think that model 1 is not the right choice. We can try it but I don't think we will get a good solution from it - it will be either allowing wierd bugs to happen or it will be just partial async.

I would personally prefer some sort of isolation which means going with 2 or 3 (my personal preference is 3 but not sure if it's more acceptable).

[zct919]

If a developer only knows basic fpm, then you may already be eliminated. Most PHP developers around me understand the concept of asynchrony, and swoole, workman, etc. are very popular in China

[deleugpn]

@zct919 that’s quite a bubble. Most PHP developers worldwide are not familiar with any async model.

[Tinywan]

@zct919 that’s quite a bubble. Most PHP developers worldwide are not familiar with any async model.那可真是有点脱离实际。全球大多数 PHP 开发者都不熟悉任何异步模型。
Then let PHP die!

[EdmondDantes]

Several cases regarding memory model 3.

Case 1. Getting the value of a coroutine or a Future.
Since the shared-nothing model forbids having the same object in different coroutines, you cannot obtain the result of a Future or coroutine if that rule is violated. This creates a risk of an exception when the programmer makes a mistake. Detecting such a situation before execution is difficult. PHP has no compiler and cannot verify the memory model the way Rust does.

This issue can be partially mitigated by cloning the object, but that immediately introduces additional pitfalls.

Case 2. Lazy-objects

I have a service, and this service has two dependencies provided as lazy objects.
Two different coroutines, possibly running in two different threads, try to use the service.
The service attempts to access a dependency and at that moment — boom! Undefined state.

Consequences: Model 3 forbids using lazy objects, and DI resolution becomes difficult unless it is solved in some other way.

Case 3. The problem of passing complex data structures.

I have a data structure that contains cyclic references.
Under memory model 3, PHP is required to throw an error if refCount > 1. PHP cannot verify that each object resides in a single memory domain.

This is a classic headache in Rust and not only Rust.

Case 4. Complex unexpected errors

There is code that passes object X into a coroutine.
The code works correctly. The reference count of the object is 1.

I write a new function and accidentally capture the object in some closure without noticing it.
During testing everything breaks, and I try to find the error.
But without special tools, I cannot diagnose it.

This problem would not exist if PHP were a compiled language with a memory model that could be validated at compile time.
But PHP is not such a language.

So, memory model 3, while solving one problem, creates another.

There is a way out: explicitly define two types of coroutines: coroutines that run in the same thread and coroutines that run in different threads.

[EdmondDantes]

I only meant to say that the title here says "Several cases for memory model 3.". But what you are describing here are "Several cases that explains difficulties to choose/implement memory model 3." While I was reading, I was trying to understand how these cases highlights why we should choose (go for) model 3, while in fact they're all describing complications of choosing model 3.

Exactly. The cases that describe model 3 from the advantageous side were already covered earlier. They are clear and straightforward.
Model 3 sounds very appealing in theory, which makes it dangerous if we don’t also consider the opposite cases.

[bukka]

This is just slightly incorrect English - that's all. It should be more Several cases regarding memory model 3.

[bukka]

About case 1 - it could still alow sharing object that are shareable (meaning those that could be potentially made thread safe like channels). There could be also concept that what is going to get returned will need to be clonnable / copyable and for object clonning will be used. We are considering just coroutines here so it should be relatively cheap as it will just bump reference (potentially just change ownership if all is 1)
Case 2 - again there should be a way to pass copy (clone) of the object so fiber would get its own copy of DI. So I guess it would require deep clonning. It's likely that this would be slower and more convenient.
Case 3 - I guess we are effectively talking about serialization. Essentially objects that cannot be serialized could not be passed to coroutine. It could be a bit different as there could be exception for resources like streams that could be potentially shared (possibly requiring it some changes in the future).
Case 4 - that should disappear if cloning is used.

So I think to make this work it would need to clone / copy data when passed and returned from the coroutine. This would result in slower performance but also introduces possibility for real parallelization in the future.

[roxblnfk]

Just repeat a thought from chats where we discussed the third model of memory management.

If we had readonly classes that guarantee immutability at all levels of nesting (let's call this "true readonly"), many issues with model 3 would be solved.

Case 2. Lazy-objects

The problem with lazy objects is only due to poor design of lazy objects. It was a fatal mistake to allow resetting them. However, this issue can be avoided by disallowing resetting "true readonly" objects.

This way, we stop counting refcount and allow transfering of "true readonly" objects without cloning or copying, with the guarantee that they won't be changed.

[EdmondDantes]

About case 1 - it could still alow sharing object that are shareable (meaning those that could be potentially made thread safe like channels). There could be also concept that what is going to get returned will need to be clonnable / copyable and for object clonning will be used. We are considering just coroutines here so it should be relatively cheap as it will just bump reference (potentially just change ownership if all is 1)

Yes, correct. And now the language ends up with three kinds of objects that can be shared:

• objects that can be moved
• objects that are explicitly made Shared
• readonly objects

From a language-design perspective this increases complexity, and in the code it becomes impossible to understand which kind of object was passed.

Case 2 - again there should be a way to pass copy (clone) of the object so fiber would get its own copy of DI. So I guess it would require deep clonning. It's likely that this would be slower and more convenient.

Cloning should theoretically solve this problem. Or simply avoid using Lazy objects between coroutines at all.

Case 3 - I guess we are effectively talking about serialization. Essentially objects that cannot be serialized could not be passed to coroutine. It could be a bit different as there could be exception for resources like streams that could be potentially shared (possibly requiring it some changes in the future).

This is the solution used in Erlang. Of course it works. And of course it leads to significant performance loss. However, for PHP this may not be a critical case.

Case 4 - that should disappear if cloning is used.

If automatic object cloning is meant, then coroutines will become a relatively expensive feature. On the other hand, it solves many problems.

[EdmondDantes]

This discussion is focused on the specific set of questions stated in the topic title.
I kindly ask you to avoid posting messages that are not directly related, as this makes the thread harder to read and analyze, and diverts attention from the core issues.

If you would like to share feedback, criticism, or general comments, please use a separate topic. Thank you!

[ioweb-gr]

@EdmondDantes

First of all, thank you sincerely for all the time and dedication you've put into this. I know the community can be very demanding, and working through all the feedback in such detail is both tedious and exhausting.

After going back through the threads and discussions, I realized that for developers who haven’t worked with asynchronous logic before, it’s genuinely difficult to grasp how it affects program flow, why shared globals are dangerous, how race conditions can arise, and why different memory models matter for safety and predictability.

A big source of confusion also seems to be understanding how existing synchronous code would need to be refactored under each memory isolation model. Without clear intuition for that, it's hard for people to assess the real-world impact of each option.

The recurring questions like “why do we even need this?” and calls for concrete use cases suggest that we’ll need to be extremely beginner friendly in all discussions where possible, if this is going to move forward and eventually gain adoption without friction.

Would it make sense to expand the initial post with a few very simple, high-level examples under <details> blocks that illustrate the common pitfalls of each memory model? Just conceptual, not tied to implementation details. For example:

• Why using shared globals can break behavior when multiple coroutines run,
• How writing to a file in order fails without coordination and how they should process this type of logic
• What happens when processing a global array concurrently without isolation.

These kinds of examples have come up in scattered places in the RFC proposals, but gathering them in one place could really help readers who aren’t used to async programming.

Lastly, would it be worthwhile to actively reach out to major players like WordPress, Laravel, Symfony, Adobe, Prestashop, etc., and invite them to weigh in? Although I believe they would need a strong nudge from the active php maintainers to actually invest time to it, their perspective could help shape the proposal and build confidence that the direction aligns with the wider ecosystem and will eventually be adopted

[bukka]

I think we are confusing here the actual RFC (which should have this info) and working group technical discussion that requires certain degree of knowledge to discuss some technical aspects. I think people that don't have this knowledge should not be asking for this in topics that have such requirement (like this memory model discussion). However, I think it's fine to open a new topic to discuss it there.

[ioweb-gr]

@bukka I totally agree, but there’s one practical issue I think is worth pointing out.

In the last discussions, Ed posted this thread link directly on the externals page. When people jump in from the RFC conversations, the first thing they encounter is this memory-model discussion. Since they don’t yet understand the broader context or the staging of the working group, they naturally start posting general questions here. When those posts get marked off-topic or dismissed, it creates frustration and makes the whole process feel unwelcoming, even if that’s not the intention.

Maybe a more structured entry point could help:
a single, centralized main thread that clearly links to:

• high-level / non-technical explanations,
• the technical deep-dives (like this one),
• decisions that have already been made,
• and open questions currently being worked on.

This way newcomers know where they’re supposed to start, and specialized threads like this one can stay clean without people feeling shut down. It also prevents the “ping-pong effect” where people bounce between scattered discussions without knowing what belongs where.

To give context, here is Ed’s announcement:

Ed’s working group announcement

Hello everyone.

Today I am officially announcing the start of the async working group.
Anyone may join the group without restrictions.
However, the discussion will follow a clear flow and will be divided
into several stages.

The discussion moderator will help keep the conversation on the right
track, separating secondary questions from the more important ones.
Anyone may propose a candidate for the role of moderator, and until
then I will fulfill this role.

https://github.com/true-async/php-true-async-rfc/discussions/8

Please note that this discussion is not tied to the RFC, is not a
continuation of it, and is essentially a discussion starting from
scratch. At this stage, PHP may choose any direction, as long as it
receives support.
The discussion stages will be finalized through voting and will also
be documented in the form of theses.

It is possible that some stages will need to be approved by the
broader PHP community. I am not yet sure how this should be done, but
I hope for support from the PHP Foundation.

In addition to the main discussion, anyone may create an independent
topic, which will be reviewed.
I also intend to bring this discussion to the attention of the
maintainers of PHP frameworks, because these changes are of primary
importance to them.
For RFC developers, it is important to identify the parts of the code
that will not work, or to understand how the code should be adapted so
that it functions correctly in asynchronous mode.
This is why collective discussion is so important.

For the developers of Swoole and FrankenPHP, this discussion is
equally important. Asynchronous execution makes little sense if PHP
does not have its own built-in web server. A built-in web server with
workers is the foundation for everything. Releasing Async without a
server is possible, but almost meaningless. This is why there is great
hope and attention directed toward your work.

Please share this message with anyone who may be able to help the project.

----
Best regards, Ed

And because Ed explicitly invited everyone to join this discussion about all aspects not tied to the RFC,

Please note that this discussion is not tied to the RFC, is not a
continuation of it, and is essentially a discussion starting from
scratch. At this stage, PHP may choose any direction, as long as it
receives support.

Without reading that they should start new topics about the rest of the questions, many people understandably assume this is the place to start and this becomes something hard to track.

[bukka]

I think couldn't predict that this will pop in and he thought that this is the right think to start with which is understandable. The discussion about the use cases is in #9 so please comment there.

[EdmondDantes]

Thank you for the feedback!
You’re right, the discussion disclaimer needs to be placed more prominently.

Maybe a more structured entry point could help:
a single, centralized main thread that clearly links to:

That’s an excellent idea, it definitely needs to be implemented!
I just can’t get to everything at once :)
It might be helpful if someone could take on the administrative tasks, reviewing and organizing the messages.

Current strategy:

1. First, we discuss the key questions that affect everything else.
2. Initially, the topic is discussed primarily by specialists with experience in async.
   If at this stage we can’t reach an agreement, there’s no point in moving further.
3. If we do manage to reach a shared understanding, then we proceed with research,
   creating use cases, and testing.
4. After that, we invite a broader group of developers, but by then we already have
   much more knowledge and understanding than before.
5. We hold a vote on the key questions.

Then we move down to the next level of detail.

The questions we are discussing now do not have clear answers.
We can invite the WP developers and others, but it will be difficult to explain things that we ourselves don’t yet fully understand.
Though I am not against their participation at all. Developers come in many types, and among them there may be specialists in other programming languages, which is very valuable for us.

[tw2066]

Support for no memory separation
In the current ecosystem, whether it's FPM, Swoole, or AMPHP, there is no isolation. If isolation were implemented, it would change how developers use these tools and severely impact backward compatibility (BC). It would also diminish the functionality of static features, lose their intended meaning, and increase the learning curve. It is recommended to isolate only those aspects that have issues in coroutine execution, such as static variables within methods,  $_GET ,  $_POST , and similar elements.

[EdmondDantes]

Support for no memory separation In the current ecosystem, whether it's FPM, Swoole, or AMPHP, there is no isolation. If isolation were implemented, it would change how developers use these tools and severely impact backward compatibility (BC). It would also diminish the functionality of static features, lose their intended meaning, and increase the learning curve. It is recommended to isolate only those aspects that have issues in coroutine execution, such as static variables within methods,  $_GET ,  $_POST , and similar elements.

Due to how static properties are implemented, I also tend to think it would be reasonable to limit isolation to GLOBALS and superglobals.

At the same time, I would suggest isolating superglobals not per coroutine but per separate scope, which would make the code much more flexible.
Superglobals could be used as a Coroutine/Scope context.

[roxblnfk]

In the context of true async, you can consider one FPM process as a single coroutine. Accordingly, each coroutine in true async can be seen as a separate FPM process in thread-safe mode for simplicity (though this is a very rough simplification). This is to say that there is isolation between FPM workers.

I think that if isolation were available, AMP would gladly use it too.

[EdmondDantes]

In the context of true async, you can consider one FPM process as a single coroutine.

I view this process exactly the same way at the highest abstract level. However, full isolation would not only reduce performance but also eliminate one of the key advantages of coroutines: memory savings. Reusing the same classes over and over again.

That’s why I’m increasingly leaning toward the middle option, which takes the best of both extremes.

[EdmondDantes]

First results of researching model 2

[EdmondDantes]

Due to how static properties are implemented, I also tend to think it would be reasonable to limit isolation to GLOBALS and superglobals.

At the same time, I would suggest isolating superglobals not per coroutine but per separate scope, which would make the code much more flexible. Superglobals could be used as a Coroutine/Scope context.

The next step is to put all these options up for a vote.

[bukka]

I wonder if it would be better to disallow access to static properties from all coroutines except the main one. Maybe later there could be option to make make some static properties per fiber using attribute but by default, it would just cause fatal error. It's a bit of an limitation but it seems more reasonable than sharing it.

[EdmondDantes]

I wonder if it would be better to disallow access to static properties from all coroutines except the main one. Maybe later there could be option to make make some static properties per fiber using attribute but by default, it would just cause fatal error. It's a bit of an limitation but it seems more reasonable than sharing it.

Forbid it, sure. But how should it be handled? Throw an exception? Return a default value?

The code that isolates function-level static variables will not have any significant performance impact. But the question is: why isolate only function statics?
Static properties usually aren’t part of I/O-heavy scenarios, and they can be useful across multiple coroutines while saving shared memory.

I think we need real-world application testing to understand the actual risks. That will likely happen soon.

Smart isolation of SuperGlobals and isolation of GLOBALS will push people away from using global variables, which is great. And I expect it to solve a major WordPress problem. But this still needs to be verified :)

P.S.
About coroutine specific var.
I'm currently considering adding something like $_CONTEXT or $_SCOPE as syntactic sugar over currentContext. It will probably be convenient to use, but it’s not a key feature.

[bukka]

I think it should throw. This is because it would otherwise keep the same sort of "foot gun" as was the case for globals (changing shared state from coroutine when it's not expected by the previously sync code).

[EdmondDantes]

I think it should throw. This is because it would otherwise keep the same sort of "foot gun" as was the case for globals (changing shared state from coroutine when it's not expected by the previously sync code).

There’s a high risk that this approach will make the language irritating to use.
Look what happens.

Suppose you wrote some great code and everything works.
Then someone else writes some code that writes to the wrong variable.
Everything breaks, you go to see why. PHP says: “I can’t write here because this violates such-and-such rule.” And you can’t actually fix the bug; you have to run around trying to find where exactly something was wrong.

So the exception only signals that there is a problem, but doesn’t help to solve it. And that’s not all. Suppose that in 50% of cases where static is used, no I/O is involved (and realistically it’s probably more like 90%). Yet we still get an error, even though the code is perfectly valid in a single-threaded context.

The more I think about it, the more it reminds me of Rust’s panic mode, which used to be considered a beautiful idea until it became obvious in practice that real applications can’t work that way.
Or the idea of “nursery”. At first it seemed like a great concept. Like the holy grail of safe programming. And what happened in practice? Nobody wants to use it.

That's why I’m so cautious about things that look like "elegant" solutions...

[bukka]

One thing to properly consider is separation of internal globals. If those are shared, we should stright away reject any future paralellization as extension already rely on the fact that those globals are thread safe so it would not be realisticly possible to change this expectation in the future.

If we separate those, then we would be doing pretty much the same thing that parallel does but also supporting coroutines (which means also changes in TSRM - similar ones that FrankenPHP needs for using goroutines).

I honestly think that the full separation is the way to go as it closes all footguns and would alow using threads in the future.

[EdmondDantes]

I honestly think that the full separation is the way to go as it closes all footguns and would alow using threads in the future.

How are you planning to address the problems described above?
I don’t mean only the performance issues (those might not be that critical).
I mean the inability to simply implement a counter-aggregation algorithm. For example:

<?php

final class Metrics
{
    public static int $requests = 0;
    public static int $errors   = 0;
}

function print_metrics_loop(): void
{
    while (true) {
        echo sprintf(
            "[metrics] requests=%d, errors=%d\n",
            Metrics::$requests,
            Metrics::$errors,
        );

        sleep(5); 
    }
}

function main(): void
{
spawn('print_metrics_loop');

//Calculation code

}

Even for this code, you would still need to create a special object.
In doing so, you lose the advantages of single-threaded concurrency.

[bukka]

well that's the limitation of the separation. One would need to use other ways (e.g. channels) instead.

[EdmondDantes]

well that's the limitation of the separation. One would need to use other ways (e.g. channels) instead.

Yeah. That’s why it seems to me that having an explicit distinction between SingleThread coroutine and Thread-coroutine would give more flexibility.
And for thread-coroutines you could use only model 3. Because there’s no other option there.

[krakjoe]

There seems to be a misunderstanding about how the threading model of PHP is supposed to work. Parallel doesn't do any "tricks", it implements the threading model as required. Talk of needing "sharable opcodes" and a new memory model betray misunderstanding; opcodes are already sharable, and there's no problem with the memory model. Talk of something called "true parallelism" does exactly the same thing.

I'm not certain that this is a productive discussion, talk of burdening lightweight concurrency primitives with not only that which is required by the engine, but also what you imagine is necessary on top, changes the nature of the primitive from something lightweight, into something that whatever else can no longer be called lightweight. The reason coroutines are suitable as a primitive in a reactive application is their nature - the very fact that they are lightweight - to reenter IO doesn't require a heavy context switch ...

This direction is a non-starter for me ...

[bukka]

The problem is that shared model (Model 1 in this case) seems like a difficult one to pass as it might not play nicely with the current sync code (this was expressed already by few voters). I tried to summarise it here: #8 (reply in thread) .

[EdmondDantes]

Talk of needing "sharable opcodes" and a new memory model betray misunderstanding; opcodes are already sharable, and there's no problem with the memory model. Talk of something called "true parallelism" does exactly the same thing.

Yes, I was wrong. The code truly does have separation from the threads.

talk of burdening lightweight concurrency primitives with not only that which is required by the engine, but also what you imagine is necessary on top

What does this mean? I don’t really understand what “on top” refers to.

to reenter IO doesn't require a heavy context switch ...

Unfortunately, I don’t understand what is meant here either.
What does “lightweight” mean, what does “heavy” mean?
How is this related to I/O?

Maybe you mean that context switching would become expensive?
There is no requirement in this discussion to make coroutines heavy.

[EdmondDantes]

Wordress here :)

Here you see many instances of WordPress running inside a single PHP process that never dies — well, almost never.

Each request creates its own personal WordPress instance, which sees only its own global variables and therefore works perfectly without conflicting with other requests.

The cherry on top is the SUPERGLOBALS: they are shared between multiple coroutines within a single request, forming a kind of sandbox.

No performance degradation.
Now WordPress has the ability to create shared data caches that can be accessed across different requests.

[theodorejb]

Does it improve performance?

[EdmondDantes]

Does it improve performance?

No magic involved.
A stateful application makes it possible to reuse certain objects many times across different requests.

But.

This needs to be written.
It can complicate the code.

The benefit is that now this capability exists.

[xingye00186]

Perhaps concurrency has been significantly enhanced