[RFC] Fibers

Peter Stalman

5 years ago

Hi Aaron, this is very interesting to me.  Can I ask why this approach as
opposed to other paradigms like promises, coroutines, etc?  You mentioned
async/await in the future scope, and I assume most of these patterns can be
implemented once there is an underlying functionality.  Basically, why
fibers instead of x?

You also mentioned this isn't really intended to be used directly, but with
a library such as AMPHP.  IS the expectation that non-blocking I/O
functionality like database drivers and file operation be provided by
libraries as well?

I hope I don't come off as critical, I am merely curious.  Thank you for
pushing this forward, as async is something PHP has been lacking and should
have IMO to compare favourably to other alternatives that do.

Regards, Peter.

On Thu, Dec 17, 2020 at 8:30 AM Aaron Piotrowski <aaron@trowski.com> wrote:

Aaron Piotrowski

5 years ago

Hi Peter,

> On Dec 17, 2020, at 1:23 PM, Peter Stalman <sarkedev@gmail.com> wrote:
>
> Hi Aaron, this is very interesting to me.  Can I ask why this approach as
> opposed to other paradigms like promises, coroutines, etc?  You mentioned
> async/await in the future scope, and I assume most of these patterns can be
> implemented once there is an underlying functionality.  Basically, why
> fibers instead of x?

Promises result in the “What color is your function” problem as described in the introduction of the RFC. Returning promises from functions means that functions calling those functions must also return promises, resulting in the entire call stack needing to return promises.

Fibers are a method of implementing coroutines or interruptible functions. Promises likely would still be used as placeholders in libraries using fibers, but coroutines written using fibers do not have to return another placeholder. Fibers allow async code to be indistinguishable from sync code, as opposed to an approach where async functions must return a promise.

>
> You also mentioned this isn't really intended to be used directly, but with
> a library such as AMPHP.  IS the expectation that non-blocking I/O
> functionality like database drivers and file operation be provided by
> libraries as well?
>

Since most code written for PHP is blocking, yes, such libraries/frameworks would need to provide functionality such as database drivers. Both AMPHP and ReactPHP already have existing async drivers available for several different popular database systems. AMPHP’s postgres, mysql, and redis drivers already have a version using fibers.

> I hope I don't come off as critical, I am merely curious.  Thank you for
> pushing this forward, as async is something PHP has been lacking and should
> have IMO to compare favourably to other alternatives that do.
>

You didn’t com off as critical at all! These were good questions to ask. I too think if PHP is to add support for async code it should compare favorably to other languages. I think fibers offer a distinct advantage to using promise for async code.

Cheers,
Aaron Piotrowski

azjezz

5 years ago

Hello Aaron,

First, I want to say that I love this proposal and would love to see it land in the next PHP release, but I have one question regarding this:

> Promises result in the “What color is your function” problem as described in the introduction of the RFC. Returning promises from functions means that functions calling those functions must also return promises, resulting in the entire call stack needing to return promises.

Hack-Lang provides `HH\Asio\join` function which allows awaiting Awaitables in sync code, so you are capable of running multiple async tasks concurrently without having to declare the entire call stack as "async" or with an "Awaitable" return type, isn't this possible?

```
use namespace HH\Asio;

async function async_task(): Awaitable<void> {
await Asio\usleep(1000000);
}

<<__EntryPoint>>
function main(): void {
$start = microtime(true);

$async = async {
concurrent {
await async_task();
await async_task();
};

return 'hello';
};

$result = Asio\join($async);

printf('Result: %s ( %f )', $result, microtime(true) - $start); // output "Result: hello ( 1.010382 )"
}

```

Regards,

Saif.


‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Thursday, December 17, 2020 8:43 PM, Aaron Piotrowski <aaron@trowski.com> wrote:

Aaron Piotrowski

5 years ago

> On Dec 17, 2020, at 4:11 PM, Saif Eddin Gmati <azjezz@protonmail.com> wrote:
>
> Hello Aaron,
>
> First, I want to say that I love this proposal and would love to see it land in the next PHP release, but I have one question regarding this:
>
>
>> Promises result in the “What color is your function” problem as described in the introduction of the RFC. Returning promises from functions means that functions calling those functions must also return promises, resulting in the entire call stack needing to return promises.
>
> Hack-Lang provides `HH\Asio\join` function which allows awaiting Awaitables in sync code, so you are capable of running multiple async tasks concurrently without having to declare the entire call stack as "async" or with an "Awaitable" return type, isn't this possible?
>
> ```
> use namespace HH\Asio;
>
> async function async_task(): Awaitable<void> {
>  await Asio\usleep(1000000);
> }
>
> <<__EntryPoint>>
> function main(): void {
>  $start = microtime(true);
>
>  $async = async {
>    concurrent {
>      await async_task();
>      await async_task();
>    };
>
>    return 'hello';
>  };
>
>  $result = Asio\join($async);
>
>  printf('Result: %s ( %f )', $result, microtime(true) - $start); // output "Result: hello ( 1.010382 )"
> }
>
> ```
>
> Regards,
>
> Saif.
>

Hi Saif,

`HH\Asio\join()` implements a synchronous await (I don't know the details of how its implemented, possibly involving entering and exiting the built-in event loop), but it does not solve the problem that functions using `await` need to be declared using `async` and return an Awaitable. Your example declares `async_task()` as async, while a similar function using the proposed fiber API would not need to change the function declaration to use `Fiber::suspend()`. There's an example in the RFC using `Amp\delay()` that is very similar to your code sample.

Fibers allow existing interfaces to be implemented using either sync or async I/O because the interface does not need to change to return promises/awaitables.

Cheers,
Aaron Piotrowski

Mike Schinkel

5 years ago

> On Dec 17, 2020, at 11:30 AM, Aaron Piotrowski <aaron@trowski.com> wrote:
>
> Hello everyone!
>
> I would like to introduce an RFC for adding full-stack fibers to PHP: https://wiki.php.net/rfc/fibers
>
> Fibers are primarily used to implement green-threads or coroutines for asynchronous I/O. Fibers are similar to threads, except fibers exist within a single thread and require cooperative scheduling of the fibers by the process. Since fibers do not require a full CPU context switch, they are lightweight and more performant than multi-processing or threading for awaiting I/O.
>
> An implementation as an extension is at https://github.com/amphp/ext-fiber
>
> Fibers are a complex feature. The RFC contains many examples and links to code using fibers to help explain and demonstrate what is possible, however I’m certain many more questions and concerns will arise. Looking forward to feedback and discussion.
>

This is interesting, and potentially very useful.

I am curious about how you propose access to shared memory across fibers?  What will happen if two fibers try to update a $GLOBALS variable at the same time?  Or a property of the same object?  How will developers manage that?

-Mike

P.S. Have you considered concurrency functionality like in GoLang[1] e.g. channels, where the mantra is "Do not communicate by sharing memory; instead, share memory by communicating?"

[1] https://medium.com/@thejasbabu/concurrency-in-go-e4a61ec96491#:~:text=Do%20not%20communicate%20by%20sharing,race%20conditions%2C%20memory%20management%20etc.

Aaron Piotrowski

5 years ago

> On Dec 21, 2020, at 4:33 PM, Mike Schinkel <mike@newclarity.net> wrote:
>
>> On Dec 17, 2020, at 11:30 AM, Aaron Piotrowski <aaron@trowski.com> wrote:
>>
>> Hello everyone!
>>
>> I would like to introduce an RFC for adding full-stack fibers to PHP: https://wiki.php.net/rfc/fibers
>>
>> Fibers are primarily used to implement green-threads or coroutines for asynchronous I/O. Fibers are similar to threads, except fibers exist within a single thread and require cooperative scheduling of the fibers by the process. Since fibers do not require a full CPU context switch, they are lightweight and more performant than multi-processing or threading for awaiting I/O.
>>
>> An implementation as an extension is at https://github.com/amphp/ext-fiber
>>
>> Fibers are a complex feature. The RFC contains many examples and links to code using fibers to help explain and demonstrate what is possible, however I’m certain many more questions and concerns will arise. Looking forward to feedback and discussion.
>>
>
> This is interesting, and potentially very useful.
>
> I am curious about how you propose access to shared memory across fibers?  What will happen if two fibers try to update a $GLOBALS variable at the same time?  Or a property of the same object?  How will developers manage that?
>
> -Mike
>
> P.S. Have you considered concurrency functionality like in GoLang[1] e.g. channels, where the mantra is "Do not communicate by sharing memory; instead, share memory by communicating?"
>
> [1] https://medium.com/@thejasbabu/concurrency-in-go-e4a61ec96491#:~:text=Do%20not%20communicate%20by%20sharing,race%20conditions%2C%20memory%20management%20etc <https://medium.com/@thejasbabu/concurrency-in-go-e4a61ec96491#:~:text=Do%20not%20communicate%20by%20sharing,race%20conditions%2C%20memory%20management%20etc>.

Hi Mike,

Fibers do not change the single-threaded nature of PHP. Only a single fiber can be running at one time, so memory cannot be modified simultaneously.

There are synchronization issues when writing asynchronous code using either stackless or stackful coroutines, as anyone who has worked with AMPHP or ReactPHP can tell you. Multiple fibers (coroutines, green-threads, whatever you want to call them) will interleave execution. Multiple interleaved fibers can change object state between pausing and resuming, which I'm guessing is more to what you were concerned about, rather than literal simultaneous modification that can occur with threads. The RFC does not provide tools to synchronize memory access, as these can be implemented in user space. Fibers don't provide the entire API, just the "bare-metal" API needed to implement various styles of concurrency in user space code.

AMPHP provides synchronization tools such as mutexes, semaphores, parcels, and channels in https://github.com/amphp/sync <https://github.com/amphp/sync> and https://github.com/amphp/parallel <https://github.com/amphp/parallel>. Other libraries could provide similar tools, though perhaps with a different approach. We too like the Go-style of concurrency and look to emulate it in AMPHP v3.

Cheers,
Aaron Piotrowski

Mike Schinkel

5 years ago

> On Dec 21, 2020, at 6:53 PM, Aaron Piotrowski <aaron@trowski.com> wrote:
>
>> On Dec 21, 2020, at 4:33 PM, Mike Schinkel <mike@newclarity.net <mailto:mike@newclarity.net>> wrote:
>>
>>> On Dec 17, 2020, at 11:30 AM, Aaron Piotrowski <aaron@trowski.com <mailto:aaron@trowski.com>> wrote:
>>>
>>> Hello everyone!
>>>
>>> I would like to introduce an RFC for adding full-stack fibers to PHP: https://wiki.php.net/rfc/fibers <https://wiki.php.net/rfc/fibers>
>>>
>>> Fibers are primarily used to implement green-threads or coroutines for asynchronous I/O. Fibers are similar to threads, except fibers exist within a single thread and require cooperative scheduling of the fibers by the process. Since fibers do not require a full CPU context switch, they are lightweight and more performant than multi-processing or threading for awaiting I/O.
>>>
>>> An implementation as an extension is at https://github.com/amphp/ext-fiber <https://github.com/amphp/ext-fiber>
>>>
>>> Fibers are a complex feature. The RFC contains many examples and links to code using fibers to help explain and demonstrate what is possible, however I’m certain many more questions and concerns will arise. Looking forward to feedback and discussion.
>>>
>>
>> This is interesting, and potentially very useful.
>>
>> I am curious about how you propose access to shared memory across fibers?  What will happen if two fibers try to update a $GLOBALS variable at the same time?  Or a property of the same object?  How will developers manage that?
>>
>> -Mike
>>
>> P.S. Have you considered concurrency functionality like in GoLang[1] e.g. channels, where the mantra is "Do not communicate by sharing memory; instead, share memory by communicating?"
>>
>> [1] https://medium.com/@thejasbabu/concurrency-in-go-e4a61ec96491#:~:text=Do%20not%20communicate%20by%20sharing,race%20conditions%2C%20memory%20management%20etc <https://medium.com/@thejasbabu/concurrency-in-go-e4a61ec96491#:~:text=Do not communicate by sharing,race conditions, memory management etc>.
>
> Hi Mike,

Thanks for the reply.

> Fibers do not change the single-threaded nature of PHP. Only a single fiber can be running at one time, so memory cannot be modified simultaneously. There are synchronization issues when writing asynchronous code using either stackless or stackful coroutines, as anyone who has worked with AMPHP or ReactPHP can tell you. Multiple fibers (coroutines, green-threads, whatever you want to call them) will interleave execution.

So if I am to understand correctly, one fiber could block all others, so fiber code will need to be well-behaved and not block much like how Node code must not block when used as a web server?  If I understand correctly, no need to reply about this.

> Multiple interleaved fibers can change object state between pausing and resuming, which I'm guessing is more to what you were concerned about, rather than literal simultaneous modification that can occur with threads.

Correct.

> The RFC does not provide tools to synchronize memory access, as these can be implemented in user space.

Would it be appropriate of me to ask for a section that discusses how that might be done in user space in the RFC, at least with some simple pseudo-code, or if is it non-trivial than a link to where it is discussed in depth?

> Fibers don't provide the entire API, just the "bare-metal" API needed to implement various styles of concurrency in user space code.
>
> AMPHP provides synchronization tools such as mutexes, semaphores, parcels, and channels in https://github.com/amphp/sync <https://github.com/amphp/sync> and https://github.com/amphp/parallel <https://github.com/amphp/parallel>. Other libraries could provide similar tools, though perhaps with a different approach. We too like the Go-style of concurrency and look to emulate it in AMPHP v3.

Thanks in advance.

-Mike

Aaron Piotrowski

5 years ago

Hi Mike,

> On Dec 21, 2020, at 7:38 PM, Mike Schinkel <mike@newclarity.net> wrote:
>
> Would it be appropriate of me to ask for a section that discusses how that might be done in user space in the RFC, at least with some simple pseudo-code, or if is it non-trivial than a link to where it is discussed in depth?

Absolutely! I added a short example implementation of a mutex to the RFC under https://wiki.php.net/rfc/fibers#how_do_various_fibers_access_the_same_memory. The code uses a simple queue of fibers waiting to access the mutex to acquire and release the lock.

A channel might be implemented with a queue of messages where the receiving fiber suspends if the queue is empty. Sending a message on the channel would resume a suspended fiber that was waiting to receive a message.

Hopefully that helps. If something is still unclear or an additional example would help, don't hesitate to ask.

Cheers!
Aaron Piotrowski

tyson andre

5 years ago

Hi Mike Schinkel,

> This is interesting, and potentially very useful.
> 
> I am curious about how you propose access to shared memory across fibers?  What will happen if two fibers try to update a $GLOBALS variable at the same time?  Or a property of the same object?  How will developers manage that?

(I started writing this before Aaron sent a different response)

[Concurrency is not parallelism](text of https://blog.golang.org/waza-talk , not the video),
and this RFC isn't adding parallelism to php.
Instead, the RFC is a proposal to make concurrency easier.
The rfc announcement email says that "fibers exist within a single thread".
A fiber would have to explicitly release control to the scheduler after modifying a $GLOBALS variable, etc.

As I understand it, this proposal is similar to how concurrency works in node for code written in JS - https://stackoverflow.com/questions/29977049/how-does-concurrency-work-in-nodejs
(in that there's only one thread, but it's different in that scheduling is done manually)

Thanks,
- Tyson

Peter Stalman

5 years ago

Hi again Aaron,

I've been playing around with this for a bit, and I have some more
questions. I see you recently changed how the `FiberScheduler` works,
making it `final` and adding some "is" functions to match it more to a
regular fiber.

Since the `FiberScheduler` is itself also a fiber, why does it need to be a
separate class?  Why did you choose to go this way and make it "special"
instead of not just making a regular fiber into a scheduler in userland.
Could you have two or more schedulers?  Could you get by without a
scheduler and call a fiber inside a fiber recursively?

Related to that, I noticed it was not possible to call `Fiber::this()` from
within the scheduler, why is that?  Is it not just another fiber?  Or is
this to prevent it from being passed to another scheduler?

Alternatively, going the other way, instead of making the scheduler a
unique class that needs to be passed around, why not go the more
"traditional" PHP route and pattern it after
`register_shutdown_function(callable $callback, mixed ...$args) : void`,
`spl_autoload_register(callable $autoload_function = ?, bool $throw = true,
bool $prepend = false) : bool`, and those type of functions?  After all,
isn't it just a callback too?  Something like
`register_fiber_scheduler(callable $callback) : void`?

This would remove the need for a special scheduler class and the need for
passing the scheduler back to the `Fiber::suspend()`.  Each `suspend()`
call would bubble up through the registered scheduler callbacks.  This
would allow competing schedulers to work nicer together, instead of one
scheduler having to finish before the higher up scheduler can run it's next
loop.

Either way, doesn't the fiber already know which scheduler it is in when it
suspends?

I think this would go along with simplifying it and keep the implementation
broad to allow for various userland implementations (as mentioned, such as
amphp and reactphp).  But it probably doesn't simplify the C
implementation...

Anyways, something like this?


$fiber = Fiber::this();
$callbacks = [
fn() => $fiber->resume("Test"),
];

register_fiber_scheduler(function() use ($callbacks) {
foreach ($callbacks as $callback) {
$callback();
}
});

$value = Fiber::suspend();

echo "After resuming main fiber: ", $value, "\n"; // Output: After
resuming main fiber: Test


Btw, this is just me vomiting my thoughts, I don't know enough about fibers
to design it one way or the other, but I hope to understand it more as well
as give a few different perspectives on it.

Thanks,
Peter



On Thu, Dec 17, 2020 at 8:30 AM Aaron Piotrowski <aaron@trowski.com> wrote:

Aaron Piotrowski

5 years ago

Hi Peter,

> On Jan 8, 2021, at 3:03 AM, Peter Stalman <sarkedev@gmail.com> wrote:
>
> I've been playing around with this for a bit, and I have some more questions. I see you recently changed how the `FiberScheduler` works, making it `final` and adding some "is" functions to match it more to a regular fiber.
>
> Since the `FiberScheduler` is itself also a fiber, why does it need to be a separate class?  Why did you choose to go this way and make it "special" instead of not just making a regular fiber into a scheduler in userland.

`FiberScheduler` is “special” because user code is not in control of suspending/resuming the underlying fiber, so `FiberScheduler` is a different class from `Fiber` without those methods. Internally a user fiber and scheduler fiber are similar, but I wanted to differentiate them in user code.

`FiberScheduler` recently changed from an interface to a class because we discovered that automatically creating a scheduler fiber internally would make it difficult for adaptors or wrappers of a `FiberScheduler` to not create multiple internal scheduler fibers when there should only be a single scheduler fiber. The API largely works the same, but now the creation of the scheduler fiber is explicit in user code instead of done internally. This way adaptors or wrappers can return a single `FiberScheduler` instance. More code is required in user libraries, but offers greater flexibility.

> Could you have two or more schedulers?

You can have two or more schedulers in a single script. Only one can ever be running at a single time.

> Could you get by without a scheduler and call a fiber inside a fiber recursively?

This design requires a scheduler fiber to be entered between suspending one user fiber and resume another user fiber. User fibers are designed to be independent. If two users fibers need to communicate, they should use something similar to Go’s channels to exchange data.

>
> Related to that, I noticed it was not possible to call `Fiber::this()` from within the scheduler, why is that?  Is it not just another fiber?  Or is this to prevent it from being passed to another scheduler?

A scheduler fiber cannot be suspended or resumed by user code so it is not useful to get a reference to that fiber.

>
> Alternatively, going the other way, instead of making the scheduler a unique class that needs to be passed around, why not go the more "traditional" PHP route and pattern it after `register_shutdown_function(callable $callback, mixed ...$args) : void`, `spl_autoload_register(callable $autoload_function = ?, bool $throw = true, bool $prepend = false) : bool`, and those type of functions?  After all, isn't it just a callback too?  Something like `register_fiber_scheduler(callable $callback) : void`?
>
> This would remove the need for a special scheduler class and the need for passing the scheduler back to the `Fiber::suspend()`.  Each `suspend()` call would bubble up through the registered scheduler callbacks.  This would allow competing schedulers to work nicer together, instead of one scheduler having to finish before the higher up scheduler can run it's next loop.

The scheduler to be entered is specific to the code calling `Fiber::suspend()`. Registering a global scheduler would require only a single scheduler to be used in a script. Only a single scheduler is entered on a call to `Fiber::suspend()`, not multiple schedulers. Registering schedulers or wrapping application code in boilerplate depending on the library being used is something this API is attempting to avoid.

>
> Either way, doesn't the fiber already know which scheduler it is in when it suspends?

No, a fiber can potentially use different schedulers at different suspend points. The scheduler that starts a fiber does not necessarily need to be the only scheduler that suspends the fiber.

Hopefully that helps in understanding how the API works. Please take a look at how amphp v3 uses fibers in these examples https://github.com/amphp/amp/tree/v3/examples/pipeline or in react-fiber https://github.com/trowski/react-fiber/tree/master/examples where the Fiber API is handled by the library rather than “application” code.

Cheers,
Aaron Piotrowski

Benjamin Eberlei

5 years ago

On Thu, Dec 17, 2020 at 5:30 PM Aaron Piotrowski <aaron@trowski.com> wrote:

> Hello everyone!
>
> I would like to introduce an RFC for adding full-stack fibers to PHP:
> https://wiki.php.net/rfc/fibers
>
> Fibers are primarily used to implement green-threads or coroutines for
> asynchronous I/O. Fibers are similar to threads, except fibers exist within
> a single thread and require cooperative scheduling of the fibers by the
> process. Since fibers do not require a full CPU context switch, they are
> lightweight and more performant than multi-processing or threading for
> awaiting I/O.
>
> An implementation as an extension is at https://github.com/amphp/ext-fiber
>
> Fibers are a complex feature. The RFC contains many examples and links to
> code using fibers to help explain and demonstrate what is possible, however
> I’m certain many more questions and concerns will arise. Looking forward to
> feedback and discussion.
>
> Aaron Piotrowski
>

Hi Aaron,

this is a very interesting and welcome piece of functionality. I have gone
through the RFC a few times now, it have never thought about or worked with
fibers before, additional feedback will be forthcoming once I grasp the
details more.

From my POV the effects on other extensions are the most important factor,
you already have a section with that for Xdebug, Parallel, and pcov. But
how does this affect Profilers that manage their own stack of frames,
either all function calls or specifically selected ones. I.e. xhprof,
tideways, datadog, newrelic and so on.

At the moment any PHP Profiler only has to manage a single stack of "open
frames". With Fibers for each fiber a new stack must be opened.

Does an extension know what the "active" fiber is so that  from a
zend_execute_data I know on which stack of open frames I need to push a new
frame?

Could you add a section to the RFC explaining how a switch between fibers
works in terms of "the next" and "previous" zend_execute_data that is run?
This  would also be interesting to understand how stack traces work, for
example debug_print_backtrace.

Aaron Piotrowski

5 years ago

> On Jan 18, 2021, at 8:59 AM, Benjamin Eberlei <kontakt@beberlei.de> wrote:
>
> Hi Aaron,
>
> this is a very interesting and welcome piece of functionality. I have gone
> through the RFC a few times now, it have never thought about or worked with
> fibers before, additional feedback will be forthcoming once I grasp the
> details more.
>
> From my POV the effects on other extensions are the most important factor,
> you already have a section with that for Xdebug, Parallel, and pcov. But
> how does this affect Profilers that manage their own stack of frames,
> either all function calls or specifically selected ones. I.e. xhprof,
> tideways, datadog, newrelic and so on.

Hi Benjamin,

Sorry for the bit of a delay in replying. I’ve been busy the last few days.

Profilers that manage their own stack of frames will have to be modified to account for fibers. The extension currently provides an API to access the current fiber and uniquely identify each different fiber.

The internal API for this will need further discussion amongst the rest of the internals contributors and hopefully the authors of those extensions. I omitted any API for this from the RFC as it does not affect user code.

>
> At the moment any PHP Profiler only has to manage a single stack of "open
> frames". With Fibers for each fiber a new stack must be opened.
>
> Does an extension know what the "active" fiber is so that  from a
> zend_execute_data I know on which stack of open frames I need to push a new
> frame?

Four functions are currently provided for determining the current executing fiber.

```
zend_fiber *zend_get_root_fiber()
zend_fiber *zend_get_current_fiber()
zend_long zend_fiber_get_id(zend_fiber *fiber)
zend_long zend_fiber_get_current_id()
```

These allow you to get the root and current fiber as well as the ID associated with the fiber.
The fiber ID is unique to the process and is never reused, so it can be used to determine which open stack frame to push a frame.

>
> Could you add a section to the RFC explaining how a switch between fibers
> works in terms of "the next" and "previous" zend_execute_data that is run?
> This  would also be interesting to understand how stack traces work, for
> example debug_print_backtrace.
>

I added a brief section under FAQs entitled "How are execution stacks swapped?”, https://wiki.php.net/rfc/fibers#how_are_execution_stacks_swapped

Backtraces are currently limited to including only the currently executing fiber. It may be possible to include backtraces of fibers further down the execution stack, but I had issues when trying to implement this in the extension during shutdown due to stacks being freed. Something I think can be addressed if adding to core with the help of someone more familiar with how references to these stacks are kept and what should be modified in the functions generating backtraces.

Cheers,
Aaron Piotrowski

Unnamed Person

5 years ago

On Sat, Jan 23, 2021 at 12:11 PM Aaron Piotrowski <aaron@trowski.com> wrote:

>
>
> > On Jan 18, 2021, at 8:59 AM, Benjamin Eberlei <kontakt@beberlei.de> wrote:
> >
> > Hi Aaron,
> >
> > this is a very interesting and welcome piece of functionality. I have gone
> > through the RFC a few times now, it have never thought about or worked with
> > fibers before, additional feedback will be forthcoming once I grasp the
> > details more.
> >
> > From my POV the effects on other extensions are the most important factor,
> > you already have a section with that for Xdebug, Parallel, and pcov. But
> > how does this affect Profilers that manage their own stack of frames,
> > either all function calls or specifically selected ones. I.e. xhprof,
> > tideways, datadog, newrelic and so on.
>
> Hi Benjamin,
>
> Sorry for the bit of a delay in replying. I’ve been busy the last few days.
>
> Profilers that manage their own stack of frames will have to be modified to account for fibers. The extension currently provides an API to access the current fiber and uniquely identify each different fiber.
>
> The internal API for this will need further discussion amongst the rest of the internals contributors and hopefully the authors of those extensions. I omitted any API for this from the RFC as it does not affect user code.
>
> >
> > At the moment any PHP Profiler only has to manage a single stack of "open
> > frames". With Fibers for each fiber a new stack must be opened.
> >
> > Does an extension know what the "active" fiber is so that  from a
> > zend_execute_data I know on which stack of open frames I need to push a new
> > frame?
>
> Four functions are currently provided for determining the current executing fiber.
>
> ```
> zend_fiber *zend_get_root_fiber()
> zend_fiber *zend_get_current_fiber()
> zend_long zend_fiber_get_id(zend_fiber *fiber)
> zend_long zend_fiber_get_current_id()
> ```
>
> These allow you to get the root and current fiber as well as the ID associated with the fiber.
> The fiber ID is unique to the process and is never reused, so it can be used to determine which open stack frame to push a frame.

I think it would probably be advantageous to have an observer that
alerts interested parties when the fiber switches. This way we can
avoid querying the current fiber on every fcall.

Aaron Piotrowski

5 years ago

> On Jan 31, 2021, at 11:29 AM, Levi Morrison via internals <internals@lists.php.net> wrote:
>
> I think it would probably be advantageous to have an observer that
> alerts interested parties when the fiber switches. This way we can
> avoid querying the current fiber on every fcall.
>

Hi Levi,

Yes, I agree and will look at implementing an observer API.

Cheers,
Aaron Piotrowski

Aaron Piotrowski

5 years ago

> On Feb 3, 2021, at 11:32 AM, Aaron Piotrowski <aaron@trowski.com> wrote:
>
>
>> On Jan 31, 2021, at 11:29 AM, Levi Morrison via internals <internals@lists.php.net> wrote:
>>
>> I think it would probably be advantageous to have an observer that
>> alerts interested parties when the fiber switches. This way we can
>> avoid querying the current fiber on every fcall.
>>
>
> Hi Levi,
>
> Yes, I agree and will look at implementing an observer API.
>
> Cheers,
> Aaron Piotrowski
>
>> --
>> PHP Internals - PHP Runtime Development Mailing List
>> To unsubscribe, visit: https://www.php.net/unsub.php <https://www.php.net/unsub.php>

Hi Levi,

I implemented an observer that notifies registered handlers whenever the current fiber context is switched.

typedef void (*zend_observer_fiber_switch_handler)(zend_fiber *from, zend_fiber *to);
PHP_FIBER_API void zend_observer_fiber_switch_register(zend_observer_fiber_switch_handler handler);

This works similar to the error observer API.

zend_fiber provides access to a unique ID, zend_execute_data, etc. that should be sufficient for code profilers, debuggers, etc. One pointer is NULL if {main} is the from or to fiber. The handlers are invoked before switching to the fiber and after suspending from a fiber.

Let me know if there’s something missing or improvements that could be made.

Cheers,
Aaron Piotrowski

tyson andre

5 years ago

Hi Aaron Piotrowski,

> I would like to introduce an RFC for adding full-stack fibers to PHP: https://wiki.php.net/rfc/fibers
>
> Fibers are primarily used to implement green-threads or coroutines for asynchronous I/O. Fibers are similar to threads, except fibers exist within a single thread and require cooperative scheduling of the fibers by the process. Since fibers do not require a full CPU context switch, they are lightweight and more performant than multi-processing or threading for awaiting I/O.
>
> An implementation as an extension is at https://github.com/amphp/ext-fiber
>
> Fibers are a complex feature. The RFC contains many examples and links to code using fibers to help explain and demonstrate what is possible, however I’m certain many more questions and concerns will arise. Looking forward to feedback and discussion.

I took a look at the amphp/ext-fiber repo a few weeks ago - nothing stood out as a major concern but I'm only moderately familiar with threading,
A minor comment is that it'd be easier to read the phpt test cases if the `--EXPECTF--`
section included the file's basename in the test output of error traces instead of just `%s`, etc.
(e.g. tests/002-throw.phpt)

```
Stack trace:
#0 %s(%d): {closure}()
#1 %s(%d): Loop->tick()
#2 %s(%d): Loop->run()
...
```

I didn't see in the test cases/rfc:
How do the `zend_try`/`zend_catch`/`zend_end_try` macros and `zend_error_noreturn` macros get handled after a fatal error?
I'm not 100% sure - I think those use setjmp/longjmp internally - will fatal errors continue to work if there's a fatal error from within a fiber.
(e.g. `require_once` on a file with a compile-time fatal error such as duplicate parameters).
I forget exactly how they work, e.g. in the context of a web server before this RFC - at a glance I'd guess an unrecoverable fatal error would cause the worker to shut down.
Would being in a different fiber and different C stack interfere with the shutdown process for fatal errors? (I guess the macros could be changed to make that switch to the main fiber if needed to fix that)

Thanks,
-Tyson Andre

twosee

5 years ago

> 2021年2月2日 上午11:04，tyson andre <tysonandre775@hotmail.com> 写道：
>
> Hi Aaron Piotrowski,
>
>> I would like to introduce an RFC for adding full-stack fibers to PHP: https://wiki.php.net/rfc/fibers
>>
>> Fibers are primarily used to implement green-threads or coroutines for asynchronous I/O. Fibers are similar to threads, except fibers exist within a single thread and require cooperative scheduling of the fibers by the process. Since fibers do not require a full CPU context switch, they are lightweight and more performant than multi-processing or threading for awaiting I/O.
>>
>> An implementation as an extension is at https://github.com/amphp/ext-fiber
>>
>> Fibers are a complex feature. The RFC contains many examples and links to code using fibers to help explain and demonstrate what is possible, however I’m certain many more questions and concerns will arise. Looking forward to feedback and discussion.
>
> I took a look at the amphp/ext-fiber repo a few weeks ago - nothing stood out as a major concern but I'm only moderately familiar with threading,
> A minor comment is that it'd be easier to read the phpt test cases if the `--EXPECTF--`
> section included the file's basename in the test output of error traces instead of just `%s`, etc.
> (e.g. tests/002-throw.phpt)
>
> ```
> Stack trace:
> #0 %s(%d): {closure}()
> #1 %s(%d): Loop->tick()
> #2 %s(%d): Loop->run()
> ...
> ```
>
> I didn't see in the test cases/rfc:
> How do the `zend_try`/`zend_catch`/`zend_end_try` macros and `zend_error_noreturn` macros get handled after a fatal error?
> I'm not 100% sure - I think those use setjmp/longjmp internally - will fatal errors continue to work if there's a fatal error from within a fiber.
> (e.g. `require_once` on a file with a compile-time fatal error such as duplicate parameters).
> I forget exactly how they work, e.g. in the context of a web server before this RFC - at a glance I'd guess an unrecoverable fatal error would cause the worker to shut down.
> Would being in a different fiber and different C stack interfere with the shutdown process for fatal errors? (I guess the macros could be changed to make that switch to the main fiber if needed to fix that)
>
> Thanks,
> -Tyson Andre
>
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: https://www.php.net/unsub.php
>
>

Hi Tyson Andre,

We have tried two different solutions in Swoole to solve this issue.

Since each coroutine has an independent C stack and the C stack location saved by bailout is illegal in other coroutines, we have to redundantly do zend_try and zend_catch in each coroutine. When a fatal error occurs, jump to the C stack corresponding to main(), and then exit through the normal process of PHP. However, this method is not a hundred percent safe. It is difficult for Swoole to recycle other surviving coroutines. In extreme cases, some memory problems may still occur.

I prefer the other one - Due to the great efforts made by PHP8, the fatal errors in PHP have been greatly reduced and replaced by exception mechanisms. Therefore, when fatal errors occur in an extremely low possibility, directly exit through exit() may be the safest way. Of course, this will cause register_shutdown_function to fail.

Regards,
Twosee

Nikita Popov

5 years ago

On Thu, Dec 17, 2020 at 5:30 PM Aaron Piotrowski <aaron@trowski.com> wrote:

> Hello everyone!
>
> I would like to introduce an RFC for adding full-stack fibers to PHP:
> https://wiki.php.net/rfc/fibers
>
> Fibers are primarily used to implement green-threads or coroutines for
> asynchronous I/O. Fibers are similar to threads, except fibers exist within
> a single thread and require cooperative scheduling of the fibers by the
> process. Since fibers do not require a full CPU context switch, they are
> lightweight and more performant than multi-processing or threading for
> awaiting I/O.
>
> An implementation as an extension is at https://github.com/amphp/ext-fiber
>
> Fibers are a complex feature. The RFC contains many examples and links to
> code using fibers to help explain and demonstrate what is possible, however
> I’m certain many more questions and concerns will arise. Looking forward to
> feedback and discussion.
>

Hi Aaron,

Thank you for the proposal. Ergonomics of async I/O in PHP certainly leave
something to be desired right now, and improvements in this area are
welcome.

Despite your explanations in the RFC and this thread, I'm still having a
hard time understanding the purpose of the FiberScheduler.

My current understanding is that the FiberScheduler is a special type of
fiber that cannot be explicitly scheduled by the user -- it is
automatically scheduled by Fiber::suspend() and automatically un-scheduled
by Fiber::resume() or Fiber::throw(). It's the fiber that runs between
fibers :) Does that sound accurate?

What's not clear to me is why the scheduling fiber needs to be
distinguished from other fibers. If we want to stick with the general
approach, why is Fiber::suspend($scheduler) not Fiber::transferTo($fiber),
where $fiber would be the fiber serving as scheduler (but otherwise a
normal Fiber)? I would expect that context-switching between arbitrary
fibers would be both most expressive, and make for the smallest interface.

The more limited alternative is to instead have Fiber::suspend() return to
the parent fiber (the one that resume()d it). Here, the parent fiber
effectively becomes the scheduler fiber. If I understand right, the reason
why you don't want to use that approach, is that it doesn't allow you to
call some AMP function from the {main} fiber, create the scheduler there
and then treat {main} just like any other fiber. Is that correct?

Regards,
Nikita

Niklas Keller

5 years ago

Hey Nikita,

Thank you for the proposal. Ergonomics of async I/O in PHP certainly leave

> something to be desired right now, and improvements in this area are
> welcome.
>
> Despite your explanations in the RFC and this thread, I'm still having a
> hard time understanding the purpose of the FiberScheduler.
>
> My current understanding is that the FiberScheduler is a special type of
> fiber that cannot be explicitly scheduled by the user -- it is
> automatically scheduled by Fiber::suspend() and automatically un-scheduled
> by Fiber::resume() or Fiber::throw(). It's the fiber that runs between
> fibers :) Does that sound accurate?
>

Yes, that's accurate. Fibers are used for cooperative multi-tasking and
there's usually a single scheduler responsible for the scheduling. Multiple
schedulers would block each other or busy wait. So having multiple
schedulers is strongly discouraged in long running applications, however,
it might be acceptable in traditional applications, i.e. PHP-FPM. In
PHP-FPM, multiple schedulers partially blocking each other is still better
than blocking entirely for every I/O operation.

> What's not clear to me is why the scheduling fiber needs to be
> distinguished from other fibers. If we want to stick with the general
> approach, why is Fiber::suspend($scheduler) not Fiber::transferTo($fiber),
> where $fiber would be the fiber serving as scheduler (but otherwise a
> normal Fiber)? I would expect that context-switching between arbitrary
> fibers would be both most expressive, and make for the smallest interface.
>

There are a few reasons to make a difference here:

- SchedulerFibers are run to completion at script end, which isn't the case
for normal fibers.
- Terminating fibers need a fiber to return to. For schedulers it's fine if
a resumed fiber terminates, for normal fibers it should be an exception if
the scheduler fiber terminates without explicitly resuming the suspended
fiber.
- Keeping the previous fiber for each suspension point is complicated if
not impossible to get right and generally complicates the implementation
and cognitive load, see following example:

main -> A -> B -> C -> A (terminates) -> C (previous) -> B (terminates) ->
C (previous, terminates) -> main

In the example above, the previous fiber linked list from C back to main
needs to be optimized at some point, otherwise A and B need to be kept in
memory and thus leak memory until C is resumed.

I'm sure Aaron can present a few other reasons to keep the separation.

> The more limited alternative is to instead have Fiber::suspend() return to
> the parent fiber (the one that resume()d it). Here, the parent fiber
> effectively becomes the scheduler fiber. If I understand right, the reason
> why you don't want to use that approach, is that it doesn't allow you to
> call some AMP function from the {main} fiber, create the scheduler there
> and then treat {main} just like any other fiber. Is that correct?
>

Correct, this wouldn't allow top-level Fiber::suspend(). It would also make
the starting / previously resuming party responsible for resuming the fiber
instead of the fiber being able to "choose" the scheduler for a specific
suspension point. One fiber would thus be effectively limited to a single
scheduler.

Best,
Niklas

Nikita Popov

5 years ago

On Tue, Feb 2, 2021 at 10:35 PM Niklas Keller <me@kelunik.com> wrote:

> Hey Nikita,
>
> Thank you for the proposal. Ergonomics of async I/O in PHP certainly leave
>> something to be desired right now, and improvements in this area are
>> welcome.
>>
>> Despite your explanations in the RFC and this thread, I'm still having a
>> hard time understanding the purpose of the FiberScheduler.
>>
>> My current understanding is that the FiberScheduler is a special type of
>> fiber that cannot be explicitly scheduled by the user -- it is
>> automatically scheduled by Fiber::suspend() and automatically un-scheduled
>> by Fiber::resume() or Fiber::throw(). It's the fiber that runs between
>> fibers :) Does that sound accurate?
>>
>
> Yes, that's accurate. Fibers are used for cooperative multi-tasking and
> there's usually a single scheduler responsible for the scheduling. Multiple
> schedulers would block each other or busy wait. So having multiple
> schedulers is strongly discouraged in long running applications, however,
> it might be acceptable in traditional applications, i.e. PHP-FPM. In
> PHP-FPM, multiple schedulers partially blocking each other is still better
> than blocking entirely for every I/O operation.
>
>
>> What's not clear to me is why the scheduling fiber needs to be
>> distinguished from other fibers. If we want to stick with the general
>> approach, why is Fiber::suspend($scheduler) not Fiber::transferTo($fiber),
>> where $fiber would be the fiber serving as scheduler (but otherwise a
>> normal Fiber)? I would expect that context-switching between arbitrary
>> fibers would be both most expressive, and make for the smallest interface.
>>
>
> There are a few reasons to make a difference here:
>
> - SchedulerFibers are run to completion at script end, which isn't the
> case for normal fibers.
> - Terminating fibers need a fiber to return to. For schedulers it's fine
> if a resumed fiber terminates, for normal fibers it should be an exception
> if the scheduler fiber terminates without explicitly resuming the suspended
> fiber.
> - Keeping the previous fiber for each suspension point is complicated if
> not impossible to get right and generally complicates the implementation
> and cognitive load, see following example:
>
> main -> A -> B -> C -> A (terminates) -> C (previous) -> B (terminates) ->
> C (previous, terminates) -> main
>
> In the example above, the previous fiber linked list from C back to main
> needs to be optimized at some point, otherwise A and B need to be kept in
> memory and thus leak memory until C is resumed.
>
> I'm sure Aaron can present a few other reasons to keep the separation.
>

Thanks, I didn't consider the terminating fiber case here. But possibly a
combination of the two would work? That is, you generally have
Fiber::suspend() return to the parent fiber, and make the parent fiber
terminating while still having children an error condition. Additionally,
you provide something like Fiber::runAsParent() to allow "adopting" a fiber
under a new parent, that serves as scheduler (this is to cover the {main}
-> scheduler use case).

If you stick to the FiberScheduler concept, then you might want to consider
inverting the API. Right now you're basically using a standard Fiber API,
with the difference that suspend() accepts a FiberScheduler, which is
unintuitive to me. If Fibers require a scheduler anyway, why are the
suspend and resume methods not on the scheduler?

class FiberScheduler {
function suspend(Fiber $fiber);
function start(Fiber $fiber);
function resume(Fiber $fiber);
}

Both methods are bound to the scheduler in that "suspend" suspends back to
a certain scheduler, while "resume" resumes a fiber such that it will
return back to this scheduler on termination. This also makes it more
obvious that, for example, it's not possible to just do a "$fiber->start()"
without having created a scheduler first (though it does not make it
obvious that the call has to be from within the scheduler).

Regards,
Nikita

Aaron Piotrowski

5 years ago

> On Feb 3, 2021, at 8:27 AM, Nikita Popov <nikita.ppv@gmail.com> wrote:
>
> If you stick to the FiberScheduler concept, then you might want to consider
> inverting the API. Right now you're basically using a standard Fiber API,
> with the difference that suspend() accepts a FiberScheduler, which is
> unintuitive to me. If Fibers require a scheduler anyway, why are the
> suspend and resume methods not on the scheduler?
>
> class FiberScheduler {
>    function suspend(Fiber $fiber);
>    function start(Fiber $fiber);
>    function resume(Fiber $fiber);
> }
>
> Both methods are bound to the scheduler in that "suspend" suspends back to
> a certain scheduler, while "resume" resumes a fiber such that it will
> return back to this scheduler on termination. This also makes it more
> obvious that, for example, it's not possible to just do a "$fiber->start()"
> without having created a scheduler first (though it does not make it
> obvious that the call has to be from within the scheduler).
>
> Regards,
> Nikita
>

Hi Nikita,

I considered adding start, resume, and throw methods on FiberScheduler as you suggested, however I’m not sure it would serve to make the API clearer. The callback (function, method, etc.) that is used to create the instance of FiberScheduler does not immediately have a reference to the created FiberScheduler object, so using the object within that callback would be awkward. One solution would be a `FiberScheduler::this()` method, but I think that adds complexity for not much gain. `FiberScheduler::this()->resume($fiber, $value)` is, in my opinion, less intuitive than `$fiber->resume($value)`. Either would have to be called within the callback provided to the FiberScheduler constructor.

Niklas otherwise outlined the reasons to have FiberScheduler a unique fiber – suspending {main}, interoperability between libraries using differing schedulers, and running schedulers to completion upon script termination.

>>>
>>> What's not clear to me is why the scheduling fiber needs to be
>>> distinguished from other fibers. If we want to stick with the general
>>> approach, why is Fiber::suspend($scheduler) not Fiber::transferTo($fiber),
>>> where $fiber would be the fiber serving as scheduler (but otherwise a
>>> normal Fiber)? I would expect that context-switching between arbitrary
>>> fibers would be both most expressive, and make for the smallest interface.

When starting or resuming a fiber, the current execution point in the current fiber is stored as the point to jump to when the starting/resuming fiber suspends or terminates. Fibers must therefore be only pushed or popped from a stack, you cannot switch to an already running fiber, as another fiber higher in the stack would no longer return to the correct execution point when suspended or terminated.

Since FiberScheduler internally is just another fiber, there’s nothing really special done to switch to a fiber scheduler. Requiring suspending to a scheduler rather than an arbitrary fiber prevents users from attempting to switch to an already running fiber.

Cheers,
Aaron Piotrowski

twosee

5 years ago

> 2020年12月18日 上午12:30，Aaron Piotrowski <aaron@trowski.com> 写道：
>
> Hello everyone!
>
> I would like to introduce an RFC for adding full-stack fibers to PHP: https://wiki.php.net/rfc/fibers
>
> Fibers are primarily used to implement green-threads or coroutines for asynchronous I/O. Fibers are similar to threads, except fibers exist within a single thread and require cooperative scheduling of the fibers by the process. Since fibers do not require a full CPU context switch, they are lightweight and more performant than multi-processing or threading for awaiting I/O.
>
> An implementation as an extension is at https://github.com/amphp/ext-fiber
>
> Fibers are a complex feature. The RFC contains many examples and links to code using fibers to help explain and demonstrate what is possible, however I’m certain many more questions and concerns will arise. Looking forward to feedback and discussion.
>
> Aaron Piotrowski
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: https://www.php.net/unsub.php
>
>

Hi there,

So glad to see that the RFC of Fiber is back on the agenda.

Let me introduce myself first. I'm a very active contributor to Swoole and also a core member of the Swoole developing group.

Though Swoole has not been mentioned in the RFC, Swoole has already worked for async/coroutine feature for many years and always doing best in order to be merged into php-src. It's extremely difficult to merge Swoole into php-src fully because Swoole is systematic and complex. Even the coroutine is a core but also a tiny part of Swoole.

In recent years, Swoole has been committed to improving stability and overcoming technical problems. It can be said that the current Swoole is very close to the ideal status. What's more, Swoole has been applied widely by advanced enterprises in China. It has been verified by a large amount of real-world applying. We may have more practical experience in many details of the implementation of coroutine technology

For the core of Fiber, the context switching of C stack and PHP VM stack, there is no more obvious problem. But there is somewhere worth improving, indeed. Furthermore, we may do more context switching, since Swoole may yield from PHP internal function, which would make the "@" not work.

As we could see, the main controversy is what about the FiberScehduler. I think it is designed for the amphp, however, it makes code hard to be comprehended, and it was unnecessary if we just want a minimum coroutine implementation.

Just having creation and switching is enough for a minimum coroutine implementation. So did the last Fiber proposal. The implementation of the scheduler is still debatable.

Otherwise, compared with the previous Fiber proposal, the most valuable change in the new Fiber proposal is the introduction of C stack coroutine technology.

In my opinion, the goal of the new Fiber proposal is not that ambitious. It just eliminates the stack pollution caused by Promise. This makes the style of PHP coroutine neither like the Nodejs which represents the pure stackless coroutine, nor the Lua and Golang which represent pure stack coroutine.

Once PHP has a stack coroutine like Fiber, we can do more than what we can do now. Since we can interrupt from PHP internal functions, then we can replace all the implementation of PHP blocking functions, such as sleep(), and we can also replace php_stream so that we can change the implementation of PDO, mysqli, and phpredis into a coroutine way, and we can also make curl become a coroutine version through libcurl's support for multiplexing.

In fact, many years ago, amphp-liked solutions were very popular in the Chinese PHP community. But they were replaced by Swoole, eventually.

Performance is one of the reasons. Because, indeed, parsing protocol by PHP is disadvantaged compared to parsing by C. Though the amphp-liked solutions dose expand the PHP's capabilities, the performance is not that satisfactory. However, Swoole performs like other static languages.

On another hand, the replacement cost. What has been proved is that most PHP programmers want to continue to use common clients such as PDO, mysqli, phpredis, and curl, and the great PHP package and library ecosystem based on these clients is hard to replace. The cost of moving a modern PHP application to Swoole is so low which was unimaginable before. Many success cases there in the community.

So what I want to explain is that once the PHP core is ready to accept the Fiber proposal, everyone will start to think, what is the next? Therefore, what Fiber's goal is what the Fiber extension is designed for are particularly important. What I really care about is - does it just simply eliminates the pollution of the stack caused by Promise, or it plans to improve the performance of PHP applications by a hundredfold at a low cost as Swoole did.

Regards,
Twosee

Aaron Piotrowski

5 years ago

> On Dec 17, 2020, at 10:30 AM, Aaron Piotrowski <aaron@trowski.com> wrote:
>
> Hello everyone!
>
> I would like to introduce an RFC for adding full-stack fibers to PHP: https://wiki.php.net/rfc/fibers
>
> Fibers are primarily used to implement green-threads or coroutines for asynchronous I/O. Fibers are similar to threads, except fibers exist within a single thread and require cooperative scheduling of the fibers by the process. Since fibers do not require a full CPU context switch, they are lightweight and more performant than multi-processing or threading for awaiting I/O.
>
> An implementation as an extension is at https://github.com/amphp/ext-fiber
>
> Fibers are a complex feature. The RFC contains many examples and links to code using fibers to help explain and demonstrate what is possible, however I’m certain many more questions and concerns will arise. Looking forward to feedback and discussion.
>
> Aaron Piotrowski
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: https://www.php.net/unsub.php
>

Hello again everyone!

Based upon feedback both on this list and elsewhere, I’ve decided to remove the FiberScheduler API from the fiber proposal.

The FiberScheduler API greatly increased the complexity of the implementation and the potential for edge cases that would need to be handled and maintained. The intent of including the FiberScheduler API as part of the proposed implementation was to require usage of fibers to be interoperable between various libraries. However, this was perhaps over-reaching, and the core should only provide the most fundamental tools for fibers.

The Fiber API proposed now resembles that of Ruby and other languages providing a fiber API. The revised RFC now proposes only a minimal API required for fibers – no more, no less. I believe this minimal, simplified API should resolve any concerns raised about adding the fiber API to core.

I would like to open voting for this RFC around the beginning of March, so please review the minimal API and provide any feedback soon.

As before, amphp v3 (https://github.com/amphp/amp/tree/v3) and trowski/react-fiber (https://github.com/trowski/react-fiber) provide real-world examples of how fibers can be used in addition to the examples in the RFC.

Cheers!
Aaron Piotrowski

Mark Randall

5 years ago

On 12/02/2021 21:40, Aaron Piotrowski wrote:

> I would like to open voting for this RFC around the beginning of March, so please review the minimal API and provide any feedback soon.

Removing the scheduler was likely a good plan, but pretty please
reconsider your future scope.

It's going to be a really big pain to push people to use this
functionality, only to have to re-write it 1 or 2 year later because a
vastly superior syntax mechanism was introduced.

If you can add async / await / delay / defer, even if it takes another
month or so, that would likely make this RFC have a significantly bigger
and more beneficial impact on userland.

Mark Randall

Aaron Piotrowski

5 years ago

> On Feb 12, 2021, at 3:47 PM, Mark Randall <marandall@php.net> wrote:
>
> On 12/02/2021 21:40, Aaron Piotrowski wrote:
>> I would like to open voting for this RFC around the beginning of March, so please review the minimal API and provide any feedback soon.
>
> Removing the scheduler was likely a good plan, but pretty please reconsider your future scope.
>
> It's going to be a really big pain to push people to use this functionality, only to have to re-write it 1 or 2 year later because a vastly superior syntax mechanism was introduced.
>
> If you can add async / await / delay / defer, even if it takes another month or so, that would likely make this RFC have a significantly bigger and more beneficial impact on userland.
>
> Mark Randall
>
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: https://www.php.net/unsub.php
>

Hi Mark,

The Fiber API is only tangentially related to async / await, etc. Adding such a feature is a much larger proposal and would not be small undertaking. The future scope proposes using fibers in an async / await implementation, but async / await would not replace fibers.

The Fiber API would conflict or prevent async / await from being added to PHP in the future. The two APIs can coexist and serve different purposes.

Cheers,
Aaron Piotrowski

Aaron Piotrowski

5 years ago

> On Feb 12, 2021, at 4:05 PM, Aaron Piotrowski <aaron@trowski.com> wrote:
>
> The Fiber API would conflict or prevent async / await from being added to PHP in the future. The two APIs can coexist and serve different purposes.
>

While probably obvious from the rest of the context of the email, for clarity, I meant to state that the Fiber API would *not* conflict or prevent async / await from being added to PHP in the future.

Sorry for the noise!

Aaron Piotrowski

Aaron Piotrowski

5 years ago

> Hello again everyone!
>
> Based upon feedback both on this list and elsewhere, I’ve decided to remove the FiberScheduler API from the fiber proposal.
>
> The FiberScheduler API greatly increased the complexity of the implementation and the potential for edge cases that would need to be handled and maintained. The intent of including the FiberScheduler API as part of the proposed implementation was to require usage of fibers to be interoperable between various libraries. However, this was perhaps over-reaching, and the core should only provide the most fundamental tools for fibers.
>
> The Fiber API proposed now resembles that of Ruby and other languages providing a fiber API. The revised RFC now proposes only a minimal API required for fibers – no more, no less. I believe this minimal, simplified API should resolve any concerns raised about adding the fiber API to core.
>
> I would like to open voting for this RFC around the beginning of March, so please review the minimal API and provide any feedback soon.
>
> As before, amphp v3 (https://github.com/amphp/amp/tree/v3) and trowski/react-fiber (https://github.com/trowski/react-fiber) provide real-world examples of how fibers can be used in addition to the examples in the RFC.
>

Hi all!

Since there has not been much feedback lately on the Fiber RFC, I wanted to notify everyone that I intend to start voting on the RFC next week Monday.

If you have any questions or concerns, please do raise them now, as it’s always more difficult to address these once voting has started.

I feel this feature deserves a lot of time to merge and test if it is to be included in 8.1, so I’d like to move forward soon.

Cheers,
Aaron Piotrowski