[RFC] Auto-capture multi-line closures and short functions take 2

Nikita Popov

5 years ago

On Wed, Mar 24, 2021 at 5:20 PM Larry Garfield <larry@garfieldtech.com>
wrote:

> Greetings, Internalians.
>
> Some months back I proposed an RFC for short functions.
>
> https://wiki.php.net/rfc/short-functions
>
> After some discussion, I put it on hold to ensure that it was compatible
> with the other discussion floating about around an alternate syntax for
> multi-line closures that did auto-capture.  It's important that the syntax
> of both proposals is consistent.
>
> Nuno Maduro and i have talked recently and worked out the syntax that is
> consistent between both proposals.  That takes 'fn' off the table for
> short-functions, for reasons discussed in both RFCs.
>
> To that end, I offer:
>
> 1) The updated short-functions RFC:
> https://wiki.php.net/rfc/short-functions
>
> 2) A new RFC, code by Nuno, for auto-capturing multi-statement closures:
> https://wiki.php.net/rfc/auto-capture-closure
>
> These are separate RFCs and at this time we plan to send them to separate
> votes, but I list them together to demonstrate that they have been
> developed in a mutually-compatible way.
>
> Further details in the auto-capture-closure RFC.
>

Could you please create two separate threads for these RFCs?

Nikita

Chase Peeler

5 years ago

On Wed, Mar 24, 2021 at 1:02 PM Nikita Popov <nikita.ppv@gmail.com> wrote:

> On Wed, Mar 24, 2021 at 5:20 PM Larry Garfield <larry@garfieldtech.com>
> wrote:
>
> > Greetings, Internalians.
> >
> > Some months back I proposed an RFC for short functions.
> >
> > https://wiki.php.net/rfc/short-functions
> >
> > After some discussion, I put it on hold to ensure that it was compatible
> > with the other discussion floating about around an alternate syntax for
> > multi-line closures that did auto-capture.  It's important that the
> syntax
> > of both proposals is consistent.
> >
> > Nuno Maduro and i have talked recently and worked out the syntax that is
> > consistent between both proposals.  That takes 'fn' off the table for
> > short-functions, for reasons discussed in both RFCs.
> >
> > To that end, I offer:
> >
> > 1) The updated short-functions RFC:
> > https://wiki.php.net/rfc/short-functions
> >
> > 2) A new RFC, code by Nuno, for auto-capturing multi-statement closures:
> > https://wiki.php.net/rfc/auto-capture-closure
> >
> > These are separate RFCs and at this time we plan to send them to separate
> > votes, but I list them together to demonstrate that they have been
> > developed in a mutually-compatible way.
> >
> > Further details in the auto-capture-closure RFC.
> >
>
> Could you please create two separate threads for these RFCs?
>
> Nikita
>

I guess my one question would be why we didn't support auto-capture when we
first implemented anonymous functions, and if there was a reason, why does
that no longer apply?

--
Chase Peeler
chasepeeler@gmail.com

Christian Schneider

5 years ago

Am 24.03.2021 um 18:15 schrieb Chase Peeler <chasepeeler@gmail.com>:

> I guess my one question would be why we didn't support auto-capture when we
> first implemented anonymous functions, and if there was a reason, why does
> that no longer apply?

My guess would be that it was seen as one of PHP's big strength that variables don't just leak into other contexts but that it has to be done explicitly.

Now with arrow functions both the benefit of auto-capturing is bigger compared to the actual function 'body', it normally only spans a line or two and (for me personally) the absence of {} also kind of make the leakage fit my mental model more easily.

For these reasons I think I lean towards -1 for both https://wiki.php.net/rfc/short-functions <https://wiki.php.net/rfc/short-functions> and https://wiki.php.net/rfc/auto-capture-closure <https://wiki.php.net/rfc/auto-capture-closure> as they add more syntax without (IMHO) a huge benefit.

You might be able to convince me otherwise though ;-)
- Chris

Chase Peeler

5 years ago

On Wed, Mar 24, 2021 at 1:34 PM Christian Schneider <cschneid@cschneid.com>
wrote:

> Am 24.03.2021 um 18:15 schrieb Chase Peeler <chasepeeler@gmail.com>:
> > I guess my one question would be why we didn't support auto-capture when
> we
> > first implemented anonymous functions, and if there was a reason, why
> does
> > that no longer apply?
>
> My guess would be that it was seen as one of PHP's big strength that
> variables don't just leak into other contexts but that it has to be done
> explicitly.
>
> Now with arrow functions both the benefit of auto-capturing is bigger
> compared to the actual function 'body', it normally only spans a line or
> two and (for me personally) the absence of {} also kind of make the leakage
> fit my mental model more easily.
>
> For these reasons I think I lean towards -1 for both
> https://wiki.php.net/rfc/short-functions <
> https://wiki.php.net/rfc/short-functions> and
> https://wiki.php.net/rfc/auto-capture-closure <
> https://wiki.php.net/rfc/auto-capture-closure> as they add more syntax
> without (IMHO) a huge benefit.
>
> You might be able to convince me otherwise though ;-)
>

I don't really have anything against auto-capture, was just kind of
curious. I guess it is somewhat similar to how x = function(){...} and x =
() => {} work in javascript with respect to "this".

I do think it makes sense to only do auto-capture on the short syntax, so
that you still have the ability to fallback to the long syntax if you don't
want auto-capture, since there are definitely use-cases where it wouldn't
be make sense.
,

> - Chris
>
>

--
Chase Peeler
chasepeeler@gmail.com

Rowan Collins

5 years ago

On 24/03/2021 17:33, Christian Schneider wrote:

> My guess would be that it was seen as one of PHP's big strength that variables don't just leak into other contexts but that it has to be done explicitly.
>
> Now with arrow functions both the benefit of auto-capturing is bigger compared to the actual function 'body', it normally only spans a line or two and (for me personally) the absence of {} also kind of make the leakage fit my mental model more easily.
>
> For these reasons I think I lean towards -1 for bothhttps://wiki.php.net/rfc/short-functions  <https://wiki.php.net/rfc/short-functions>  andhttps://wiki.php.net/rfc/auto-capture-closure  <https://wiki.php.net/rfc/auto-capture-closure>  as they add more syntax without (IMHO) a huge benefit.
>
> You might be able to convince me otherwise though ;-)

This pretty much sums up my view.

While making the syntaxes consistent is a valid aim, some focus seems to
have been lost on the actual use cases being addressed.


With "function" spelled out in full, the "short functions" are shorter
by exactly the length of the word "return", which is rather uninspiring.
The examples exaggerate the benefit by sticking to PSR-12 formatting for
the "long" version, when the language itself is perfectly happy for you
to write this:

public function getFirstName(): string { return $this->firstName; }
public function getLastName(): string { return $this->lastName; }


As Christian says, automatic capture is a dramatic change to the
language's scoping rules, and IMHO requires a more thorough
justification on why the current syntax is burdensome. As I've said
previously, my naive impression is that a long list of captured
variables ought to be as bad a code smell as a long list of parameters;
I'm willing to be proven wrong on this, but nobody has yet stepped
forward with a real-life example.

If we do agree that we want to support auto-capture, I am also not
convinced that "fn" is the right keyword to indicate it. I would be
happier leaving that to mean "expression becoming a function", and
extend the current "function ... use" syntax, e.g. "function($x) use
(*)  { ... }". This also has the benefit of being extensible to
supporting reference capture, e.g. "function($x) use (&$x, *)  { ... }"
as short-hand for "function($x) use (&$x, $a, $b, $c, $d, $etc)  { ... }"


Regards,

--
Rowan Tommins
[IMSoP]

Mark Randall

5 years ago

On 24/03/2021 21:00, Rowan Tommins wrote:

> As Christian says, automatic capture is a dramatic change to the
> language's scoping rules, and IMHO requires a more thorough
> justification on why the current syntax is burdensome. As I've said
> previously, my naive impression is that a long list of captured
> variables ought to be as bad a code smell as a long list of parameters;
> I'm willing to be proven wrong on this, but nobody has yet stepped
> forward with a real-life example.

Automatic capture ceased to be a dramatic change from PHP the day after
short closurers were introduced. So they've been a part of PHP for years
now.

I hit long lists of use() repeatedly, almost always because I need to
perform a series of operations within a callback, prime examples are
database transactions and using throw-aware buffering handlers.

In general, anything that needs to wrap a code block in a complex set of
instructions before and after.

To give my own example, earlier this week I wrote the following:

$x = function () use ($to, $library, $thread, $author, $title,
$library_name, $top_post) { ... }

That was just to get those variables inside a callback that could be
invoked inside a throw-aware buffering helper.

I believe that by explicitly stating my intent to use auto capture by
using fn() { ... } that my code would have been cleaner with less noise.

If I thought otherwise, I would be under no obligation to use them and
could use function() { ... } instead.

For me auto capture is a solid +1.

Mark Randall

Peter Stalman

5 years ago

On Wed, Mar 24, 2021 at 10:15 AM Chase Peeler <chasepeeler@gmail.com> wrote:

> I guess my one question would be why we didn't support auto-capture when we
> first implemented anonymous functions, and if there was a reason, why does
> that no longer apply?
>

I believe this was the original discussion:

https://externals.io/message/38290

Thanks,
Peter

Larry Garfield

5 years ago

On Thu, Mar 25, 2021, at 12:47 AM, Peter Stalman wrote:

> On Wed, Mar 24, 2021 at 10:15 AM Chase Peeler <chasepeeler@gmail.com> wrote:
>
> > I guess my one question would be why we didn't support auto-capture when we
> > first implemented anonymous functions, and if there was a reason, why does
> > that no longer apply?
> >
>
> I believe this was the original discussion:
>
> https://externals.io/message/38290
>
> Thanks,
> Peter

Thanks!

I just skimmed that thread looking for scope discussions.  I didn't read everything so I may have missed a comment somewhere, but the gist seems to be:

* The original proposal included manual capture because it originally was going to capture everything by reference, not by value, and so it was seen as "safer" to not surprise someone with references leaking about.  (Especially comparisons to Javascript.)

* There was mention in the thread of there being a performance advantage of manual capture, but I didn't quite follow it.  Also, that was back in 5.3 and so much has changed since then in the engine that I would consider any comments about performance from that era no longer relevant.

* There was ample debate about what syntax to use for manual capture, but very little discussion of whether it should be manual or automatic in the first place.

* In the end, we ended up with the use ($byVal, &$byRef) syntax we have today... but with the switch to by-val by default there was no discussion of whether auto-capture should be used since that was already safer.  Arguably the need to differentiate by val or by ref made the explicit list necessary.

So in conclusion, it seems the reasons closures weren't auto-capture always were:

1) Possible performance concerns that are no longer relevant.
2) Avoid surprise references.
3) Allow users to capture by value or by reference.

Point 1 is no longer relevant as the engine has changed so much.
Point 2 is no longer relevant since forever, since capture is by-val by default.
Point 3 is only relevant in the cases where you need to capture by reference, which in practice it turns out are really quite rare.

So, I would argue that all of the reasons for manual capture in the first place are no longer relevant, except in the edge case where you really do want to capture by reference.

Which is exactly what the RFC currently says. :-)

--Larry Garfield

Christian Schneider

5 years ago

Am 25.03.2021 um 17:25 schrieb Larry Garfield <larry@garfieldtech.com>:

> So in conclusion, it seems the reasons closures weren't auto-capture always were:
>
> 1) Possible performance concerns that are no longer relevant.
> 2) Avoid surprise references.
> 3) Allow users to capture by value or by reference.
>
> Point 1 is no longer relevant as the engine has changed so much.
> Point 2 is no longer relevant since forever, since capture is by-val by default.
> Point 3 is only relevant in the cases where you need to capture by reference, which in practice it turns out are really quite rare.
>
> So, I would argue that all of the reasons for manual capture in the first place are no longer relevant, except in the edge case where you really do want to capture by reference.

While the discussion back then revolved a lot around performance and references I don't think

4) Local-as-default scoping and explicit captures are the PHP way and make it a better language

which was mentioned in this thread should be buried ;-)

- Chris

Rowan Collins

5 years ago

On 25/03/2021 16:25, Larry Garfield wrote:

> So in conclusion, it seems the reasons closures weren't auto-capture always were:
>
> 1) Possible performance concerns that are no longer relevant.
> 2) Avoid surprise references.
> 3) Allow users to capture by value or by reference.

I wasn't involved in the discussion when anonymous functions were first
added, but I was during the discussions that led to the current arrow
functions.

I think it's important context that multi-statement closures were
originally included and explicitly removed, precisely because automatic
capture was always controversial.

It is perfectly reasonable to re-visit the topic now, but it is not, as
some seem to be implying, one that has already been agreed in principle.


Bob's original "Short Closures" RFC [1] in 2015 included both single
expressions and full function bodies. As in subsequent discussions, some
people thought implicit capture was a great feature, and others were
concerned by it. Rasmus described it [2] as "breaking [a] longstanding
basic tenet of the language".

There were other reasons that RFC was not accepted, mostly around
syntax, but several people in the voting thread [3] gave automatic
capture as a reason for voting no.

The subsequent "Arrow Functions" RFC [4] by Bob and Levi in 2017
explicitly removed multi-statement bodies from the proposal, citing
feedback from previous discussions. Bob confirmed in the discussion
thread [5] that this was a deliberate design choice, because requiring
"use" for most Closures "vastly improves readability and debuggability".

Rasmus agreed [6] that automatic capture was acceptable in the new RFC
specifically because single-expression closures "don't typically need a
local scope at all".

Finally, in 2019 Nikita wrote a new version of the RFC [7] which again
left out multi-statement bodies. During the discussion [8] he considered
adding them back in, but felt it would "add a number of complications"
which should be addressed in a future RFC.

Several people who had previously expressed strong concerns about
automatic capture voted "Yes" to Nikita's RFC . At least some presumably
did so precisely because multi-statement bodies were not included.


[1] https://wiki.php.net/rfc/short_closures
[2] https://externals.io/message/87958#87969
[3] https://externals.io/message/88394
[4] https://wiki.php.net/rfc/arrow_functions
[5] https://externals.io/message/98045#98069
[6] https://externals.io/message/98045#98108
[7] https://wiki.php.net/rfc/arrow_functions_v2
[8] https://externals.io/message/104693#104738


Regards,

--
Rowan Tommins
[IMSoP]

Deleu

5 years ago

On Wed, Mar 24, 2021, 17:20 Larry Garfield <larry@garfieldtech.com> wrote:

> Greetings, Internalians.
>
> Some months back I proposed an RFC for short functions.
>
> https://wiki.php.net/rfc/short-functions
>
> After some discussion, I put it on hold to ensure that it was compatible
> with the other discussion floating about around an alternate syntax for
> multi-line closures that did auto-capture.  It's important that the syntax
> of both proposals is consistent.
>
> Nuno Maduro and i have talked recently and worked out the syntax that is
> consistent between both proposals.  That takes 'fn' off the table for
> short-functions, for reasons discussed in both RFCs.
>
> To that end, I offer:
>
> 1) The updated short-functions RFC:
> https://wiki.php.net/rfc/short-functions
>
> 2) A new RFC, code by Nuno, for auto-capturing multi-statement closures:
> https://wiki.php.net/rfc/auto-capture-closure
>
> These are separate RFCs and at this time we plan to send them to separate
> votes, but I list them together to demonstrate that they have been
> developed in a mutually-compatible way.
>
> Further details in the auto-capture-closure RFC.
>
> --
>   Larry Garfield
>   larry@garfieldtech.com
>
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: https://www.php.net/unsub.php

These RFC are an amazing addition to PHP. I have been working with PHP 8
for the past few months and small syntax improvements such as short
closure, private constructor promotion, nullsafe operator, etc. are game
changers to my day to day work.

These RFC close some gaps that are natural expectations like short closure
with multiple statements and the ability to simplify even further simple
methods on Data Transfer objects.

I wish the best luck to both RFCs and hope they get approved, if not for
their wonderful merits, maybe at least for the sake of consistent syntax
throughout the language.

Peter Stalman

5 years ago

On Wed, Mar 24, 2021 at 9:20 AM Larry Garfield <larry@garfieldtech.com>
wrote:

> 1) The updated short-functions RFC:
> https://wiki.php.net/rfc/short-functions
>
> 2) A new RFC, code by Nuno, for auto-capturing multi-statement closures:
> https://wiki.php.net/rfc/auto-capture-closure
>
> These are separate RFCs and at this time we plan to send them to separate
> votes, but I list them together to demonstrate that they have been
> developed in a mutually-compatible way.
>

Hi Larry (and Nuno) ,

Well done on the RFCs and making sure they are compatible and consistent.

That said, I agree with the sentiment that multi-statement auto-capturing
closures can add noise.  It's not just a multi-line `fn()`, as their usage
would be different.  With the single statement and return of `fn()` it was
intended to be used as a function argument where a callback is needed.
There autocapture makes sense to me.

But in the end, it's a matter of preference to use it or not.  I just don't
want PHP to drift closer to the callback hell that is JS.

Looking at the two proposals, I boil it down to this combined example:

```
class Short
{
public function foo($a, $b, $c): int => match ($a) {
1, 3, 5 => (fn() {
$val = $a * $b;
return $val * $c;
})(),
2, 4, 6 => (fn() {
$val = $a + $b;
return $val + $c;
})(),
};
}

class Long
{
public function foo($a, $b, $c): int
{
return match ($a) {
1, 3, 5 => self::bar($a, $b, $c),
2, 4, 6 => self::baz($a, $b, $c),
};
}

private static function bar($a, $b, $c): int
{
$val = $a * $b;
return $val * $c;
}

private static function baz($a, $b, $c): int
{
$val = $a + $b;
return $val + $c;
}
}
```

The Short class is "fine", but I would still prefer the Long class.

Thanks,
Peter

A.L.E.C

5 years ago

On 24.03.2021 17:19, Larry Garfield wrote:

> 1) The updated short-functions RFC: https://wiki.php.net/rfc/short-functions
> 2) A new RFC, code by Nuno, for auto-capturing multi-statement closures: https://wiki.php.net/rfc/auto-capture-closure

I'm not sure just the keyword (`fn`) should indicate the change in
variables scope handling (auto-capturing). Note that in Javascript it is
a new syntax not just the keyword.

Looks like a counter-proposal could be this syntax:

$foo = function($a, $b) => {
$val = $a * $b;
return $val * c;
}

then whether it is `fn` or `function` keyword does not really matter.
Here it is `=>` that indicates the auto-capturing in the block on the
right side.

I don't know. Another approach might be something that makes mind to
think this is something different than a function. A completely new
keyword that makes the code block to auto-capture. E.g. `procedure` or
`macro`.

--
Aleksander Machniak
Kolab Groupware Developer        [https://kolab.org]
Roundcube Webmail Developer  [https://roundcube.net]
----------------------------------------------------
PGP: 19359DC1 # Blog: https://kolabian.wordpress.com

Larry Garfield

5 years ago

On Wed, Mar 24, 2021, at 11:19 AM, Larry Garfield wrote:

> Greetings, Internalians.
>
> Some months back I proposed an RFC for short functions.
>
> https://wiki.php.net/rfc/short-functions
>
> After some discussion, I put it on hold to ensure that it was
> compatible with the other discussion floating about around an alternate
> syntax for multi-line closures that did auto-capture.  It's important
> that the syntax of both proposals is consistent.
>
> Nuno Maduro and i have talked recently and worked out the syntax that
> is consistent between both proposals.  That takes 'fn' off the table
> for short-functions, for reasons discussed in both RFCs.
>
> To that end, I offer:
>
> 1) The updated short-functions RFC: https://wiki.php.net/rfc/short-functions
>
> 2) A new RFC, code by Nuno, for auto-capturing multi-statement
> closures: https://wiki.php.net/rfc/auto-capture-closure
>
> These are separate RFCs and at this time we plan to send them to
> separate votes, but I list them together to demonstrate that they have
> been developed in a mutually-compatible way.
>
> Further details in the auto-capture-closure RFC.

Hi folks.

We've recently added some additional reasoning to the auto-capture-closure RFC, so it's worth giving another read-over.  It was also recently featured on an episode of PHP Internals if you want a more verbal case made. :-)

Unless there is any significant feedback, we'll be calling a vote for this RFC sometime late this week (Thursday/Friday-ish).

Cheers.

--Larry Garfield

Nikita Popov

5 years ago

On Wed, Mar 24, 2021 at 5:20 PM Larry Garfield <larry@garfieldtech.com>
wrote:

> Greetings, Internalians.
>
> Some months back I proposed an RFC for short functions.
>
> https://wiki.php.net/rfc/short-functions
>
> After some discussion, I put it on hold to ensure that it was compatible
> with the other discussion floating about around an alternate syntax for
> multi-line closures that did auto-capture.  It's important that the syntax
> of both proposals is consistent.
>
> Nuno Maduro and i have talked recently and worked out the syntax that is
> consistent between both proposals.  That takes 'fn' off the table for
> short-functions, for reasons discussed in both RFCs.
>
> To that end, I offer:
>
> 1) The updated short-functions RFC:
> https://wiki.php.net/rfc/short-functions
>
> 2) A new RFC, code by Nuno, for auto-capturing multi-statement closures:
> https://wiki.php.net/rfc/auto-capture-closure
>
> These are separate RFCs and at this time we plan to send them to separate
> votes, but I list them together to demonstrate that they have been
> developed in a mutually-compatible way.
>
> Further details in the auto-capture-closure RFC.
>

Hey Larry,

I'm generally in favor of supporting auto-capture for multi-line closures.
I think that extensive experience in other programming languages has shown
that auto-capture does not hinder readability of code, and I don't see any
particular reason why the effects in PHP would be different. I also think
there is value in having a consistent syntax for closures -- currently, you
have two options, and may need to switch between them as you add or remove
code.

There are some caveats though, which this RFC should address:

1. Auto-capture in arrow functions works by-value. This is sufficient for
the purpose of single-line arrow functions, where you would be hard-pressed
to perform any meaningful state mutation. For multi-line closures, it
becomes much more likely that you want to modify a value from the outer
scope. It should be possible to do this without switching to the function()
notation.

I do believe that by-value capture is the correct default, and should
remain as such, but there should be an opt-in for by-reference capture. My
suggestion would be to allow use(&$count) on fn(), which allows capturing
certain variables by reference rather than by value. An alternative that
has previously been discussed is to allow changing the default capture mode
using something like use(&), but I believe it would be better if variables
captured by reference had to be spelled out explicitly.

2. For much the same reason, capture analysis for arrow functions is very
primitive. It basically finds all variables that are used inside the
closure body and tries to import them, silently failing if they are not
available in the outer scope. For multi-line closures, this would commonly
end up importing too many variables. For example:

fn() {
$tmp = foo();
bar($tmp);
return $tmp;
}

Here, there is no need to import $tmp from the outer scope, but a naive
implementation (which I assume you're using) will still try to capture it.
Now, this has two effects:

a) Performance, as we're trying to capture unnecessary variables. This may
be negligible, but it would be good to at least quantify. I would rather
there not be recommendations along the line of "you should use function()
for performance-critical code, because it's faster than fn()".

b) Subtle side-effects, visible in debugging functionality, or through
destructor effects (the fact that a variable is captured may be
observable). I think it nothing else, the RFC should at least make clear
that this behavior is explicitly unspecified, and a future implementation
may no longer capture variables where any path from entry to read passes a
write.

Regards,
Nikita

Rowan Collins

5 years ago

On 28/04/2021 15:31, Nikita Popov wrote:

> My suggestion would be to allow use(&$count) on fn(), which allows capturing
> certain variables by reference rather than by value.

It might be rather confusing for the use block to sometimes mean
"capture exactly these variables", and sometimes mean "capture all
variables, but modify *how* you capture these ones", based on if it
follows "function" or "fn".

In other words, it's not obvious at a glance that these two closures
have completely different effects, and why:

$increment = 1;
$count = 0;
$a = function() use (&$count) { $count += $increment; }
$b = fn() use (&$count) { $count += $increment; }

> I think that extensive experience in other programming languages has shown
> that auto-capture does not hinder readability of code, and I don't see any
> particular reason why the effects in PHP would be different.

The difference between PHP and a lot of those other languages is that
variables are not imported automatically from other scopes.

In PHP, the following global function and closure would not use the same
definition of $foo:

$foo = 42;
function echoFoo { echo $foo; }
$echoFoo = fn() { echo $foo; }

That's not automatically a bad thing, but it is an extra concern that
PHP has which other languages don't.


Regards,

--
Rowan Tommins
[IMSoP]

Nuno Maduro

5 years ago

On Wed, 28 Apr 2021 at 15:31, Nikita Popov <nikita.ppv@gmail.com> wrote:

> On Wed, Mar 24, 2021 at 5:20 PM Larry Garfield <larry@garfieldtech.com>
> wrote:
>
> > Greetings, Internalians.
> >
> > Some months back I proposed an RFC for short functions.
> >
> > https://wiki.php.net/rfc/short-functions
> >
> > After some discussion, I put it on hold to ensure that it was compatible
> > with the other discussion floating about around an alternate syntax for
> > multi-line closures that did auto-capture.  It's important that the
> syntax
> > of both proposals is consistent.
> >
> > Nuno Maduro and i have talked recently and worked out the syntax that is
> > consistent between both proposals.  That takes 'fn' off the table for
> > short-functions, for reasons discussed in both RFCs.
> >
> > To that end, I offer:
> >
> > 1) The updated short-functions RFC:
> > https://wiki.php.net/rfc/short-functions
> >
> > 2) A new RFC, code by Nuno, for auto-capturing multi-statement closures:
> > https://wiki.php.net/rfc/auto-capture-closure
> >
> > These are separate RFCs and at this time we plan to send them to separate
> > votes, but I list them together to demonstrate that they have been
> > developed in a mutually-compatible way.
> >
> > Further details in the auto-capture-closure RFC.
> >
>
> Hey Larry,
>
> I'm generally in favor of supporting auto-capture for multi-line closures.
> I think that extensive experience in other programming languages has shown
> that auto-capture does not hinder readability of code, and I don't see any
> particular reason why the effects in PHP would be different. I also think
> there is value in having a consistent syntax for closures -- currently, you
> have two options, and may need to switch between them as you add or remove
> code.
>
> There are some caveats though, which this RFC should address:
>
> 1. Auto-capture in arrow functions works by-value. This is sufficient for
> the purpose of single-line arrow functions, where you would be hard-pressed
> to perform any meaningful state mutation. For multi-line closures, it
> becomes much more likely that you want to modify a value from the outer
> scope. It should be possible to do this without switching to the function()
> notation.
>
> I do believe that by-value capture is the correct default, and should
> remain as such, but there should be an opt-in for by-reference capture. My
> suggestion would be to allow use(&$count) on fn(), which allows capturing
> certain variables by reference rather than by value. An alternative that
> has previously been discussed is to allow changing the default capture mode
> using something like use(&), but I believe it would be better if variables
> captured by reference had to be spelled out explicitly.
>
> 2. For much the same reason, capture analysis for arrow functions is very
> primitive. It basically finds all variables that are used inside the
> closure body and tries to import them, silently failing if they are not
> available in the outer scope. For multi-line closures, this would commonly
> end up importing too many variables. For example:
>
>     fn() {
>         $tmp = foo();
>         bar($tmp);
>         return $tmp;
>     }
>
> Here, there is no need to import $tmp from the outer scope, but a naive
> implementation (which I assume you're using) will still try to capture it.
> Now, this has two effects:
>
> a) Performance, as we're trying to capture unnecessary variables. This may
> be negligible, but it would be good to at least quantify. I would rather
> there not be recommendations along the line of "you should use function()
> for performance-critical code, because it's faster than fn()".
>
> b) Subtle side-effects, visible in debugging functionality, or through
> destructor effects (the fact that a variable is captured may be
> observable). I think it nothing else, the RFC should at least make clear
> that this behavior is explicitly unspecified, and a future implementation
> may no longer capture variables where any path from entry to read passes a
> write.
>
> Regards,
> Nikita
>

Hey Nikita,

Concerning point 1: Agreed. Going to update both the pull request and the
RFC accordingly.

Concerning point 2: Developing some sort of static analysis that can infer
what needs to be captured may be problematic. First, that won't be
consistent with the way auto-capture currently works in PHP (one-line short
closures). Second, what needs to be captured may be determined at runtime
(thinking about eval, variables names computed at runtime, etc). Of course,
I am willing to hear more thoughts on this.

Regards,
Nuno.

Rowan Collins

5 years ago

On 28/04/2021 23:00, Nuno Maduro wrote:

> Concerning point 2: Developing some sort of static analysis that can infer
> what needs to be captured may be problematic. First, that won't be
> consistent with the way auto-capture currently works in PHP (one-line short
> closures).

The question is, how do we decide which variables are local, and which
are captured? Options include:

1) All variables are captured, except those declared local. This is
common in other languages, but has no syntax in PHP.
2) All variables are local, except those declared captured. This is what
the "use" clause is for in the current "long" closure syntax.
3) All variables are captured, there is no local scope. This is
effectively what PHP's single-expression closures use.
4) All variables are captured, unless provably written before read. This
is what I think Nikita is suggesting.

Option 3 is the easy option, and is currently "good enough" because it's
rare to need a purely local variable within a single expression. For a
full function body, local variables are much more likely, so we now need
to think about the question more carefully.

Without a keyword like "var" to declare it, I think most users would
assume that an initialisation like "$foo=null;" at the top of their
closure would make a variable local.

Regards,

--
Rowan Tommins
[IMSoP]

Christian Schneider

5 years ago

Am 29.04.2021 um 10:14 schrieb Rowan Tommins <rowan.collins@gmail.com>:

> On 28/04/2021 23:00, Nuno Maduro wrote:
> 3) All variables are captured, there is no local scope. This is effectively what PHP's single-expression closures use.
>
> Option 3 is the easy option, and is currently "good enough" because it's rare to need a purely local variable within a single expression. For a full function body, local variables are much more likely, so we now need to think about the question more carefully.
>
> Without a keyword like "var" to declare it, I think most users would assume that an initialisation like "$foo=null;" at the top of their closure would make a variable local.

This assumption by the users would be true as we are capturing by value, not by reference.

At the same time it feels like abusing a side-effect. But adding some sort of explicit local variable declaration seems even worse (as in even less PHP-like) to me.

This illustrates why I'm wary of / opposed to auto-capture for multi-line closures.

- Chris

Rowan Collins

5 years ago

On 29/04/2021 10:25, Christian Schneider wrote:

>> Without a keyword like "var" to declare it, I think most users would assume that an initialisation like "$foo=null;" at the top of their closure would make a variable local.
> This assumption by the users would be true as we are capturing by value, not by reference.

Not entirely. As currently implemented, the value would still be
captured, and so use memory, prevent destructors firing, etc.

Compare 3v4l.org/Utd9j <https://3v4l.org/Utd9j> vs 3v4l.org/fpjbX
<https://3v4l.org/fpjbX>


Regards,

--
Rowan Tommins
[IMSoP]

Nikita Popov

5 years ago

On Thu, Apr 29, 2021 at 12:00 AM Nuno Maduro <enunomaduro@gmail.com> wrote:

> On Wed, 28 Apr 2021 at 15:31, Nikita Popov <nikita.ppv@gmail.com> wrote:
>
>> On Wed, Mar 24, 2021 at 5:20 PM Larry Garfield <larry@garfieldtech.com>
>> wrote:
>>
>> > Greetings, Internalians.
>> >
>> > Some months back I proposed an RFC for short functions.
>> >
>> > https://wiki.php.net/rfc/short-functions
>> >
>> > After some discussion, I put it on hold to ensure that it was compatible
>> > with the other discussion floating about around an alternate syntax for
>> > multi-line closures that did auto-capture.  It's important that the
>> syntax
>> > of both proposals is consistent.
>> >
>> > Nuno Maduro and i have talked recently and worked out the syntax that is
>> > consistent between both proposals.  That takes 'fn' off the table for
>> > short-functions, for reasons discussed in both RFCs.
>> >
>> > To that end, I offer:
>> >
>> > 1) The updated short-functions RFC:
>> > https://wiki.php.net/rfc/short-functions
>> >
>> > 2) A new RFC, code by Nuno, for auto-capturing multi-statement closures:
>> > https://wiki.php.net/rfc/auto-capture-closure
>> >
>> > These are separate RFCs and at this time we plan to send them to
>> separate
>> > votes, but I list them together to demonstrate that they have been
>> > developed in a mutually-compatible way.
>> >
>> > Further details in the auto-capture-closure RFC.
>> >
>>
>> Hey Larry,
>>
>> I'm generally in favor of supporting auto-capture for multi-line closures.
>> I think that extensive experience in other programming languages has shown
>> that auto-capture does not hinder readability of code, and I don't see any
>> particular reason why the effects in PHP would be different. I also think
>> there is value in having a consistent syntax for closures -- currently,
>> you
>> have two options, and may need to switch between them as you add or remove
>> code.
>>
>> There are some caveats though, which this RFC should address:
>>
>> 1. Auto-capture in arrow functions works by-value. This is sufficient for
>> the purpose of single-line arrow functions, where you would be
>> hard-pressed
>> to perform any meaningful state mutation. For multi-line closures, it
>> becomes much more likely that you want to modify a value from the outer
>> scope. It should be possible to do this without switching to the
>> function()
>> notation.
>>
>> I do believe that by-value capture is the correct default, and should
>> remain as such, but there should be an opt-in for by-reference capture. My
>> suggestion would be to allow use(&$count) on fn(), which allows capturing
>> certain variables by reference rather than by value. An alternative that
>> has previously been discussed is to allow changing the default capture
>> mode
>> using something like use(&), but I believe it would be better if variables
>> captured by reference had to be spelled out explicitly.
>>
>> 2. For much the same reason, capture analysis for arrow functions is very
>> primitive. It basically finds all variables that are used inside the
>> closure body and tries to import them, silently failing if they are not
>> available in the outer scope. For multi-line closures, this would commonly
>> end up importing too many variables. For example:
>>
>>     fn() {
>>         $tmp = foo();
>>         bar($tmp);
>>         return $tmp;
>>     }
>>
>> Here, there is no need to import $tmp from the outer scope, but a naive
>> implementation (which I assume you're using) will still try to capture it.
>> Now, this has two effects:
>>
>> a) Performance, as we're trying to capture unnecessary variables. This may
>> be negligible, but it would be good to at least quantify. I would rather
>> there not be recommendations along the line of "you should use function()
>> for performance-critical code, because it's faster than fn()".
>>
>> b) Subtle side-effects, visible in debugging functionality, or through
>> destructor effects (the fact that a variable is captured may be
>> observable). I think it nothing else, the RFC should at least make clear
>> that this behavior is explicitly unspecified, and a future implementation
>> may no longer capture variables where any path from entry to read passes a
>> write.
>>
>> Regards,
>> Nikita
>>
>
> Hey Nikita,
>
> Concerning point 1: Agreed. Going to update both the pull request and the
> RFC accordingly.
>
> Concerning point 2: Developing some sort of static analysis that can infer
> what needs to be captured may be problematic. First, that won't be
> consistent with the way auto-capture currently works in PHP (one-line short
> closures). Second, what needs to be captured may be determined at runtime
> (thinking about eval, variables names computed at runtime, etc). Of course,
> I am willing to hear more thoughts on this.
>

Just to clarify: The way arrow functions *currently* work, capture analysis
is performed statically. Only variables that are used directly in the arrow
function will be captured. Variables used indirectly via include, eval,
variable-variables, compact() etc are not captured. It would be possible to
support this, but it isn't supported right now.

Regards,
Nikita