Everything

php

Home » Project » Newsgroups » php.internals » Proposal: Adding functions any(iterable $input, ?callable $cb = null, int $use_flags=0) and all(...) »

Proposal: Adding functions any(iterable $input, ?callable $cb = null, int $use_flags=0) and all(...)

php.internals

tyson andre

6 years ago

Hi internals,

The primitives any() and all() are a common part of many programming languages and help in avoiding verbosity or unnecessary abstractions.

- https://hackage.haskell.org/package/base-4.14.0.0/docs/Prelude.html#v:any
- https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/some
- https://docs.python.org/3/library/functions.html#all
- https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html#allMatch-java.util.function.Predicate-

For example, the following code could be shortened significantly

```
// Old version
$satisifes_predicate = false;
foreach ($items as $item) {
    if (API::satisfiesCondition($item)) {
        $satisfies_predicate = true;
        break;
    }
}
if (!$satisfies_predicate) {
    throw new APIException("No matches found");
}

// New version is much shorter and readable
if (!any($items, fn($x) => API::satisfiesCondition($x))) {
    throw new APIException("No matches found");
}
```

That example doesn't have any suitable helpers already in the standard library.
Using array_filter would unnecessarily call satisfiesCondition even after the first item was found,
and array_search doesn't take a callback.

A proposed implementation is https://github.com/php/php-src/pull/6053 - it takes similar flags and param orders to array_filter().

Previous discussion was in https://externals.io/message/103357#103373

- New contributors to projects wouldn't know about any() and all() if it was reimplemented with different semantics and only occasionally used
   (e.g. MyArrayUtil::any()) in various projects)
- If this was provided only in userland, there'd be low adoption and code such as the first example would remain common.
   If the standard library provided it, then polyfills would as well, making cleaner code easier to write.

Thanks,
- Tyson

Alex Dowad

6 years ago

I like it!

What is the $use_flags parameter for?

On Sat, Aug 29, 2020 at 10:24 PM tyson andre <tysonandre775@hotmail.com>
wrote:

Josh Bruce

6 years ago

Hello!

I’m still finding my understanding of actual internals, so can only comment from a PHP developer perspective.

The “any” check is just to if anything in the itrerable passes the predicate, yeah??

What I find myself doing more often is wanting the first thing to satisfy the predicate - a “first” function, if you will.

This could skip the step of iterating to find of something satisfies the predicate. Then iterating again to get one or more items that do satisfy it.

Trying to think of a use case where I would want to check, but not do anything with that knowledge.

Again, might not be completely following.

Cheers,
Josh

tyson andre

6 years ago

Josh Bruce

6 years ago

Think I’m following.

The compelling feature here for me is the idea of an “array walker” that can be broken out of.

all() could be and() at which point this should be equivalent.

$collection = [1, 2, 3];

If (count($collection) === count(array_filter($collection, “is_int”))

any() could be or() at which point this should be equivalent, short of the breaking early.

If (count(array_filter($collection, “is_int”)) > 0)

The benefit I see of the first (using count === count) is that I could store the result of the filter to use later based on the passing of the condition without iterating again. (See str_replace, where the function returns the result, while making count available to the caller as well: https://www.php.net/manual/en/function.str-replace.php <https://www.php.net/manual/en/function.str-replace.php> - using pass by reference.)

The drawback of the second (using count > 0) is that I have to iterate over the entire collection despite only needing one element to pass the predicate.

Would there also be an xor() or equivalent?? one() or something??

If (count(array_filter($collection, “is_int”)) === 1)

Here’s an implementation of each() I made that allows for breaking by the caller using a passed by ref third argument in the closure: https://github.com/8fold/php-shoop/blob/2a8b6fc41c545ff71690562ac2f35a12457b1514/src/Traits/EachImp.php <https://github.com/8fold/php-shoop/blob/2a8b6fc41c545ff71690562ac2f35a12457b1514/src/Traits/EachImp.php> - I’ve since deprecated that functionality and not sure if I’ll be bringing that functionality back because of frustrations and philosophical arguments with myself. :)

Part of me wishes the “all()” and “any()” names were more descriptive of what’s going on.

When I call for “all” in a database or some ORM, I *get* all - I’m not verifying “all pass [a predicate]”.

Cheers,
Josh

tyson andre

6 years ago

Hi Josh Bruce,

> The compelling feature here for me is the idea of an “array walker” that can be broken out of. 
>
> all() could be and() at which point this should be equivalent.

Yes, you can write `all($set, $predicate_fn)` as `!any($set, fn($x) => !$predicate_fn($x))` or some other common primitive such as and(),
but as an end user or a reviewer of code, I really don't want to do the latter, which is why I proposed adding both of these. (also see below snippet)

> Would there also be an xor() or equivalent?? one() or something??
> 
> If (count(array_filter($collection, “is_int”)) === 1)

I don't plan to expand the scope of the RFC more - there are various features such as none() or `exactly_n(int $n, $iterable, $callback)` I'm also not proposing.
RFC votes require a 2/3 majority and I'd think more would advocate doing one() in userland than any()

> Here’s an implementation of each() I made that allows for breaking by the caller using a passed by ref third argument in the closure: https://github.com/8fold/php-shoop/blob/2a8b6fc41c545ff71690562ac2f35a12457b1514/src/Traits/EachImp.php - I’ve since deprecated that functionality and not sure if I’ll be bringing that functionality back because of frustrations and philosophical arguments with myself. :)

Modifying a reference instead could be done to achieve the same result
as setting a reference boolean in the cases where you needed to know what element it was.

```
if (any($values, function ($v) use (&$result) {
        if ($v->satisfiesPredicate()) {$result = $v; return true;} return false; })
) {
    process($result);
}
```

> Part of me wishes the “all()” and “any()” names were more descriptive of what’s going on. 
> 
> When I call for “all” in a database or some ORM, I *get* all - I’m not verifying “all pass [a predicate]”.

The naming is based on mathematical notation.
- any($list, $predicate) "Determines whether any element of the iterable satisfies the predicate."
- all($list, $predicate) "Determines whether all elements of the iterable satisfy the predicate."

every() and some() are alternative names,
but all()/any() are my preference and appear more commonly used elsewhere.

- https://hackage.haskell.org/package/base-4.14.0.0/docs/Prelude.html#v:any
- https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/some
- https://docs.python.org/3/library/functions.html#all
- https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html#allMatch-java.util.function.Predicate-

Thanks,
- Tyson

Josh Bruce

6 years ago

Thanks.

Re the name, I see that.

I think part of the approachability of PHP is in the language being more common - though not being consistent in use of math, computer science, or common can be frustrating as all.

join() (math) -> alias of implode() (cheeky common??) -> could be array_as_string() (descriptive common) -> or concatenate_strings() (descriptive CS) -> or sum() (math)

I tend toward the descriptive common for self-documenting code purposes. I don’t need to know the implementation behind something like array_as_string() or concatenate_strings() - I do for join() (less) or implode() (more). sum(), for this example, would be very confusing.

Naming things is hard. lol

Cheers,
Josh

ps. I don’t know the origins for some of most of the functions in the standard library, but it is fascinating from a communication perspective.

tyson andre

6 years ago

Hi Alex,

> I like it! 
>
> What is the $use_flags parameter for?

It's for deciding what parameters to pass to callback, like https://www.php.net/array_filter - the reflection name should probably just be $flag

Occasionally, the key might be useful to check against, e.g. the identifier of an item in a map, or SplObjectStorage.

- the default is `any($iterable, fn ($value) => ...)`
- or `any($iterable, fn ($key) => expr, ARRAY_FILTER_USE_KEY)`
- or `any($iterable, fn ($value, $key) => expr, ARRAY_FILTER_USE_BOTH)`

Thanks,
- Tyson

Mike Schinkel

6 years ago

> On Aug 29, 2020, at 7:18 PM, tyson andre <tysonandre775@hotmail.com> wrote:
>
> Hi Alex,
>
>> I like it!
>>
>> What is the $use_flags parameter for?
>
> It's for deciding what parameters to pass to callback, like https://www.php.net/array_filter - the reflection name should probably just be $flag
>
> Occasionally, the key might be useful to check against, e.g. the identifier of an item in a map, or SplObjectStorage.
>
> - the default is `any($iterable, fn ($value) => ...)`
> - or `any($iterable, fn ($key) => expr, ARRAY_FILTER_USE_KEY)`
> - or `any($iterable, fn ($value, $key) => expr, ARRAY_FILTER_USE_BOTH)`

Doesn't the occasional required use of long constant names like ARRAY_FILTER_USE_KEY and ARRAY_FILTER_USE_BOTH somewhat negative the benefit of a more concise syntax?

I know that many existing functions in PHP have a collection of long constant names that can be passed, but is that a style of programming that PHP really wants to continue to proliferate, or would it not be better to address that use-case in a different manner?   (Admittedly doing so might require having this feature dependent on other critical path language features.)

One approach might be if PHP could allow for skipped parameters in callables — where I am using a single underscore to indicate an unused parameter (GoLang has a "blank identifier" for similar purposes so there is some prior art to consider) — e.g:

- the default is `any($iterable, fn ( $value ) => ...)`
- or `any($iterable, fn ( _, $key ) => expr )`
- or `any($iterable, fn ( $value, $key ) => expr )`

PHP could handle it more generically, is possible, or recognize this and branch to different implementations of any() and all() if the goal is to optimize performance by only passing one value in key-only or value-only calls.

Optionally we could use null instead:

- the default is `any($iterable, fn ($value) => ...)`
- or `any($iterable, fn (null, $key) => expr )`
- or `any($iterable, fn ($value, $key) => expr )`

-Mike

[1] https://www.geeksforgeeks.org/what-is-blank-identifierunderscore-in-golang/#:~:text=_(underscore)%20in%20Golang%20is,Blank%20Identifier. <https://www.geeksforgeeks.org/what-is-blank-identifierunderscore-in-golang/#:~:text=_(underscore)%20in%20Golang%20is,Blank%20Identifier.>

Josh Bruce

6 years ago

>>
>> - the default is `any($iterable, fn ($value) => ...)`
>> - or `any($iterable, fn ($key) => expr, ARRAY_FILTER_USE_KEY)`
>> - or `any($iterable, fn ($value, $key) => expr, ARRAY_FILTER_USE_BOTH)`
>
> Doesn't the occasional required use of long constant names like ARRAY_FILTER_USE_KEY and ARRAY_FILTER_USE_BOTH somewhat negative the benefit of a more concise syntax?

For what it’s worth, the constants do give at least two benefits:

1. Flexibility for adding functionality without signature change.
2. Readability at the call site even if the output isn’t different in most cases.

example(true, true, false);

example(USE_BLUE, DROP_FIRST...

With names arguments in 8 we do get helped, I think.

mb_case_switch had 3 flags in the beginning, now gas 6 - if I’m remembering and interpreting the docs correctly.

Cheers,
Josh

tyson andre

6 years ago

Hi Mike Schinkel,

> Doesn't the occasional required use of long constant names like ARRAY_FILTER_USE_KEY and ARRAY_FILTER_USE_BOTH somewhat negative the benefit of a more concise syntax?  
> 
> I know that many existing functions in PHP have a collection of long constant names that can be passed, but is that a style of programming that PHP really wants to continue to proliferate, or would it not be better to address that use-case in a different manner?   (Admittedly doing so might require having this feature dependent on other critical path language features.)
> 
> One approach might be if PHP could allow for skipped parameters in callables — where I am using a single underscore to indicate an unused parameter (GoLang has a "blank identifier" for similar purposes so there is some prior art to consider) — e.g:
> 
> - the default is `any($iterable, fn ( $value ) => ...)`
> - or `any($iterable, fn ( _, $key ) => expr )`
> - or `any($iterable, fn ( $value, $key ) => expr )`
>
> PHP could handle it more generically, is possible, or recognize this and branch to different implementations of any() and all() if the goal is to optimize performance by only passing one value in key-only or value-only calls.
> 
> Optionally we could use null instead:
> 
> - the default is `any($iterable, fn ($value) => ...)`
> - or `any($iterable, fn (null, $key) => expr )`
> - or `any($iterable, fn ($value, $key) => expr )`

Supporting "blank identifiers" is out of the scope of this RFC,
and would have some implementation concerns of how it'd work with named parameters
and how backtrace generation would be affected. I don't think there'd be a significant performance improvement over not using parameters.

I'm taking $flag out of the rfc because there's many proposed alternatives for doing this,
and checking values of an iterable is the most common use case.

Repeating the response to Marco Pivetta,
I find that checking parameter counts would be inconsistent with how other internal functions accepting callbacks such as `array_filter` already behave.
It would also be unexpected if users or tooling wrap callbacks with other callbacks.
Developers would also expect `func_get_args()` to work.

```
<?php
function my_none(iterable $values, callable $predicate) {
    return all($values, fn(...$args) => $predicate(...$args));
}
// PHP can't distinguish between these when given `fn(...$args)`
// - would throw if passed 1 parameter
my_none($values, fn($value, $key) => $value == true || $key == 'x');
// - would throw if passed 2 parameters (too many)
my_none($values, 'is_string');
```

Cheers,
- Tyson

Mike Schinkel

6 years ago

> On Aug 30, 2020, at 12:31 PM, tyson andre <tysonandre775@hotmail.com> wrote:
>
> Hi Mike Schinkel,
>
>> Doesn't the occasional required use of long constant names like ARRAY_FILTER_USE_KEY and ARRAY_FILTER_USE_BOTH somewhat negative the benefit of a more concise syntax?
>>
>> I know that many existing functions in PHP have a collection of long constant names that can be passed, but is that a style of programming that PHP really wants to continue to proliferate, or would it not be better to address that use-case in a different manner?   (Admittedly doing so might require having this feature dependent on other critical path language features.)
>>
>> One approach might be if PHP could allow for skipped parameters in callables — where I am using a single underscore to indicate an unused parameter (GoLang has a "blank identifier" for similar purposes so there is some prior art to consider) — e.g:
>>
>> - the default is `any($iterable, fn ( $value ) => ...)`
>> - or `any($iterable, fn ( _, $key ) => expr )`
>> - or `any($iterable, fn ( $value, $key ) => expr )`
>>
>> PHP could handle it more generically, is possible, or recognize this and branch to different implementations of any() and all() if the goal is to optimize performance by only passing one value in key-only or value-only calls.
>>
>> Optionally we could use null instead:
>>
>> - the default is `any($iterable, fn ($value) => ...)`
>> - or `any($iterable, fn (null, $key) => expr )`
>> - or `any($iterable, fn ($value, $key) => expr )`
>
> Supporting "blank identifiers" is out of the scope of this RFC,
> and would have some implementation concerns of how it'd work with named parameters
> and how backtrace generation would be affected. I don't think there'd be a significant performance improvement over not using parameters.
>
> I'm taking $flag out of the rfc because there's many proposed alternatives for doing this,
> and checking values of an iterable is the most common use case.

Acknowledged.

If not a significant performance improvement why not pass both value and key to the callable in your RFC in case someone needs access to the key?

Adding the key as a second parameter would not preclude passing a flag to modify behavior in a future version of PHP if that is the internals consensus.

-Mike

P.S. John Bafford recently suggested the use of void in foreach loops for essentially the same reason as I proposed with underscore or null (and frankly void is better than underscore or null):

https://externals.io/message/111774 <https://externals.io/message/111774>

If his RFC were to pass then that would open the door to using it here, right?

Marco Pivetta

6 years ago

Hey Tyson,
<http://ocramius.github.com/>


On Sat, Aug 29, 2020 at 10:24 PM tyson andre <tysonandre775@hotmail.com>
wrote:

> Hi internals,
>
> The primitives any() and all() are a common part of many programming
> languages and help in avoiding verbosity or unnecessary abstractions.
>
> -
> https://hackage.haskell.org/package/base-4.14.0.0/docs/Prelude.html#v:any
> -
> https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/some
> - https://docs.python.org/3/library/functions.html#all
> -
> https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html#allMatch-java.util.function.Predicate-
>
> For example, the following code could be shortened significantly
>
> ```
> // Old version
> $satisifes_predicate = false;
> foreach ($items as $item) {
>     if (API::satisfiesCondition($item)) {
>         $satisfies_predicate = true;
>         break;
>     }
> }
> if (!$satisfies_predicate) {
>     throw new APIException("No matches found");
> }
>
> // New version is much shorter and readable
> if (!any($items, fn($x) => API::satisfiesCondition($x))) {
>     throw new APIException("No matches found");
> }
> ```
>
> That example doesn't have any suitable helpers already in the standard
> library.
> Using array_filter would unnecessarily call satisfiesCondition even after
> the first item was found,
> and array_search doesn't take a callback.
>
> A proposed implementation is https://github.com/php/php-src/pull/6053 -
> it takes similar flags and param orders to array_filter().
>
> Previous discussion was in https://externals.io/message/103357#103373
>
> - New contributors to projects wouldn't know about any() and all() if it
> was reimplemented with different semantics and only occasionally used
>    (e.g. MyArrayUtil::any()) in various projects)
> - If this was provided only in userland, there'd be low adoption and code
> such as the first example would remain common.
>    If the standard library provided it, then polyfills would as well,
> making cleaner code easier to write.
>
> Thanks,
> - Tyson
>

Would it make sense, instead of having a third boolean parameter (causing
two parameters to be coupled together - already quite messy with existing
array functions) for `any()` and `all()` to just detect if the given
callback requires >1 parameter?

That would make this much simpler.

Marco Pivetta

http://twitter.com/Ocramius

http://ocramius.github.com/

tyson andre

6 years ago

Hi Marco Pivetta,

> Would it make sense, instead of having a third boolean parameter (causing two parameters to be coupled together - already quite messy with existing array functions) for `any()` and `all()` to just detect if the given callback requires >1 parameter?
> 
> That would make this much simpler.

I find that to be inconsistent with how other internal functions accepting callbacks such as `array_filter` behave.
It would also be unexpected if users or tooling wrap callbacks with other callbacks.
Developers would also expect `func_get_args()` to work.

```
<?php
function my_none(iterable $values, callable $predicate) {
    return all($values, fn(...$args) => $predicate(...$args));
}
// PHP can't distinguish between these when given `fn(...$args)`
// - would throw if passed 1 parameter
my_none($values, fn($value, $key) => $value == true || $key == 'x');
// - would throw if passed 2 parameters (too many)
my_none($values, 'is_string');
```

Thanks,
- Tyson

Larry Garfield

6 years ago

On Sat, Aug 29, 2020, at 3:24 PM, tyson andre wrote:

> Hi internals,
>
> The primitives any() and all() are a common part of many programming
> languages and help in avoiding verbosity or unnecessary abstractions.
>
> -
> https://hackage.haskell.org/package/base-4.14.0.0/docs/Prelude.html#v:any
> -
> https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/some
> - https://docs.python.org/3/library/functions.html#all
> -
> https://docs.oracle.com/javase/8/docs/api/java/util/stream/Stream.html#allMatch-java.util.function.Predicate-
>
> For example, the following code could be shortened significantly
>
> ```
> // Old version
> $satisifes_predicate = false;
> foreach ($items as $item) {
>     if (API::satisfiesCondition($item)) {
>         $satisfies_predicate = true;
>         break;
>     }
> }
> if (!$satisfies_predicate) {
>     throw new APIException("No matches found");
> }
>
> // New version is much shorter and readable
> if (!any($items, fn($x) => API::satisfiesCondition($x))) {
>     throw new APIException("No matches found");
> }
> ```
>
> That example doesn't have any suitable helpers already in the standard
> library.
> Using array_filter would unnecessarily call satisfiesCondition even
> after the first item was found,
> and array_search doesn't take a callback.
>
> A proposed implementation is https://github.com/php/php-src/pull/6053 -
> it takes similar flags and param orders to array_filter().
>
> Previous discussion was in https://externals.io/message/103357#103373
>
> - New contributors to projects wouldn't know about any() and all() if
> it was reimplemented with different semantics and only occasionally used
>    (e.g. MyArrayUtil::any()) in various projects)
> - If this was provided only in userland, there'd be low adoption and
> code such as the first example would remain common.
>    If the standard library provided it, then polyfills would as well,
> making cleaner code easier to write.
>
> Thanks,
> - Tyson

I like this, but I do not like the flags.  I don't think they're at all useful.  A lot of the other discussion in the thread seems to be needlessly complicating it, too.

all() and any() only need return booleans.  Their callbacks only need return booleans.  That's the point.  first() makes sense to add, and it would return the first value that matches.

For the callback itself, there is work to, hopefully, add partial function application to 8.1.  (No idea if it will be successful, but the effort is in progress.)  If so, the upshot is that turning an arbitrary function into a single-parameter function becomes silly easy, which means functions like this can just expect a single parameter callback and be done with it.  No need for extra-args or flags or whatnot.

If you want to check the keys of an array, call array_keys() first and use that.

if (any(array_keys($foo), fn($k) => $k %2)) { ... }

all(), any(), and first() all sound like good things to include, but let's not over-complicate them.  We can do better today than we could in 1999...

--Larry Garfield

tyson andre

6 years ago

Weirdan

6 years ago

On Sun, Aug 30, 2020 at 6:13 PM tyson andre <tysonandre775@hotmail.com>
wrote:

> > I like this, but I do not like the flags.  I don't think they're at all
> useful.  A lot of the other discussion in the thread seems to be needlessly
> complicating it, too.
> >
> > all() and any() only need return booleans.  Their callbacks only need
> return booleans.  That's the point.  first() makes sense to add, and it
> would return the first value that matches.
>
> What would first() return on failure? Would it throw (inefficient)?
> Would it set an optional output reference to distinguish between returning
> the value null from an iterable and the null from no matches?
>

If it took the default value as well it could return that. While it's
useful in itself it also would enable you to pass a marker object and check
the identity of that to know if no matches have been found:

$none = new stdClass;
$element = first($collection, fn($elt) => ...);
if ($element === $none) {
// nothing found
}

--
Best regards,
Bruce Weirdan                                     mailto:
weirdan@gmail.com

Weirdan

6 years ago

> On Sun, Aug 30, 2020 at 6:13 PM tyson andre <tysonandre775@hotmail.com>
> wrote:
>
>> > I like this, but I do not like the flags.  I don't think they're at all
>> useful.  A lot of the other discussion in the thread seems to be needlessly
>> complicating it, too.
>> >
>> > all() and any() only need return booleans.  Their callbacks only need
>> return booleans.  That's the point.  first() makes sense to add, and it
>> would return the first value that matches.
>>
>> What would first() return on failure? Would it throw (inefficient)?
>> Would it set an optional output reference to distinguish between
>> returning the value null from an iterable and the null from no matches?
>>
>
> If it took the default value as well it could return that. While it's
> useful in itself it also would enable you to pass a marker object and check
> the identity of that to know if no matches have been found:
>
> $none = new stdClass;
> $element = first($collection, fn($elt) => ...);
> if ($element === $none) {
>    // nothing found
> }
>

Of course it should have been `$element = first($collection, fn($elt)
=>..., $none);`

--
Best regards,
Bruce Weirdan                                     mailto:
weirdan@gmail.com

tyson andre

6 years ago

Hi Bruce Weirdan,

> If it took the default value as well it could return that. While it's useful in itself it also would enable you to pass > a marker object and check the identity of that to know if no matches have been found:
> 
> $none = new stdClass;
> $element = first($collection, fn($elt) => ..., $none);
> if ($element === $none) {
>    // nothing found
> }

Oh, right, I'd forgotten about that option - that would work.

There's still the question of what it would do without a predicate.
Making the predicate mandatory in the initial proposal would help avoid the confusion of whether `first() should behave like array_filter() and any() and filter for the first truthy value.

Calling it [iterable_]search_callback() or first_matching() or find() might help distinguish this from the reset()/end()/next()/prev() family of functions - not happy with any of my naming ideas

Thanks,
- Tyson

Dik Takken

6 years ago

On 29-08-2020 22:24, tyson andre wrote:

> Hi internals,
>
> The primitives any() and all() are a common part of many programming languages and help in avoiding verbosity or unnecessary abstractions.

I would love to see this come to PHP. I also do a lot of Python
development and I really like its operators module, which provides
function equivalents to the intrinsic Python operators. Have a look:

https://docs.python.org/3/library/operator.html

Although the any() and all() functions are my favorites, having the full
range of operator functions would be great. That could ultimately yield
a proper counterpart to the range of array_* functions which support any
iterable in stead of just arrays. Think of a keys() function that is a
generalization of array_keys().

This reminds me of the iter library that Nikita created:

https://github.com/nikic/iter

So yes, this can also be done in user space. Having it built into the
language has advantages though:

* High adoption rates, making lots of existing PHP code more concise
* Possibly better performance

Regarding performance: Since we already have opcodes for handling
operators, it may be possible to extend these to consume arbitrary
numbers of operands from iterables. Then, an operator function like
any() would compile into its equivalent operator opcode. Finally, that
opcode can probably be JIT compiled into a tight loop in machine code.

So a +1 from me for adding any() and all() but let us also consider the
general problem of lacking iterable support in PHP. Perhaps that could
be the subject of a followup RFC.

Regards,
Dik Takken

tyson andre

6 years ago

Hi Dik Takken,

> I would love to see this come to PHP. I also do a lot of Python
> development and I really like its operators module, which provides
> function equivalents to the intrinsic Python operators. Have a look:
> 
> https://docs.python.org/3/library/operator.html
> 
> Although the any() and all() functions are my favorites, having the full
> range of operator functions would be great. That could ultimately yield
> a proper counterpart to the range of array_* functions which support any
> iterable in stead of just arrays. Think of a keys() function that is a
> generalization of array_keys().

Operator overloading was the subject of https://wiki.php.net/rfc/userspace_operator_overloading#vote
which had a 38-28 vote, which didn't meet the 2/3 voting threshold.

> This reminds me of the iter library that Nikita created:
> 
> https://github.com/nikic/iter
> 
> So yes, this can also be done in user space. Having it built into the
language has advantages though:
> 
> * High adoption rates, making lots of existing PHP code more concise
> * Possibly better performance

Strangely, if I remember correctly, I've had better performance looping inline and calling closures than calling array_map
when there were a large number of elements. I think opcache's inferences and the fact there's no switch
from internals back to the php vm was part of the reason for it.
(and the tracking of stack frames?)

The readability's my main reason for making it native. I'd wondered if there'd be a benefit to preloading where we replace C functions with native php functions (optionally JITted) if there's an expected benefit,
like HHVM does for parts of its standard library, but again out of scope.

> Regarding performance: Since we already have opcodes for handling
> operators, it may be possible to extend these to consume arbitrary
> numbers of operands from iterables. Then, an operator function like
> any() would compile into its equivalent operator opcode. Finally, that
> opcode can probably be JIT compiled into a tight loop in machine code.

That's definitely something I'd want to see for array_map/array_filter.
There's the question of whether it'd be reasonable to omit stack frames or whether it's practical to create fake frames with parameter values
for debug_backtrace().

Larry Garfield also did some work on python-like list comprehensions,
though it's still in draft.
Automatically rewriting array_map(), array_filter(), all(), etc. to list comprehensions and then to JITted code
could see a nice performance benefit in benefit, but this is out of scope of the RFC I'm working on
See https://externals.io/message/104637

> So a +1 from me for adding any() and all() but let us also consider the
> general problem of lacking iterable support in PHP. Perhaps that could
> be the subject of a followup RFC.

Part of the issue is what the return type would be.
For example, reduce(iterable, callable, $initial = null) could work well, but what should map() do

`map(callable, iterable $values): iterable` - should it preserve keys, should it return an array|Generator, array|CustomTraversable, something depending on the object type, etc.
(Generator performance seems like it'd be worse than eager evaluation)

Regards,
Tyson

Larry Garfield

6 years ago

On Sun, Aug 30, 2020, at 3:43 PM, tyson andre wrote:

> Hi Dik Takken,
>
> > I would love to see this come to PHP. I also do a lot of Python
> > development and I really like its operators module, which provides
> > function equivalents to the intrinsic Python operators. Have a look:
> >
> > https://docs.python.org/3/library/operator.html
> >
> > Although the any() and all() functions are my favorites, having the full
> > range of operator functions would be great. That could ultimately yield
> > a proper counterpart to the range of array_* functions which support any
> > iterable in stead of just arrays. Think of a keys() function that is a
> > generalization of array_keys().
>
> Operator overloading was the subject of
> https://wiki.php.net/rfc/userspace_operator_overloading#vote
> which had a 38-28 vote, which didn't meet the 2/3 voting threshold.
>
> > This reminds me of the iter library that Nikita created:
> >
> > https://github.com/nikic/iter
> >
> > So yes, this can also be done in user space. Having it built into the
> language has advantages though:
> >
> > * High adoption rates, making lots of existing PHP code more concise
> > * Possibly better performance
>
> Strangely, if I remember correctly, I've had better performance looping
> inline and calling closures than calling array_map
> when there were a large number of elements. I think opcache's
> inferences and the fact there's no switch
> from internals back to the php vm was part of the reason for it.
> (and the tracking of stack frames?)
>
> The readability's my main reason for making it native. I'd wondered if
> there'd be a benefit to preloading where we replace C functions with
> native php functions (optionally JITted) if there's an expected benefit,
> like HHVM does for parts of its standard library, but again out of
> scope.
>
> > Regarding performance: Since we already have opcodes for handling
> > operators, it may be possible to extend these to consume arbitrary
> > numbers of operands from iterables. Then, an operator function like
> > any() would compile into its equivalent operator opcode. Finally, that
> > opcode can probably be JIT compiled into a tight loop in machine code.
>
> That's definitely something I'd want to see for array_map/array_filter.
> There's the question of whether it'd be reasonable to omit stack frames
> or whether it's practical to create fake frames with parameter values
> for debug_backtrace().
>
> Larry Garfield also did some work on python-like list comprehensions,
> though it's still in draft.
> Automatically rewriting array_map(), array_filter(), all(), etc. to
> list comprehensions and then to JITted code
> could see a nice performance benefit in benefit, but this is out of
> scope of the RFC I'm working on
> See https://externals.io/message/104637

I was just considering mentioning that. :-)  I still would like to see comprehensions happen, but I'd need to partner with someone with more engine skill than I to actually implement it, plus the reception was somewhat lukewarm.  I do think they're useful and would help with any/all/first type use cases, though.

If someone reading this is interested, you know where to find me...

--Larry Garfield

Unnamed Person

6 years ago

I was actually working on this sort of thing recently. _Technically_,
you can support `all`, `any`, and `first` by using a single function:

function find_first(iterable $of, callable($value, $key): bool
$thatSatistifes): Iterator

It converts the $iterable into an Iterator, then calls the callback
for each key/value pair until one returns true, and then always
returns the iterator at the current position.

1. This allows you to know both key and value when making a decision.
2. By returning an iterator the caller can get both key and value.
3. By returning an iterator it can handle both the empty case and not
found cases with $result->valid() === false.
4. By returning an iterator it might be useful for processing the
remainder of the list somehow.

I'm not sure that in practice it would be that friendly, but it's
worth pointing out for discussion at least.

tyson andre

6 years ago

Hi Levi Morrison,

> I was actually working on this sort of thing recently. _Technically_,
> you can support `all`, `any`, and `first` by using a single function:
> 
>     function find_first(iterable $of, callable($value, $key): bool
$thatSatistifes): Iterator
> 
> It converts the $iterable into an Iterator, then calls the callback
> for each key/value pair until one returns true, and then always
> returns the iterator at the current position.
>
> I'm not sure that in practice it would be that friendly, but it's
> worth pointing out for discussion at least.

I'd find the `find($iterable, $callable, $default = null): mixed` to be more user-friendly,
personally, and would worry about accidentally increasing memory usage if I left one of those iterables around as a saved value or a long-lived local variable.

There's plenty of time until the php 8.1 feature freeze, so I don't feel like increasing the scope of this RFC
(https://wiki.php.net/rfc/any_all_on_iterable)
even though I would find some version of `find()` with a callback useful.

Thanks,
- Tyson