[Concept] Flip relative function lookup order (global, then local)

Rob Landers

2 years ago

On Fri, Aug 2, 2024, at 18:51, Ilija Tovilo wrote:

> Hi everyone
>
> As you probably know, a common performance optimization in PHP is to
> prefix global function calls in namespaced code with a `\`. In
> namespaced code, relative function calls (meaning, not prefixed with
> `\`, not imported and not containing multiple namespace components)
> will be looked up in the current namespace before falling back to the
> global namespace. Prefixing the function name with `\` disambiguates
> the called function by always picking the global function.
>
> Not knowing exactly which function is called at compile time has a
> couple of downsides to this:
>
> * It leads to the aforementioned double-lookup.
> * It prevents compile-time-evaluation of pure internal functions.
> * It prevents compiling to specialized  opcodes for specialized
> internal functions (e.g. strlen()).
> * It requires branching for frameless functions [1].
> * It prevents an optimization that looks up internal functions by
> offset rather than by name [2].
> * It prevents compiling to more specialized argument sending opcodes
> because of unknown by-value/by-reference passing.
>
> All of these are enabled by disambiguating the call. Unfortunately,
> prefixing all calls with `\`, or adding a `use function` at the top of
> every file is annoying and noisy. We recently got a feature request to
> change how functions are looked up [3]. The approach that appears to
> cause the smallest backwards incompatibility is to flip the order in
> which functions are looked up: Check in global scope first, and only
> then in local scope. With this approach, if we can find a global
> function at compile-time, we know this is the function that will be
> picked at run-time, hence automatically enabling the optimizations
> above. I created a PoC implementing this approach [4].
>
> Máté has kindly benchmarked the patch, measuring an improvement of
> ~3.9% for Laravel, and ~2.1% for Symfony
> (https://gist.github.com/kocsismate/75be09bf6011630ebd40a478682d6c17).
> This seems quite significant, given that no changes were required in
> either of these two codebases.

So, what you’re saying is that symfony and laravel can get a performance increase by simply adding a \ in the right places? Why don’t they do that instead of changing the language?

>
> There are a few noteworthy downsides:
>
> * Unqualified calls to functions in the same namespace would be
> slightly slower, because they now involve checking global scope first.
> I believe that unqualified, global calls are much more common, so this
> change should still result in a net positive. It's also possible to
> avoid this cost by adding a `use function` to the top of the file.

For functions/classes in the same exact namespace, you don’t need a use statement. But after this change, you do in certain cases?

namespace Foo;

function array_sum($bar) {}

function baz($bar) {
return array_sum($bar);
}

So, how do you use that function in the same file?

> * Introducing new functions in the global namespace could cause a BC
> break for unqualified calls, if the function happens to have the same
> name. This is unfortunate, but likely rare. Since new functions are
> only introduced in minor/major versions, this should be manageable,
> but must be considered for every PHP upgrade.

We can only see open source code when doing impact analysis. This means picking even a slightly “popular” name could go very poorly.

> * Some mocking libraries (e.g. Symfony's ClockMock [5]) intentionally
> declare functions called from some file in the files namespace to
> intercept these calls. This use-case would break. That said, it is
> somewhat of a fragile approach to begin with, given that it wouldn't
> work for fully qualified calls, or unnamespaced code.

See above. I’ve seen this “trick” used on many closed source projects. I’ve also seen it used when PHP has a bug and the workaround is to implement it in php like this.

>
> I performed a small impact analysis [6]. There are 484 namespaced
> functions shadowing global, internal functions in the top 1000
> composer packages. However, the vast majority (464) of these functions
> come from thecodingmachine/safe, whose entire purpose is offering
> safer wrappers around internal functions. Excluding this library,
> there are only 20 shadowing functions, which is surprisingly little.
> Furthermore, the patch would have no impact on users of
> thecodingmachine/safe, only on the library code itself.
>
> As for providing a migration path: One approach might be to introduce
> an INI setting that performs the function lookup in both local and
> global scope at run-time, and informs the user about the behavioral
> change in the future. To mitigate it, an explicit `use function` would
> need to be added to the top of the file, or the call would need to be
> prefixed with `namespace\`. The impact analysis [6] also provides a
> script that looks for shadowing functions in your project. It does not
> identify uses of these functions (yet), just their declarations.
>
> Lastly, I've already raised this idea in the PHP Foundations internal
> chat but did not receive much positive feedback, mostly due to fear of
> the potential BC impact. I'm not particularly convinced this is an
> issue, given the impact analysis. Given the surprisingly large
> performance benefits, I was inclined to raise it here anyway. It also
> sparked some related ideas, like providing modules that lock
> namespaces and optimize multiple files as a singular unit. That said,
> such approaches would likely be significantly more complex than the
> approach proposed here (~30 lines of C code).
>
> Anyway, please let me know about possible concerns, broken use-cases,
> or any alternative approaches that may come to mind. I'm looking
> forward to your feedback.
>
> Ilija
>
> [1] https://github.com/php/php-src/pull/12461
> [2] https://github.com/php/php-src/pull/13634
> [3] https://github.com/php/php-src/issues/13632
> [4] https://github.com/php/php-src/pull/14529
> [5] https://github.com/symfony/symfony/blob/7.1/src/Symfony/Bridge/PhpUnit/ClockMock.php
> [6] https://gist.github.com/iluuu1994/4b83481baac563f8f0d3204c697c5551
>

— Rob

Ilija Tovilo

2 years ago

Hi Rob

On Fri, Aug 2, 2024 at 7:10 PM Rob Landers <rob@bottled.codes> wrote:

>
> So, what you’re saying is that symfony and laravel can get a performance increase by simply adding a \ in the right places? Why don’t they do that instead of changing the language?

Nothing, of course. However, a Symfony maintainer has expressed
uninterest in prefixing all internal function calls, including
automated use statements at the top of the file. Even if they did,
most users will not.

> For functions/classes in the same exact namespace, you don’t need a use statement. But after this change, you do in certain cases?
>
> namespace Foo;
>
> function array_sum($bar) {}
>
> function baz($bar) {
>   return array_sum($bar);
> }
>
> So, how do you use that function in the same file?

Yes. But I'm not sure how that's different from today? If there's a
local and global function declared with the same name, and you intend
to call the global one, you'll already need to disambiguate the call
with a \.

With this change, your two options would be to:

* Prefix your calls with namespace\. That's quite ugly, but is the
syntax we currently offer.
* Add a `use array_sum;` to the top of the file.

An explicit use has upsides too. It makes it much more obvious that
the global function is shadowed.

> We can only see open source code when doing impact analysis. This means picking even a slightly “popular” name could go very poorly.

Yes, and there are many more than 10 000 composer repositories. An
impact analysis can give you an approximation for breakage, not
absolute numbers.

Ilija

Alexandru Pătrănescu

2 years ago

On Sun, Aug 4, 2024 at 8:43 PM Ilija Tovilo <tovilo.ilija@gmail.com> wrote:

> Hi Rob
>
> On Fri, Aug 2, 2024 at 7:10 PM Rob Landers <rob@bottled.codes> wrote:
> >
> > So, what you’re saying is that symfony and laravel can get a performance
> increase by simply adding a \ in the right places? Why don’t they do that
> instead of changing the language?
>
> Nothing, of course. However, a Symfony maintainer has expressed
> uninterest in prefixing all internal function calls, including
> automated use statements at the top of the file. Even if they did,
> most users will not.
>
>

Function lookup in either global or local scope is problematic, and
probably that's why we don't have autoloading for functions yet.
How about we change the language so that in PHP 9.0 there will be a notice
that gets triggered when a fallback to the global namespace gets triggered.
We would upgrade that to a warning in PHP 9.2, and it would end up being an
error on PHP 10 and have a BC break.

I don't think adding a \ to each function call is ugly, that's what we have
for classes, and it works fine; or an use statement.
So, why do we think that after people get used to it, they would still
consider it ugly? Never heard the "ugliness" mentioned for classes.

Now, I know this would be a big BC break, but it brings consistency to the
language and forces everyone to improve their code performance.

If that's not acceptable, then maybe considering all unqualified functions
as belonging to global namespace only might be an alternative solution.
That means that non-global functions must be imported or be fully
qualified, and that would be a BC break as well, but a smaller one.

To sum up, I think we need to remove the fallback behavior, so we can have
better things in the future.
Either keep only local with a bigger BC break but a better language
consistency.
Or keep only global with a smaller BC break.

Regards,
Alex

Nick Lockheart

2 years ago

> We would upgrade that to a warning in PHP 9.2, and it would end up
> being an error on PHP 10 and have a BC break.
>
> I don't think adding a \ to each function call is ugly, that's what
> we have for classes, and it works fine; or an use statement.
>
> So, why do we think that after people get used to it, they would
> still consider it ugly? Never heard the "ugliness" mentioned for
> classes.

Respectfully, I think `\` is ugly for both functions and classes.

> Now, I know this would be a big BC break, but it brings consistency
> to the language and forces everyone to improve their code
> performance.

There should be a directive for this, like:

namespace foo using global functions;

...which automatically acts as if all functions have a \ in front of
them, unless they are fully qualified.

Rob Landers

2 years ago

On Tue, Aug 20, 2024, at 10:41, Nick Lockheart wrote:

>
> > We would upgrade that to a warning in PHP 9.2, and it would end up
> > being an error on PHP 10 and have a BC break.
> >
> > I don't think adding a \ to each function call is ugly, that's what
> > we have for classes, and it works fine; or an use statement.
> >
> > So, why do we think that after people get used to it, they would
> > still consider it ugly? Never heard the "ugliness" mentioned for
> > classes.
>
>
> Respectfully, I think `\` is ugly for both functions and classes.
>
>
> > Now, I know this would be a big BC break, but it brings consistency
> > to the language and forces everyone to improve their code
> > performance.
>
> There should be a directive for this, like:
>
>     namespace foo using global functions;
>
> ...which automatically acts as if all functions have a \ in front of
> them, unless they are fully qualified.
>

Respectfully, I feel like this gets into the heart of a problem with RFCs, where if someone wants to implement something, they have to solve everyone’s problems.

In this case, there is a problem with performance issues due to multiple lookups (though, I’m not convinced fully), so if someone wants to implement function autoloading, they also have to solve this problem (Gina and I have both independently solved it in various ways).

Personally, I’m of the opinion that if you want performance, you know what to do: fully qualify your names. If you don’t care (which is what I gather from the first email in this thread where maintainers were not willing to change their code), then “deal with it.”

The vast majority of performance issues won’t be caused by function lookups, but by databases and poorly written code. Maybe I am wrong, but I rather like what we currently have, whatever benchmarks have to say on the matter.

— Rob

Levi Morrison

2 years ago

> To sum up, I think we need to remove the fallback behavior, so we can have better things in the future.
> Either keep only local with a bigger BC break but a better language consistency.
> Or keep only global with a smaller BC break.

I have long been in favor of a larger BC break with better language
consistency. Class lookup and function lookup with respect to
namespaces should be treated the same. The difficulty is getting a
majority of people to vote yes for this. Keep in mind that qualifying
every global function is annoying but probably can be somewhat
automated, and will bring better performance. So again, this improves
the existing code even without upgrading.

Yes, there would be complaints about it. Yes, there are probably some
people or projects who wouldn't upgrade. I don't particularly care, as
there are increasingly more operating systems and companies providing
LTS support for long periods of time. Probably Zend.com will offer LTS
support for the last PHP 8.X release, and possibly there will be some
distro which also has it. I believe it's the right thing to do
because:

1. It's faster.
2. It enables function autoloading in a similar manner to class autoloading.
3. It's more consistent, and simpler to teach and maintain.

It's rare that you get all of these together, often you have to make
tradeoffs within them.

Ilija Tovilo

2 years ago

Hi Levi

On Tue, Aug 20, 2024 at 5:14 PM Levi Morrison
<levi.morrison@datadoghq.com> wrote:

>
> I have long been in favor of a larger BC break with better language
> consistency. Class lookup and function lookup with respect to
> namespaces should be treated the same. The difficulty is getting a
> majority of people to vote yes for this. Keep in mind that qualifying
> every global function is annoying but probably can be somewhat
> automated, and will bring better performance. So again, this improves
> the existing code even without upgrading.
>
> Yes, there would be complaints about it. Yes, there are probably some
> people or projects who wouldn't upgrade. I don't particularly care, as
> there are increasingly more operating systems and companies providing
> LTS support for long periods of time. Probably Zend.com will offer LTS
> support for the last PHP 8.X release, and possibly there will be some
> distro which also has it. I believe it's the right thing to do
> because:
>
>  1. It's faster.
>  2. It enables function autoloading in a similar manner to class autoloading.
>  3. It's more consistent, and simpler to teach and maintain.
>
> It's rare that you get all of these together, often you have to make
> tradeoffs within them.

The approach I originally proposed also solves 1. and 2. (mostly) with
very little backwards incompatibility. Consistency is absolutely
something to strive for, but not at the cost of breaking most PHP
code.

To clarify on 2.: The main issue with function autoloading today is
that the engine needs to trigger the autoloader for every unqualified
call to global functions, given that the autoloader might declare the
function in local scope. As most unqualified calls are global calls,
this adds a huge amount of overhead.

Gina solved this in part by aliasing the local function to the global
one after the first lookup. However, that still means that the
autoloader will trigger for every new namespace the function is called
in, and will also pollute the function table.

Reversing the lookup order once again avoids local lookup when calling
global functions in local scope, which also means dodging the
autoloader. The caveat is that calling local functions in local scope
triggers the autoloader on first encounter, but at least it can be
marked as undeclared in the symbol table once, instead of in every
namespace, which also means triggering the autoloader only once.

Ilija

Faizan Akram Dar

2 years ago

On Tue, Aug 20, 2024 at 11:34 PM Ilija Tovilo <tovilo.ilija@gmail.com>
wrote:

> Hi Levi
>
> On Tue, Aug 20, 2024 at 5:14 PM Levi Morrison
> <levi.morrison@datadoghq.com> wrote:
> >
> > I have long been in favor of a larger BC break with better language
> > consistency. Class lookup and function lookup with respect to
> > namespaces should be treated the same. The difficulty is getting a
> > majority of people to vote yes for this. Keep in mind that qualifying
> > every global function is annoying but probably can be somewhat
> > automated, and will bring better performance. So again, this improves
> > the existing code even without upgrading.
> >
> > Yes, there would be complaints about it. Yes, there are probably some
> > people or projects who wouldn't upgrade. I don't particularly care, as
> > there are increasingly more operating systems and companies providing
> > LTS support for long periods of time. Probably Zend.com will offer LTS
> > support for the last PHP 8.X release, and possibly there will be some
> > distro which also has it. I believe it's the right thing to do
> > because:
> >
> >  1. It's faster.
> >  2. It enables function autoloading in a similar manner to class
> autoloading.
> >  3. It's more consistent, and simpler to teach and maintain.
> >
> > It's rare that you get all of these together, often you have to make
> > tradeoffs within them.
>
> The approach I originally proposed also solves 1. and 2. (mostly) with
> very little backwards incompatibility. Consistency is absolutely
> something to strive for, but not at the cost of breaking most PHP
> code.
>
> To clarify on 2.: The main issue with function autoloading today is
> that the engine needs to trigger the autoloader for every unqualified
> call to global functions, given that the autoloader might declare the
> function in local scope. As most unqualified calls are global calls,
> this adds a huge amount of overhead.
>
> Gina solved this in part by aliasing the local function to the global
> one after the first lookup. However, that still means that the
> autoloader will trigger for every new namespace the function is called
> in, and will also pollute the function table.
>
> Reversing the lookup order once again avoids local lookup when calling
> global functions in local scope, which also means dodging the
> autoloader. The caveat is that calling local functions in local scope
> triggers the autoloader on first encounter, but at least it can be
> marked as undeclared in the symbol table once, instead of in every
> namespace, which also means triggering the autoloader only once.
>
> Ilija
>

Hi,

I completely agree with Levi's perspective, aligning class and function
lookup with respect
to namespaces seems a very sensible option.
It will improve consistency and pave the road for autoloading functions
without quirks.

The impact of fixing functions look up is overstated. For instance,
PHP-CS-Fixer can add
"global namespace qualifiers" to all global functions in a matter of
minutes, it is not like
people have to go through code and change it manually.


To ease the transition, PHP can ship a small fixer with the next PHP
version for changing
global function usage (prepending \ or adding use statements) and be done
with the
inconsistency once and for all.


Kind regards,
Faizan

Rob Landers

2 years ago

On Tue, Aug 20, 2024, at 23:56, Faizan Akram Dar wrote:

>
>
> On Tue, Aug 20, 2024 at 11:34 PM Ilija Tovilo <tovilo.ilija@gmail.com> wrote:
>> Hi Levi
>>
>> On Tue, Aug 20, 2024 at 5:14 PM Levi Morrison
>> <levi.morrison@datadoghq.com> wrote:
>> >
>> > I have long been in favor of a larger BC break with better language
>> > consistency. Class lookup and function lookup with respect to
>> > namespaces should be treated the same. The difficulty is getting a
>> > majority of people to vote yes for this. Keep in mind that qualifying
>> > every global function is annoying but probably can be somewhat
>> > automated, and will bring better performance. So again, this improves
>> > the existing code even without upgrading.
>> >
>> > Yes, there would be complaints about it. Yes, there are probably some
>> > people or projects who wouldn't upgrade. I don't particularly care, as
>> > there are increasingly more operating systems and companies providing
>> > LTS support for long periods of time. Probably Zend.com will offer LTS
>> > support for the last PHP 8.X release, and possibly there will be some
>> > distro which also has it. I believe it's the right thing to do
>> > because:
>> >
>> >  1. It's faster.
>> >  2. It enables function autoloading in a similar manner to class autoloading.
>> >  3. It's more consistent, and simpler to teach and maintain.
>> >
>> > It's rare that you get all of these together, often you have to make
>> > tradeoffs within them.
>>
>> The approach I originally proposed also solves 1. and 2. (mostly) with
>> very little backwards incompatibility. Consistency is absolutely
>> something to strive for, but not at the cost of breaking most PHP
>> code.
>>
>> To clarify on 2.: The main issue with function autoloading today is
>> that the engine needs to trigger the autoloader for every unqualified
>> call to global functions, given that the autoloader might declare the
>> function in local scope. As most unqualified calls are global calls,
>> this adds a huge amount of overhead.
>>
>> Gina solved this in part by aliasing the local function to the global
>> one after the first lookup. However, that still means that the
>> autoloader will trigger for every new namespace the function is called
>> in, and will also pollute the function table.
>>
>> Reversing the lookup order once again avoids local lookup when calling
>> global functions in local scope, which also means dodging the
>> autoloader. The caveat is that calling local functions in local scope
>> triggers the autoloader on first encounter, but at least it can be
>> marked as undeclared in the symbol table once, instead of in every
>> namespace, which also means triggering the autoloader only once.
>>
>> Ilija
>
> Hi,
>
> I completely agree with Levi's perspective, aligning class and function lookup with respect
> to namespaces seems a very sensible option.
> It will improve consistency and pave the road for autoloading functions without quirks.
>
> The impact of fixing functions look up is overstated. For instance, PHP-CS-Fixer can add
>  "global namespace qualifiers" to all global functions in a matter of minutes, it is not like
>  people have to go through code and change it manually.
>
>
> To ease the transition, PHP can ship a small fixer with the next PHP version for changing
>  global function usage (prepending \ or adding use statements) and be done with the
> inconsistency once and for all.
>
>
> Kind regards,
> Faizan
>
>

I am currently working on benchmarks specifically related to my function autoloading RFC, and I'm (not yet) certain there will be any performance impacts related to function autoloading. I may end up eating my hat here, but in any case, there is only speculation at this point.

If this change improves performance; that's great. However, I don't think we should be changing things just for the sake of performance though (or the opposite). It's great to be aware of how things affect performance, but I don't think we should make decisions purely based on it; otherwise we will never add any new features to PHP.

— Rob

Christian Schneider

2 years ago

Am 20.08.2024 um 17:14 schrieb Levi Morrison <levi.morrison@datadoghq.com>:

> Keep in mind that qualifying
> every global function is annoying but probably can be somewhat
> automated, and will bring better performance. So again, this improves
> the existing code even without upgrading.

Just to be sure: Would code *not* using namespaces also have to qualify global function calls? I admit that I somewhat skimmed the discussion so I might have missed that point.

The point where I think we disagree is that it improves the code. It may improve performance of the code (even though I somewhat doubt this has a *significant* impact on most projects) but it IMHO hurts readability. Writing the additional \ is less of a problem but as code is read a lot more often than written I think the additional "line-noise" is something I'd like to avoid.

Regards,
- Chris

Levi Morrison

2 years ago

On Tue, Aug 20, 2024 at 8:26 PM Christian Schneider
<cschneid@cschneid.com> wrote:

>
> Am 20.08.2024 um 17:14 schrieb Levi Morrison <levi.morrison@datadoghq.com>:
> > Keep in mind that qualifying
> > every global function is annoying but probably can be somewhat
> > automated, and will bring better performance. So again, this improves
> > the existing code even without upgrading.
>
> Just to be sure: Would code *not* using namespaces also have to qualify global function calls? I admit that I somewhat skimmed the discussion so I might have missed that point.

Code that isn't in a namespace is in the global namespace. So no, such
code does not have to qualify the global function calls.

Faizan Akram Dar

2 years ago

On Wed, Aug 21, 2024, 9:34 AM Christian Schneider <cschneid@cschneid.com>
wrote:

> Am 20.08.2024 um 17:14 schrieb Levi Morrison <levi.morrison@datadoghq.com
> >:
> > Keep in mind that qualifying
> > every global function is annoying but probably can be somewhat
> > automated, and will bring better performance. So again, this improves
> > the existing code even without upgrading.
>
> Just to be sure: Would code *not* using namespaces also have to qualify
> global function calls? I admit that I somewhat skimmed the discussion so I
> might have missed that point.
>
> The point where I think we disagree is that it improves the code. It may
> improve performance of the code (even though I somewhat doubt this has a
> *significant* impact on most projects) but it IMHO hurts readability.
> Writing the additional \ is less of a problem but as code is read a lot
> more often than written I think the additional "line-noise" is something
> I'd like to avoid.
>
> Regards,
> - Chris
>

Hi Chris,

You don't have to write additional \,
you can add "use function" statements
if you prefer that style.
It's no different from referencing global
classes, they either need to be prefixed
with \ or need to have a corresponding "use" statement.



Kind regards,
Faizan

Christian Schneider

2 years ago

Am 21.08.2024 um 09:44 schrieb Faizan Akram Dar <hello@faizanakram.me>:

> On Wed, Aug 21, 2024, 9:34 AM Christian Schneider <cschneid@cschneid.com> wrote:
>> The point where I think we disagree is that it improves the code. It may improve performance of the code (even though I somewhat doubt this has a *significant* impact on most projects) but it IMHO hurts readability. Writing the additional \ is less of a problem but as code is read a lot more often than written I think the additional "line-noise" is something I'd like to avoid.
>
> You don't have to write additional \,
> you can add "use function" statements
>  if you prefer that style.

I think that is trading one problem for another:
- Having to declare all global functions like strlen with 'use' is (IMHO) unnecessary boilerplate which also needs to be kept in sync with the rest of the code below
- I am generally wary of top declarations changing "semantics" of code further down the line. Being able to tell what is being done without (far away) context is a feature and that's why I e.g. prefer foo($GLOBALS['bar']) to global $bar; ... foo($bar).

Regards,
- Chris

Rowan Tommins [IMSoP]

2 years ago

On 04/08/2024 18:41, Ilija Tovilo wrote:

> * Prefix your calls with namespace\. That's quite ugly, but is the
> syntax we currently offer.

I was thinking about this earlier, and how the migration is pretty much
the same (and equally automatable) in either direction:

* If unqualified calls become always local, then every global function
call needs a use statement or prefixing with "\".
* If they become always global, then every local function call needs a
use statement or prefixing with "namespace\".

But the first option probably requires changes in the majority of PHP
files in use anywhere; whereas the second only affects a small minority
of code bases, and a small minority of code in those.


BUT, if people already complain about "\" being ugly, having to write
"namespace\" is going to make them REALLY grumpy...

So maybe at the same time (or, probably, in advance) we need to come up
with a nicer syntax for explicitly referencing the current namespace.

Unfortunately, finding unused syntax is hard, which is why we have "\"
in the first place (and for the record, I think it works just fine), but
maybe something like "_\" could work? Giving us:

namespace Foo;

$native_length = strlen('hello'); # same as \strlen('hello')
$foo_length = _\strlen('hello'); #  same as \Foo\strlen('hello')


If I had a time machine, I'd campaign for "unqualified means local" in
PHP 5.3, and we'd all be used to writing "\strlen" by now; but
"unqualified means global" feels much more achievable from where we are.

--
Rowan Tommins
[IMSoP]

Mike Schinkel

2 years ago

> On Aug 22, 2024, at 5:32 PM, Rowan Tommins [IMSoP] <imsop.php@rwec.co.uk> wrote:
> I was thinking about this earlier, and how the migration is pretty much the same (and equally automatable) in either direction:
>
> * If unqualified calls become always local, then every global function call needs a use statement or prefixing with "\".
> * If they become always global, then every local function call needs a use statement or prefixing with "namespace\".
>
> But the first option probably requires changes in the majority of PHP files in use anywhere; whereas the second only affects a small minority of code bases, and a small minority of code in those.
>
> BUT, if people already complain about "\" being ugly, having to write "namespace\" is going to make them REALLY grumpy...
>
> So maybe at the same time (or, probably, in advance) we need to come up with a nicer syntax for explicitly referencing the current namespace.
>
> Unfortunately, finding unused syntax is hard, which is why we have "\" in the first place (and for the record, I think it works just fine), but maybe something like "_\" could work? Giving us:
>
> namespace Foo;
>
> $native_length = strlen('hello'); # same as \strlen('hello')
> $foo_length = _\strlen('hello'); #  same as \Foo\strlen('hello')
>

If having to type `\strlen()` is ugly — and I agree that is it — then having to type `_\strlen()` is what in university we would call "fugly," to emphasize just how much worse something was vs. just run-of-the-mill "ugly."

Having to prefix with a name like Foo, e.g. Foo\strlen() is FAR PREFERABLE to _\strlen() because at least it provides satiating information rather than the empty calories of a cryptic shorthand.  #jmtcw, anyway.

> If I had a time machine, I'd campaign for "unqualified means local" in PHP 5.3, and we'd all be used to writing "\strlen" by now; but "unqualified means global" feels much more achievable from where we are.
>

If I had a time machine I would campaign for real packages instead of what namespaces turned out to me, and that used sigils that do not double as the escape character for strings, but then both of us digress.

-Mike

Rowan Tommins [IMSoP]

2 years ago

On 23 August 2024 00:15:19 BST, Mike Schinkel <mike@newclarity.net> wrote:

>Having to prefix with a name like Foo, e.g. Foo\strlen() is FAR PREFERABLE to _\strlen() because at least it provides satiating information rather than the empty calories of a cryptic shorthand.  #jmtcw, anyway.

I knew I'd regret keeping the example short. Realistically, it's not a substitute for "\Foo\strlen", it's a substitute for "\AcmeComponents\SplineReticulator\Utilities\Text\strlen".

Having a syntax for "relative to current" is incredibly common in other path-like syntaxes. The most common marker is ".", and ".\foo" is literally how you'd refer to something in the current directory under DOS/Windows. But unfortunately, we don't have "." available, so I wondered if "_" would feel similar enough.

Another option would be to find a shorter keyword than "namespace" to put it in front. "ns\strlen(...)" is an obvious step from what we have currently, but it's not very obvious what it means, so maybe there's a different word we could use.

Rowan Tommins
[IMSoP]

Nick Lockheart

2 years ago

On Fri, 2024-08-23 at 07:39 +0100, Rowan Tommins [IMSoP] wrote:

>
>
> On 23 August 2024 00:15:19 BST, Mike Schinkel <mike@newclarity.net>
> wrote:
> > Having to prefix with a name like Foo, e.g. Foo\strlen() is FAR
> > PREFERABLE to _\strlen() because at least it provides satiating
> > information rather than the empty calories of a cryptic shorthand. 
> > #jmtcw, anyway.
>
> I knew I'd regret keeping the example short. Realistically, it's not
> a substitute for "\Foo\strlen", it's a substitute for
> "\AcmeComponents\SplineReticulator\Utilities\Text\strlen".
>
> Having a syntax for "relative to current" is incredibly common in
> other path-like syntaxes. The most common marker is ".", and ".\foo"
> is literally how you'd refer to something in the current directory
> under DOS/Windows. But unfortunately, we don't have "." available, so
> I wondered if "_" would feel similar enough.
>
> Another option would be to find a shorter keyword than "namespace" to
> put it in front. "ns\strlen(...)" is an obvious step from what we
> have currently, but it's not very obvious what it means, so maybe
> there's a different word we could use.
>
> Rowan Tommins
> [IMSoP]

Could be mistaken, but I think the way PHP handles namespaces
internally is sort of the same as a long string, rather than as a
tree/hierarchy.

ie. \AcmeComponents\SplineReticulator\Utilities\Text\strlen

is really like:

class AcmeComponentsSplineReticulatorUtilitiesTextstrlen {

public function __construct(){

}

}

And the "AcmeComponentsSplineReticulatorUtilitiesText" just kind of
gets appended to the front when the class name is registered.

I haven't done work on the namespace code, but I recall reading this
somewhere recently.

Rob Landers

2 years ago

On Fri, Aug 23, 2024, at 09:27, Nick Lockheart wrote:

> On Fri, 2024-08-23 at 07:39 +0100, Rowan Tommins [IMSoP] wrote:
> >
> >
> > On 23 August 2024 00:15:19 BST, Mike Schinkel <mike@newclarity.net>
> > wrote:
> > > Having to prefix with a name like Foo, e.g. Foo\strlen() is FAR
> > > PREFERABLE to _\strlen() because at least it provides satiating
> > > information rather than the empty calories of a cryptic shorthand.
> > > #jmtcw, anyway.
> >
> > I knew I'd regret keeping the example short. Realistically, it's not
> > a substitute for "\Foo\strlen", it's a substitute for
> > "\AcmeComponents\SplineReticulator\Utilities\Text\strlen".
> >
> > Having a syntax for "relative to current" is incredibly common in
> > other path-like syntaxes. The most common marker is ".", and ".\foo"
> > is literally how you'd refer to something in the current directory
> > under DOS/Windows. But unfortunately, we don't have "." available, so
> > I wondered if "_" would feel similar enough.
> >
> > Another option would be to find a shorter keyword than "namespace" to
> > put it in front. "ns\strlen(...)" is an obvious step from what we
> > have currently, but it's not very obvious what it means, so maybe
> > there's a different word we could use.
> >
> > Rowan Tommins
> > [IMSoP]
>
> Could be mistaken, but I think the way PHP handles namespaces
> internally is sort of the same as a long string, rather than as a
> tree/hierarchy.
>
> ie. \AcmeComponents\SplineReticulator\Utilities\Text\strlen
>
> is really like:
>
> class AcmeComponentsSplineReticulatorUtilitiesTextstrlen {
>
>    public function __construct(){
>
>    }
>
> }
>
> And the "AcmeComponentsSplineReticulatorUtilitiesText" just kind of
> gets appended to the front when the class name is registered.
>
> I haven't done work on the namespace code, but I recall reading this
> somewhere recently.

This is mostly correct, the only thing missing from your strings is the `\` character. I believe this even happens during compilation. Meaning it sees your namespace/uses and then rewrites the function/class calls during compile time. Thus an unqualified call is prepended with the current namespace defined at the top of the file.

If we were to go with any major change in the current lookup where it is perf or nothing, this is what I would propose for php 9.0 (starting with an immediate deprecation):
1. any unqualified call simply calls the current namespace
2. >= php 9.0: no fallback to global
3. < php 9.0: emit deprecation notice if falls back to global
This is how classes work (pretty sure), so it would be consistent.

Going the other way (global first) doesn't really make sense because it is inconsistent, IMHO. Will it suck? Probably. Will it be easy to fix? Probably via Rector.

— Rob

Nick Lockheart

2 years ago

>
> If we were to go with any major change in the current lookup where it
> is perf or nothing, this is what I would propose for php 9.0
> (starting with an immediate deprecation):
>    1. any unqualified call simply calls the current namespace
>    2. >= php 9.0: no fallback to global
>    3. < php 9.0: emit deprecation notice if falls back to global
> This is how classes work (pretty sure), so it would be consistent.
>
> Going the other way (global first) doesn't really make sense because
> it is inconsistent, IMHO. Will it suck? Probably. Will it be easy to
> fix? Probably via Rector.
>
> — Rob

A third option, which I haven't seen come up on the list yet, is that
unqualified functions that are PHP built-ins are treated as global, and
using a function having the same name as a built-in, in a namespace
scope, requires a fully qualified name to override the built-in.

It seems that if someone is writing `array_key_exists()` or similar
they probably mean the built-in function, and in the rare cases where
they do mean `\foo\array_key_exists()`, they can write it explicitly.

Functions that are *not* on the built-in function list could default to
the local namespace.

Rob Landers

2 years ago

On Fri, Aug 23, 2024, at 10:08, Nick Lockheart wrote:

>
> >
> > If we were to go with any major change in the current lookup where it
> > is perf or nothing, this is what I would propose for php 9.0
> > (starting with an immediate deprecation):
> >    1. any unqualified call simply calls the current namespace
> >    2. >= php 9.0: no fallback to global
> >    3. < php 9.0: emit deprecation notice if falls back to global
> > This is how classes work (pretty sure), so it would be consistent.
> >
> > Going the other way (global first) doesn't really make sense because
> > it is inconsistent, IMHO. Will it suck? Probably. Will it be easy to
> > fix? Probably via Rector.
> >
> > — Rob
>
>
> A third option, which I haven't seen come up on the list yet, is that
> unqualified functions that are PHP built-ins are treated as global, and
> using a function having the same name as a built-in, in a namespace
> scope, requires a fully qualified name to override the built-in.
>
> It seems that if someone is writing `array_key_exists()` or similar
> they probably mean the built-in function, and in the rare cases where
> they do mean `\foo\array_key_exists()`, they can write it explicitly.
>
> Functions that are *not* on the built-in function list could default to
> the local namespace.

I was actually thinking of doing something like this for function autoloading, where extensions could register global functions that bypass the autoloader and go straight to global if it isn't defined in the local namespace already. I decided not to even bring it up because it felt controversial (it would effectively be global first, except for user functions). Though, it might be a nice compromise?

— Rob

Mike Schinkel

2 years ago

> On Aug 23, 2024, at 4:08 AM, Nick Lockheart <lists@ageofdream.com> wrote:
> A third option, which I haven't seen come up on the list yet, is that
> unqualified functions that are PHP built-ins are treated as global, and
> using a function having the same name as a built-in, in a namespace
> scope, requires a fully qualified name to override the built-in.
>
> It seems that if someone is writing `array_key_exists()` or similar
> they probably mean the built-in function, and in the rare cases where
> they do mean `\foo\array_key_exists()`, they can write it explicitly.
>
> Functions that are *not* on the built-in function list could default to
> the local namespace.

I was going back and forth on this.

On one hand it could be confusing for developers to learn when to use `\` and when not to.

OTOH, once they learn it would create a clear indication of which functions are userland code and which are standard library, and I this could have significant benefit for readability and maintainability.

Yet OTOH, this would create a bifurcation between userland and standard library that is encouraged in some languages and discouraged in others, i.e. the latter being the "keep the language as small as possible and then let the standard library be implemented in the language no differently than any code a user could write" type of languages.

Yet OTOH still, PHP does not allow core functions to be implemented in userland, at least without monkey patching so the bifurcation already exists.

Go, for example, has both. Most of the standard library is just Go code anyone could replace with their own, but a handful of functions are special and built into the language, e.g. append, new, close, etc.  Basically anything in lowercase that can be used globally without qualification is special. And after using Go, I think it is a great design as it reserves future enhancements without BC concerns.

In theory it would be nice to open up PHP to allow overriding core functions, but that could also open a Pandora's box, the kind that makes Ruby code so fragile. At least in Go you have to omit the standard lib import and use your own import to override a standard library package.

So in practice PHP may never change to allow core functions to be overridden and thus pining for that to block this idea would be a missed opportunity. (That said, PHP could allow a userland function to be "registered" to be called instead of the core function, and if that were allowed then Nick's proposal would cause no problems. Of course I doubt internals would ever bless that idea.)

Anyway — in summary — I think Nick's 3rd option has wings. And along with automatic `use` statements for each namespace might just be the best possible solution.

-Mike

P.S. If PHP ever added a set of standard library functions written in PHP to the core distribution, they should rightly IMO need to be namespaced, per this proposal. But here I digress. I only mention in hopes to keep this specific dream alive for some future day.

Stephen Reay

2 years ago

> On 23 Aug 2024, at 15:08, Nick Lockheart <lists@ageofdream.com> wrote:
>
> A third option, which I haven't seen come up on the list yet, is that
> unqualified functions that are PHP built-ins are treated as global, and
> using a function having the same name as a built-in, in a namespace
> scope, requires a fully qualified name to override the built-in.
>
> It seems that if someone is writing `array_key_exists()` or similar
> they probably mean the built-in function, and in the rare cases where
> they do mean `\foo\array_key_exists()`, they can write it explicitly.
>
> Functions that are *not* on the built-in function list could default to
> the local namespace.

This doesn't solve the "future hidden BC break" aspect, at all.

If I write a namespaced function `http_parse_url` to adhere more strictly to whatever relevant RFC, when I write it, it will work as expected using a relative name (and/or itself quite likely using helper functions in the same namespace)

If an RFC then approves a "core" `http_parse_url` function to serve as a better replacement for the old `parse_url`, suddenly my function won't work the same way with the new version of PHP... because of a global function that didn't exist when I wrote mine.

If this "global first" change is made, *any* use of unqualified functions that refer to the current namespace will *have* to use some form of current-namespace indicator to be future version-safe.

If you want to make global functions resolve without a `\` the only realistically safe solution is to completely remove unqualified lookup of local function (and constant) symbols, and force them to always use a name that resolves absolutely, because otherwise any future version could break what they do completely.

Rowan Tommins [IMSoP]

2 years ago

On Fri, 23 Aug 2024, at 08:27, Nick Lockheart wrote:

> Could be mistaken, but I think the way PHP handles namespaces
> internally is sort of the same as a long string, rather than as a
> tree/hierarchy.

Just to be clear, PHP already has a syntax for explicitly resolving a name relative to the current namespace, it's just not needed very often. See e.g. https://3v4l.org/Xfma5 and https://3v4l.org/3o2TD (You're right that underneath it's all just string concatenation, but that's all you need in this case.)

All I was talking about was alternative syntax that would behave in exactly the same way that "namespace\Foo" already does.

--
Rowan Tommins
[IMSoP]

Mike Schinkel

2 years ago

Hi Rowan,

> On Aug 23, 2024, at 2:39 AM, Rowan Tommins [IMSoP] <imsop.php@rwec.co.uk> wrote:
> On 23 August 2024 00:15:19 BST, Mike Schinkel <mike@newclarity.net> wrote:
>> Having to prefix with a name like Foo, e.g. Foo\strlen() is FAR PREFERABLE to _\strlen() because at least it provides satiating information rather than the empty calories of a cryptic shorthand.  #jmtcw, anyway.
>
> I knew I'd regret keeping the example short. Realistically, it's not a substitute for "\Foo\strlen", it's a substitute for "\AcmeComponents\SplineReticulator\Utilities\Text\strlen".

And similarly, I too regret keeping my answer short. I was assuming what I omitted would be obvious.

(And I am not being snarky, I literally thought about including this next but then felt I did not need to. Hindsight!)

So, long namespaces is why PHP has the `use` statement, making references in functions short and sweet, e.g:

namespace \AcmeComponents\SplineReticulator\Utilities\Text
use \AcmeComponents\SplineReticulator\Utilities\Text

function Foo():int {
return Text\strlen("Hello World");
}

(Of course, that is a lot of redundant boilerplate.)

> Another option would be to find a shorter keyword than "namespace" to put it in front. "ns\strlen(...)" is an obvious step from what we have currently, but it's not very obvious what it means, so maybe there's a different word we could use.

So rather than all that boilerplate, and rather than yet another special set of characters developers would need to learn and remember — and tooling would need to adjust to — we could instead easily add an automatic `use` statement for every namespace, as long as no existing use statement conflicts with it.

An automatic `use` would then give us the following, which provides a strong information sent and is really consistent with the nature of the PHP language:

namespace \AcmeComponents\SplineReticulator\Utilities\Text

function Foo():int {
return Text\strlen("Hello World");
}

The above of course could result in BC breaks IF there happened to be existing code that referenced Text\strlen() where Text was a top-level namespace, AND that code was not remediated when this change takes place. However, I am guessing those collisions would be pretty rare as both the namespace and the symbol would have to match to be in conflict.

> Having a syntax for "relative to current" is incredibly common in other path-like syntaxes. The most common marker is ".", and ".\foo" is literally how you'd refer to something in the current directory under DOS/Windows. But unfortunately, we don't have "." available, so I wondered if "_" would feel similar enough.

I'll be honest, the association with the relative path of `.\` did not occur to me when you presented `_\` so after you stating this I pondered if from that perspective.

However, frankly, I am not sold on that perspective. Something about it does not feel right. I can't currently give any more objective arguments than that, so I will just leave it as #jmctw.

I will say if we were going with relative path, I think `\\strlen()` would be preferable to `_\strlen()`. Subjectively `\\` is easier for me to "see" and thus does not look so out of place to me.

OTOH the objective arguments for `\\` over `_\` are it is much easier to type:  slash+slash vs. shift-underscore+nonshift-slash. There is a precedent in URIs with `//`, albeit not exactly equivalent. And finally, it also does not use a sigil that could be better used elsewhere in some as-yet-to be agreed or envisioned future use. #fwiw

-Mike

Rowan Tommins [IMSoP]

2 years ago

On Fri, 23 Aug 2024, at 12:29, Mike Schinkel wrote:

> namespace \AcmeComponents\SplineReticulator\Utilities\Text
>
> function Foo():int {
> 	return Text\strlen("Hello World");
> }
>
> The above of course could result in BC breaks IF there happened to be
> existing code that referenced Text\strlen() where Text was a top-level
> namespace

It wouldn't be a top-level namespace that would cause a conflict, but a nested one: currently the above code resolves the function name as "AcmeComponents\SplineReticulator\Utilities\Text\Text\strlen" (note the "...\Text\Text\...").

It's an interesting suggestion, but I'm not totally sold on "use the end of the current namespace" being easier to remember than "use this symbol or keyword".

return namespace\strlen("Hello World"); # current syntax, rather long and unclear
return _\strlen("Hello World"); # short, but maybe a bit cryptic
return Text\strlen("Hello World"); # variable length, relies on current context
return NS\strlen("Hello World"); # shortening of current keyword
return self\strlen("Hello World"); # maybe confusing to reuse a keyword?
return current\strlen("Hello World"); # clear, but a bit long

--
Rowan Tommins
[IMSoP]

Mike Schinkel

2 years ago

> On Aug 23, 2024, at 7:50 AM, Rowan Tommins [IMSoP] <imsop.php@rwec.co.uk> wrote:
>
> On Fri, 23 Aug 2024, at 12:29, Mike Schinkel wrote:
>> namespace \AcmeComponents\SplineReticulator\Utilities\Text
>>
>> function Foo():int {
>> 	return Text\strlen("Hello World");
>> }
>>
>> The above of course could result in BC breaks IF there happened to be
>> existing code that referenced Text\strlen() where Text was a top-level
>> namespace
>
> It wouldn't be a top-level namespace that would cause a conflict, but a nested one: currently the above code resolves the function name as "AcmeComponents\SplineReticulator\Utilities\Text\Text\strlen" (note the "...\Text\Text\...").

How often does that really occur in the wild?

And how can it occur without an explicit `use AcmeComponents\SplineReticulator\Utilities\Text\Text` statement, which I proposed would override the automatic `use`, anyway?

> It's an interesting suggestion, but I'm not totally sold on "use the end of the current namespace" being easier to remember than "use this symbol or keyword".
>
> return namespace\strlen("Hello World"); # current syntax, rather long and unclear
> return _\strlen("Hello World"); # short, but maybe a bit cryptic
> return Text\strlen("Hello World"); # variable length, relies on current context
> return NS\strlen("Hello World"); # shortening of current keyword
> return self\strlen("Hello World"); # maybe confusing to reuse a keyword?
> return current\strlen("Hello World"); # clear, but a bit long

The only one of those that has a strong analog to existing PHP code is to "use the end of the current namespace" as people frequently do with explicit `use` statements.

Yes, `self` has a weak analog to `self::`, but none of the others even come close, IMO. And adding `self\` may have unintended to the language, some slight BC concerns, and/or downstream consequences for other projects  vs. a simple automatic `use`.

Lastly, no comment on `\\`?

-Mike

Rowan Tommins [IMSoP]

2 years ago

On 23 August 2024 13:04:22 BST, Mike Schinkel <mike@newclarity.net> wrote:

>
>> It wouldn't be a top-level namespace that would cause a conflict, but a nested one: currently the above code resolves the function name as "AcmeComponents\SplineReticulator\Utilities\Text\Text\strlen" (note the "...\Text\Text\...").
>
>How often does that really occur in the wild?

Oh, I think it would be much rarer than colliding with a global namespace. I was pointing out that your suggestion was *better* than you thought.

>And how can it occur without an explicit `use AcmeComponents\SplineReticulator\Utilities\Text\Text` statement, which I proposed would override the automatic `use`, anyway?

I'm not sure what you mean. Right now, that's the function name that would be looked up for your example code (other than a couple of unrelated typos in your example). So if, for some reason, someone was relying on that, their code would break with your "automatic use".

>The only one of those that has a strong analog to existing PHP code is to "use the end of the current namespace" as people frequently do with explicit `use` statements.

True. I just don't love the context-sensitive nature of it.

>Lastly, no comment on `\\`?

Ah, yes, I forgot to say: I'm not keen on that because in other contexts it means exactly the opposite: it refers to the absolute root in a context where a relative name would be assumed.

For example, \\domain\username and \\server\fileshare on Windows, or //example.com/foo in a URL

Rowan Tommins
[IMSoP]

Nick Lockheart

2 years ago

>
> BUT, if people already complain about "\" being ugly, having to write
> "namespace\" is going to make them REALLY grumpy...
> So maybe at the same time (or, probably, in advance) we need to come
> up with a nicer syntax for explicitly referencing the current
> namespace.
> Unfortunately, finding unused syntax is hard, which is why we have
> "\" in the first place (and for the record, I think it works just
> fine), but maybe something like "_\" could work? Giving us:
> namespace Foo;
> $native_length = strlen('hello'); # same as \strlen('hello')
>  $foo_length = _\strlen('hello'); #  same as \Foo\strlen('hello')
>

namespace foo using global functions;

- or -

namespace foo using local functions;

Tell PHP what you want at the per-file level.

Rowan Tommins [IMSoP]

2 years ago

On 23 August 2024 01:42:38 BST, Nick Lockheart <lists@ageofdream.com> wrote:

>
>>
>> BUT, if people already complain about "\" being ugly, having to write
>> "namespace\" is going to make them REALLY grumpy...
>> So maybe at the same time (or, probably, in advance) we need to come
>> up with a nicer syntax for explicitly referencing the current
>> namespace.
>
>    namespace foo using global functions;
>
> - or -
>
>    namespace foo using local functions;
>
>
>Tell PHP what you want at the per-file level.

This doesn't seem mutually exclusive to me. If you have a file where you've opted for "using global functions", you might want a way to reference a function in the current namespace.

It also doesn't address my other point, that having global as the default mode (even if we provide an option for local) is much less disruptive to existing code.

Regards,
Rowan Tommins
[IMSoP]

Nick Lockheart

2 years ago

On Fri, 2024-08-23 at 09:16 +0100, Rowan Tommins [IMSoP] wrote:

>
>
> On 23 August 2024 01:42:38 BST, Nick Lockheart <lists@ageofdream.com>
> wrote:
> >
> > >
> > > BUT, if people already complain about "\" being ugly, having to
> > > write
> > > "namespace\" is going to make them REALLY grumpy...
> > > So maybe at the same time (or, probably, in advance) we need to
> > > come
> > > up with a nicer syntax for explicitly referencing the current
> > > namespace.
> >
> >    namespace foo using global functions;
> >
> > - or -
> >
> >    namespace foo using local functions;
> >
> >
> > Tell PHP what you want at the per-file level.
>
>
> This doesn't seem mutually exclusive to me. If you have a file where
> you've opted for "using global functions", you might want a way to
> reference a function in the current namespace.

Correct, so if you use the example:

namespace foo using global functions;

you can write:

array_key_exists();

and it will be resolved as global without a namespace lookup and will
use the dedicated opcode.

But if you need to use a local function you can do:

\foo\sort();

The proposed global/local declaration as part of the namespace
declaration just turns off namespace lookups and sets the default
resolution for **unqualified** names.

Fully qualified names are not affected.

> It also doesn't address my other point, that having global as the
> default mode (even if we provide an option for local) is much less
> disruptive to existing code.

They are compatible, but related decisions.

I think it would be easier for people to accept a new PHP version where
unqualified names were always global, if we also had an option to make
local/namespaced the default resolution for *unqualified* names, on a
per-file basis, for those who need that.

Thus, there are multiple decision points:

1. Should we do namespace lookups on unqualified function calls at all?

2. If yes to 1, should we lookup in global first or local first?

3. Regardless of 1 or 2, should we let developers explicitly specify a
behavior for unqualified calls in the namespace declaration?

4. If yes to 1, should the behavior of namespace lookups change for
user-defined functions vs PHP built-in function names?

These aren't mutually exclusive, but they all work together to create a
complete behavior.

There are several ways that the above options could be combined:

### OPTION ONE ###

Using a regular namespace declaration still does an NS lookup, in the
same order, just like it normally works now.

That means that code that uses:

namespace foo;

will behave exactly the same as today, with no BC breaks.

Developers using the new PHP version could opt-in to explicit namespace
behavior with:

namespace foo using global functions;

or

namespace foo using local functions;

In both cases, *fully-qualified* names still work the same.

Only *unqualified* names are affected by this directive, and they use
local only or global only, depending on the declaration.

### OPTION TWO ###

Namespace lookup is removed from a future version of PHP.

Code that uses the current namespace declaration:

namespace foo;

will assume that all unqualified function calls are global scope.

To use a function in the local namespace, it can be fully qualified
with:

\foo\MyFunction();

But, developers could also write:

namespace foo using local functions;

And all unqualified function names would be resolved to local at
compile time. Global functions could still be accessed with a `\` if
this directive was used:

\array_key_exists();

### OPTION THREE ###

Namespace lookup is removed from a future version of PHP.

Code that uses the current namespace declaration:

namespace foo;

...will assume that an *unqualified* function name is a global function
*IF* it is a PHP built-in function.

Otherwise, *unqualified* function names that are *not* PHP built-in
functions will be presumed to be local to the namespace.

With Option Three, developers can still fully-qualify their functions:

\foo\array_key_exists();

...to override a built-in name with a user function in the current
namespace.

Likewise, a fully-qualified:

\MyFunction();

called from inside a namespace will still call the global function.

Only unqualified names are affected.

As an additional optional feature of Option Three, developers can
change this behavior with:

namespace foo using global functions;

or

namespace foo using local functions;

Only *unqualified* names are affected by this directive, and they use
local only or global only, depending on the namespace declaration.

In both cases, *fully-qualified* names still work the same.

Of course, there are many other possibilities that can be mixed-and-
matched.

Rob Landers

2 years ago

On Fri, Aug 23, 2024, at 11:27, Nick Lockheart wrote:

> On Fri, 2024-08-23 at 09:16 +0100, Rowan Tommins [IMSoP] wrote:
> >
> >
> > On 23 August 2024 01:42:38 BST, Nick Lockheart <lists@ageofdream.com>
> > wrote:
> > >
> > > >
> > > > BUT, if people already complain about "\" being ugly, having to
> > > > write
> > > > "namespace\" is going to make them REALLY grumpy...
> > > > So maybe at the same time (or, probably, in advance) we need to
> > > > come
> > > > up with a nicer syntax for explicitly referencing the current
> > > > namespace.
> > >
> > >    namespace foo using global functions;
> > >
> > > - or -
> > >
> > >    namespace foo using local functions;
> > >
> > >
> > > Tell PHP what you want at the per-file level.
> >
> >
> > This doesn't seem mutually exclusive to me. If you have a file where
> > you've opted for "using global functions", you might want a way to
> > reference a function in the current namespace.
>
> Correct, so if you use the example:
>
>     namespace foo using global functions;
>
> you can write:
>
>     array_key_exists();
>
> and it will be resolved as global without a namespace lookup and will
> use the dedicated opcode.
>
> But if you need to use a local function you can do:
>
>     \foo\sort();
>
>
> The proposed global/local declaration as part of the namespace
> declaration just turns off namespace lookups and sets the default
> resolution for **unqualified** names.
>
> Fully qualified names are not affected.
>
>
> > It also doesn't address my other point, that having global as the
> > default mode (even if we provide an option for local) is much less
> > disruptive to existing code.
>
>
> They are compatible, but related decisions.
>
> I think it would be easier for people to accept a new PHP version where
> unqualified names were always global, if we also had an option to make
> local/namespaced the default resolution for *unqualified* names, on a
> per-file basis, for those who need that.
>
>
> Thus, there are multiple decision points:
>
> 1. Should we do namespace lookups on unqualified function calls at all?
>
> 2. If yes to 1, should we lookup in global first or local first?
>
> 3. Regardless of 1 or 2, should we let developers explicitly specify a
> behavior for unqualified calls in the namespace declaration?
>
> 4. If yes to 1, should the behavior of namespace lookups change for
> user-defined functions vs PHP built-in function names?
>
>
> These aren't mutually exclusive, but they all work together to create a
> complete behavior.
>
> There are several ways that the above options could be combined:
>
>
>
> ### OPTION ONE ###
>
> Using a regular namespace declaration still does an NS lookup, in the
> same order, just like it normally works now.
>
> That means that code that uses:
>
>     namespace foo;
>
> will behave exactly the same as today, with no BC breaks.
>
> Developers using the new PHP version could opt-in to explicit namespace
> behavior with:
>
>     namespace foo using global functions;
>
> or
>
>     namespace foo using local functions;
>
> In both cases, *fully-qualified* names still work the same.
>
> Only *unqualified* names are affected by this directive, and they use
> local only or global only, depending on the declaration.
>
>
>
> ### OPTION TWO ###
>
> Namespace lookup is removed from a future version of PHP.
>
> Code that uses the current namespace declaration:
>
>     namespace foo;
>
> will assume that all unqualified function calls are global scope.
>
> To use a function in the local namespace, it can be fully qualified
> with:
>
>     \foo\MyFunction();
>
>
> But, developers could also write:
>
>      namespace foo using local functions;
>
> And all unqualified function names would be resolved to local at
> compile time. Global functions could still be accessed with a `\` if
> this directive was used:
>
>     \array_key_exists();
>
>
>
> ### OPTION THREE ###
>
> Namespace lookup is removed from a future version of PHP.
>
> Code that uses the current namespace declaration:
>
>     namespace foo;
>
> ...will assume that an *unqualified* function name is a global function
> *IF* it is a PHP built-in function.
>
> Otherwise, *unqualified* function names that are *not* PHP built-in
> functions will be presumed to be local to the namespace.
>
> With Option Three, developers can still fully-qualify their functions:
>
>     \foo\array_key_exists();
>
> ...to override a built-in name with a user function in the current
> namespace.
>
> Likewise, a fully-qualified:
>
>     \MyFunction();
>
> called from inside a namespace will still call the global function.
>
> Only unqualified names are affected.
>
> As an additional optional feature of Option Three, developers can
> change this behavior with:
>
>     namespace foo using global functions;
>
> or
>
>     namespace foo using local functions;
>
>
> Only *unqualified* names are affected by this directive, and they use
> local only or global only, depending on the namespace declaration.
>
> In both cases, *fully-qualified* names still work the same.
>
>
>
> Of course, there are many other possibilities that can be mixed-and-
> matched.
>

I personally would find option 3 to be the best of both worlds, and you don't even need the `namespace ... using ... functions` stuff.

— Rob

Rob Landers

2 years ago

On Fri, Aug 23, 2024, at 14:16, Rob Landers wrote:

> On Fri, Aug 23, 2024, at 11:27, Nick Lockheart wrote:
>> On Fri, 2024-08-23 at 09:16 +0100, Rowan Tommins [IMSoP] wrote:
>> >
>> >
>> > On 23 August 2024 01:42:38 BST, Nick Lockheart <lists@ageofdream.com>
>> > wrote:
>> > >
>> > > >
>> > > > BUT, if people already complain about "\" being ugly, having to
>> > > > write
>> > > > "namespace\" is going to make them REALLY grumpy...
>> > > > So maybe at the same time (or, probably, in advance) we need to
>> > > > come
>> > > > up with a nicer syntax for explicitly referencing the current
>> > > > namespace.
>> > >
>> > >    namespace foo using global functions;
>> > >
>> > > - or -
>> > >
>> > >    namespace foo using local functions;
>> > >
>> > >
>> > > Tell PHP what you want at the per-file level.
>> >
>> >
>> > This doesn't seem mutually exclusive to me. If you have a file where
>> > you've opted for "using global functions", you might want a way to
>> > reference a function in the current namespace.
>>
>> Correct, so if you use the example:
>>
>>     namespace foo using global functions;
>>
>> you can write:
>>
>>     array_key_exists();
>>
>> and it will be resolved as global without a namespace lookup and will
>> use the dedicated opcode.
>>
>> But if you need to use a local function you can do:
>>
>>     \foo\sort();
>>
>>
>> The proposed global/local declaration as part of the namespace
>> declaration just turns off namespace lookups and sets the default
>> resolution for **unqualified** names.
>>
>> Fully qualified names are not affected.
>>
>>
>> > It also doesn't address my other point, that having global as the
>> > default mode (even if we provide an option for local) is much less
>> > disruptive to existing code.
>>
>>
>> They are compatible, but related decisions.
>>
>> I think it would be easier for people to accept a new PHP version where
>> unqualified names were always global, if we also had an option to make
>> local/namespaced the default resolution for *unqualified* names, on a
>> per-file basis, for those who need that.
>>
>>
>> Thus, there are multiple decision points:
>>
>> 1. Should we do namespace lookups on unqualified function calls at all?
>>
>> 2. If yes to 1, should we lookup in global first or local first?
>>
>> 3. Regardless of 1 or 2, should we let developers explicitly specify a
>> behavior for unqualified calls in the namespace declaration?
>>
>> 4. If yes to 1, should the behavior of namespace lookups change for
>> user-defined functions vs PHP built-in function names?
>>
>>
>> These aren't mutually exclusive, but they all work together to create a
>> complete behavior.
>>
>> There are several ways that the above options could be combined:
>>
>>
>>
>> ### OPTION ONE ###
>>
>> Using a regular namespace declaration still does an NS lookup, in the
>> same order, just like it normally works now.
>>
>> That means that code that uses:
>>
>>     namespace foo;
>>
>> will behave exactly the same as today, with no BC breaks.
>>
>> Developers using the new PHP version could opt-in to explicit namespace
>> behavior with:
>>
>>     namespace foo using global functions;
>>
>> or
>>
>>     namespace foo using local functions;
>>
>> In both cases, *fully-qualified* names still work the same.
>>
>> Only *unqualified* names are affected by this directive, and they use
>> local only or global only, depending on the declaration.
>>
>>
>>
>> ### OPTION TWO ###
>>
>> Namespace lookup is removed from a future version of PHP.
>>
>> Code that uses the current namespace declaration:
>>
>>     namespace foo;
>>
>> will assume that all unqualified function calls are global scope.
>>
>> To use a function in the local namespace, it can be fully qualified
>> with:
>>
>>     \foo\MyFunction();
>>
>>
>> But, developers could also write:
>>
>>      namespace foo using local functions;
>>
>> And all unqualified function names would be resolved to local at
>> compile time. Global functions could still be accessed with a `\` if
>> this directive was used:
>>
>>     \array_key_exists();
>>
>>
>>
>> ### OPTION THREE ###
>>
>> Namespace lookup is removed from a future version of PHP.
>>
>> Code that uses the current namespace declaration:
>>
>>     namespace foo;
>>
>> ...will assume that an *unqualified* function name is a global function
>> *IF* it is a PHP built-in function.
>>
>> Otherwise, *unqualified* function names that are *not* PHP built-in
>> functions will be presumed to be local to the namespace.
>>
>> With Option Three, developers can still fully-qualify their functions:
>>
>>     \foo\array_key_exists();
>>
>> ...to override a built-in name with a user function in the current
>> namespace.
>>
>> Likewise, a fully-qualified:
>>
>>     \MyFunction();
>>
>> called from inside a namespace will still call the global function.
>>
>> Only unqualified names are affected.
>>
>> As an additional optional feature of Option Three, developers can
>> change this behavior with:
>>
>>     namespace foo using global functions;
>>
>> or
>>
>>     namespace foo using local functions;
>>
>>
>> Only *unqualified* names are affected by this directive, and they use
>> local only or global only, depending on the namespace declaration.
>>
>> In both cases, *fully-qualified* names still work the same.
>>
>>
>>
>> Of course, there are many other possibilities that can be mixed-and-
>> matched.
>>
>
> I personally would find option 3 to be the best of both worlds, and you don't even need the `namespace ... using ... functions` stuff.
>
> — Rob

Totally sent that before finishing...

My only two concerns are:
1. Calling functions in the current namespace. I don't want that syntax to change.
2. Changing the order might make function autoloading impossible; forever.
If these concerns can be ameliorated, then I don't really care much about the specifics.

— Rob

Stephen Reay

2 years ago

> On 23 Aug 2024, at 15:29, Rowan Tommins [IMSoP] <imsop.php@rwec.co.uk> wrote:
>
> having global as the default mode (even if we provide an option for local) is much less disruptive to existing code.

Hi Rowan,

I don't disagree with this summary of the current state, but I think this misses an important factor: namespaced functions are currently nowhere near as popular as namespaced classes, and a significant part of that is almost certainly because we don't have function autoloading, nor any kind of visibility controls for functions (eg package private).

Making relative function names do the opposite of relative class names sounds like a great way to permanently kill any prospects of encouraging developers to use regular namespaced functions in place of static classes as "bag of functions", which is what we keep hearing we should use - most notably on a recent RFC to embody the concept of a static class.

So we're told "no don't use classes for static functions like that, use proper functions".

We already can't autoload them which makes them less appealing, and less practical.

In a world where global functions take precedence over local ones because some people don't like writing a single \ character, autoloading would be a moot point because if you preference global functions you're implicitly telling developers they shouldn't write namespaced functions, by making them harder and less intuitive to use.

Cheers

Stephen

Paul Dragoonis

2 years ago

On Fri, 23 Aug 2024, 11:02 Stephen Reay, <php-lists@koalephant.com> wrote:

>
>
> > On 23 Aug 2024, at 15:29, Rowan Tommins [IMSoP] <imsop.php@rwec.co.uk>
> wrote:
> >
> > having global as the default mode (even if we provide an option for
> local) is much less disruptive to existing code.
>
> Hi Rowan,
>
> I don't disagree with this summary of the current state, but I think this
> misses an important factor: namespaced functions are currently nowhere near
> as popular as namespaced classes, and a significant part of that is almost
> certainly because we don't have function autoloading, nor any kind of
> visibility controls for functions (eg package private).
>
> Making relative function names do the opposite of relative class names
> sounds like a great way to permanently kill any prospects of encouraging
> developers to use regular namespaced functions in place of static classes
> as "bag of functions", which is what we keep hearing we should use - most
> notably on a recent RFC to embody the concept of a static class.
>
> So we're told "no don't use classes for static functions like that, use
> proper functions".
>
> We already can't autoload them which makes them less appealing, and less
> practical.
>
> In a world where global functions take precedence over local ones because
> some people don't like writing a single \ character, autoloading would be a
> moot point because if you preference global functions you're implicitly
> telling developers they shouldn't write namespaced functions, by making
> them harder and less intuitive to use.
>
>
> Cheers
>
> Stephen

I've taken the time to carefully read Ilija's proposal and all followup
messages.

This is a great proposal, Ilija, it will immediately benefit 95%+ userbase.

It looks to me that the pro's outweigh the con's, as well as Ilija having
done good research here already.

As for next steps, I'm suggesting that Ilija reach out to the core team at
Symfony, Zend, Laravel as well as WordPress, Magento, Drupal teams .. for
their consideration and input, and bring it back here/RFC.

The latter group of project's codebases can be quite "quirky", and quite
creative, in how they use PHP compared to a traditional "framework".

We need to understand how this could negatively impact how their systems
are put together, and we definitely want to, upfront, identify anything we
wouldn't normally think of, that they can spot, since they know their own
codebases better than we do.

We already have the composer analysis, so after we have these
framework/project analysis then we can make a strongly informed decision on
the impact, positively or negatively, this change will make.

Many thanks,
Paul

Rowan Tommins [IMSoP]

2 years ago

On Fri, 23 Aug 2024, at 10:58, Stephen Reay wrote:

> Making relative function names do the opposite of relative class names
> sounds like a great way to permanently kill any prospects of
> encouraging developers to use regular namespaced functions in place of
> static classes as "bag of functions", which is what we keep hearing we
> should use - most notably on a recent RFC to embody the concept of a
> static class.

That's why I brought up the point about making it easy to explicitly say "relative to current", just as you can explicitly say "relative to global" with a single "\".

It's also worth remembering that this whole discussion has no effect on using functions that are defined in a *different* namespace. The below code might benefit from function autoloading, but would not be helped, hurt, or changed in any way by any of the proposals in this thread:

namespace Acme\Foo\Controller;

use function Acme\StringUtils\better_strlen;
use Acme\StandardUtils as Std;

$foo = Std\generate_something();
$len = better_strlen($foo);
\Acme\Debug\out($len);

Regards,

--
Rowan Tommins
[IMSoP]

Stephen Reay

2 years ago

> On 23 Aug 2024, at 17:29, Rowan Tommins [IMSoP] <imsop.php@rwec.co.uk> wrote:
>
> On Fri, 23 Aug 2024, at 10:58, Stephen Reay wrote:
>> Making relative function names do the opposite of relative class names
>> sounds like a great way to permanently kill any prospects of
>> encouraging developers to use regular namespaced functions in place of
>> static classes as "bag of functions", which is what we keep hearing we
>> should use - most notably on a recent RFC to embody the concept of a
>> static class.
>
> That's why I brought up the point about making it easy to explicitly say "relative to current", just as you can explicitly say "relative to global" with a single "\".
>
> It's also worth remembering that this whole discussion has no effect on using functions that are defined in a *different* namespace. The below code might benefit from function autoloading, but would not be helped, hurt, or changed in any way by any of the proposals in this thread:
>
> namespace Acme\Foo\Controller;
>
> use function Acme\StringUtils\better_strlen;
> use Acme\StandardUtils as Std;
>
> $foo = Std\generate_something();
> $len = better_strlen($foo);
> \Acme\Debug\out($len);
>
>
>
> Regards,
> --
> Rowan Tommins
> [IMSoP]
>

I understand what you are proposing, and I understand that it doesn't affect using *other* namespaces.

But let's be realistic here:  if adopted this would mean developers have to understand and internalise two concepts of how relative-local (or unqualified if you prefer that term) symbol names work, that are completely opposite to each other, essentially forever, unless you expect yet another flop to follow this flip?

This change would also break existing code that does "the right thing", and has the potential to arbitrarily break perfectly valid userland code *any time a new global function is added*, forever.

And the reason for all this was...

> prefixing all calls with `\`, or adding a `use function` at the top of every file is annoying and noisy.

For all the handwringing this list does over backwards compatibility breaks, it absolutely astounds me that there is even consideration of such a massive BC break, with the added bonus of guaranteed future BC breaks, to save typing one character per function call (or realistically, configuring your IDE/code linter to do it for you).

This whole discussion is predicated on "people don't use functions much so its not a big BC break" literally at the same time as not one but two different discussions about an RFC for function autoloading, specifically because we're constantly being told that static classes are "wrong" and namespaced functions are "right".

> Sorry to reply to the same message twice, but as a concrete example, consider this code:

I realise you replied to me twice, but for the sake of clarity let me be perfectly clear:

I understand absolutely what you are proposing.

I'm saying it's a ridiculous idea (the whole concept, not just your specific "solution") to propose that such a core facet of the language be changed, because some people both "want the absolute best performance" but also "don't like a leading backslash". Hidden BC breaks *forever* to gain 2-4% performance benefit without typing "\". If you told me my calendar is wrong and today is actually April 1st I would 100% believe you based the contents of this thread.

Cheers

Stephen

Rowan Tommins [IMSoP]

2 years ago

On Fri, 23 Aug 2024, at 13:43, Stephen Reay wrote:

> This change would also break existing code that does "the right thing",
> and has the potential to arbitrarily break perfectly valid userland
> code *any time a new global function is added*, forever.

You replied to me, but you seem to be commenting on one of the other proposals. My preference is for "unqualified = global", which is a one-off breaking change, which only affects user-defined functions, which are declared in a namespace, and used in that same namespace.

You're right that it would mean classes and functions resolve differently, and that's why I said that if I had a time machine, I would support a different option. But, personally, I don't think the small long-term inconsistency outweighs the huge short-term disruption of defaulting to local.

Regards,

--
Rowan Tommins
[IMSoP]

Rowan Tommins [IMSoP]

2 years ago

On Fri, 23 Aug 2024, at 10:58, Stephen Reay wrote:

> In a world where global functions take precedence over local ones
> because some people don't like writing a single \ character,
> autoloading would be a moot point because if you preference global
> functions you're implicitly telling developers they shouldn't write
> namespaced functions, by making them harder and less intuitive to use.

Sorry to reply to the same message twice, but as a concrete example, consider this code:

// Definition
namespace Acme\Foo;
class Utils {
public static function magic(string $x): int {
return \strlen($x);
}
public static function more_magic(string $x): int {
return self::magic($x) * 2;
}
}

// Caller
namespace Acme\MyApp\SearchPage;
use Acme\Foo\Utils;
echo Utils::more_magic($_GET['query']);


Rewritten as namespaced functions, with current PHP:

// Definition
namespace Acme\Foo\Utils;
function magic(string $x): int {
return strlen($x);
}
function more_magic(string $x): int {
return magic($x) * 2;
}

// Caller
namespace Acme\MyApp\SearchPage;
use Acme\Foo\Utils;
echo Utils\more_magic($_GET['query']);



With "unqualified names are global", but a new "_\" shorthand for "relative to current", the caller is completely unaffected, but the definition becomes:

namespace Acme\Foo\Utils;
function magic(string $x): int {
return strlen($x);
}
function more_magic(string $x): int {
return _\magic($x) * 2;
}

Note how the "_\" is used in all the same places as "self::" was in the "static class" version.

With "unqualified names are local", the change is very similar, but "the other way around":

namespace Acme\Foo\Utils;
function magic(string $x): int {
return \strlen($x);
}
function more_magic(string $x): int {
return magic($x) * 2;
}

Regards,

--
Rowan Tommins
[IMSoP]

Rob Landers

2 years ago

On Fri, Aug 2, 2024, at 18:51, Ilija Tovilo wrote:

> It also
> sparked some related ideas, like providing modules that lock
> namespaces and optimize multiple files as a singular unit. That said,
> such approaches would likely be significantly more complex than the
> approach proposed here (~30 lines of C code).

There was an entire thread about modules and packages and shenanigans not too long ago. It’s rather fascinating. Highly recommend participating or starting a new thread. It seems that people are interested in it, and want it.

— Rob

Nick Lockheart

2 years ago

On Fri, 2024-08-02 at 18:51 +0200, Ilija Tovilo wrote:

> Hi everyone
>
> As you probably know, a common performance optimization in PHP is to
> prefix global function calls in namespaced code with a `\`. In
> namespaced code, relative function calls (meaning, not prefixed with
> `\`, not imported and not containing multiple namespace components)
> will be looked up in the current namespace before falling back to the
> global namespace. Prefixing the function name with `\` disambiguates
> the called function by always picking the global function.
>
> Not knowing exactly which function is called at compile time has a
> couple of downsides to this:
>
> * It leads to the aforementioned double-lookup.
> * It prevents compile-time-evaluation of pure internal functions.
> * It prevents compiling to specialized  opcodes for specialized
> internal functions (e.g. strlen()).
> * It requires branching for frameless functions [1].
> * It prevents an optimization that looks up internal functions by
> offset rather than by name [2].
> * It prevents compiling to more specialized argument sending opcodes
> because of unknown by-value/by-reference passing.
>
> All of these are enabled by disambiguating the call. Unfortunately,
> prefixing all calls with `\`, or adding a `use function` at the top
> of
> every file is annoying and noisy. We recently got a feature request
> to
> change how functions are looked up [3].

I think there should be some way to use globals first at compile time.

I had suggested a per-file directive in a post to this list a while
back. Something like:

namespace foo;
use global functions;

class MyClass {

// do stuff.

}

Where `use global functions` would be a special token that the compiler
uses to skip the ns lookup and use dedicated opcodes when available.

Rowan Tommins [IMSoP]

2 years ago

On 2 August 2024 18:19:41 BST, Nick Lockheart <lists@ageofdream.com> wrote:

>I had suggested a per-file directive in a post to this list a while
>back. Something like:
>
>namespace foo;
>use global functions;

There was a proposal for exactly this a few years ago, which ended up in an RFC with a slightly different syntax (using a declare() statement), but was declined in voting by 35 votes to 2.

I can't remember much about the discussion, so am not sure what changes would make a new attempt more likely to pass.

Regards,
Rowan Tommins
[IMSoP]

Nick Lockheart

2 years ago

On Fri, 2024-08-02 at 18:53 +0100, Rowan Tommins [IMSoP] wrote:

>
>
> On 2 August 2024 18:19:41 BST, Nick Lockheart <lists@ageofdream.com>
> wrote:
> > I had suggested a per-file directive in a post to this list a while
> > back. Something like:
> >
> > namespace foo;
> > use global functions;
>
> There was a proposal for exactly this a few years ago, which ended up
> in an RFC with a slightly different syntax (using a declare()
> statement), but was declined in voting by 35 votes to 2.
>
> I can't remember much about the discussion, so am not sure what
> changes would make a new attempt more likely to pass.
>
> Regards,
> Rowan Tommins
> [IMSoP]

In all likelihood, it was the syntax that was disfavored.

What about an RFC where we vote on if the feature should exist, without
any syntax?

ie. "Should there be a way for developers to signal to the parser that
all functions should be treated as global and skip NS lookup, and use
dedicated opcodes? The specific syntax would be decided in a different
RFC/vote if this one passes."

Yes: We should do this, let's discuss syntax possibilities.
No: This should not be a feature at all.

Rowan Tommins [IMSoP]

2 years ago

On Fri, 2024-08-02 at 18:53 +0100, Rowan Tommins [IMSoP] wrote:

> There was a proposal for exactly this a few years ago, which ended up
> in an RFC with a slightly different syntax (using a declare()
> statement), but was declined in voting by 35 votes to 2.

Sorry, I forgot the link: https://wiki.php.net/rfc/use_global_elements

On 2 August 2024 19:46:03 BST, Nick Lockheart <lists@ageofdream.com> wrote:

>In all likelihood, it was the syntax that was disfavored.

There was a lot of discussion beforehand about the syntax, and the RFC attempts to summarise some of it, but skimming through the voting thread <https://externals.io/message/108306> I don't think that was what made it fail. The objections seem to be mostly about the general approach, not the details.


Rowan Tommins
[IMSoP]

Thomas Nunninger

2 years ago

Hi,

Am 02.08.24 um 18:51 schrieb Ilija Tovilo:

...

> There are a few noteworthy downsides:
>
> * Unqualified calls to functions in the same namespace would be
> slightly slower, because they now involve checking global scope first.
> I believe that unqualified, global calls are much more common, so this
> change should still result in a net positive. It's also possible to
> avoid this cost by adding a `use function` to the top of the file.
> * Introducing new functions in the global namespace could cause a BC
> break for unqualified calls, if the function happens to have the same
> name. This is unfortunate, but likely rare. Since new functions are
> only introduced in minor/major versions, this should be manageable,
> but must be considered for every PHP upgrade.
> * Some mocking libraries (e.g. Symfony's ClockMock [5]) intentionally
> declare functions called from some file in the files namespace to
> intercept these calls. This use-case would break. That said, it is
> somewhat of a fragile approach to begin with, given that it wouldn't
> work for fully qualified calls, or unnamespaced code.

Similar to Symfony's ClockMock this "feature" was propagated some years
ago to e.g. intercept calls to the file system when running tests where
the application was not designed with test-ability in mind.

Regards,
Thomas

Claude Pache

2 years ago

Hi,

I propose the following alternative approach:

* establish a restricted whitelist of global functions for which the performance gain would be noteworthy if there wasn’t any need to look at local scope first;

* for those functions, disallow to define a function of same name in any namespace, e.g.: https://3v4l.org/RKnZt

That way, those functions could be optimised, but the current semantics of namespace lookup would remain unchanged.

—Claude

Ilija Tovilo

2 years ago

Hi Claude

On Fri, Aug 2, 2024 at 9:02 PM Claude Pache <claude.pache@gmail.com> wrote:

>
> I propose the following alternative approach:
>
> * establish a restricted whitelist of global functions for which the performance gain would be noteworthy if there wasn’t any need to look at local scope first;
>
> * for those functions, disallow to define a function of same name in any namespace, e.g.: https://3v4l.org/RKnZt
>
> That way, those functions could be optimised, but the current semantics of namespace lookup would remain unchanged.

That would be an improvement over the status quo. However, if you look
at the bullet points in my original email, while some of the
optimizations apply only to some functions (CTE, custom opcodes and
frameless calls), others apply to all internal, global functions
(double-lookup, lookup by offset, specialized argument passing).
Hence, we may only get a fraction of the benefits by restricting the
optimization to a handful of functions. I also wonder if the impact is
actually bigger, as then there's no workaround for redeclaring the
function, requiring much bigger refactoring.

Ilija

Nick Lockheart

2 years ago

On Sun, 2024-08-04 at 19:53 +0200, Ilija Tovilo wrote:

> Hi Claude
>
> On Fri, Aug 2, 2024 at 9:02 PM Claude Pache <claude.pache@gmail.com>
> wrote:
> >
> > I propose the following alternative approach:
> >
> > * establish a restricted whitelist of global functions for which
> > the performance gain would be noteworthy if there wasn’t any need
> > to look at local scope first;
> >
> > * for those functions, disallow to define a function of same name
> > in any namespace, e.g.: https://3v4l.org/RKnZt
> >
> > That way, those functions could be optimised, but the current
> > semantics of namespace lookup would remain unchanged.
>
> That would be an improvement over the status quo. However, if you
> look
> at the bullet points in my original email, while some of the
> optimizations apply only to some functions (CTE, custom opcodes and
> frameless calls), others apply to all internal, global functions
> (double-lookup, lookup by offset, specialized argument passing).
> Hence, we may only get a fraction of the benefits by restricting the
> optimization to a handful of functions. I also wonder if the impact
> is
> actually bigger, as then there's no workaround for redeclaring the
> function, requiring much bigger refactoring.
>
> Ilija

Also, overriding any default function is one of the benefits of name
spacing in the first place. If we say, "built-in functions can't be
overridden, then you are basically saying that all built-in functions
are global.

But there is a valid use case for overriding built-ins. You may want to
disable built-in functionality with a stub for unit testing.

There should probably be a per-file way of setting the default either
way without ns lookups.

use global functions
- or -
use local functions
- or -
omit directive to use dynamic NS lookup for BC.

Bilge

2 years ago

Hi Ilija,

I think this proposal has legs, and you are right to rekindle it,
instead of letting it die quietly.

On 02/08/2024 17:51, Ilija Tovilo wrote:

> * Some mocking libraries (e.g. Symfony's ClockMock [5]) intentionally
> declare functions called from some file in the files namespace to
> intercept these calls. This use-case would break. That said, it is
> somewhat of a fragile approach to begin with, given that it wouldn't
> work for fully qualified calls, or unnamespaced code.
>

My only concern is there needs to be an alternative way to do this:
intercepting internal calls. Sometimes, whether due to poor architecture
or otherwise, we just need to be able to replace an internal function
call. One example I can think of recently is where I had to replace
`header()` with a void function in tests, just to stop some legacy code
emitting headers before the main framework kicked in, then unable to
emit its own response because HTTP headers had already been sent. In a
perfect world it shouldn't be necessary, but sometimes it is, so I think
for this proposal to be palpable there must still be a way to achieve this.

Cheers,
Bilge

Nick Lockheart

2 years ago

On Fri, 2024-08-02 at 21:37 +0100, Bilge wrote:

> Hi Ilija,
> I think this proposal has legs, and you are right to rekindle it,
> instead of letting it die quietly.
> On 02/08/2024 17:51, Ilija Tovilo wrote:
>  
> > * Some mocking libraries (e.g. Symfony's ClockMock [5])
> > intentionally
> > declare functions called from some file in the files namespace to
> > intercept these calls. This use-case would break. That said, it is
> > somewhat of a fragile approach to begin with, given that it
> > wouldn't
> > work for fully qualified calls, or unnamespaced code.
> >
> My only concern is there needs to be an alternative way to do this:
> intercepting internal calls. Sometimes, whether due to poor
> architecture or otherwise, we just need to be able to replace an
> internal function call. One example I can think of recently is where
> I had to replace `header()` with a void function in tests, just to
> stop some legacy code emitting headers before the main framework
> kicked in, then unable to emit its own response because HTTP headers
> had already been sent. In a perfect world it shouldn't be necessary,
> but sometimes it is, so I think for this proposal to be palpable
> there must still be a way to achieve this.
> Cheers,
>  Bilge
>  

I was thinking about a similar problem this week.

If class A relies on class B, but you want to swap out
class B with a stub to test class A in isolation,
is there a way to make every call to class B,
from class A, actually call a different class
during the test, without modifying class A's code?


Minimal code for discussion purposes:


// conf class in global namespace
abstract class CONF {
const DATABASE_HOST_NAME = 'db.example.com';
const DATABASE_NAME = 'production';
const DATABASE_USER_NAME = 'prod_user';
const DATABASE_PASSWORD = '123';
}


// conf class in test namespace:
namespace test;
abstract class CONF {
const DATABASE_HOST_NAME = 'db.sandbox.com';
const DATABASE_NAME = 'test';
const DATABASE_USER_NAME = 'test_user';
const DATABASE_PASSWORD = 'abc';
}


// SQL class in global namespace
class SQL {

private function Init(){
self::$oPDO = new PDO(
'mysql:host='.CONF::DATABASE_HOST_NAME.
';dbname='.CONF::DATABASE_NAME.';charset=utf8mb4',
CONF::DATABASE_USER_NAME,
CONF::DATABASE_PASSWORD,
[]
);
}
}

// Testing class in test namespace:
namespace test;
class SQLTester {

// How do I make the SQL class see \test\CONF instead of
// \CONF, when SQL calls for CONF in this test scope,
/// without changing anything inside of the SQL class?
}


I think some kind of sandboxing tools would be useful for
build/test/deployment.

Christoph Becker

2 years ago

On 03.08.2024 at 00:00, Nick Lockheart wrote:

> I think some kind of sandboxing tools would be useful for
> build/test/deployment.

There are uopz[1] and runkit7[2] available on PECL which can be used to
unit-test untestable code (and more), but you are likely better off to
refactor such code sooner than possible, since such extensions may
easily break for new minor PHP versions (and occasionally, such breaks
may not be fixable at all[3]), and often are completely broken for new
major PHP versions (uopz got a completely different API for PHP 7, and
runkit was even provided as new extension named runkit7).  And
maintainig such extensions is a PITA[4], and as such, compatibility with
new PHP versions may not be available when you need it.

[1] <https://pecl.php.net/package/uopz>
[2] <https://pecl.php.net/package/runkit7>
[3] <https://github.com/krakjoe/uopz/issues/176>
[4] <https://github.com/zenovich/runkit/issues/87>

Cheers,
Christoph

Deleu

2 years ago

On Fri, Aug 2, 2024 at 7:03 PM Nick Lockheart <lists@ageofdream.com> wrote:

> I was thinking about a similar problem this week.
>
> If class A relies on class B, but you want to swap out
> class B with a stub to test class A in isolation,
> is there a way to make every call to class B,
> from class A, actually call a different class
> during the test, without modifying class A's code?
>
>
> Minimal code for discussion purposes:
>
>
> // conf class in global namespace
> abstract class CONF {
>    const DATABASE_HOST_NAME = 'db.example.com';
>    const DATABASE_NAME = 'production';
>    const DATABASE_USER_NAME = 'prod_user';
>    const DATABASE_PASSWORD = '123';
> }
>
>
> // conf class in test namespace:
> namespace test;
> abstract class CONF {
>    const DATABASE_HOST_NAME = 'db.sandbox.com';
>    const DATABASE_NAME = 'test';
>    const DATABASE_USER_NAME = 'test_user';
>    const DATABASE_PASSWORD = 'abc';
> }
>
>
> // SQL class in global namespace
> class SQL {
>
>    private function Init(){
>       self::$oPDO = new PDO(
>          'mysql:host='.CONF::DATABASE_HOST_NAME.
>          ';dbname='.CONF::DATABASE_NAME.';charset=utf8mb4',
>          CONF::DATABASE_USER_NAME,
>          CONF::DATABASE_PASSWORD,
>          []
>       );
>    }
> }
>
> // Testing class in test namespace:
> namespace test;
> class SQLTester {
>
>    // How do I make the SQL class see \test\CONF instead of
>    // \CONF, when SQL calls for CONF in this test scope,
>    /// without changing anything inside of the SQL class?
> }
>
>
> I think some kind of sandboxing tools would be useful for
> build/test/deployment.
>

You could hack this out using the autoloader, but it's something that the
PHP community frowns upon, imo. A much prevalent practice in the PHP
ecosystem is a Dependency Injection container. A somewhat similar concept
exists in the Javascript ecosystem with hoisting import statements and
mocking modules, but if you don't understand the system and are unaware
that order of import execution will matter on whether the mock succeeds or
not plays a huge role in making it a cumbersome and awkward system.

Regardless, it's not possible for functions as users don't control function
autoloader.

--
Marco Deleu

Nick Lockheart

2 years ago

Good morning,

I am writing to request RFC karma for the wiki account with username
`nlockheart`.

I would like to write an RFC for community discussion and
consideration.

Thank you,
Nick Lockheart

Christoph Becker

2 years ago

On 04.08.2024 at 08:19, Nick Lockheart wrote:

> I am writing to request RFC karma for the wiki account with username
> `nlockheart`.
>
> I would like to write an RFC for community discussion and
> consideration.

RFC karma granted.  Good luck with the RFC!

Christoph

Rowan Tommins [IMSoP]

2 years ago

On 02/08/2024 23:00, Nick Lockheart wrote:

> If class A relies on class B, but you want to swap out
> class B with a stub to test class A in isolation,
> is there a way to make every call to class B,
> from class A, actually call a different class
> during the test, without modifying class A's code?

There are libraries that do exactly this, such as Mockery's "overload"
and "alias"; and some that do other transparent manipulations, like
https://github.com/dg/bypass-finals

They work by generating code dynamically, based on the real code, and
executing it before the real definition is loaded. The same approach
could definitely be taken to replace every call to a global function,
and would actually be more reliable than shadowing, because it could
rewrite even calls with a leading "\" or "use function" statement.

Obviously, shadowing a function in one namespace is currently a lot
easier than setting up such a rewriter; but I don't think we should let
that convenience for a few use cases outweigh the benefits in
performance that a change in behaviour could bring, particularly when
combined with function autoloading.

--
Rowan Tommins
[IMSoP]

John Coggeshall

2 years ago

On Aug 2 2024, at 4:37 pm, Bilge <bilge@scriptfusion.com> wrote:

> My only concern is there needs to be an alternative way to do this: intercepting internal calls. Sometimes, whether due to poor architecture or otherwise, we just need to be able to replace an internal function call. One example I can think of recently is where I had to replace `header()` with a void function in tests, just to stop some legacy code emitting headers before the main framework kicked in, then unable to emit its own response because HTTP headers had already been sent. In a perfect world it shouldn't be necessary, but sometimes it is, so I think for this proposal to be palpable there must still be a way to achieve this.
>

Just a tangent thought to the above, but I've always been a little concerned with the idea that a malicious composer package could potentially do nasty things because PHP looks at the local namespace first for functions. For example, if a composer package focused on Laravel that defines malicious versions of internal functions for common namespaces like App\Models , App\Http\Controllers , etc. it could do some nasty stuff -- and supply-chain attacks aren't exactly uncommon. Even worse is Wordpress or any other PHP-based software package that allows arbitrary plugins to be installed by non-technical users who really would have no idea if the package was safe even if they were looking at the code.
<?php
// something.php
namespace App\Models;

function password_hash(string $password, string|int|null $algo, array $options = []): string
{
print("Hello");
return $password;
}

<?php
// my code
namespace App\Models;

include "something.php";
password_hash('foobar', PASSWORD_DEFAULT);
I don't recall why local namespace first won, but IMO it wasn't a great call out the gate for that reason alone. Yes, you can always use \password_hash instead of password_hash , but making the default insecure and slower is silly IMO -- and not fixing it because of BC seems like the weaker argument here.
John

Rob Landers

2 years ago

On Wed, Aug 21, 2024, at 10:23, John Coggeshall wrote:

>
>
> On Aug 2 2024, at 4:37 pm, Bilge <bilge@scriptfusion.com> wrote:
>> My only concern is there needs to be an alternative way to do this: intercepting internal calls. Sometimes, whether due to poor architecture or otherwise, we just need to be able to replace an internal function call. One example I can think of recently is where I had to replace `header()` with a void function in tests, just to stop some legacy code emitting headers before the main framework kicked in, then unable to emit its own response because HTTP headers had already been sent. In a perfect world it shouldn't be necessary, but sometimes it is, so I think for this proposal to be palpable there must still be a way to achieve this.
>
> Just a tangent thought to the above, but I've always been a little concerned with the idea that a malicious composer package could potentially do nasty things because PHP looks at the local namespace first for functions. For example, if a composer package focused on Laravel that defines malicious versions of internal functions for common namespaces like `App\Models` , `App\Http\Controllers` , etc. it could do some nasty stuff -- and supply-chain attacks aren't exactly uncommon.  Even worse is Wordpress or any other PHP-based software package that allows arbitrary plugins to be installed by non-technical users who really would have no idea if the package was safe even if they were looking at the code.
>
> <?php
> // something.php
> namespace App\Models;
>
> function password_hash(string $password, string|int|null $algo, array $options = []): string
> {
>    print("Hello");
>    return $password;
> }
>
> <?php
> // my code
> namespace App\Models;
>
> include "something.php";
>
> password_hash('foobar', PASSWORD_DEFAULT);

If this is an attack vector for your application, then fully qualified names is the way to go (WordPress does this nearly everywhere, for example).

>
> I don't recall why local namespace first won, but IMO it wasn't a great call out the gate for that reason alone.  Yes, you can always use `\password_hash`  instead of `password_hash` , but making the default insecure and slower is silly IMO -- and not fixing it because of BC seems like the weaker argument here.
>
> John

It's not (at least for me) the BC break. It's being able to override global functions. There are legitimate use-cases outside of testing. For example, consider when a global function signature changes. In your library, you have to check the php version. You can change this 100 times for every single call, or you can just wrap it in a function that supports the old signature and proxies it to the new signature. In other words, it provides options that may be better than the alternative.

— Rob

John Coggeshall

2 years ago

On Aug 21 2024, at 8:03 am, Rob Landers <rob@bottled.codes> wrote:

>
> If this is an attack vector for your application, then fully qualified names is the way to go (WordPress does this nearly everywhere, for example).

This is an attack vector for every application and I would argue should be a real concern for the vast majority of applications out there -- any which rely on namespace-based frameworks and composer packages from untrustworthy sources. It's not just Wordpress -- literally every single PHP application that uses a publicly available framework and consumes external composer packages should be FQing their internal function calls. The natural behavior of the language shouldn't be the insecure way of doing things for the sake of maintaining BC compatibility with existing, insecure, code.
Cheers,
John

Ilija Tovilo

2 years ago

Hi John

On Wed, Aug 21, 2024 at 8:02 PM John Coggeshall <john@coggeshall.org> wrote:

>
> This is an attack vector for every application and I would argue should be a real concern for the vast majority of applications  out there -- any which rely on namespace-based frameworks and composer packages from untrustworthy sources. It's not just Wordpress -- literally every single PHP application that uses a publicly available framework and consumes external composer packages should be FQing their internal function calls. The natural behavior of the language shouldn't be the insecure way of doing things for the sake of maintaining BC compatibility with existing, insecure, code.

Including a malicious composer package already allows for arbitrary
code execution, do you really need more than that?

Ilija

John Coggeshall

2 years ago

On Aug 21 2024, at 2:10 pm, Ilija Tovilo <tovilo.ilija@gmail.com> wrote:

>
> Including a malicious composer package already allows for arbitrary
> code execution, do you really need more than that?
>

Of course. We've seen many examples in the wild of 3rd party libraries getting hijacked to inject malicious code (e.g. the whole xz attack). This behavior in PHP is not obvious, and provides a way to covertly target and hijack specific highly sensitive functions without an obvious way to detect it -- while otherwise behaving exactly as a developer would expect.
Why possibly would we want to make it easier to perform such an attack, which as Illija pointed out is actually making PHP slower, in the name of backward compatibility? Defense in depth is a cornerstone of application security.
John

John Coggeshall

2 years ago

Forgive me, s/Illija/you :)

Rob Landers

2 years ago

On Wed, Aug 21, 2024, at 20:32, John Coggeshall wrote:

>
>
> On Aug 21 2024, at 2:10 pm, Ilija Tovilo <tovilo.ilija@gmail.com> wrote:
>>
>> Including a malicious composer package already allows for arbitrary
>> code execution, do you really need more than that?
>
> Of course. We've seen many examples in the wild of 3rd party libraries getting hijacked to inject malicious code (e.g. the whole `xz`  attack). This behavior in PHP is not obvious, and provides a way to covertly target and hijack specific highly sensitive functions without an obvious way to detect it -- while otherwise behaving exactly as a developer would expect.
>
> Why possibly would we want to make it easier to perform such an attack, which as Illija pointed out is actually making PHP slower, in the name of backward compatibility? Defense in depth is a cornerstone of application security.
>
> John

If you have the ability to inject arbitrary code, you've already lost. It doesn't matter whether they use this feature, or just register a shutdown function, autoloader, replace classes/functions/methods entirely, or whatever. Should we remove those features as well?

— Rob

John Coggeshall

2 years ago

On Aug 22 2024, at 4:09 am, Rob Landers <rob@bottled.codes> wrote:

>
> If you have the ability to inject arbitrary code, you've already lost. It doesn't matter whether they use this feature, or just register a shutdown function, autoloader, replace classes/functions/methods entirely, or whatever. Should we remove those features as well?

I think it's a fallacy to claim "well if they got this far the game is over" when it comes to application security. There are a million ways an attacker could use this feature to covertly gain access to things like passwords before they are encrypted, etc. that would enable lateral movement within an organization that otherwise they might have difficulty achieving even with RCE in a properly locked down system (e.g. PHP doesn't have the ability to write to the filesystem / overwrite existing classes, etc.)
Regarding the subject at hand I've made my case here and we can agree to disagree -- changing the function lookup order is an easy win with security benefits and, according to Ilija, performance benefits. I think it should be seriously considered.
John

Derick Rethans

2 years ago

On Fri, 2 Aug 2024, Ilija Tovilo wrote:

> As for providing a migration path: One approach might be to introduce
> an INI setting that performs the function lookup in both local and
> global scope at run-time, and informs the user about the behavioral
> change in the future.

That INI setting would control the *warning*, and not the
*functionlity*, right?

> Lastly, I've already raised this idea in the PHP Foundations internal
> chat but did not receive much positive feedback, mostly due to fear of
> the potential BC impact. I'm not particularly convinced this is an
> issue, given the impact analysis. Given the surprisingly large
> performance benefits, I was inclined to raise it here anyway.

I am surprised that it is that much of a performance benefit as well,
but I am also concerned about the BC impact. But if that isn't too much,
then I guess we need to consider this, but only for a major version. Not
something I believe we can change in a 8.x version.

cheers,
Derick

Ilija Tovilo

2 years ago

On Mon, Aug 5, 2024 at 1:23 PM Derick Rethans <derick@php.net> wrote:

>
> On Fri, 2 Aug 2024, Ilija Tovilo wrote:
>
> > As for providing a migration path: One approach might be to introduce
> > an INI setting that performs the function lookup in both local and
> > global scope at run-time, and informs the user about the behavioral
> > change in the future.
>
> That INI setting would control the *warning*, and not the
> *functionlity*, right?

Yes, that was my suggestion. First, in a future minor version, an INI
option could be added that would warn when finding both a local and
global function when performing an unqualified function call. The only
reason to hide this behind a setting is to avoid the cost of a double
lookup in production code when one isn't necessary, i.e. when calling
local functions in some namespace.

> I am surprised that it is that much of a performance benefit as well,
> but I am also concerned about the BC impact. But if that isn't too much,
> then I guess we need to consider this, but only for a major version. Not
> something I believe we can change in a 8.x version.

Sure, waiting for 9.0 sounds reasonable if we were to choose this approach.

Ilija