Allowing NULL for some internal functions

php.internals

Craig Francis

4 years ago
On Thu, 2 Dec 2021 at 15:19, Sara Golemon <pollita@php.net> wrote:
> On Thu, Dec 2, 2021 at 8:48 AM Craig Francis <craig@craigfrancis.co.uk> > wrote: > >> Is there any value in me proposing an RFC to update *some* internal >> functions so they can accept NULL? >> > > I'm not hard against this idea. The interpretation of null in these > contexts as being equivalent to empty string isn't unreasonable. I guess > the only objection I could have would be an academic one and I can't really > defend that. So yeah, sure... why not? >
Thanks Sara, I've spent a few days coming up with a short(ish) list of parameters that are likely to receive `NULL`. The focus is on making it easier for developers to upgrade to newer versions of PHP. I'm thinking of the typical developer, probably using WordPress, isn't using Psalm (at levels 1 to 3, with no baseline), and isn't using `strict_types`; where they don't really have the time to add strval() to everything that could be NULL. Draft RFC: https://wiki.php.net/rfc/allow_null And the list, where I'm proposing we only relax this requirement for the *bold* parameters: https://github.com/craigfrancis/php-allow-null-rfc/blob/main/functions-change.md I'll give it a few weeks to see if there are any parameter suggestions (welcome via email, pull request, carrier pigeon, etc), then I'll start looking for the best way to implement this. Craig PS; starting new thread, as I have gone a bit off topic, original one at: https://externals.io/message/116519#116556

AllenJB

4 years ago
On 01/01/2022 23:17, Craig Francis wrote:
> I've spent a few days coming up with a short(ish) list of parameters that > are likely to receive `NULL`. > > The focus is on making it easier for developers to upgrade to newer > versions of PHP. I'm thinking of the typical developer, probably using > WordPress, isn't using Psalm (at levels 1 to 3, with no baseline), and > isn't using `strict_types`; where they don't really have the time to add > strval() to everything that could be NULL. > > Draft RFC: > https://wiki.php.net/rfc/allow_null > > And the list, where I'm proposing we only relax this requirement for the > *bold* parameters: > https://github.com/craigfrancis/php-allow-null-rfc/blob/main/functions-change.md > > I'll give it a few weeks to see if there are any parameter suggestions > (welcome via email, pull request, carrier pigeon, etc), then I'll start > looking for the best way to implement this. > > Craig > > > PS; starting new thread, as I have gone a bit off topic, original one at: > https://externals.io/message/116519#116556
This RFC has come out of a similar discussion around deprecating dynamic properties (and this isn't the first time "upgrading is hard" / "the language is moving too fast" has come up), so it's not just this one change that's affected. I think this RFC is trying to solve the wrong problem / only fixing one small portion of the "immediate" problem without paying attention to the bigger picture. Alternative solutions: I feel that effort would be better spent on improving and encouraging use of tooling such as the PHPCompatibility CodeSniffer ruleset (which has unfortunately fallen behind for changes in PHP 8+) and Rector to improve the upgrading process for everyone. I recognize that there's arguments against the project being seen to endorse / contribute to specific tools / libraries / frameworks, but I feel that in this area the benefits outweigh the costs. The project could provide guides (or link to selected existing articles) for upgrading older code. Whilst significant documentation does already exist on this it's not always easy to find or contains errors (a lot of - probably copy-paste - blog articles that I've seen focus on PHPCompatibility for upgrading to 8.0 / 8.1 even tho its coverage of the changes in these versions is poor). This could be done via something like a user-contributed wiki similar to what Python and other projects do: https://wiki.python.org/moin/FrontPage - I recognize that there's significant work in this suggestion, but I feel it also offers good opportunities for further improvements to the PHP ecosystem / community. The migration guides could include documentation on updating existing code for each change (could / should more be done to encourage this being done at the time the change is made rather than trying to compile migration docs just before release?) - they do this sometimes but I feel this is an area with room for improvement. The above mentioned alternatives solve the entire class of problem whilst also maintaining PHP's work towards being a stricter language that's that much harder to create bugs with and needs less "boilerplate" for more defensive programming. Which leads to the argument that in trying to solve a one-time problem for some developers, this RFC creates work for developers both in debugging code, and for developers with defensive programming styles to (re-)add checks to detect the exact class of problem these deprecations / warnings highlight. Developers also already have tools to selectively silence deprecation notices using custom error handlers. On the proposed changes / RFC content: The RFC uses retrieving request parameters (get/post/cookie) values using various frameworks as a (primary) example. With the exception of CodeIgniter (it may be possible, but wasn't immediately obvious from the quickest of glances at their docs) all the frameworks used in the example have an additional default value parameter. This makes all of these (and the null coalesce alternative) relatively trivial to update using a regular expression find and replace, that most text editors can do. The RFC focuses on changing string parameters, but uses mysqli_fetch_row(), json_decode() and error_get_last() in its arguments - none of which are usually used to return a string (or anything you'd want to treat as a string). Accessing properties of non-objects (since PHP 5.0) and accessing non-existant array indexes (since PHP 7.4) have been warnings / notices for a signifcant time already. Some of the suggested functions (parameters) to change are strange selections in my opinion - gz functions, bcmath parameters, password_hash() and mail() (subject / message) are not usually values you want to pass either null or an empty value to (and are almost certainly bugs if the code does that). (And similarly from the maybe list, I would apply this to most of the date_create functions, dns functions hostname and logging functions message parameter) If this change were to be made, I think it would be better to severely restrict the list to only the most commonly used functions. How many codebases affected by the null parameter deprecation are using sodium or MessageFormatter? While this might leave some work for developers upgrading codebases, I feel the long list of functions affected by this RFC (even without considering the maybe list) goes too far. How would code that declares strict_types be affected by this RFC? This isn't clearly defined in the RFC. I would prefer it if these function continue to not accept null (and throw a TypeError) if strict_types is declared.

Craig Francis

4 years ago
Thanks Allen, Before I start, the RFC is still a Draft, and I'm trying to find a solution... I'll also use a bit of sarcasm, just to be entertaining/funny, so it's not a page of boring technical stuff, I'm not arguing or saying you're wrong, and I genuinely appreciate your views on this. On Sun, 2 Jan 2022 at 11:27, AllenJB <php.lists@allenjb.me.uk> wrote:
> I think this RFC is trying to solve the wrong problem / only fixing one > small portion of the "immediate" problem without paying attention to the > bigger picture. >
Dynamic properties do cause problems, and I'm convinced they should deprecated, whereas NULL is a useful value. e.g. when a value has not been set; and then providing NULL to a function like htmlspecialchars(), I don't think should be a problem. I feel that effort would be better spent on improving and encouraging
> use of tooling such as the PHPCompatibility CodeSniffer ruleset.
This one requires variable tracing from source to sink, not an easy task (it's why I noted "Psalm at level 3, with no baseline"). Neither PHPCompatibility or Rector have rules for this at the moment, and I'm pretty sure they never will; other than the really obvious ones, e.g. `str_replace('a', NULL, $name)`, which is clearly a terrible thing to do :-) As an aside, PHPCompatibility did the bad thing of using trim() on getConfigData(), which can return NULL: https://github.com/PHPCompatibility/PHPCompatibility/commit/ab7b0b82d95c82e62f9cffb0f1960f3ccbf0805e https://github.com/squizlabs/PHP_CodeSniffer/blob/d6a58bdd8707bab1032344f33d01c0627e0f3f97/src/Config.php#L1479 As noted in the RFC, while these don't take long to fix, finding them is tricky, it nearly always results in more code (yay?), and it is being seen as wasting time on a fairly questionable improvement. The migration guides could include documentation on updating existing
> code for each change
While migration guides are useful (I appreciate them), I'm focusing on this change to avoid unnecessary work. I don't think nullable variables should need to be explicitly cast to a string before calling these functions - doing so doesen't seem to make the code better (if anything, it's making it more complex). ... this RFC creates work for developers both in debugging code, and for
> developers with > defensive programming styles to (re-)add checks to detect the exact > class of problem these deprecations / warnings highlight. >
For example? e.g. `$name`, if it's set to NULL, why should `ucwords()` have a problem with that? Won't it cause extra work having to explicitly convert it to a string? Consider those who aren't using Static Analysis (or they are using the entry levels, with a baseline), it requires that everyone remembers that while their HTML form would normally provide this value, the user can edit the <form> in their browser DOM, or a browser extension could do something weird (e.g. a password manager), or a network issue results in a partial page load... and kaboom, future Fatal TypeError in the middle of processing? yay? party time? ...all the frameworks used in the example have an additional
> default value parameter. This makes all of these (and the null > coalesce alternative) relatively trivial to update using a regular > expression find and replace, that most text editors can do.
They use a default value of NULL so you can determine when the value has not been provided... should we tell them to stop doing this? default to returning an empty string instead? I'm not sure they will accept that one. Some of the suggested functions (parameters) to change are strange
> selections in my opinion
It's a draft, I just started with as many candidates as I could find, on the basis that it's easier to cut back if NULL can cause a problem. But as to your examples (gz compressing, bcmath, password_hash, and mail); they can all be provided with user input, and other sources of NULL. e.g. `password_hash()`, while I wouldn't advise anyone to use a blank password, or anything less than 8 characters, this function does accept it. And because passwords often come from a POST request, if the value is not present (framework function returns NULL)... well, assuming this deprecation indicates a future where this is not acceptable, what happens? future Fatal TypeError? This is why I have the "Future Scope" section - noting that some parameters could be updated (in a different RFC) to complain when an Empty String or NULL is provided. e.g. `setcookie()` already complains when `$name` is empty, and `strpos()` probably should complain when `$needle` is empty.
> If this change were to be made, I think it would be better to severely > restrict the list to only the most commonly used functions.
I'm fine with that, but what should be removed? Feel free to do this on or off-list, pull request, etc. If you can, I'd really appreciate examples where the NULL would cause a problem but an empty string is fine - i.e. assume developers will be lazy, and just use strval() to fix the hideous crime they have been committing. How would code that declares strict_types be affected by this RFC? Good question... personally I think a trim() on NULL should return an empty string, not a fatal error, so maybe we are looking at this problem in a slightly different way? Craig

Christoph Becker

4 years ago
On 02.01.2022 at 00:17, Craig Francis wrote:
> On Thu, 2 Dec 2021 at 15:19, Sara Golemon <pollita@php.net> wrote: > >> On Thu, Dec 2, 2021 at 8:48 AM Craig Francis <craig@craigfrancis.co.uk> >> wrote: >> >>> Is there any value in me proposing an RFC to update *some* internal >>> functions so they can accept NULL? >> >> I'm not hard against this idea. The interpretation of null in these >> contexts as being equivalent to empty string isn't unreasonable. I guess >> the only objection I could have would be an academic one and I can't really >> defend that. So yeah, sure... why not? > > I've spent a few days coming up with a short(ish) list of parameters that > are likely to receive `NULL`. > > The focus is on making it easier for developers to upgrade to newer > versions of PHP. I'm thinking of the typical developer, probably using > WordPress, isn't using Psalm (at levels 1 to 3, with no baseline), and > isn't using `strict_types`; where they don't really have the time to add > strval() to everything that could be NULL. > > Draft RFC: > https://wiki.php.net/rfc/allow_null > > And the list, where I'm proposing we only relax this requirement for the > *bold* parameters: > https://github.com/craigfrancis/php-allow-null-rfc/blob/main/functions-change.md > > I'll give it a few weeks to see if there are any parameter suggestions > (welcome via email, pull request, carrier pigeon, etc), then I'll start > looking for the best way to implement this.
In my opinion, it makes no sense to special case some functions in this regard; the selection of which parameters of which function to special case will always be too long for some, and to short for others. And even if we find an acceptable compromise, what about functions of external extensions (e.g. PECL packages, but also others, maybe private packages). We have not much, if any, say on these, but they will be affected by the deprecation anyway. And then it's totally unclear to me how this is supposed to affect strict_types=1. Either the respective string parameters are made nullable unconditionally, so null would be accepted in strict mode as well, what doesn't appear to be favorable. Or do we want to special case this kind of nullable parameters, so they are only nullable for strict_types=0? In my opinion, that would be the worst possible outcome. How should this even be documented? If the BC break is deemed to serious (maybe for string parameters only), we better consider to undeprecate this, although that would make internal functions behave differently to userland functions for strict_types=0. But frankly, in that case the original RFC[1] should not have passed, but it did, with 46 votes in favor, and *none* against. [1] <https://wiki.php.net/rfc/deprecate_null_to_scalar_internal_arg> Christoph

Craig Francis

4 years ago
On Sun, 2 Jan 2022 at 13:41, Christoph M. Becker <cmbecker69@gmx.de> wrote:
> [...] > And then it's totally unclear to me how this is supposed to affect > strict_types=1. > [...] > If the BC break is deemed to serious (maybe for string parameters only), > we better consider to undeprecate this, although that would make > internal functions behave differently to userland functions for > strict_types=0. But frankly, in that case the original RFC[1] should > not have passed, but it did, with 46 votes in favor, and *none* against. >
I'm open to suggestions on how this should be handled, my RFC is just a draft. As to the original RFC, there wasn't really a discussion on it: https://externals.io/message/112327 The only person who raised a concern was Craig Duncan. And tbh, just scanning the RFC text, it's very reasonable, as consistency is ideal... but I didn't realise how much code would need to be changed until a few projects I work on started trying to use 8.1 (one team has gone back to 8.0, although I'm hoping they will accept a custom error handler to ignore these deprecation notices). Craig

Andreas Leathley

4 years ago
On 02.01.22 00:17, Craig Francis wrote:
> I've spent a few days coming up with a short(ish) list of parameters that > are likely to receive `NULL`. > > The focus is on making it easier for developers to upgrade to newer > versions of PHP. I'm thinking of the typical developer, probably using > WordPress, isn't using Psalm (at levels 1 to 3, with no baseline), and > isn't using `strict_types`; where they don't really have the time to add > strval() to everything that could be NULL.
I don't think the current deprecation notices are a real hindrance in upgrading to a newer PHP version - there have been many such changes which require possibly many small adaptions in code in the past, like making undefined array keys a warning in PHP 8.0 (which seems a lot more radical than adding the current deprecation notices). Singling out this specific deprecation/change as impactful seems a bit overblown. I also feel like such an un-deprecation would be premature. PHP 8.1 has only just come out, the road until PHP 9.0 is still quite long, and deprecation notices can easily be ignored for now, for those who do not want to change their code right away. If in a year or two this is a big problem in real code, then there would at least be a basis for changing it, but I suspect most codebases will see more benefit in handling their types more properly than if the language "sometimes" automatically converts null to an empty string without complaining about it. I did have some occurances of this deprecation notice in my applications when upgrading to 8.1, and each time it was clearly a bug or oversight. Removing the deprecation notices would also remove those insights into possibly unreliable code.

Craig Francis

4 years ago
On Sun, 2 Jan 2022 at 14:22, Andreas Leathley <a.leathley@gmx.net> wrote:
> I don't think the current deprecation notices are a real hindrance in > upgrading to a newer PHP version.
Nice, but I suspect your code doesen't exactly represents most code :-) And while these may be "small adaptions", I work with some people where the quantity is, impressive... and tbh I can't actually explain what the point for this change is. As an aside, it is making it even harder for me to get them to use static analysis, as they keep saying this stuff is elitist bull... I'll leave it there :-) I did have some occurances of this deprecation notice in my applications
> when upgrading to 8.1, and each time it was clearly a bug or oversight. > Removing the deprecation notices would also remove those insights into > possibly unreliable code. >
Would you mind sharing some of them, as I've not seen anything like that myself (that's genuine interest). Craig

Craig Francis

4 years ago
On Sat, 1 Jan 2022 at 23:17, Craig Francis <craig@craigfrancis.co.uk> wrote:
> Draft RFC: > https://wiki.php.net/rfc/allow_null >
I know, I'm a glutton for punishment, but I've started writing a patch for this: https://github.com/craigfrancis/php-src/compare/master...allow-null It does not affect any Fatal Error exceptions for scripts using 'strict_types'. For now it simply defines `Z_PARAM_STR_ALLOW_NULL`; it's not used by any functions yet (that's the next step). It works a bit like `Z_PARAM_STR_OR_NULL` (but it will return an empty string instead of NULL). It should be a fairly easy drop in replacement for `Z_PARAM_STR`. e.g. To update `htmlspecialchars()`, and related functions, I'd simply update this line: https://github.com/php/php-src/blob/7b90ebeb3f954123915f6d62fb7b2cd3fdf3c6ec/ext/standard/html.c#L1324 Please note I'm not a C programmer, and I barely understand the code base, so constructive/friendly feedback is welcome. Thanks, Craig Extra notes... No-one has given a single reason why any one of the listed functions should not allow `NULL`; that said, I appreciate that developers using `strict_types` do prefer this (and they will not be affected by this). I've not seen any suggestion on how Rector, PHPCompatibility, or anything else could provide a way to fix this issue automatically. Only very strict static analysis (e.g. Psalm at levels 1 to 3, with no baseline) currently picks up this issue. I am simply trying to "relax certain type requirements on a case by case basis" (which will help upgrading to PHP 9); ref the original discussion: https://externals.io/message/112327 I know one person simply said this was a "terrible idea", but I'm still waiting to hear any details on why. In response to the "it's a bit late" position; we only have a deprecation at the moment (can be ignored), it will be too late when PHP 9 starts raising Fatal Error exceptions for anyone committing the heinous crime of passing NULL to these functions (for reference, WordPress aren't exactly rushing to fix this one). The change in behaviour is done in `zend_parse_arg_str_weak()`, so I don't think it will make much of a performance difference; but considering how much effort went in to testing `is_literal()`, and how people still managed to mis-read that, and how other RFCs don't mention performance *details* at all, I'm skipping that bit of pain. And yes, I have seen the -4 on externals.io... thank you for that </sarcasm>.

Mark Randall

4 years ago
On 07/02/2022 01:27, Craig Francis wrote:
> I know one person simply said this was a "terribl > idea", but I'm still > waiting to hear any details on why.
The changes you propose are not something that I am comfortable with either. I understand your motivations in proposing them, but to my mind it goes against the direction that PHP is developing, which I think is the right one, where errors and likely errors result in stopping execution rather than allowing it continue, potentially silently (dependent on error handling settings). If a parameter expects a string, that is what it should be given, and its the callers' responsibility to ensure that is the case. If they fail to do so then it's an error just like any other. IMHO reverting to "If it's a null we'll just pretend its a string" is contrary to how the language should be progressing. It sucks that it was ever allowed in the first place. PHP has a long history of making descisions to try to make things 'just work', and if history teaches us anything, its that we inevitably come to regret these descisions down the line. Mark Randall

Unnamed Person

4 years ago
Den 2022-01-02 kl. 00:17, skrev Craig Francis:
> On Thu, 2 Dec 2021 at 15:19, Sara Golemon <pollita@php.net> wrote: > >> On Thu, Dec 2, 2021 at 8:48 AM Craig Francis <craig@craigfrancis.co.uk> >> wrote: >> >>> Is there any value in me proposing an RFC to update *some* internal >>> functions so they can accept NULL? >>> >> >> I'm not hard against this idea. The interpretation of null in these >> contexts as being equivalent to empty string isn't unreasonable. I guess >> the only objection I could have would be an academic one and I can't really >> defend that. So yeah, sure... why not? >> > > > Thanks Sara, > > I've spent a few days coming up with a short(ish) list of parameters that > are likely to receive `NULL`. > > The focus is on making it easier for developers to upgrade to newer > versions of PHP. I'm thinking of the typical developer, probably using > WordPress, isn't using Psalm (at levels 1 to 3, with no baseline), and > isn't using `strict_types`; where they don't really have the time to add > strval() to everything that could be NULL. > > Draft RFC: > https://wiki.php.net/rfc/allow_null > > And the list, where I'm proposing we only relax this requirement for the > *bold* parameters: > https://github.com/craigfrancis/php-allow-null-rfc/blob/main/functions-change.md > > I'll give it a few weeks to see if there are any parameter suggestions > (welcome via email, pull request, carrier pigeon, etc), then I'll start > looking for the best way to implement this. > > Craig > > > PS; starting new thread, as I have gone a bit off topic, original one at: > https://externals.io/message/116519#116556 >
Hi, Thanks for this RFC! I think it could interesting to see how libraries that needs to adapt to this are doing it IRL. One example I have seen how to fix this for e.g. substr is to replace: $result = substr($string, 0, $length) with: $result = substr($string ?? '', 0, $lenght) So the null coalescing operator comes in quite handy ;-) Now if this is a good way to solve it is another matter. But if you are a maintainer of a library that works fine and wants to support PHP 8.1 this will solve the upgrade in a smooth way. Would be interesting to see how common this approach is for solving PHP 8.1 upgrades for non-nullable arguments with internal functions? I mean, it shows the value with the RFC from a slightly another angle. Regards //Björn

Craig Francis

4 years ago
On Sat, 1 Jan 2022 at 23:17, Craig Francis <craig@craigfrancis.co.uk> wrote:
> Draft RFC: > https://wiki.php.net/rfc/allow_null >
I've been over-complicating this... why should NULL be treated so differently? <?php $values = ['a', 1, 2.3, false, NULL]; foreach ($values as $value) { echo urlencode($value); } ?> It just struck me, this whole thing is about type coercion when not using `strict_types=1`; and the 8.1 change has introduced an inconstancy by deprecating the coercion of NULL, a frequently used value. Craig

Christoph Becker

4 years ago
On 20.02.2022 at 21:38, Craig Francis wrote:
> On Sat, 1 Jan 2022 at 23:17, Craig Francis <craig@craigfrancis.co.uk> wrote: > >> Draft RFC: >> https://wiki.php.net/rfc/allow_null >> > > > I've been over-complicating this... why should NULL be treated so > differently? > > <?php > $values = ['a', 1, 2.3, false, NULL]; > foreach ($values as $value) { > echo urlencode($value); > } > ?> > > It just struck me, this whole thing is about type coercion when not using > `strict_types=1`; and the 8.1 change has introduced an inconstancy by > deprecating the coercion of NULL, a frequently used value.
That "inconsistency" had been introduced with PHP 7.0.0, i.e. right when scalar type declarations have been introduced. Passing a null to a non-nullable parameter of a *userland* function throws a TypeError: <https://3v4l.org/lbF4h>. As of PHP 8.1.0, internal functions behave the same as userland function in this regard.
-- Christoph M. Becker

Craig Francis

4 years ago
On Mon, 21 Feb 2022 at 09:04, Christoph M. Becker <cmbecker69@gmx.de> wrote:
> That "inconsistency" had been introduced with PHP 7.0.0, i.e. right when > scalar type declarations have been introduced. Passing a null to a > non-nullable parameter of a *userland* function throws a TypeError >
Thanks Christoph, that's a good way of looking at it. Considering NULL has been passed to these internal function parameters for years, for a variety of reasons, and PHP has coerced it into an empty string, integer 0, float 0, or boolean false... I'm wondering if my RFC should focus on matching that expectation, so anyone not using `strict_types=1` does not need to make loads of hard-to-find/unnecessary edits; that way NULL would be treated the same as the other variable types (I appreciate arrays and objects don't get coerced, but I don't think anyone expects them to). Craig

Alexandru Pătrănescu

4 years ago
On Mon, Feb 21, 2022 at 5:32 PM Craig Francis <craig@craigfrancis.co.uk> wrote:
> On Mon, 21 Feb 2022 at 09:04, Christoph M. Becker <cmbecker69@gmx.de> > wrote: > > > That "inconsistency" had been introduced with PHP 7.0.0, i.e. right when > > scalar type declarations have been introduced. Passing a null to a > > non-nullable parameter of a *userland* function throws a TypeError > > > > > Thanks Christoph, that's a good way of looking at it. > > Considering NULL has been passed to these internal function parameters for > years, for a variety of reasons, and PHP has coerced it into an empty > string, integer 0, float 0, or boolean false... I'm wondering if my RFC > should focus on matching that expectation, so anyone not using > `strict_types=1` does not need to make loads of hard-to-find/unnecessary > edits; that way NULL would be treated the same as the other variable types > (I appreciate arrays and objects don't get coerced, but I don't think > anyone expects them to). >
Yes, exactly that. A few weeks ago when the issue was mentioned again I had to remember by trying on 3v4l myself that using null for a string parameter wasn't an automatically coerced case when strict_types = 0. In my view, false would have the same problem for those internal functions. Why should it be coerced to an empty string? https://wiki.php.net/rfc/deprecate_null_to_scalar_internal_arg#proposal is all about removing the automatic coercion that was happening for null in the internal functions Planning that in PHP 9 the type interpretation for internal function will be done the same as for user defined functions. Consistency. Right now, if a parameter type is defined as string in an user function, no matter what strict_type is configured, it would still result in a TypeError when passing null. For internal functions, even if the parameter type is defined as string, null would be passed and a coercion would happen. As I mentioned, coercing false to int would be for me almost as bad as coercing null to int. But when types are not considered important I think it's worth pursuing extending the coercion from null to the 4 other types where it's happening right now: - int as 0, - float as 0.0, - string as an empty string - bool as false. I don't like it and I have no good idea how that would work as it would be a pretty big BC break. Maybe it would be easier to change strict_types to default to 1 in PHP 9 along with adding the null coercion for null when strict_types is 0 rather than inventing a new strict_types value like -1 that would allow null coercion as well. Starting with a notice in PHP 8.2 when the directive is not present might be an interesting starting point. And no more warning for implicit coercion when strict_types=0 directive is explicitly defined as it would not change anymore in PHP 9. Regards, Alex

Guilliam Xavier

4 years ago
On Tue, Feb 22, 2022 at 3:59 AM Alexandru Pătrănescu <drealecs@gmail.com> wrote:
> On Mon, Feb 21, 2022 at 5:32 PM Craig Francis <craig@craigfrancis.co.uk> > wrote: > > > On Mon, 21 Feb 2022 at 09:04, Christoph M. Becker <cmbecker69@gmx.de> > > wrote: > > > > > That "inconsistency" had been introduced with PHP 7.0.0, i.e. right > when > > > scalar type declarations have been introduced. Passing a null to a > > > non-nullable parameter of a *userland* function throws a TypeError > > > > > > > > > Thanks Christoph, that's a good way of looking at it. > > > > Considering NULL has been passed to these internal function parameters > for > > years, for a variety of reasons, and PHP has coerced it into an empty > > string, integer 0, float 0, or boolean false... I'm wondering if my RFC > > should focus on matching that expectation, so anyone not using > > `strict_types=1` does not need to make loads of hard-to-find/unnecessary > > edits; that way NULL would be treated the same as the other variable > types > > (I appreciate arrays and objects don't get coerced, but I don't think > > anyone expects them to). > > > > Yes, exactly that. > A few weeks ago when the issue was mentioned again I had to remember by > trying on 3v4l myself that using null for a string parameter wasn't an > automatically coerced case when strict_types = 0. >
For history: https://wiki.php.net/rfc/scalar_type_hints_v5#behaviour_of_weak_type_checks (**emphasis** mine):
> A weakly type-checked call to an extension or built-in PHP function has
exactly the same behaviour as it did in previous PHP versions.
> > The weak type checking rules for the new scalar type declarations are
**mostly** the same as those of extension and built-in PHP functions. The only **exception** to this is the handling of `NULL`: in order to be **consistent with our existing type declarations for classes, callables and arrays**, `NULL` is not accepted by default (...) In hindsight, maybe that wasn't the most "fortunate" decision... (although, fair enough, `NULL` was never considered to be "scalar")
> > In my view, false would have the same problem for those internal functions. > Why should it be coerced to an empty string? > https://wiki.php.net/rfc/deprecate_null_to_scalar_internal_arg#proposal is > all about removing the automatic coercion that was happening for null in > the internal functions > Planning that in PHP 9 the type interpretation for internal function will > be done the same as for user defined functions. Consistency. > > (...) > > As I mentioned, coercing false to int would be for me almost as bad as > coercing null to int. > But when types are not considered important I think it's worth pursuing > extending the coercion from null to the 4 other types where it's happening > right now: > - int as 0, > - float as 0.0, > - string as an empty string > - bool as false. >
Indeed, that could also be a way to solve the original (PHP 7.0) inconsistency between internal and user-defined functions (although PHP 8.1 started to take the opposite route...)
> > I don't like it and I have no good idea how that would work as it would be > a pretty big BC break. > > Maybe it would be easier to change strict_types to default to 1 in PHP 9 > along with adding the null coercion for null when strict_types is 0 rather > than inventing a new strict_types value like -1 that would allow null > coercion as well. > Starting with a notice in PHP 8.2 when the directive is not present might > be an interesting starting point. And no more warning for implicit coercion > when strict_types=0 directive is explicitly defined as it would not change > anymore in PHP 9. >
Or maybe (if going directly from error to implicit coercion is deemed too "risky") the current TypeError in non-strict_types mode (when passing NULL to a user-defined function expecting a scalar) could first be "demoted" to some kind of Notice [who said E_STRICT] in 8.2 (along with reverting the Deprecation added in 8.1 for internal functions) and removed in 9.0?
-- Guilliam Xavier

Craig Francis

4 years ago
On Wed, 23 Feb 2022 at 14:00, Guilliam Xavier <guilliam.xavier@gmail.com> wrote:
> On Tue, Feb 22, 2022 at 3:59 AM Alexandru Pătrănescu <drealecs@gmail.com> > wrote: > > But when types are not considered important I think it's worth pursuing > > extending the coercion from null to the 4 other types where it's > happening > > right now: > > - int as 0, > > - float as 0.0, > > - string as an empty string > > - bool as false. > > > > Indeed, that could also be a way to solve the original (PHP 7.0) > inconsistency between internal and user-defined functions > (although PHP 8.1 started to take the opposite route...) >
Yep, I'm hoping to find a solution that keeps as many people happy as possible, and improves the language. Personally I think `strict_types=1` is fine for my code, but I would never want to force that style on everyone, because doing so would be fairly hostile for a language that's popular and well known for being easy to use/learn. And for those who like strict environments, I'm always interested in what `Content-Security-Policy` they use, because that's a really useful form of strict coding, especially when using `default-src 'none'; script-src https://static.example.com; require-trusted-types-for 'script'; trusted-types 'none';` and `Content-Type: application/xhtml+xml; charset=UTF-8` to ensure quoted attributes :-) Craig

Mark Randall

4 years ago
On 28/02/2022 01:46, Craig Francis wrote:
> Personally I think `strict_types=1` is fine for my code, but I would never > want to force that style on everyone, because doing so would be fairly > hostile for a language that's popular and well known for being easy to > use/learn.
Magically coercing things and hiding things under the hood is useful for the first 10 minutes of learning. After that it just becomes a hindrance difficult because you have to know all of the secret rules about type juggling.

Dik Takken

4 years ago
On 21-02-2022 10:04, Christoph M. Becker wrote:
> That "inconsistency" had been introduced with PHP 7.0.0, i.e. right when > scalar type declarations have been introduced. Passing a null to a > non-nullable parameter of a *userland* function throws a TypeError: > <https://3v4l.org/lbF4h>. As of PHP 8.1.0, internal functions behave > the same as userland function in this regard. >
In my view, consistency between internal and userland functions brings a lot of value, and not only for the language itself. As soon as internal and userland become fully consistent it will become a lot easier to write "internal" functions in PHP rather than C. Not only will that make developing the standard library easier, it may also make the optimizer and JIT compiler more effective. The more consistency the better. Approaches 2 and 3 from the RFC are a step in the right direction. Regards, Dik Takken

Guilliam Xavier

4 years ago
On Mon, Feb 28, 2022 at 5:41 PM Dik Takken <dik.takken@gmail.com> wrote:
> > In my view, consistency between internal and userland functions brings a > lot of value, and not only for the language itself. As soon as internal > and userland become fully consistent it will become a lot easier to > write "internal" functions in PHP rather than C. Not only will that make > developing the standard library easier, it may also make the optimizer > and JIT compiler more effective. The more consistency the better. >
Yes, and we see two possible ways to make them consistent w.r.t. handling of null argument passed into scalar parameter: 1. implicit coercion by default, error with strict_types=1 2. error (independent of strict_types) AFAIK, internal functions have been doing 1 since like forever, but PHP 7.0 chose 2 for userland functions when introducing scalar parameter type declarations (see my previous message for history) and PHP 8.1 continued in that direction by deprecating 1 for internal functions (and planning to change them to 2 in PHP 9.0). Call me devil's advocate, but is it too late to discuss revisiting past decisions and consider changing direction towards 1 for userland functions (esp. in implications of BC impact)? Regards,
-- Guilliam Xavier

Larry Garfield

4 years ago
On Mon, Feb 28, 2022, at 11:35 AM, Guilliam Xavier wrote:
> On Mon, Feb 28, 2022 at 5:41 PM Dik Takken <dik.takken@gmail.com> wrote: > >> >> In my view, consistency between internal and userland functions brings a >> lot of value, and not only for the language itself. As soon as internal >> and userland become fully consistent it will become a lot easier to >> write "internal" functions in PHP rather than C. Not only will that make >> developing the standard library easier, it may also make the optimizer >> and JIT compiler more effective. The more consistency the better. >> > > Yes, and we see two possible ways to make them consistent w.r.t. handling > of null argument passed into scalar parameter: > > 1. implicit coercion by default, error with strict_types=1 > 2. error (independent of strict_types) > > AFAIK, internal functions have been doing 1 since like forever, but PHP 7.0 > chose 2 for userland functions when introducing scalar parameter type > declarations (see my previous message for history) and PHP 8.1 continued in > that direction by deprecating 1 for internal functions (and planning to > change them to 2 in PHP 9.0). > Call me devil's advocate, but is it too late to discuss revisiting past > decisions and consider changing direction towards 1 for userland functions > (esp. in implications of BC impact)?
Under absolutely no circumstances should parameters become implicitly nullable. That way lies madness. And yes, coercing null to a magic zero-value is a form of implicit nullability. Absolutely not, under any circumstances. Bringing internal functions into line with user-space was the correct move. There may be internals functions that make sense to be nullable on their own right, on a case by case basis. We can evaluate that case by case. --Larry Garfield

Craig Francis

4 years ago
On Mon, 28 Feb 2022 at 17:42, Larry Garfield <larry@garfieldtech.com> wrote:
> Bringing internal functions into line with user-space was the correct > move. There may be internals functions that make sense to be nullable on > their own right, on a case by case basis. We can evaluate that case by > case. >
Thanks Larry, I agree about bringing them into line, but I think our understanding of NULL may be different... these are PHP scripts which have used NULL as a distinct value for, well, forever? and many developers expect it to be coerced like the other values. And while some developers use strict_types=1 (like myself) because we like the type to match up without values being coerced (I should be the one that manually chooses to convert)... I don't think I should force that strict coding style onto everyone, because there is nothing technically wrong with passing NULL into functions like urlencode(), it just implies, iff you use strict_types=1, that something may have gone wrong earlier. As an aside, I often note that many developers who talk about how strict their code is... obv not you, but many still use 'unsafe-inline' JavaScript on their websites (mixing content), and don't use Trusted Types to disable unsafe JS APIs, which I consider a much bigger security concern :-) And after all of this, no-one has come up with a way to find or address this problem, e.g. <?php $nullable = ($_GET['a'] ?? NULL); echo htmlentities($nullable); ?> ./vendor/bin/psalm --init ./public/ 4 ./vendor/bin/psalm No errors found! ./vendor/bin/phpstan analyse -l 9 ./public/ [OK] No errors ./vendor/bin/phpcs -p public/ --standard=PHPCompatibility . 1 / 1 (100%) ./vendor/bin/rector process ./public/ [OK] Rector is done! Craig

Christoph Becker

4 years ago
On 28.02.2022 at 21:51, Craig Francis wrote:
> And after all of this, no-one has come up with a way to find or address > this problem, e.g. > > <?php > $nullable = ($_GET['a'] ?? NULL); > echo htmlentities($nullable); > ?>
<?php function my_htmlentities(string|null $value) { return htmlentities($value ?? ""); } $nullable = ($_GET['a'] ?? NULL); echo my_htmlentities($nullable); ?> The BC break doesn't appear to be that serious after all.
-- Christoph M. Becker

Christian Schneider

4 years ago
Am 28.02.2022 um 22:05 schrieb Christoph M. Becker <cmbecker69@gmx.de>:
> On 28.02.2022 at 21:51, Craig Francis wrote: > >> And after all of this, no-one has come up with a way to find or address >> this problem, e.g. >> >> <?php >> $nullable = ($_GET['a'] ?? NULL); >> echo htmlentities($nullable); >> ?> > > <?php > function my_htmlentities(string|null $value) { > return htmlentities($value ?? ""); > } > > $nullable = ($_GET['a'] ?? NULL); > echo my_htmlentities($nullable); > ?> > > The BC break doesn't appear to be that serious after all.
I'm not sure I get your point here: If you provide a user-land implementation of the previous behavior under a different name then the BC break cannot be serious? - Chris

Craig Francis

4 years ago
On Mon, 28 Feb 2022 at 22:11, Christian Schneider <cschneid@cschneid.com> wrote:
> Am 28.02.2022 um 22:05 schrieb Christoph M. Becker <cmbecker69@gmx.de>: > > The BC break doesn't appear to be that serious after all. > > I'm not sure I get your point here: If you provide a user-land > implementation of the previous behavior under a different name then the BC > break cannot be serious? > > - Chris >
Yeah, I'm not sure what the point is either... sorry, I thought the suggestion was a joke, and replied off-list as such, but creating replacement userland functions for all of these functions, and to update all code to use those functions... maybe I'm missing something. And because I'm back to tired pissed-off sarcasm masking depression (why bother spending days on a solution that works for everyone when we can make a hostile/elitist environment/language)... maybe we could simply suggest that everyone affected by this should use strval() for everything? Admittedly the following only does 1 parameter, with a single variable, but at least this is one way to dismiss/ignore the problem: sed -i -E 's/(htmlspecialchars)\((\$[a-zA-Z_\x80-\xff][a-zA-Z0-9_\x80-\xff]*)\)/\1(strval(\2))/i' index.php As to voting at: https://quiz.craigfrancis.co.uk/ So far it's 10 votes to continue supporting null coercing for those not using `strict_types=1` (as PHP always has always done with internal functions, and non-strict comparison operators)... 10 votes to ignore the problem and make a hard backwards compatibility break for PHP 9.0... and 4 votes to update some functions to explicitly allow null (going to round two, you get 1 more vote each, and 2 which drop out because they either didn't provide a second choice, or selected "Don't Mind")... also, while I have counted them, 7 people didn't enter a name, or entered "N/A"... and obviously this vote only helps me setup the RFC, it will be the RFC that will be the official record of who voted for what. Craig

Christoph Becker

4 years ago
On 28.02.2022 at 23:11, Christian Schneider wrote:
> Am 28.02.2022 um 22:05 schrieb Christoph M. Becker <cmbecker69@gmx.de>: > >> On 28.02.2022 at 21:51, Craig Francis wrote: >> >>> And after all of this, no-one has come up with a way to find or address >>> this problem, e.g. >>> >>> <?php >>> $nullable = ($_GET['a'] ?? NULL); >>> echo htmlentities($nullable); >>> ?> >> >> <?php >> function my_htmlentities(string|null $value) { >> return htmlentities($value ?? ""); >> } >> >> $nullable = ($_GET['a'] ?? NULL); >> echo my_htmlentities($nullable); >> ?> >> >> The BC break doesn't appear to be that serious after all. > > I'm not sure I get your point here: If you provide a user-land implementation of the previous behavior under a different name then the BC break cannot be serious?
I said, the BC break doesn't appear to be *that* serious. To elaborate: in my opinion, it is a good thing if internal functions and userland functions behave the same regarding parameter types. As such I'm fine with bringing internal functions inline with userland functions. A somewhat reasonable alternative would be to bring userland functions inline with internal functions (prior to PHP 8.1.0). However, picking *some* functions and make *some* of their parameters nullable does not look reasonable to me; that is just a hack to not break *some* code, but still would not work for a lot of other code (e.g. code using PECL extensions, which may not be changed in this regard), and would make it harder to remember which parameters of which functions are nullable now (and which have special meaning for null, and which do simply treat null as an empty string). Since it has been brought up multiple times that it is hard to find all occurrences which would have to be adapted, I suggested a somewhat viable solution for userland code (namely to define userland functions which establish the old behavior; and of course, you'd need some tooling to replace the original function calls). So everybody *could* use this technique for those functions where it's relevant to the code base at hand.
-- Christoph M. Becker

Craig Francis

4 years ago
On Mon, 28 Feb 2022 at 17:35, Guilliam Xavier <guilliam.xavier@gmail.com> wrote:
> Call me devil's advocate, but is it too late to discuss revisiting past > decisions and consider changing direction towards 1 for userland functions >
Hi Guilliam, tbh, for those who use `strict_types=1` nothing changes, so we can ignore them... for everyone else, nothing would break if user functions did start coercing NULL (if anything it just makes PHP more consistent)... whereas the current direction would add a Type Error exception for developers who aren't familiar with them, and those developers will just get annoyed at having to convert those NULL values manually (see the complaints from developers upgrading to PHP 8.1). Craig

Craig Francis

4 years ago
On Mon, 28 Feb 2022 at 16:41, Dik Takken <dik.takken@gmail.com> wrote:
> In my view, consistency between internal and userland functions brings a > lot of value, and not only for the language itself.
Thanks Dik, I agree that consistency is very important, and I do not want to stop that... I just recognise that many scripts use NULL coercion a lot, as they do with other variable types (when not using `strict_types=1`), and that's being broken just for NULL (which creates a form of inconsistency vs int/float/string/bool types). I started with preferring 2, but I think 1 is the best route to keep the spirit of the original RFC, while not breaking backwards compatibility. Craig

Craig Francis

4 years ago
On Sat, 1 Jan 2022 at 23:17, Craig Francis <craig@craigfrancis.co.uk> wrote:
> Draft RFC: > https://wiki.php.net/rfc/allow_null >
To get a better idea on how I should progress this RFC, I've created a simple quiz (well, modified an old script). This is to decide if my RFC should either - continue to allow NULL to be coerced when not using strict_types=1; or update some ~335 parameters to explicitly allow NULL; or do we just ignore backwards compatibility and let the current deprecation lead to a Fatal Error for everyone? https://quiz.craigfrancis.co.uk/ As an aside, Juliette (@jrfnl) has confirmed that getting PHPCompatibility to solve this problem will be "pretty darn hard to do" because it's "not reliably sniffable" [1]. Craig [1] https://twitter.com/jrf_nl/status/1497937320766496772