Stop replacing dots with underscores in query, post and cookie parameters for PHP 8?

php.internals

Arnold Daniels

7 years ago
Hi, PHP replaces dots with underscores for $_GET, $_POST and $_COOKIE. This behavior once made sense because of Register globals. The explanation in the manual also still implies that query and post parameters are converted to variables (see https://php.net/manual/en/language.variables.external.php#language.variables.external.dot-in-names). Register globals has been removed since 5.4.0 and thus this behavior serves little purpose. I think it would be good to remove the conversion in PHP 8, as it's a general cause of confusion and annoyance for anyone who comes across it. Is there a good reason to keep this behavior in PHP 8? Yours, Arnold [Arnold Daniels - Chat @ Spike](https://www.spikenow.com/?ref=spike-organic-signature&_ts=2oznd) [2oznd]

Sara Golemon

7 years ago
On Mon, Jul 15, 2019 at 8:39 PM Arnold Daniels <arnold.adaniels.nl@gmail.com> wrote:
> PHP replaces dots with underscores for $_GET, $_POST and $_COOKIE. > This behavior once made sense because of Register globals. > The explanation in the manual also still implies that query and > post parameters are converted to variables > (see
https://php.net/manual/en/language.variables.external.php#language.variables.external.dot-in-names ).
> Register globals has been removed since 5.4.0 and thus this behavior
serves little purpose.
> > I think it would be good to remove the conversion in PHP 8, as it's a
general cause of confusion and annoyance for anyone who comes across it.
> > Is there a good reason to keep this behavior in PHP 8? >
IMO, we can safely kill this. If anyone needs this behavior preserved, it can be mimicked with half a dozen lines of PHP, or heck we can include a function to call. I will, however, be very surprised if anyone misses this by now. -Sara

Guilliam Xavier

7 years ago
On Tue, Jul 16, 2019 at 4:39 AM Sara Golemon <pollita@php.net> wrote:
> > On Mon, Jul 15, 2019 at 8:39 PM Arnold Daniels <arnold.adaniels.nl@gmail.com> > wrote: > > PHP replaces dots with underscores for $_GET, $_POST and $_COOKIE. > > This behavior once made sense because of Register globals. > > The explanation in the manual also still implies that query and > > post parameters are converted to variables > > (see > https://php.net/manual/en/language.variables.external.php#language.variables.external.dot-in-names > ). > > Register globals has been removed since 5.4.0 and thus this behavior > serves little purpose. > > > > I think it would be good to remove the conversion in PHP 8, as it's a > general cause of confusion and annoyance for anyone who comes across it. > > > > Is there a good reason to keep this behavior in PHP 8? > > > IMO, we can safely kill this. If anyone needs this behavior preserved, it > can be mimicked with half a dozen lines of PHP, or heck we can include a > function to call. I will, however, be very surprised if anyone misses this > by now. > > -Sara
Indeed, it seems that people are rather strugling to work around the current behavior (and have been for years)... Some examples: - https://bugs.php.net/bug.php?id=40000 - https://stackoverflow.com/questions/68651/get-php-to-stop-replacing-characters-in-get-or-post-arrays - https://github.com/symfony/symfony/issues/9009 - https://github.com/api-platform/core/issues/509 - https://github.com/api-platform/core/blob/v2.4.5/src/Util/RequestParser.php
-- Guilliam Xavier

Nikita Popov

7 years ago
On Tue, Jul 16, 2019 at 3:40 AM Arnold Daniels <arnold.adaniels.nl@gmail.com> wrote:
> Hi, > > PHP replaces dots with underscores for $_GET, $_POST and $_COOKIE. This > behavior once made sense because of Register globals. The explanation in > the manual also still implies that query and post parameters are converted > to variables (see > https://php.net/manual/en/language.variables.external.php#language.variables.external.dot-in-names). > Register globals has been removed since 5.4.0 and thus this behavior serves > little purpose. > > I think it would be good to remove the conversion in PHP 8, as it's a > general cause of confusion and annoyance for anyone who comes across it. > > Is there a good reason to keep this behavior in PHP 8? >
This has been discussed a few times already, and I think that everyone agrees that this behavior should go, but not necessarily on the migration path. There is an RFC here: https://wiki.php.net/rfc/on_demand_name_mangling I think that the latest version of that RFC, that basically proposes to drop the behavior and tell people to use a polyfill is fine. Nikita

Bishop Bettini

7 years ago
On Tue, Jul 16, 2019 at 3:51 AM Nikita Popov <nikita.ppv@gmail.com> wrote:
> On Tue, Jul 16, 2019 at 3:40 AM Arnold Daniels < > arnold.adaniels.nl@gmail.com> > wrote: > > > Hi, > > > > PHP replaces dots with underscores for $_GET, $_POST and $_COOKIE. This > > behavior once made sense because of Register globals. The explanation in > > the manual also still implies that query and post parameters are > converted > > to variables (see > > > https://php.net/manual/en/language.variables.external.php#language.variables.external.dot-in-names > ). > > Register globals has been removed since 5.4.0 and thus this behavior > serves > > little purpose. > > > > I think it would be good to remove the conversion in PHP 8, as it's a > > general cause of confusion and annoyance for anyone who comes across it. > > > > Is there a good reason to keep this behavior in PHP 8? > > > > This has been discussed a few times already, and I think that everyone > agrees that this behavior should go, but not necessarily on the migration > path. There is an RFC here: > https://wiki.php.net/rfc/on_demand_name_mangling > I think that the latest version of that RFC, that basically proposes to > drop the behavior and tell people to use a polyfill is fine. >
I've readied the proposal for formal discussion. As proposed: - PHP 8.0 will no longer mangle variable names in any super-global. - The changelog will recommend auditing super-global access for mangled names, and replacing with the actual variable name. - No INI settings will engage the behavior. - No warnings or notices will be emitted. - A polyfill will be made available to emulate that original behavior. - Applications requiring name mangling shall invoke the polyfill during bootstrap phase. https://wiki.php.net/rfc/on_demand_name_mangling

Zeev Suraski

7 years ago
On Tue, Jul 16, 2019 at 5:34 AM Bishop Bettini <bishop@php.net> wrote:
> On Tue, Jul 16, 2019 at 3:51 AM Nikita Popov <nikita.ppv@gmail.com> wrote: > > > On Tue, Jul 16, 2019 at 3:40 AM Arnold Daniels < > > arnold.adaniels.nl@gmail.com> > > wrote: > > > > > Hi, > > > > > > PHP replaces dots with underscores for $_GET, $_POST and $_COOKIE. This > > > behavior once made sense because of Register globals. The explanation > in > > > the manual also still implies that query and post parameters are > > converted > > > to variables (see > > > > > > https://php.net/manual/en/language.variables.external.php#language.variables.external.dot-in-names > > ). > > > Register globals has been removed since 5.4.0 and thus this behavior > > serves > > > little purpose. > > > > > > I think it would be good to remove the conversion in PHP 8, as it's a > > > general cause of confusion and annoyance for anyone who comes across > it. > > > > > > Is there a good reason to keep this behavior in PHP 8? > > > > > > > This has been discussed a few times already, and I think that everyone > > agrees that this behavior should go, but not necessarily on the migration > > path. There is an RFC here: > > https://wiki.php.net/rfc/on_demand_name_mangling > > I think that the latest version of that RFC, that basically proposes to > > drop the behavior and tell people to use a polyfill is fine. > > > > I've readied the proposal for formal discussion. As proposed: > > - PHP 8.0 will no longer mangle variable names in any super-global. > - The changelog will recommend auditing super-global access for mangled > names, and replacing with the actual variable name. > - No INI settings will engage the behavior. > - No warnings or notices will be emitted. > - A polyfill will be made available to emulate that original behavior. > - Applications requiring name mangling shall invoke the polyfill during > bootstrap phase. > > https://wiki.php.net/rfc/on_demand_name_mangling
I think it looks good and well thought-through. The only thing I'd add is having the userland polyfill function implementation available/referenced in the Upgrade notes, so that folks can get it without Composer (in addition of having it available in Composer). I suspect the ones who'd actually won't to use it are quite likely to not be Composer users. Zeev

Nikita Popov

7 years ago
On Tue, Jul 16, 2019 at 2:34 PM Bishop Bettini <bishop@php.net> wrote:
> On Tue, Jul 16, 2019 at 3:51 AM Nikita Popov <nikita.ppv@gmail.com> wrote: > > > On Tue, Jul 16, 2019 at 3:40 AM Arnold Daniels < > > arnold.adaniels.nl@gmail.com> > > wrote: > > > > > Hi, > > > > > > PHP replaces dots with underscores for $_GET, $_POST and $_COOKIE. This > > > behavior once made sense because of Register globals. The explanation > in > > > the manual also still implies that query and post parameters are > > converted > > > to variables (see > > > > > > https://php.net/manual/en/language.variables.external.php#language.variables.external.dot-in-names > > ). > > > Register globals has been removed since 5.4.0 and thus this behavior > > serves > > > little purpose. > > > > > > I think it would be good to remove the conversion in PHP 8, as it's a > > > general cause of confusion and annoyance for anyone who comes across > it. > > > > > > Is there a good reason to keep this behavior in PHP 8? > > > > > > > This has been discussed a few times already, and I think that everyone > > agrees that this behavior should go, but not necessarily on the migration > > path. There is an RFC here: > > https://wiki.php.net/rfc/on_demand_name_mangling > > I think that the latest version of that RFC, that basically proposes to > > drop the behavior and tell people to use a polyfill is fine. > > > > I've readied the proposal for formal discussion. As proposed: > > - PHP 8.0 will no longer mangle variable names in any super-global. > - The changelog will recommend auditing super-global access for mangled > names, and replacing with the actual variable name. > - No INI settings will engage the behavior. > - No warnings or notices will be emitted. > - A polyfill will be made available to emulate that original behavior. > - Applications requiring name mangling shall invoke the polyfill during > bootstrap phase. > > https://wiki.php.net/rfc/on_demand_name_mangling
Can you please start a new [RFC] thread for this? Nikita

Rasmus Schultz

7 years ago
On Tue, Jul 16, 2019 at 2:34 PM Bishop Bettini <bishop@php.net> wrote:
> On Tue, Jul 16, 2019 at 3:51 AM Nikita Popov <nikita.ppv@gmail.com> wrote: > > > On Tue, Jul 16, 2019 at 3:40 AM Arnold Daniels < > > arnold.adaniels.nl@gmail.com> > > wrote: > > > > > Hi, > > > > > > PHP replaces dots with underscores for $_GET, $_POST and $_COOKIE. This > > > behavior once made sense because of Register globals. The explanation > in > > > the manual also still implies that query and post parameters are > > converted > > > to variables (see > > > > > > https://php.net/manual/en/language.variables.external.php#language.variables.external.dot-in-names > > ). > > > Register globals has been removed since 5.4.0 and thus this behavior > > serves > > > little purpose. > > > > > > I think it would be good to remove the conversion in PHP 8, as it's a > > > general cause of confusion and annoyance for anyone who comes across > it. > > > > > > Is there a good reason to keep this behavior in PHP 8? > > > > > > > This has been discussed a few times already, and I think that everyone > > agrees that this behavior should go, but not necessarily on the migration > > path. There is an RFC here: > > https://wiki.php.net/rfc/on_demand_name_mangling > > I think that the latest version of that RFC, that basically proposes to > > drop the behavior and tell people to use a polyfill is fine. > > > > I've readied the proposal for formal discussion. As proposed: > > - PHP 8.0 will no longer mangle variable names in any super-global. > - The changelog will recommend auditing super-global access for mangled > names, and replacing with the actual variable name. > - No INI settings will engage the behavior. > - No warnings or notices will be emitted. > - A polyfill will be made available to emulate that original behavior. > - Applications requiring name mangling shall invoke the polyfill during > bootstrap phase. > > https://wiki.php.net/rfc/on_demand_name_mangling
What about libraries that need to support both PHP 7 and 8? Many libraries will likely need to, potentially for a long time. How will these libraries be able to detect whether the names have been mangled or not? Libraries that "unmangle" run the risk of "unmangling" names that aren't actually mangled, and could end up mangling them instead. This is a particular concern with the proposed polyfill - if there's a polyfill that mangles the names, you can't just use something like php_version() to check if it's version 8, since the polyfill could have been applied. Maybe consider adding something like $_SERVER["REAL_HEADER_NAMES"] which, if set, will indicate the names have not been mangled? The polyfill could then unset() this value - it could even check first if the names are already non-mangled, whether this is because it's running under PHP 8, because the polyfill gets bootstrapped twice (which it'll probably need to check somehow anyway) or because somebody else polyfilled the same change in some other way.

Dan Ackroyd

7 years ago
On Thu, 18 Jul 2019 at 11:08, Rasmus Schultz <rasmus@mindplay.dk> wrote:
> > What about ...
Although there might be real problems for this, please can you describe a concrete problem that will occur, in a library people use, rather than a vague possible problem? For most framework users, all of the request variables are only accessed through either a PSR7 request object or their framework specific equivalent, and the problem of mangling/demangling can be solved there. So for most people this isn't going to be a problem imo. Without a clear description of what the problem is, coming up with solutions is not an effective exercise. cheers Dan Ack

Rasmus Schultz

7 years ago
On Thu, Jul 18, 2019 at 4:28 PM Dan Ackroyd <Danack@basereality.com> wrote:
> On Thu, 18 Jul 2019 at 11:08, Rasmus Schultz <rasmus@mindplay.dk> wrote: > > > > What about ... > > Although there might be real problems for this, please can you > describe a concrete problem that will occur, in a library people use, > rather than a vague possible problem? >
Well, both underscores and hyphens are permitted in header names - so a library that replaces underscores with hyphens potentially mangles real header names. (Mangling and unmangling are both lossy operations.)
> For most framework users, all of the request variables are only > accessed through either a PSR7 request object or their framework > specific equivalent, and the problem of mangling/demangling can be > solved there. >
Yes, that's my concern - if they don't know whether the names are already mangled or not, they might apply the unmangling, unintentionally mangling actual valid header names. Besides also wasting some CPU time - no point in processing header-names if they're already properly formatted. Mainly, all I'm suggesting is that there be some way to tell if the names were either never mangled, or have already been unmangled, so we can avoid single or multiple (redundant and/or lossy) mangling/unmangling operations in the polyfill and PSR-7 libraries. Since this has to do with $_SERVER, it seems natural to keep it in $_SERVER["REAL_HEADER_NAMES"], which would be set to true on PHP 8. So in the polyfill: if (isset($_SERVER["REAL_HEADER_NAMES"])) { // mangle names... unset($_SERVER["REAL_HEADER_NAMES"]); } And in the PSR-7 libraries: if (isset($_SERVER["REAL_HEADER_NAMES"])) { // use names as is } else { // unmangle names } This is just an example - I'm indifferent as to the name and location of this value. Someone suggested using a constant, but you can't unset a constant - for the polyfill and PHP 8 both to be seen by a PSR-7 library as having real header names, the polyfill needs to be able to clear that flag. (With a constant, you'd need an additional check against php_version() - just trying to keep this as simple as possible.)

Andrew Faulds

7 years ago
Hi all, Rasmus Schultz wrote:
> > What about libraries that need to support both PHP 7 and 8? Many libraries > will likely need to, potentially for a long time. > > How will these libraries be able to detect whether the names have been > mangled or not? Libraries that "unmangle" run the risk of "unmangling" > names that aren't actually mangled, and could end up mangling them instead. > > This is a particular concern with the proposed polyfill - if there's a > polyfill that mangles the names, you can't just use something like > php_version() to check if it's version 8, since the polyfill could have > been applied. > > Maybe consider adding something like $_SERVER["REAL_HEADER_NAMES"] which, > if set, will indicate the names have not been mangled? The polyfill could > then unset() this value - it could even check first if the names are > already non-mangled, whether this is because it's running under PHP 8, > because the polyfill gets bootstrapped twice (which it'll probably need to > check somehow anyway) or because somebody else polyfilled the same change > in some other way. >
Years ago I also wanted to remove the name mangling, but for backwards-compatibility reasons I think we should make it as easy as possible to migrate. My preferred solution would be to add a new built-in function that re-does the mangling exactly as it used to be done. It would be no great maintenance burden on us to maintain such a function for the future, but it would avoids userland having to reimplement it multiple times and less performantly or perhaps with bugs. Imagine: array php_bc_mangle_request_vars(array $input); Then an app relying on the legacy behaviour could do this in its index.php and breathe easy: $_GET = php_bc_mangle_request_vars($_GET); $_POST = php_bc_mangle_request_vars($_POST); // etc Or for example, something using some form of Request object that didn't do de-mangling could use this function on its $post array. What does everyone think? Andrea

Nicolas Grekas

7 years ago
Le mar. 16 juil. 2019 à 09:51, Nikita Popov <nikita.ppv@gmail.com> a écrit :
> On Tue, Jul 16, 2019 at 3:40 AM Arnold Daniels < > arnold.adaniels.nl@gmail.com> > wrote: > > > Hi, > > > > PHP replaces dots with underscores for $_GET, $_POST and $_COOKIE. This > > behavior once made sense because of Register globals. The explanation in > > the manual also still implies that query and post parameters are > converted > > to variables (see > > > https://php.net/manual/en/language.variables.external.php#language.variables.external.dot-in-names > ). > > Register globals has been removed since 5.4.0 and thus this behavior > serves > > little purpose. > > > > I think it would be good to remove the conversion in PHP 8, as it's a > > general cause of confusion and annoyance for anyone who comes across it. > > > > Is there a good reason to keep this behavior in PHP 8? > > > > This has been discussed a few times already, and I think that everyone > agrees that this behavior should go, but not necessarily on the migration > path. There is an RFC here: > https://wiki.php.net/rfc/on_demand_name_mangling > I think that the latest version of that RFC, that basically proposes to > drop the behavior and tell people to use a polyfill is fine. >
The RFC doesn't tell about parse_str(), while this is the userland variant of this. I think it could be useful to make parse_str accept a third argument starting with PHP 7.4! That would allow userland code to opt-in for the new behavior asap, and would provide the core implementation for ppl that want to keep the current behavior for PHP8. Makes sense? Nicolas

Lauri Kenttä

7 years ago
On 2019-07-16 04:39, Arnold Daniels wrote:
> PHP replaces dots with underscores for $_GET, $_POST and $_COOKIE. > Is there a good reason to keep this behavior in PHP 8? >
There's a BC break that should be noted: <input type=image name=i> produces i.x=1&i.y=2 Since the dots can't be changed with HTML, this would then require different code to support both PHP 7 and 8. Also, now that we're at it, what about the following: i[].x=1 currently produces ['i' => [0 => '1']], losing the x. i[x].y=1&i[x].z=2 produces ['i' => ['x' => '2']]. Maybe something to be done to address these cases?
-- Lauri Kenttä