[Concept] declare(strict_identifiers=1)

php.internals

otzelot2021@outlook.de

6 days ago
Hi internals, I would like to gauge reaction before writing an RFC. PHP's scanner defines identifiers on bytes rather than code points:     LABEL  [a-zA-Z_\x80-\xff][a-zA-Z0-9_\x80-\xff]* Every byte >= 0x80 is accepted, so UTF-8 identifiers work by accident; the manual says as much. There is no encoding validation, no normalisation requirement and no UAX #31 conformance. As a result ${"\xFF\xFE"} is a valid variable name, and $x and $x<U+00A0> are two distinct variables that render identically. I am not proposing that PHP accept more characters -- it already accepts everything. I am proposing a per-file declare under which the accepted set is specified: well-formed UTF-8, UAX31-R1-2 with the standard Default-Ignorable Exclusion Profile, and NFC required rather than applied. Identifiers consisting only of bytes below 0x80 are never examined, so existing code pays nothing. To find out what this would break I surveyed the 250 most-downloaded Packagist packages and 250 GitHub repositories -- 168,604 PHP files -- using ext/tokenizer. The Packagist corpus contains exactly one non-ASCII identifier, and no identifier in either corpus is non-NFC. Tooling, raw data and per-identifier CSVs are here:     https://github.com/Otzie2023/PHP The patch would not touch the re2c scanner: the byte rule is already maximally permissive and cannot split a UTF-8 sequence, so a strict mode only ever rejects and the check can run after the token is formed. About 13.7 KiB of generated tables, with no dependency on intl, ICU or mbstring. I would write and maintain it. Is this worth an RFC, or is there an objection I should know about before I spend the time? Regards, Luca Rodenhaeuser

Juris Evertovskis

6 days ago
> -----Original Message----- > From: otzelot2021@outlook.de <otzelot2021@outlook.de> > Sent: Wednesday, August 26, 2026 6:15 PM > To: internals@lists.php.net > Subject: [PHP-DEV] [Concept] declare(strict_identifiers=1) > > [..] I am proposing a per-file declare under which the accepted set > is specified: well-formed UTF-8, UAX31-R1-2 with the standard Default- > Ignorable Exclusion Profile, and NFC required rather than applied. Identifiers > consisting only of bytes below 0x80 are never examined, so existing code > pays nothing.
Hey Luca, To prevent errors? I must admit I don't rly understand all the terms. I assume it implies identifiers should be more visible/readable, right?
> To find out what this would break I surveyed the 250 most-downloaded
Why would anything break if it's per-file?
> Packagist packages and 250 GitHub repositories -- 168,604 PHP files -- using > ext/tokenizer. The Packagist corpus contains exactly one non-ASCII identifier, > and no identifier in either corpus is non-NFC.
Do I understand it correctly that by adding that declare to 168604 you would uncover a single risky identifier? Not that convincing... Would it be fair to say that the same constraints can be enforced by linters/cs tooling? BR, Juris

otzelot2021@outlook.de

5 days ago
Hey Juris,
> To prevent errors? I must admit I don't rly understand all the terms. > I assume it implies identifiers should be more visible/readable, right?
Not readability -- unambiguity. Three concrete things, no jargon: 1. Two identifiers that look identical on screen are currently two    different identifiers. $x and $x followed by U+00A0 NO-BREAK SPACE    are separate variables. So are "a" plus a combining diaeresis and the    single character U+00E4; both display as a-umlaut. 2. Text that is not valid UTF-8 at all is currently a valid identifier.    ${"\xFF\xFE"} = 1; compiles. 3. Nobody can state what a PHP identifier is except by copying the byte    class out of the scanner. The manual does exactly that, and so does    PHP-Parser. The declare says: in this file, an identifier is well-formed UTF-8, is a Unicode identifier in the sense of UAX #31 (the Unicode annex that defines this for programming languages), and is written in one canonical spelling. Anything else is a compile error.
> Why would anything break if it's per-file?
Nothing breaks. That was bad wording on my part. The survey answers a different question: if a maintainer adds the declare to a file they already have, does it still compile? That is adoption friction, not breakage. It is also a calibration check -- a rule that rejected a lot of legitimate existing code would be the wrong rule, and I wanted to know that before proposing it rather than after.
> Do I understand it correctly that by adding that declare to 168604 you > would uncover a single risky identifier? Not that convincing...
That number is the cost, not the benefit, and I should have separated the two more clearly. The single Packagist finding is what would stop compiling: symfony/cache declares a class whose entire name is the single byte 0xA9. That is the whole measured adoption cost across the 250 most-installed packages. What the rule catches can only be measured where non-ASCII identifiers actually occur, which is not in the top Packagist packages. In the GitHub corpus, 33 of 136 non-ASCII identifiers fail the rule and 11 contain an invisible character. One is live code: the Alipay OpenAPI SDK has     $chrtext<U+00A0> = null;     // ...     openssl_public_encrypt($block, $chrtext<U+00A0>, $res); with a no-break space inside the name. It works only because the typo is consistent throughout the function. Anyone who types $chrtext normally gets a different variable, passed by reference, that stays null. Five vendored copies across four unrelated projects in my sample. But you are right that these are small numbers, and I would rather say so than dress them up. The case does not rest on the bug count.
> Would it be fair to say that the same constraints can be enforced by > linters/cs tooling?
Partly yes, and I will concede that plainly: UTF-8 validity, NFC and UAX #31 conformance are all statically checkable. My survey tool is exactly such a linter, written in PHP, and it is in the repository. Three things it cannot do. It cannot define the language. "What is a valid PHP identifier" currently has no answer other than "whatever bytes the scanner happened to accept", which is why the manual, PHP-Parser, every IDE and every static analyser separately reverse-engineer the same byte class. A declare makes it a versioned, testable statement. It does not travel with the code. A declare is in the source file; a lint configuration is in someone's toolchain. The file keeps its guarantee after composer install, and the rule also covers generated and eval'd code that never reaches a linter. And it cannot touch semantics. Case-insensitive symbol lookup folds ASCII only, so Stra<U+00DF>e and STRA<U+00DF>E are the same class while Stra<U+1E9E>e is not, and Strasse and STRASSE are. No linter can fix that, and I do not think it can sensibly be fixed before there is a definition of what an identifier is. I deliberately kept case folding out of this proposal, but that is the thing underneath it. The same objection would apply to strict_types -- static analysers check types, so why does the engine need a per-file declaration? I do not think the answer there was "it doesn't", though I accept the parallel is not exact, since strict_types changes runtime behaviour and this does not. This is the weakest point of the proposal and you found it on the first reading. If the list's view is that specifying the rule and leaving enforcement to tooling is the right scope, that is a smaller and possibly better RFC, and I would rather establish that now than after writing the patch. Thanks for the questions. Regards, Luca

Larry Garfield

5 days ago
On Wed, Aug 26, 2026, at 2:42 PM, otzelot2021@outlook.de wrote:
> The survey answers a different question: if a maintainer adds the declare > to a file they already have, does it still compile? That is adoption > friction, not breakage. It is also a calibration check -- a rule that > rejected a lot of legitimate existing code would be the wrong rule, and I > wanted to know that before proposing it rather than after. > > > Do I understand it correctly that by adding that declare to 168604 you > > would uncover a single risky identifier? Not that convincing... > > That number is the cost, not the benefit, and I should have separated the > two more clearly. > > The single Packagist finding is what would stop compiling: symfony/cache > declares a class whose entire name is the single byte 0xA9. That is the > whole measured adoption cost across the 250 most-installed packages.
If this is so rarely seen in the wild (something that should be verified with more than 250 packages), why make it an option? Just plan that PHP 9 will enforce UTF-8-or-GTFO rules on identifiers, Symfony updates one oddball class, and we move on with life. 99.99% of developers won't notice anything happened. --Larry Garfield

Derick Rethans

5 days ago
On 26 August 2026 22:24:06 BST, Larry Garfield <larry@garfieldtech.com> wrote:
>On Wed, Aug 26, 2026, at 2:42 PM, otzelot2021@outlook.de wrote: > >> The survey answers a different question: if a maintainer adds the declare >> to a file they already have, does it still compile? That is adoption >> friction, not breakage. It is also a calibration check -- a rule that >> rejected a lot of legitimate existing code would be the wrong rule, and I >> wanted to know that before proposing it rather than after. >> >> > Do I understand it correctly that by adding that declare to 168604 you >> > would uncover a single risky identifier? Not that convincing... >> >> That number is the cost, not the benefit, and I should have separated the >> two more clearly. >> >> The single Packagist finding is what would stop compiling: symfony/cache >> declares a class whose entire name is the single byte 0xA9. That is the >> whole measured adoption cost across the 250 most-installed packages. > >If this is so rarely seen in the wild (something that should be verified with more than 250 packages), why make it an option? Just plan that PHP 9 will enforce UTF-8-or-GTFO rules on identifiers, Symfony updates one oddball class, and we move on with life. 99.99% of developers won't notice anything happened. > >--Larry Garfield
Is it important enough to have this memory footprint added to each PHP process though?
> About 13.7 KiB of generated tables
cheers Derick

سپهر محمودی

5 days ago
در تاریخ پنجشنبه ۲۷ اوت ۲۰۲۶، ۰۱:۲۳ Derick Rethans <derick@php.net> نوشت:
> On 26 August 2026 22:24:06 BST, Larry Garfield <larry@garfieldtech.com> > wrote: > >On Wed, Aug 26, 2026, at 2:42 PM, otzelot2021@outlook.de wrote: > > > >> The survey answers a different question: if a maintainer adds the > declare > >> to a file they already have, does it still compile? That is adoption > >> friction, not breakage. It is also a calibration check -- a rule that > >> rejected a lot of legitimate existing code would be the wrong rule, and > I > >> wanted to know that before proposing it rather than after. > >> > >> > Do I understand it correctly that by adding that declare to 168604 > you > >> > would uncover a single risky identifier? Not that convincing... > >> > >> That number is the cost, not the benefit, and I should have separated > the > >> two more clearly. > >> > >> The single Packagist finding is what would stop compiling: symfony/cache > >> declares a class whose entire name is the single byte 0xA9. That is the > >> whole measured adoption cost across the 250 most-installed packages. > > > >If this is so rarely seen in the wild (something that should be verified > with more than 250 packages), why make it an option? Just plan that PHP 9 > will enforce UTF-8-or-GTFO rules on identifiers, Symfony updates one > oddball class, and we move on with life. 99.99% of developers won't notice > anything happened. > > > >--Larry Garfield > > Is it important enough to have this memory footprint added to each PHP > process though? > > > About 13.7 KiB of generated tables > > cheers > Derick
------------ Hi Derick, Good point. The ~13.7 KiB footprint comes from the static lookup tables generated for fast classification. A couple of aspects regarding how this is handled / can be optimized: 1. Shared Memory (`.rodata`): Since these tables are declared as `static const`, in standard multi-process setups (e.g. PHP-FPM), they reside in read-only memory pages shared across processes rather than allocating per-process heap memory. 2. Compacting / Range Encoding: We can definitely look into compressing the lookup tables (e.g., using run-length/interval encoding or two-stage lookup tables) to bring the table size well under a few kilobytes if the raw table footprint is a concern. I’m happy to explore compressing the tables or benchmarking the memory impact across different setups to ensure the footprint remains negligible. Best regards, Sepehr

otzelot2021@outlook.de

5 days ago
> Is it important enough to have this memory footprint added to each PHP > process though? > >> About 13.7 KiB of generated tables
It is not added to each process, and I should have been clearer about what the number is. The tables are static const arrays, so they land in .rodata. That section is mapped read-only, and every process started from the same binary maps the same physical pages. The marginal cost of an additional FPM worker is page table entries, not the data. For scale, measured on Ubuntu 24.04 with PHP 8.3.6:     mbstring.so              1,209,216 bytes       of which .rodata         780,276 bytes     php binary               5,784,016 bytes PHP therefore already carries roughly 760 KiB of read-only Unicode tables in mbstring alone, for character set conversion. On the source side the largest single files in php-src are unicode_table_uhc.h at 402 KB, unicode_table_cns11643.h at 373 KB, unicode_table_jis.h at 308 KB and unicode_data.h at 267 KB. 13.7 KiB is 1.16 % of mbstring.so and 0.24 % of the php binary. Two caveats I would rather state than have found. The 13.7 KiB is a naive figure: sorted uint32 range pairs, 693 ranges for Start, 807 for Continue, 251 for the NFC quick check, with the profile subtraction already applied. A two-stage table would be smaller. I have not measured by how much and would rather not quote a number I have not measured. The tables are linked unconditionally, so the footprint does not depend on whether any file actually uses the declare. For the same reason it does not scale with the number of processes either. Runtime cost is a separate question and is zero for ASCII. The first thing the check does is scan for a byte >= 0x80; an identifier made only of ASCII returns immediately, which is every identifier in essentially all existing code. One clarification for the list, since messages have appeared in this thread that read as if speaking for the proposal: I am the only person working on it, none of those replies were coordinated with me, and I would ask that my own messages be taken as its position. Regards, Luca

سپهر محمودی

5 days ago
در تاریخ پنجشنبه ۲۷ اوت ۲۰۲۶، ۰۰:۵۶ Larry Garfield <larry@garfieldtech.com> نوشت:
> On Wed, Aug 26, 2026, at 2:42 PM, otzelot2021@outlook.de wrote: > > > The survey answers a different question: if a maintainer adds the declare > > to a file they already have, does it still compile? That is adoption > > friction, not breakage. It is also a calibration check -- a rule that > > rejected a lot of legitimate existing code would be the wrong rule, and I > > wanted to know that before proposing it rather than after. > > > > > Do I understand it correctly that by adding that declare to 168604 you > > > would uncover a single risky identifier? Not that convincing... > > > > That number is the cost, not the benefit, and I should have separated the > > two more clearly. > > > > The single Packagist finding is what would stop compiling: symfony/cache > > declares a class whose entire name is the single byte 0xA9. That is the > > whole measured adoption cost across the 250 most-installed packages. > > If this is so rarely seen in the wild (something that should be verified > with more than 250 packages), why make it an option? Just plan that PHP 9 > will enforce UTF-8-or-GTFO rules on identifiers, Symfony updates one > oddball class, and we move on with life. 99.99% of developers won't notice > anything happened. > > --Larry Garfield >
------------ Fair point, Larry. Moving to enforce strict UTF-8 identifier rules in PHP 9 would definitely simplify things and eliminate edge cases cleanly. The main goal here was to highlight the current ambiguity and explore whether a transitional path or immediate strictness is preferred. Doing a broader ecosystem check before finalizing the PHP 9 deprecation path makes total sense.

otzelot2021@outlook.de

5 days ago
> If this is so rarely seen in the wild (something that should be verified > with more than 250 packages), why make it an option?  Just plan that
PHP 9
> will enforce UTF-8-or-GTFO rules on identifiers, Symfony updates one > oddball class, and we move on with life.  99.99% of developers won't > notice anything happened.
I ran it: the top 5,000 Packagist packages by downloads, 4,863 of them resolvable and non-empty, 520,802 PHP files, 2.4 GB. Raw scanner output, per-identifier CSVs and the corpus manifest are in the repository. It is not one oddball class. 1,447 non-ASCII identifiers across 25 packages. 1,312 of them -- 91 % -- are in markrogoyski/math-php, and they are neither fixtures nor accidents. They are variable names in src/, and they spell the formula:     $n! = self::factorial($n);     $∑  = 0;     protected $d₁;     $π  = \M_PI;     $│∑│      = $∑->det();     $√⟮2π⟯ᵏ│∑│ = \sqrt((2 * $π) ** $k * $│∑│); 1,247 of the 1,312 are T_VARIABLE. 888 of them would stop compiling: the offenders are U+27EE/U+27EF mathematical flattened parentheses, subscript and superscript digits, U+2211 summation, U+2212 minus. None of those is in XID_Continue. So the honest answer to "99.99 % won't notice" is that one library would notice 888 times, and its author chose that style deliberately and has shipped it for years. The rest of the picture argues the other way, though, and I would rather give you both halves. Outside math-php there are 135 non-ASCII identifiers, and only 25 fail -- 15 of which are one test file in hoa/console with arrow characters in method names (case_move_↑, U+2191). Everything else already conforms:   mjaschen/phpgeo     22, all conforming: $φ, $λ, $sinλ, $cos2σM in                       src/Bearing/BearingEllipsoidal.php -- Vincenty                       geodesy, Greek letters only, no symbols   tracy/tracy         14, all conforming: $ʟ_tmp, $ʟ_tag in Latte                       templates, U+029F as a namespace prefix so internal                       variables cannot collide with user ones   wsdltophp/...       44, all conforming: setСубъектРФ, ApiАдресРФ --                       generated accessors over a Russian WSDL schema,                       where the names come from the schema And across all 520,802 files, zero identifiers are not in NFC. So the cost of making this mandatory is specific and nameable rather than a long tail: it outlaws mathematical symbol notation in identifiers. Two packages do that. Everything else in the top 5,000 that uses non-ASCII identifiers already satisfies the rule and would not notice. That may still be the right call -- "PHP identifiers are letters, not notation" is a defensible position and I am not arguing against it. But it is a decision to withdraw something that works today, not a cleanup of an oddity, and I would rather the list took it with the number in front of it. One correction to something I said in the other subthread: I told Chris there was not one identifier mixing Latin with Cyrillic or Greek. At 250 packages that was true. At 5,000 there are 41 outside math-php, and they are benign -- a code generator putting an ASCII "set" in front of a Cyrillic schema element. My mixed-script metric flags the script pair, not the risk, and I should have said so. Regards, Luca

Unnamed Person

5 days ago
On 26-8-2026 21:42, otzelot2021@outlook.de wrote:
> Not readability -- unambiguity. Three concrete things, no jargon: > > 1. Two identifiers that look identical on screen are currently two > different identifiers. $x and $x followed by U+00A0 NO-BREAK SPACE > are separate variables. So are "a" plus a combining diaeresis and the > single character U+00E4; both display as a-umlaut. > > 2. Text that is not valid UTF-8 at all is currently a valid identifier. > ${"\xFF\xFE"} = 1; compiles. > > 3. Nobody can state what a PHP identifier is except by copying the byte > class out of the scanner. The manual does exactly that, and so does > PHP-Parser. > > The declare says: in this file, an identifier is well-formed UTF-8, is a > Unicode identifier in the sense of UAX #31 (the Unicode annex that > defines this for programming languages), and is written in one canonical > spelling. Anything else is a compile error. >
Hi Luca & list, Reading this and seeing you talk about making it a compile time error, raises the question for me of how this will interact with variable variables which don't comply with the proposed rules - AFAICS those wouldn't be able to be a compile time error and they also wouldn't have been found in the scan of Packagist files. I imagine "on the fly" class creation, like when mocking code may also run into issues with this up to a point ? Those are also the things which static analysis of code would not be able to find or flag (if this were left to static analysis). Curious to hear your thoughts on this. Smile, Juliette

otzelot2021@outlook.de

4 days ago
> raises the question for me of how this will interact with variable > variables which don't comply with the proposed rules - AFAICS those > wouldn't be able to be a compile time error and they also wouldn't have > been found in the scan of Packagist files.
Correct on both counts. Claude Pache gave me better vocabulary for the first half than I had, and I went and measured the second half rather than guessing at it. PHP distinguishes names from identifiers. An identifier is a lexical token the scanner produces from source text; a name is any string that reaches a symbol table, and that set is far larger -- you can create a variable named "+!" or a class alias named "" today. What I described governs identifiers only. $$name, define(), class_alias() and property names materialised by json_decode() are untouched. That is not a hole I carved out for convenience; it is a line PHP already draws and already enforces syntactically. On your second point: you were right that my scan could not see these, so I extended it. Names built from a run-time string remain unmeasurable by anyone, but the statically visible subset is not -- literals in name-creating and name-looking-up positions: ${'...'}, ->{'...'}, ?->{'...'}, Foo::${'...'}, define(), constant(), class_alias(), property_exists(), method_exists() and friends. Across the same 4,863 packages and 520,802 files, that turns up **five** non-ASCII names, in two packages. All five, in full: halaxa/json-machine, src/TokensWithDebugging.php:39, under the author's own comment "Treat UTF-8 BOM bytes as whitespace":     ${"\xEF"} = ${"\xBB"} = ${"\xBF"} = 0; Three variables named after the individual bytes of the UTF-8 BOM, used as a lookup table alongside ${' '}, ${"\n"}, ${'{'} and so on. None of the three is valid UTF-8 on its own, and none of them could be written as an identifier at all. rowbot/url (vendored into wp-php-toolkit/data-liberation), tests/WhatWg/URLSearchParamsConstructorTest.php:209-210:     $obj3->{"c\u{D83D}"} = '23';     $obj3->{"d\u{1234}"} = 'foo'; WHATWG URL conformance test data. U+D83D is a lone high surrogate, which PHP encodes as ED A0 BD and which is not well-formed UTF-8; U+1234 is ordinary Ethiopic. Four of the five would fail the identifier rule, and that is the point rather than a problem: every one of them is a deliberate use of the name syntax precisely because it is not an identifier. So as far as I can measure it, the existing split is doing its job. People who need non-identifier names reach for the name syntax, rarely and on purpose. I would not have believed that number before running it, and I would not have run it if you had not asked.
> I imagine "on the fly" class creation, like when mocking code may also > run into issues with this up to a point ?
This one has a definite answer, which I checked rather than assumed: code passed to eval() does not inherit strict_types from the calling file. It is compiled as its own unit and may carry its own declare; include behaves the same way. So under an opt-in model, PHPUnit, Mockery and Prophecy generating classes at run time sit outside it entirely. Under a mandatory model they would be checked, but a generated mock name derives from the mocked class, so a non-conforming name only appears if the class being mocked already had one.
> Those are also the things which static analysis of code would not be able > to find or flag (if this were left to static analysis).
That is the strongest argument for putting any of this in the engine that anyone has made in this thread, including me, and I do not want to claim more from it than it gives. A linter can never reach run-time names. But the engine as I described it does not reach them either. The difference is that the engine could be extended there and a linter could not -- and extending it means validating every symbol table insertion, a cost paid on every dynamic property write, and json_decode() throwing on untrusted input. I think that is a much larger and probably worse proposal. It is, though, a real option rather than an impossible one, which is more than the tooling route offers. The scanner change and the raw output are in the repository. Regards, Luca

Christian Schneider

5 days ago
Am 26.08.2026 um 17:15 schrieb otzelot2021@outlook.de:
> I am not proposing that PHP accept more characters -- it already accepts > everything. I am proposing a per-file declare under which the accepted > set is specified: well-formed UTF-8, UAX31-R1-2 with the standard > Default-Ignorable Exclusion Profile, and NFC required rather than > applied. Identifiers consisting only of bytes below 0x80 are never > examined, so existing code pays nothing.
What problem would this restriction solve? Is it about an code smuggling attack vector using code obfuscation with indistinguishable Unicode sequences?
> The patch would not touch the re2c scanner: the byte rule is already > maximally permissive and cannot split a UTF-8 sequence, so a strict > mode only ever rejects and the check can run after the token is formed. > About 13.7 KiB of generated tables, with no dependency on intl, ICU or > mbstring. I would write and maintain it. > > Is this worth an RFC, or is there an objection I should know about > before I spend the time?
Not sure if I think it is worth the effort but maybe you can shine some light on why we want this. I'm currently leaning to -1 on this, - Chris

otzelot2021@outlook.de

5 days ago

Christian Schneider

5 days ago
Am 26.08.2026 um 21:57 schrieb Luca Rodenhäuser <otzelot2021@outlook.de>:
> What it does solve, ordered by how much I think each is actually worth: > > 1. PHP has no definition of an identifier. The only answer to "what is a > valid PHP identifier" is "whatever bytes the scanner accepted", which > is why the manual, PHP-Parser, every IDE and every static analyser > each copy out the same byte class. That is a language-definition gap, > not a bug report.
I'm not sure why you consider a formal definition like LABEL [a-zA-Z_\x80-\xff][a-zA-Z0-9_\x80-\xff]* not a definition. Personally I find this a simpler definition (and easier to implement in a parser) than something like UTF-8, UAX31-R1-2 with the standard Default-Ignorable Exclusion Profile But then again I'm not really using non-ASCII identifiers today.
> 2. Identifiers that render identically are different identifiers. A > no-break space or a decomposed umlaut inside a name is invisible in > every editor. Real, but rare: 11 instances in 168,604 files.
I understand your point. But I'm not so worried about accidental mixups here. And this is also something an LSP or Linter can help you with if it a real concern for you.
> 3. Case-insensitive lookup folds ASCII only. Stra<U+00DF>e and > STRA<U+00DF>E are the same class; Stra<U+1E9E>e is not, and Strasse is > not. That rule is coherent only if identifiers are ASCII.
Case-insensitive folding adds another problem: Would you be using IntlChar::FOLD_CASE_DEFAULT or IntlChar::FOLD_CASE_EXCLUDE_SPECIAL_I to fold "I"? Or would you base it on a language setting? In general I think most people consider the case-folding for identifiers nowadays to be a bug, not a feature, so I would probably rather try to reduce than extend it. Regards, - Chris

سپهر محمودی

5 days ago
در تاریخ پنجشنبه ۲۷ اوت ۲۰۲۶، ۰۲:۵۷ Christian Schneider < cschneid@cschneid.com> نوشت:
> Am 26.08.2026 um 21:57 schrieb Luca Rodenhäuser <otzelot2021@outlook.de>: > > What it does solve, ordered by how much I think each is actually worth: > > > > 1. PHP has no definition of an identifier. The only answer to "what is a > > valid PHP identifier" is "whatever bytes the scanner accepted", which > > is why the manual, PHP-Parser, every IDE and every static analyser > > each copy out the same byte class. That is a language-definition gap, > > not a bug report. > > I'm not sure why you consider a formal definition like > LABEL [a-zA-Z_\x80-\xff][a-zA-Z0-9_\x80-\xff]* > not a definition. Personally I find this a simpler definition (and easier > to implement in a parser) than something like > UTF-8, UAX31-R1-2 with the standard Default-Ignorable Exclusion > Profile > > But then again I'm not really using non-ASCII identifiers today. > > > 2. Identifiers that render identically are different identifiers. A > > no-break space or a decomposed umlaut inside a name is invisible in > > every editor. Real, but rare: 11 instances in 168,604 files. > > I understand your point. But I'm not so worried about accidental mixups > here. And this is also something an LSP or Linter can help you with if it a > real concern for you. > > > 3. Case-insensitive lookup folds ASCII only. Stra<U+00DF>e and > > STRA<U+00DF>E are the same class; Stra<U+1E9E>e is not, and Strasse is > > not. That rule is coherent only if identifiers are ASCII. > > Case-insensitive folding adds another problem: Would you be using > IntlChar::FOLD_CASE_DEFAULT or IntlChar::FOLD_CASE_EXCLUDE_SPECIAL_I to > fold "I"? Or would you base it on a language setting? > > In general I think most people consider the case-folding for identifiers > nowadays to be a bug, not a feature, so I would probably rather try to > reduce than extend it. > > Regards, > - Chris >
--------- Hi Chris, Thanks for the solid points. Let me clarify the perspective behind these: 1. Lexer simplicity vs. Semantic definition: [a-zA-Z_\x80-\xff] is indeed trivial for the lexer, but it isn't an identifier specification in terms of character semantics—it's essentially "ASCII identifiers plus any high byte". This was originally a pragmatic way to allow Latin-1 / UTF-8 bytes to pass through unchanged. The problem arises when we consider what an identifier semantically is across tooling, ASTs, and static analyzers versus raw byte streaming. 2. Invisible characters and Linters: You're right that linters/LSPs can catch these, but language specifications usually define identifier boundaries (such as TR31 / UAX #31) precisely so that the baseline definition of a valid symbol doesn't require third-party tooling to reject canonically confusing or invisible code points. 3. Case Folding: I completely agree with your assessment here. Extending ASCII case-folding to full Unicode casing (with all the locale subtleties like the dotted/dotless Turkish I) would be opening Pandora's box. The argument wasn't necessarily to expand case-folding to Unicode, but rather to highlight the existing inconsistency: PHP treats identifiers as case-insensitive on the ASCII plane while allowing non-ASCII bytes that are strictly case-sensitive. If the consensus leans toward treating case-insensitivity as historical baggage, clarifying the identifier grammar and transition paths (especially looking ahead to PHP 9 / UTF-8 requirements) is exactly the right discussion to have. Best regards, Sepehr

otzelot2021@outlook.de

4 days ago
> I'm not sure why you consider a formal definition like >         LABEL  [a-zA-Z_\x80-\xff][a-zA-Z0-9_\x80-\xff]* > not a definition. Personally I find this a simpler definition (and easier > to implement in a parser) than something like >         UTF-8, UAX31-R1-2 with the standard Default-Ignorable Exclusion >         Profile
You are right and I overreached. It is a definition, it is precise, and it is far simpler to implement -- one character class against roughly 13.7 KiB of generated tables. That is a real cost and I should not have written as if PHP had nothing. What I should have said is that it is a complete definition of *bytes* and a silent one about *characters*. It fixes no encoding, so the same file is two different programs depending on how you read it, and it cannot tell a character from half of one. The part that actually bothers me is that PHP does not stay at the byte level. Case-insensitive symbol lookup is defined over characters -- ASCII characters. So there is already one character-level rule bolted onto a byte-level definition, and the seam between them is where Stra<U+00DF>e/STRA<U+00DF>E/Stra<U+1E9E>e/Strasse comes from.
> But then again I'm not really using non-ASCII identifiers today.
Then nothing here would ever fire for you. The check I described never looks at an identifier whose bytes are all below 0x80, which is every identifier in most codebases. That is not a footnote -- it is why the thing can be considered at all.
> And this is also something an LSP or Linter can help you with if it a
real
> concern for you.
I answered that at length to Juris and to Rowan and will not repeat it. Two things I did not know then and do now: the measurement is 68 identifiers with an invisible character across 520,802 files, and not all of them are accidents -- math-php puts U+00A0 inside variable names deliberately. That makes it warning-shaped rather than error-shaped, which is closer to your position than to my original one.
> Case-insensitive folding adds another problem: Would you be using > IntlChar::FOLD_CASE_DEFAULT or IntlChar::FOLD_CASE_EXCLUDE_SPECIAL_I to > fold "I"? Or would you base it on a language setting?
PHP has already answered this, which I think settles it in your favour. The 8.2 RFC "Locale-independent case conversion" used precisely the Turkish dotted I as its motivating example: before PHP 8.0 the engine inherited the system locale, so case folding varied by installation. The resolution was to make folding ASCII-only everywhere, deliberately and by vote. So the answer to "which folding" is that PHP tried locale-sensitive folding, found it unworkable, and retreated. I would not want to walk back into it.
> In general I think most people consider the case-folding for identifiers > nowadays to be a bug, not a feature, so I would probably rather try to > reduce than extend it.
Agreed, and I want to be clear I never proposed extending it. I listed it as evidence that the current rule is incoherent, not as something to grow. Incoherent is equally an argument for reducing. Worth saying though that reducing is not cheap either: making class and function names case-sensitive is a far larger break than anything discussed in this thread. I have no proposal there, only the observation that the rule as it stands makes sense only if identifiers are ASCII, and 1,447 of them in the top 5,000 packages are not. Regards, Luca

Claude Pache

5 days ago
> Le 26 août 2026 à 17:15, otzelot2021@outlook.de a écrit : > > Hi internals, > > I would like to gauge reaction before writing an RFC. > > PHP's scanner defines identifiers on bytes rather than code points: > > LABEL [a-zA-Z_\x80-\xff][a-zA-Z0-9_\x80-\xff]* > > Every byte >= 0x80 is accepted, so UTF-8 identifiers work by accident; > the manual says as much. There is no encoding validation, no > normalisation requirement and no UAX #31 conformance. As a result > ${"\xFF\xFE"} is a valid variable name, and $x and $x<U+00A0> are two > distinct variables that render identically.
Hi, There is some confusion here: A valid name is made up of any string, not just strings that are identifiers. Example of use of a variable named "+!" : https://3v4l.org/QSJmh Example of use of a class (more precisely a class alias) named "" (yes, the empty string): https://3v4l.org/TQL18 Of course, names that are not identifiers cannot appear in constructs that accept an identifier only, such as an `extends` clause.
> > (...). I am proposing a per-file declare under which the accepted > set is specified: well-formed UTF-8, UAX31-R1-2 with the standard > Default-Ignorable Exclusion Profile, and NFC required rather than > applied. (...)
I don’t think that a per-file declare is a reasonable option. I would love if the PHP compiler could complain with a clearer error whenever I type accidentally a non-breaking space. But I wouldn’t add a declare clause at the top of each and every file just for that. —Claude

otzelot2021@outlook.de

5 days ago
> There is some confusion here: A valid name is made up of any string, not > just strings that are identifiers. > Example of use of a variable named "+!" (...) > Of course, names that are not identifiers cannot appear in constructs
that
> accept an identifier only, such as an `extends` clause.
You are right, and I have been sloppy about it throughout this thread. PHP does distinguish the two: an identifier is a lexical token the scanner produces from source text, a name is whatever string ends up in a symbol table, and the second set is much larger than the first. Your ${'+!'} and the empty-string class alias make that concrete in a way my wording did not. That distinction helps the proposal rather than undermining it. What I am describing governs identifiers only. Names created at run time through $$name, define(), class_alias() or json_decode() are untouched -- and that is not a gap I invented to make the scope convenient, it is a line PHP already draws and enforces syntactically, exactly as you say with `extends`. Juliette raised the same question from the other side in another subthread; I will answer her with your terminology rather than mine. I will fix the wording in the draft.
> I don't think that a per-file declare is a reasonable option. I would
love
> if the PHP compiler could complain with a clearer error whenever I type > accidentally a non-breaking space. But I wouldn't add a declare clause at > the top of each and every file just for that.
You are the third person to push back on the declare. Larry asked why it is optional at all, and Rowan is circling the same ground from the direction of rejecting versus normalising. Three people arriving there independently is a signal, and I am no longer confident the declare is the right vehicle. What you describe wanting is also narrower than what I proposed, and the two come apart cleanly. A rule that *rejects* has to be opt-in: the 5,000-package survey turned up a live, maintained library whose variable names are mathematical formulae, which a mandatory rule would break 888 times. A diagnostic that *warns* about an accidental invisible character needs no opt-in at all, and would catch your no-break space without asking anything of you. I would rather not choose between those in a reply. Rowan has asked me to state the problem before the solution, which is fair, and your message is evidence for the same point -- I started from a mechanism and have been arguing backwards from it ever since. Let me answer him properly first, and then come back to whether a declare is what any of this actually needs. Regards, Luca

Rowan Tommins [IMSoP]

5 days ago
Hi Luca, On 26 August 2026 16:15:08 BST, "otzelot2021@outlook.de" <otzelot2021@outlook.de> wrote:
>... well-formed UTF-8, UAX31-R1-2 with the standard >Default-Ignorable Exclusion Profile, and NFC required rather than >applied
A couple of people have touched on this, but I don't think it's been directly addressed, so I'll ask it more explicitly: what do these terms mean? - UAX31-R1-2 - the standard Default-Ignorable Exclusion Profile - NFC I think that's important context for this discussion, but it's also relevant to the eventual user experience: what is the summary that goes into the manual and error messages? "Class name doesn't meet UAX31-R1-2" would be about as meaningful to most people as the infamous "Unexpected T_PAAMAYIM_NEKUDOTAYIM". The other thing that I'm not entirely clear on is how much of this is or should be about *rejecting* names, and how much about *normalising* them - bearing in mind we already perform some normalisation in the form of ASCII case folding. Perhaps we need to step back and define the *problem statement* more clearly, rather than starting with a *solution* and trying to define its benefits? Regards, Rowan Tommins [IMSoP]

otzelot2021@outlook.de

4 days ago
> A couple of people have touched on this, but I don't think it's been > directly addressed, so I'll ask it more explicitly: what do these terms > mean? > - UAX31-R1-2 > - the standard Default-Ignorable Exclusion Profile > - NFC
In plain terms: **NFC.** Unicode can write some characters more than one way. An o-umlaut is either one character, U+00F6, or two: a plain o followed by a combining diaeresis. Normalization Form C is the form that uses the single character wherever one exists. Requiring NFC means one spelling per name. **UAX #31.** The Unicode annex that says which characters a programming language should allow in identifiers: letters, digits, marks and connecting punctuation, but not general punctuation, symbols or formatting characters. "R1-2" only means "we follow it with a stated list of changes" rather than "we follow it exactly". **Default-Ignorable Exclusion Profile.** One of those stated changes, and a standard one that Unicode itself defines rather than something I made up: drop the characters that are invisible by design -- zero-width joiners, variation selectors and the like.
> what is the summary that goes into the manual and error messages? > "Class name doesn't meet UAX31-R1-2" would be about as meaningful to most > people as the infamous "Unexpected T_PAAMAYIM_NEKUDOTAYIM".
You are right that this is the actual deliverable, and I had not written it. Attempting it, with the standard's name appearing nowhere:     Identifier contains U+00A0 NO-BREAK SPACE, which is not allowed in a     name     Identifier is not valid UTF-8 (invalid byte 0xA9 at offset 0)     Identifier "gro<U+0308>sse" is not in Unicode normalization form C;     write it as "gr<U+00F6>sse" The third one bears on your second question.
> The other thing that I'm not entirely clear on is how much of this is or > should be about *rejecting* names, and how much about *normalising* them > - bearing in mind we already perform some normalisation in the form of > ASCII case folding.
That is the sharpest thing anyone has said in this thread, and I did not have it clear in my own head. Sorting the four things I have been bundling together along that axis:   not valid UTF-8     only rejectable; there is nothing to normalise to   invisible chars     only rejectable; removing them would change meaning   not NFC             either -- and because the engine knows the composed                       spelling, a rejection can print it, which gets most                       of the benefit of normalising without the engine                       quietly editing your source   ASCII case folding  already normalisation, and incomplete:                       Stra<U+00DF>e and STRA<U+00DF>E are one class,                       Stra<U+1E9E>e is another, Strasse is a third They also have four different costs, which I can now put numbers to, from 4,863 packages and 520,802 files:   not valid UTF-8     2 identifiers   invisible chars     68, and not all accidental -- math-php spells                       variables like <U+27EE>1<U+00A0><U+2212><U+00A0>p<U+27EF><U+02E3>,                       with U+00A0 inside the name, on purpose   not NFC             0, across 627,515 files in both corpora   case divergence     19, in case-insensitive positions Bundling four rules with four cost profiles behind one mechanism was the mistake. Larry, Claude Pache and you have each pushed on a different corner of the same thing.
> Perhaps we need to step back and define the *problem statement* more > clearly, rather than starting with a *solution* and trying to define its > benefits?
Yes. I started from a mechanism and have been arguing backwards from it all week. Trying it the other way round: **PHP's identifier rule is expressed in bytes and says nothing about characters.** Three things follow. Two identifiers a reader cannot tell apart may be distinct to the engine. An identifier may be text that is not well-formed in any encoding. And the engine's own case-insensitive matching, which is a normalisation, is defined over a 26-letter subset of what an identifier may contain. Underneath that sits a question nobody has answered: **are non-ASCII identifiers a supported feature of PHP?** The manual says they are not, and explains that they work because of how UTF-8 happens to encode. 1,447 of them, in 25 of the 5,000 most-installed packages, say otherwise -- Vincenty geodesy in mjaschen/phpgeo, Latte's U+029F prefix in tracy/tracy, Russian schema accessors in wsdltophp/packagegenerator, and mathematical formulae as variable names in markrogoyski/math-php. I do not think any one of those four items justifies a language change on its own, and I would rather say so than keep hunting for an argument that makes it sound bigger. Together they say that PHP's identifier rule was never designed, only inherited, and that the ecosystem has quietly built on it anyway. Whether that is worth fixing, and in which direction, is a question for the list rather than for me. One observation, and then I will stop reaching for mechanisms. What falls out of your reject/normalise split is not one feature but three, with three different audiences: a diagnostic for invisible characters, which needs no opt-in and is exactly what Claude Pache described wanting; a rule about well-formedness, which has to reckon with symfony/cache; and a conformance rule about which characters are permitted at all, which is the only part that would break math-php 888 times and therefore the only part that plausibly needs opting into. NFC costs nothing either way and can ride along with whichever of those happens. But take the problem statement first. I owe the thread that much before proposing anything further. Tooling, raw scanner output and per-identifier CSVs, if anyone wants to check the numbers rather than take them: https://github.com/Otzie2023/PHP Regards, Luca