[RFC] [Discussion] Shorter Attribute Syntax Change

Le Tue, 28 Jul 2020 09:46:38 -0500, Joe Ferguson <joe@joeferguson.me> a écrit :

The text uses « instead of <<, it’s confusing. It should be made clearer what #[ ] has over << >>, which does not cause any BC break. Côme

On Tue, 28 Jul 2020, Joe Ferguson wrote:

This is an excellent RFC highlighting the important deficiencies of the @@ syntax. I hope you will all read this and also conclude that we can still pick this better syntax. Remember that it is not only about how it looks. It is much more important that it is functional and causes the least amount of friction in the community. cheers, Derick

On Tue, July 28, 2020 at 9:46 AM Joe Ferguson <joe@joeferguson.me> wrote: > Hello Internals, > > I've been working with Derick Rethans and others (thanks all!) on a Shorter > Attribute Syntax Change RFC which outlines reasons why the "#[]" syntax > would be preferred over the currently agreed upon "@@" syntax for Shorter > Attribute Syntax. > > An important part of the research that went into this proposal demonstrates > how "@@" will make handling attributes more difficult for PHP_CodeSniffer > (and similar tools) and shows how the alternative "#[]" syntax is more > friendly for these tools. > > You can find the RFC > https://wiki.php.net/rfc/shorter_attribute_syntax_change and I'm happy to > answer any questions as best I can. Hi Joe, From the perspective of looks alone I don't care much one way or the other between @@ and #[]. However, I don't find the arguments for #[] in this RFC very compelling, and it ignores some of the other downsides of #[] compared to @@ that should be highlighted. Let's go through the arguments from the RFC: > It [@@] currently causes a parser conflict This isn't true now that namespaced names are treated as tokens. The @@ implementation has been merged into php-src and there is no parser conflict. > Various luminaires [sic] consider @@ a joke Who does this refer to? A random Reddit user? > Feedback to Derick's tweet (https://twitter.com/derickr/status/1285912223639130114) > were [sic] overwhelmingly positive Are you sure? I took a look at the thread and it seems like the responses were pretty mixed. Some people expressed support for #[], but almost as many were happier with the @@ syntax. If anything the majority of responses were neutral or expressing dislike for all the syntax options. > It has the distinct possibility to cause further parsing issues Like what? How is it any more likely to cause parsing issues than the @ syntax used by the majority of other C family languages? > Having a closing ] makes it easier to extend Attributes with more syntax This might be a good argument if there were actually a need to extend attributes with more syntax. What other languages have found a need for this? Even Rust doesn't allow additional syntax inside the brackets. > There is no “end symbol” to make finding occurrences easier, > including by reading or using regular expressions. Can you provide an example? When reading code I personally don't find the ending bracket very helpful - my brain tends to parse it as the end of an array, not the end of an attribute. Even when using a regex to search code, the ending bracket doesn't seem very helpful since arrays can also be passed as attribute arguments. For example, to match this attribute the regex for @@ is actually simpler than the one for #[]: ```php # regex: /@@\w+\(.+\)/ @@Attr([1, 2]) # vs. # regex: /#\[\w+\(.+\)\]/ #[Attr([1, 2])] ``` > A syntax with a closing demarcation is much easier to back fill with > tools such as PHPCS. I'm not sure I understand this. What does it mean to "back fill" a syntax with PHPCS, and what does a closing demarcation have to do with it? As a user, I'd expect to have to update PHPCS to work with PHP 8 code - doesn't it have to be updated anyway to correctly support the syntax for named arguments, union types, the nullsafe operator, match expression, etc.? And from a perspective of compatibility with existing tools, couldn't @@ be considered superior since people can use it in code examples on a blog without the syntax highlighter treating it as a comment? > It is a syntax no other language uses. Rust chose to use #[] even though it wasn't used by any other language. Does that make it a bad fit for Rust? No. But just because Rust uses a syntax also doesn't mean it's a good fit for PHP. @@ fits well with PHP since there is virtually no BC break, and the syntax is close to the @annotation syntax currently used in docblocks, so it's easily recognizable for what it is. On the other hand, #[] would be a larger BC break and can easily be confused for a comment. Best regards, Theodore

Hi internals, For #[, my main objection is the various ways this can change the lexing in a way that is impractical to (efficiently) backfill, and that the proposed patch doesn't address the fact that the syntax may change the syntax of php 7 code in unexpected ways. This syntax would help phpcs with easy examples **in the short term**, when attributes are on a single line, but would make more complicated refactorings buggy and error-prone unless phpcbf was run in the same major version. When attributes span multiple lines, the lexing is completely different for `#[]`. For example, the below syntax would yield false in php 7, but a generator in php 8. (There's precedent at least - the lexing of heredocs changed in php 7.3 or 7.4) ``` // Aside: This code snippet seems to have an assert failure in the lexer with the patch in the RFC // Zend/zend_compile.c:1794: zendlex: Assertion `!(executor_globals.exception) || ret == T_ERROR' failed. function generator() {     yield #[MyCustomAttribute('         false;         // ']function() {}; } ``` And another example which would cause problems for phpcs in php 7 - the comment syntax can cause code to be treated as inline html instead of php tokens. ``` <?php // This example echoes the rest of the source code in php 7 // and echoes "Test" in php 8. #[DeprecationReason('reason: <https://some-website/reason?>')] function main() {} const APP_SECRET = 'app-secret'; echo "Test\n"; ``` I'd posted another example in https://externals.io/message/111101#111133 One way I'd thought of to avoid this ambiguity would be to assert in PHP 8.0 with an E_COMPILE_ERROR (or E_COMPILE_WARNING) that: 1. All tokens of the #[...] annotation are on the same line 2. No non-whitespace token follows the ] on the same line. It may be permissible to allow other `//` comments or multiple attributes after it on the same line, though. 3. No `?>` substrings within the rest of the line after #[, maybe Although I'm not sure if others are actually concerned about this ambiguity and these are really artificial examples for code. As for `<<`, I'm assuming people may have meant `yield <<JIT>> function() {};`, but `yield` already has a precedence and didn't have issues - otherwise `yield+2;` could be adding 2 to the result of a yield. (It's unambiguously `yield(+2);` right now). I forget how it'd be parsed, but it wouldn't be ambiguous. As for `@@`, all of its known issues seem to have been resolved, and there is still the potential for future issues, but I still prefer it over the `#[` implementation in that patch. P.S. I'd like to note that 1. A lot more discussion has occurred since the initial vote. 2. Since `@@` was obviously passing at the time, fewer voters would put much thought into detailed tradeoffs of `#[` 3. I haven't seen those specific drawbacks to `#[` of potentially significantly changing lexing (not just losing tokens) mentioned, but the shorter attributes syntax RFC did seem to mention params were commented out. One idea I'd have on voting would be to have the exact same 3-way vote, again, and pick the attribute syntax with the same ranked choice procedure. I assume proponents of `#[` would have similar objections if an RFC with a two-way vote for `<<>>` and `@@` was started first. Regards, - Tyson

Hi Joe, Personally I favor #[] myself, but there has been a vote with a substantial participation choosing @@. Overturning this democratic outcome should require **significant** technical arguments, otherwise this RFC would provide problematic precedent for any RFC to be overturned by arbitrary revoting. The arguments the RFC brings forward don't convince me that we should pick #[] over @@. 1. The parser conflict was my major reason why @@ should not have been picked from a technical POV. But this issue has been fixed by an independent improvement since then. These kinds of small problems are found regularly after RFCs are accepted and are subsequently fixed. The RFC is formulated as the problem still exists, but it does not. Yes, the namespace token RFC vote has not yet finished, but there is no chance it will be rejected at this point. Since the namespace token RFC is passing, and I have seen the simplicity of the new parser rules, my opinion has changed to keep pushing for #[] and I accept the @@ outcome. 2. The phpcs argument also does not convince me. Summed up it is essentially that a current version of phpcs does not support parsing @@ or <<>> yet and incorrectly changes code using them, however for #[] it does nothing because it parses it as a comment. To me this sounds like a bug in PHPCS. This argument also breaks down completely on multi line attributes. Why should we even expect PHPCS to support an unreleased version? If someone uses PHP 8 code, why do we assume they keep using an old version of phpcs that does not support PHP 8 yet? 3. We already established that the syntax question is extremely contentious, and I was getting tons of messages on it on Twitter, Reddit, Email et al during the initial Attributes RFC. So of course people would be against @@ (as I am myself), however it had a majority in votes. For me putting these subjective opinions into the RFC makes the case for overturning less convincing, because it makes the argument that overturning a valid democratic vote is somehow OK if you are loud. The only reason why we change this outcome must be purely technical imho. 4. You cannot write a regex to detect the end symbols for both #[] or <<>>, because both end symbols are allowed *inside* the attribute syntax already, so it's not easier to find occurrences with regexes with the other syntaxes: #[Foo([1 => 2])] <<Foo(1 >> 4)>> Simple detection of attributes will always need to grep only for the start token, and both @@ and #[ are equally unique to find here. 5. For readability an end token would be nicer (which is why i prefer #[]). But this information was already known during the first vote on the syntax and rejected by majority. 6. An end token might indeed allow extending attributes with new stuff, but nobody has brought up this use-case before. No other language has additional attribute keywords, why is this brought as a positive argument, when the choice of syntax no other language uses is brought up as a negative point later? 7. Future potential parsing problems are brought forward for @@, but # being a comment token makes it likely that #[] will also cause problems. Tyson has a few examples in his email to this thread. Other confusing things you can do with #[] for example: #[Foo( 1 # 1 ] bar )] function foo () { } Which is #[Foo(1)] without comments and line breaks.

Hello Internals, Here is a small comparison table based on current feedback, maybe it will bring some objective clarity to the discussion: (markdown below) Impact|`@@`|`#[]` ---|---|--- BC break|virtualy nonexistent|slightly broader: `##[` comments are now broken. Parser|no technical problems by now|no technical problems by now Extendability|less future-proof, arguably|more future-proof, arguably Grepability|harder, arguably|easier, arguably (however, consider an Attribute taking an array `[]` parameter.) Perception|similar to PHPDoc annotations, more familiar to PHP devs|more familiar to Rust devs Other issues are either speculative or insignificant (as for me): - causing future parsing issues: no evidence - "backfilling": there is, arguably, no use case for it. Code using PHP 8 features will target PHP 8 anyway. Code, compatible with both PHP 7 and PHP 8 would most probably use classic PHPDoc annotations. - "syntax no other language uses": this is probably insignificant. - "possibility of @@ moving to @ is near to 0": i am not sure anyone planned to move @@ to @, and if so, it would be an unnecessary BC break. I am personally in favor of `@@`, but just because it looks visually nicer to me. I see no strong objective reason to favor one over the other. If you want an ending token, maybe consider `@@Attribute@@`, personally I like those round-ish shapes :P. On Tue, Jul 28, 2020 at 5:47 PM Joe Ferguson <joe@joeferguson.me> wrote:

Hi internals, I had thought of another alternative syntax - `/** @@MyAttribute */`, which would solve some of the problems I mentioned about `#[`. ``` namespace My\NS; use Other\MyAttribute; /** * Use @@ or << at the start of a line. (To be determined) * Resolve the names in the comment relative to the scope of the function, like the current attribute implementation.  * @@MyAttribute(expr, expr) * Extra *s are allowed *** @@AnotherAttribute * Tokenizing should continue until a matching ) is found * @@YetAnotherAttribute( * expr *)  */ function my_function() { } ``` Pros: - Doc comments are already the recommended way to associate information with a named element. - Most code modifying tools will preserve doc comments - This can go on the same line as a closure/short arrow function without lexing ambiguity. - Frameworks can use `Reflection*->getDocComment()` and update their checks to also allow `@@` for attribute-like annotations when run in PHP 7. They don't have to repeat both `@@MyAttribute` and `/** @MyAttribute */` or risk it going out of sync (Symfony would already have to parse the `use` clauses to find out what `@MyAttribute` resolves to for Symfony attributes) - Does not affect lexing outside of the doc comment - I don't believe @@ is common. To minimize the impact on ascii art, this can be initially limited to @@ at the start of a comment line (spaces and `*`s) immediately followed by \ or the start of an identifier Cons: - Need to run the lexer twice on a T_DOC_COMMENT token, but only if the comment string contains `@@` and is associated with an element. Emitting T_DOC_COMMENT_START and T_DOC_COMMENT_END and T_DOC_COMMENT_INNER and T_DOC_COMMENT_ATTRIBUTE T_DOC_COMMENT_ATTRIBUTE_ERROR might make this technically feasible (similar to heredoc having complicated logic) - I prefer `@@` and `<<>>` over this proposal long-term, after php 7.4 support is dropped. But I prefer `/** @@MyAttr */` over `#[MyAttr]`. - Would end up emitting parse errors for some existing code in the second lexer pass. - Syntax highlighting might be hard to implement in editors, but probably tractable. I don't remember seeing this proposed before. I believe @@ isn't used by mainstream frameworks in doc comments right now, but correct me if it is. Thanks, - Tyson

Hi Joe, wt., 28 lip 2020 o 16:47 Joe Ferguson <joe@joeferguson.me> napisał(a):

Thank you for bringing this up. TIL something new about popular framework templating component which I wanna share with you. There is already use of @@ in files with ".php" extension in Blade templates from Laravel framework. In Laravel 7.x double @@ is used as an escape sequence allowing template compiler to passthrough TypeScript expressions with @decorator() which is described in the Templating section of manual[1]. I realize that it's not the same cause the double @@ is used here inside ".php" files in HTML mode and not in PHP code mode but since this feature exist it may cause issues on a text search lookup using for eg. grep | sed | awk like tools. TL;DR Now please bear in mind that I'm just saying that people from Larevil community already can use the same syntax with double "@@name()" in ".php" files but outside of PHP code context. [1] https://laravel.com/docs/7.x/blade#blade-and-javascript-frameworks Cheers, Michał Marcin Brzuchalski

I think it has become clear that we need to revisit this syntax question again, including the elephpant in the room of delaying this feature to 8.1. The reason is not only Joe's desire to revote on #[], but also that there are now more syntax proposals such as @[] by Derick or @@ in comments by Tyson (though no patch exists for it yet). At this point a lot of syntaxes are potentially viable (except single @, please don't suggest it). I feel the choice between many syntaxes is 80% subjective, 20% objective facts, so tensions are to be expected. As the author of the original RFC and patch, I hope I have some klout in suggesting the following procedure (RMs would need to extend their approval for revote to this). 1. we collect syntax proposals once again, with the requirement of a simple patch being available against php-src/master for viability in 8.0. 2. RMs are the arbiter to decide the patch is acceptable to be included for 8.0 or if its selection would delay entire attributes to 8.1. 3. I would make a feature matrix for the vote / RFC page and sort each proposed syntax into it, seeking input from the proposers. 4. We would then hold another vote on syntax using STV where the choice is a combination of syntax and target version, examples: - <<>> in 8.0, #[] in 8.0, @@ in 8.0 (all these patches are viable) - @@ in doc blocks for 8.1 - §[] in 8.1 (weird example to demonstrate the point) and so on. The STV vote would run with potentially 5-10 different syntaxes. On Time Frame: Sara allowed to extend this decision into feature freeze, but I believe it shouldn't be later than Beta 2 (August 20th), especially if the outcome could be delay until 8.1. What do RMs think of voting from 10th to 17th of August, giving us 3 days to merge. This gives everyone until August 8-9th to work on their syntax proposal. greetings Benjamin On Tue, Jul 28, 2020 at 4:47 PM Joe Ferguson <joe@joeferguson.me> wrote: