Everything
php
Learn Community Releases The PHP Project
Home » Project » Newsgroups » php.internals » [RFC] Generators »

[RFC] Generators

php.internals

Nikita Popov

14 years ago
Hi internals! I think there already was a lot of discussion on the generators, so it's time to move to the next step. I'd like to vote on the feature in two weeks, so this the "announce[ment] on internals@, by the author, with the intention of voting on it". https://wiki.php.net/rfc/generators If you have any further feedback, now would be a good time to raise it. If there is something you previously posted, but which I didn't yet address, please let me know. There were around 150 mails and I sure missed some of them. Thanks, Nikita

Brian Moon

14 years ago
Hi Nikita, I admit, I have ignored these threads as there was no RFC. So, some of this may have been covered. Do you have a good example usage other than a file? I don't find fopen/fgets/fclose all that complicated. What are the other valid use cases for such a thing? Also, not allowing rewinding is unintuitive for something that is an iterator in PHP. If I can foreach() it and I can call next() on it, I expect to be able to reset() it as well. IMO, you would need to issue a FATAL PHP error if that simply is not allowed. Or you have to have a second syntax for what to do in that case. At that point, you are implementing Iterator. While I am glad that PHP has borrowed syntax from many languages, I find the yield keyword to be very WTF when I first saw it. It is also poorly explained in your RFC. You use terms like "sending and receiving". That does not say "returns from function execution" to me. I had to basically figure it out from the code examples. Lastly, because you cite needing this for sending data to PHPUnit, I think this is a user land problem and not a core problem. In about 5 minutes I implemented a reusable Generator object that does exactly what you need. http://pastebin.com/V336rEpR At least, it does what your examples show you need. Again, file IO is very easy and perhaps that example does not explain your true need very well. Brian brianlmoon@php.net http://brian.moonspot.net/ On 8/11/12 8:42 AM, Nikita Popov wrote:

Levi Morrison

14 years ago
On Sun, Aug 12, 2012 at 2:08 PM, Brian Moon <brian@moonspot.net> wrote:
> Do you have a good example usage other than a file? I don't find fopen/fgets/fclose all that complicated. What are the other valid use cases for such a thing?
One fabulous use case is creating an iterator for a Binary Search Tree. A post order done without generators looks like: https://github.com/morrisonlevi/PHP-Datastructures/blob/master/src/Spl/PostOrderIterator.php. An iterator using a generator looks something like: public function getIterator() { if ($this->left) yield* $this->left; yield $this->value; if ($this->right) yield* $this->right; } This is 5 lines. The fully commented version of the post-order iterator previously mentioned is 106 lines of code and is considerably harder to understand.

Yahav Gindi Bar

14 years ago
On 13 באוג 2012, at 01:33, Levi Morrison <morrison.levi@gmail.com> wrote:
> On Sun, Aug 12, 2012 at 2:08 PM, Brian Moon <brian@moonspot.net> wrote: >> Do you have a good example usage other than a file? I don't find fopen/fgets/fclose all that complicated. What are the other valid use cases for such a thing? > > One fabulous use case is creating an iterator for a Binary Search > Tree. A post order done without generators looks like: > https://github.com/morrisonlevi/PHP-Datastructures/blob/master/src/Spl/PostOrderIterator.php. > An iterator using a generator looks something like: > > public function getIterator() { > if ($this->left) yield* $this->left; > yield $this->value; > if ($this->right) yield* $this->right; > } > > This is 5 lines. The fully commented version of the post-order > iterator previously mentioned is 106 lines of code and is considerably > harder to understand. > > -- > PHP Internals - PHP Runtime Development Mailing List > To unsubscribe, visit: http://www.php.net/unsub.php >
Each Iterator implementation in PHP is longer and require a class. The yield keyword can reduce the code and make things simpler. If my vote count, though I can't believe it is, I'd vote for it - clean syntax, make things simpler to develop after you understand the concept (for those who against adding because of the learning issue and beginners - we'll have beginners anytime that'll mess up with the concept, so I don't think that it should restrict us. Put that aside, we should make the documentation clear and easy as possible in order to help beginners understand the concept).

Brian Moon

14 years ago
On 8/12/12 5:33 PM, Levi Morrison wrote:
> On Sun, Aug 12, 2012 at 2:08 PM, Brian Moon <brian@moonspot.net> wrote: >> Do you have a good example usage other than a file? I don't find fopen/fgets/fclose all that complicated. What are the other valid use cases for such a thing? > > One fabulous use case is creating an iterator for a Binary Search > Tree. A post order done without generators looks like: > https://github.com/morrisonlevi/PHP-Datastructures/blob/master/src/Spl/PostOrderIterator.php. > An iterator using a generator looks something like: > > public function getIterator() { > if ($this->left) yield* $this->left; > yield $this->value; > if ($this->right) yield* $this->right; > } > > This is 5 lines. The fully commented version of the post-order > iterator previously mentioned is 106 lines of code and is considerably > harder to understand.
Well, it's 52 lines, not 106. The 5 lines above are not commented nor are they spaced at all like the ones in the class. In the above example, what sets $this->right? or $this->left? There has to be more calling code around this. I don't consider this a very good example. Brian. brianlmoon@php.net

Levi Morrison

14 years ago
On Sun, Aug 12, 2012 at 5:56 PM, Brian Moon <brian@moonspot.net> wrote:
> On 8/12/12 5:33 PM, Levi Morrison wrote: >> >> On Sun, Aug 12, 2012 at 2:08 PM, Brian Moon <brian@moonspot.net> wrote: >>> >>> Do you have a good example usage other than a file? I don't find >>> fopen/fgets/fclose all that complicated. What are the other valid use cases >>> for such a thing? >> >> >> One fabulous use case is creating an iterator for a Binary Search >> Tree. A post order done without generators looks like: >> >> https://github.com/morrisonlevi/PHP-Datastructures/blob/master/src/Spl/PostOrderIterator.php. >> An iterator using a generator looks something like: >> >> public function getIterator() { >> if ($this->left) yield* $this->left; >> yield $this->value; >> if ($this->right) yield* $this->right; >> } >> >> This is 5 lines. The fully commented version of the post-order >> iterator previously mentioned is 106 lines of code and is considerably >> harder to understand. > > > Well, it's 52 lines, not 106. The 5 lines above are not commented nor are > they spaced at all like the ones in the class.
No, Brian, it is 106 lines of actual code I would find in the wild. And sure, I'd have spaced the generator code more like: public function getIterator() { if ($this->left) { yield* $this->left; } if ($this->right) { yield* $this->right; } yield $this->value; } However, that's 11 lines of code compared to 106.
> In the above example, what > sets $this->right? or $this->left?
Iterators don't set $this->right and $this->left; they only traverse the tree.
> I don't consider this a very good example.
That's fine. You are entitled to your opinion. However, you didn't have to figure out the 106 lines of code that is the Post-Order iterator. That was a NIGHTMARE and I'm still not 100% confident that it works as it is supposed to. The generator, on the other hand, is simple and follows the definition very closely. MUCH better.

Levi Morrison

14 years ago
I guess to be completely honest, there would be a docblock above the generator code as well. So 14 lines.

Brian Moon

14 years ago
>> I don't consider this a very good example. > That's fine. You are entitled to your opinion. However, you didn't > have to figure out the 106 lines of code that is the Post-Order > iterator. That was a NIGHTMARE and I'm still not 100% confident that > it works as it is supposed to. The generator, on the other hand, is > simple and follows the definition very closely. MUCH better.
Could you show the whole class that this public function belongs to? I don't see how this function fits in and compares at all to the class you linked. Brian.

Levi Morrison

14 years ago
On Sun, Aug 12, 2012 at 8:25 PM, Brian Moon <brian@moonspot.net> wrote:
>>> I don't consider this a very good example. >> >> That's fine. You are entitled to your opinion. However, you didn't >> have to figure out the 106 lines of code that is the Post-Order >> iterator. That was a NIGHTMARE and I'm still not 100% confident that >> it works as it is supposed to. The generator, on the other hand, is >> simple and follows the definition very closely. MUCH better. > > > Could you show the whole class that this public function belongs to? I don't > see how this function fits in and compares at all to the class you linked. > > Brian.
It renders the iterator class completely obsolete; it could be part of the Binary Search Tree class. If you still wanted a separate class (separation of concerns, perhaps) it would look something like this (not sure on exact yield syntax): class PostOrderIterator implements IteratorAggregate { /** * @var BinaryTree */ protected $root; public function __construct(BinaryTree $root) { $this->root = $root; } /** * @link http://php.net/manual/en/iteratoraggregate.getiterator.php * @return Traversable */ public function getIterator() { yield $this->traverse($this->root); } private function traverse(BinaryTree $node) { if ($node->left) { yield* $node->left; } if ($node->right) { yield* $node->right; } yield $node->value; } } All this does is take the node that acts as root in it's constructor and then calls the generator with root as the starting point. That's 34 lines of code including comments and whitespace.

Anthony Ferrara

14 years ago
Brian, On Sun, Aug 12, 2012 at 4:08 PM, Brian Moon <brian@moonspot.net> wrote:
> Hi Nikita, > > I admit, I have ignored these threads as there was no RFC. So, some of > this may have been covered. >
There was an RFC in those posts... It was just being iterated over.
> Do you have a good example usage other than a file? I don't find > fopen/fgets/fclose all that complicated. What are the other valid use cases > for such a thing? >
Here's a quick set of examples: http://blog.ircmaxell.com/2012/07/what-generators-can-do-for-you.html
> Also, not allowing rewinding is unintuitive for something that is an > iterator in PHP. If I can foreach() it and I can call next() on it, I > expect to be able to reset() it as well. IMO, you would need to issue a > FATAL PHP error if that simply is not allowed. Or you have to have a second > syntax for what to do in that case. At that point, you are implementing > Iterator. >
I partially agree. rewinding the generator might be possible, but it's hard to tell in those cases. It's hard to tell if resetting it is even possible from the code level. So I'm pretty much OK with living with that restriction for the time being...
> While I am glad that PHP has borrowed syntax from many languages, I find > the yield keyword to be very WTF when I first saw it. It is also poorly > explained in your RFC. You use terms like "sending and receiving". That > does not say "returns from function execution" to me. I had to basically > figure it out from the code examples. >
It's absolutely something that takes getting used to. But it's also quite intuitive once you think about it. You're not "returning" back to the parent scope (exiting your scope), you're "yielding" a value to the parent scope, expecting to continue on later (think of it as a stop light where you let the other code run for a while until you go to the next one).
> Lastly, because you cite needing this for sending data to PHPUnit, I think > this is a user land problem and not a core problem. In about 5 minutes I > implemented a reusable Generator object that does exactly what you need. > http://pastebin.com/V336rEpR At least, it does what your examples show > you need. Again, file IO is very easy and perhaps that example does not > explain your true need very well. >
Well, there's one thing that should be made clear. There's nothing (and I mean that) that generators provide that you can't do already. You can build iterators that do exactly the same thing. The trick that generators provide you is a really short, and really easy way of implementing iterators. You don't need to worry about maintaining state or anything like that. All you need to do is code it how you would normally, and replace your executing code with a yield. No more needing to split out and maintain state in weird ways, or using literally hundreds of lines of code (because of the boilerplate involved) where a dozen would do... Anthony

Jared Williams

14 years ago
> -----Original Message----- > From: Anthony Ferrara [mailto:ircmaxell@gmail.com] > Sent: 13 August 2012 03:49 > To: Brian Moon > Cc: Nikita Popov; PHP internals > Subject: Re: [PHP-DEV] [RFC] Generators > > Brian, > > On Sun, Aug 12, 2012 at 4:08 PM, Brian Moon > <brian@moonspot.net> wrote: > > > Hi Nikita, > > > > I admit, I have ignored these threads as there was no RFC. > So, some of > > this may have been covered. > > > > There was an RFC in those posts... It was just being iterated over. > > > > Do you have a good example usage other than a file? I don't find > > fopen/fgets/fclose all that complicated. What are the other > valid use > > cases for such a thing? > > > > Here's a quick set of examples: >
http://blog.ircmaxell.com/2012/07/what-generators-can-do-for-you.html
> > > > Also, not allowing rewinding is unintuitive for something > that is an > > iterator in PHP. If I can foreach() it and I can call > next() on it, I > > expect to be able to reset() it as well. IMO, you would > need to issue > > a FATAL PHP error if that simply is not allowed. Or you > have to have a > > second syntax for what to do in that case. At that point, you are > > implementing Iterator. > > > > I partially agree. rewinding the generator might be possible, > but it's hard to tell in those cases. It's hard to tell if > resetting it is even possible from the code level. So I'm > pretty much OK with living with that restriction for the time
being...
> > > > While I am glad that PHP has borrowed syntax from many languages,
I
> > find the yield keyword to be very WTF when I first saw it. > It is also > > poorly explained in your RFC. You use terms like "sending and > > receiving". That does not say "returns from function > execution" to me. > > I had to basically figure it out from the code examples. > > > > It's absolutely something that takes getting used to. But > it's also quite intuitive once you think about it. You're not > "returning" back to the parent scope (exiting your scope), > you're "yielding" a value to the parent scope, expecting to > continue on later (think of it as a stop light where you let > the other code run for a while until you go to the next one). > > > > Lastly, because you cite needing this for sending data to > PHPUnit, I > > think this is a user land problem and not a core problem. > In about 5 > > minutes I implemented a reusable Generator object that does > exactly what you need. > > http://pastebin.com/V336rEpR At least, it does what your > examples show > > you need. Again, file IO is very easy and perhaps that example
does
> > not explain your true need very well. > > > > Well, there's one thing that should be made clear. There's > nothing (and I mean that) that generators provide that you > can't do already. You can build iterators that do exactly the > same thing.
Not true. Iterator::current() cannot return references, but generators can yield them. Eg function &map(array &$row, array $order) { foreach($order as $index) yield $row[$index]; } foreach(map($row, [1,3,5,7,9]) as &$ref) ....
> > Anthony >
Jared

Alex Aulbach

14 years ago
2012/8/13 Anthony Ferrara <ircmaxell@gmail.com>:
> It's absolutely something that takes getting used to. But it's also quite > intuitive once you think about it. You're not "returning" back to the > parent scope (exiting your scope), you're "yielding" a value to the parent > scope, expecting to continue on later (think of it as a stop light where > you let the other code run for a while until you go to the next one).
I think for people, which are - like me - more used with databases, the cursor-concept is a good comparison. Perhaps this could help a little bit to explain it...
-- Alex Aulbach

Nikita Popov

14 years ago
On Sun, Aug 12, 2012 at 10:08 PM, Brian Moon <brian@moonspot.net> wrote:
> Also, not allowing rewinding is unintuitive for something that is an > iterator in PHP. If I can foreach() it and I can call next() on it, I expect > to be able to reset() it as well. IMO, you would need to issue a FATAL PHP > error if that simply is not allowed. Or you have to have a second syntax for > what to do in that case. At that point, you are implementing Iterator.
Currently I'm planning to implement the following behavior for rewind(): * If before first yield: Resume to first yield (this priming behavior is common to all the Iterator methods) * At first yield: No-op * After first yield: Recoverable fatal error So this would allow you to call ->rewind() after creating the generator, but will throw an error if you try to do so later. Another thing that I'd like to do is drop the ->close() method. It doesn't really make sense to explicitly close generators. Any objects to those two things? Nikita

Alex Aulbach

14 years ago
2012/8/13 Nikita Popov <nikita.ppv@gmail.com>:
> * After first yield: Recoverable fatal error > > So this would allow you to call ->rewind() after creating the > generator, but will throw an error if you try to do so later.
Hm. I think this looks like a workaround over conceptual problems. The (in my eyes) "right" behavior would be to call a new instance of the generator if you rewind and destroy the current (= replace current generator with a new). I would prefer to say "this is not an real iterator"; if you want to make it an real iterator, then implement it yourself. Otherwise it's like "Ok, we have here a new type of function which looks enough like a function to be a function, which implements an iterator which looks enough like an iterator to be an iterator.". :) I also recommend to say, that generators aren't a good idea if used on resources, which can't be shared. For example: the file-read-example doesn't lock. To implement locking new functions or a new class is needed - hiding complexity. Generators are (in my opinion) really nice for things which are already in the memory. Not more or less.
-- Alex Aulbach

hakre

14 years ago
If I may chime in, not all iterators support rewind, at least not as outlined. There exists also the NoRewindIterator which you can just use to decorate an iterator out of a generator to not make it do a fatal. However as far as a generator is concerned, it probably should just behave like NoRewindIterator already so not give any errors on rewind. Because you can not rewind what you can not rewind. No need to error, no need to do something. The function is just there because of the interface, but it must not do anything. If different behavior is needed it could be decorated, but right now I can not imagine why rewind would be called (apart from using in foreach() for the first time which implies a rewind and that one is already like I suggest.) My 2 cents. -- hakre ----- Ursprüngliche Message ----- > Von: Nikita Popov <nikita.ppv@gmail.com> > An: Brian Moon <brian@moonspot.net> > CC: PHP internals <internals@lists.php.net> > Gesendet: 18:09 Montag, 13.August 2012 > Betreff: Re: [PHP-DEV] [RFC] Generators > > On Sun, Aug 12, 2012 at 10:08 PM, Brian Moon <brian@moonspot.net> wrote: >> Also, not allowing rewinding is unintuitive for something that is an >> iterator in PHP. If I can foreach() it and I can call next() on it, I > expect >> to be able to reset() it as well. IMO, you would need to issue a FATAL PHP >> error if that simply is not allowed. Or you have to have a second syntax > for >> what to do in that case. At that point, you are implementing Iterator. > > Currently I'm planning to implement the following behavior for rewind(): > > * If before first yield: Resume to first yield (this priming behavior > is common to all the Iterator methods) > * At first yield: No-op > * After first yield: Recoverable fatal error > > So this would allow you to call ->rewind() after creating the > generator, but will throw an error if you try to do so later. > > Another thing that I'd like to do is drop the ->close() method. It > doesn't really make sense to explicitly close generators. > > Any objects to those two things? > > Nikita > > -- > PHP Internals - PHP Runtime Development Mailing List > To unsubscribe, visit: http://www.php.net/unsub.php >

Aaron Holmes

14 years ago
On 8/13/12 9:09 AM, Nikita Popov wrote:
> On Sun, Aug 12, 2012 at 10:08 PM, Brian Moon <brian@moonspot.net> wrote: >> Also, not allowing rewinding is unintuitive for something that is an >> iterator in PHP. If I can foreach() it and I can call next() on it, I expect >> to be able to reset() it as well. IMO, you would need to issue a FATAL PHP >> error if that simply is not allowed. Or you have to have a second syntax for >> what to do in that case. At that point, you are implementing Iterator. > Currently I'm planning to implement the following behavior for rewind(): > > * If before first yield: Resume to first yield (this priming behavior > is common to all the Iterator methods) > * At first yield: No-op > * After first yield: Recoverable fatal error > > So this would allow you to call ->rewind() after creating the > generator, but will throw an error if you try to do so later. >
My perspective is that generators are intended to generate and yield a result, not so much to iterate over a known set of results. Thus, the results not really being known, there is nothing to rewind to. Rewinding would also presumably require more state control, or keeping track of previous results, and defeat the memory advantages of generators. Perhaps this is erroneous? That said, rewind() should behave consistently. I don't feel it makes sense to have rewind() succeed at one point, and fail at another. It would only cause confusion when not familiar with the behavior. Either allow it, or don't. Not both. Thanks, Aaron Holmes

Stas Malyshev

14 years ago
Hi!
> That said, rewind() should behave consistently. I don't feel it makes > sense to have rewind() succeed at one point, and fail at another. It > would only cause confusion when not familiar with the behavior. Either > allow it, or don't. Not both.
It does, since foreach uses rewind. So first rewind should succeed if you want iterators be usable in foreach. OTOH, on something like DB result set, next rewind does not make any sense, if you have non-seekable cursor, since the results consumed are gone and there's no way to get them back (you could rerun the query, but it might have side effects and nobody guarantees you'd get the same result anyway). So it makes sense for the generator to succeed on first rewind but fail on next ones. Note that generators by nature are stateful objects, so it is not unexpected that they would produce different result in different states.
-- Stanislav Malyshev, Software Architect SugarCRM: http://www.sugarcrm.com/ (408)454-6900 ext. 227

Aaron Holmes

14 years ago
On 8/14/12 10:36 AM, Stas Malyshev wrote:
> Hi! > >> That said, rewind() should behave consistently. I don't feel it makes >> sense to have rewind() succeed at one point, and fail at another. It >> would only cause confusion when not familiar with the behavior. Either >> allow it, or don't. Not both. > It does, since foreach uses rewind. So first rewind should succeed if > you want iterators be usable in foreach. OTOH, on something like DB > result set, next rewind does not make any sense, if you have > non-seekable cursor, since the results consumed are gone and there's no > way to get them back (you could rerun the query, but it might have side > effects and nobody guarantees you'd get the same result anyway). So it > makes sense for the generator to succeed on first rewind but fail on > next ones. Note that generators by nature are stateful objects, so it is > not unexpected that they would produce different result in different > states. >
Thanks for clarifying. It makes sense now, considering foreach's behavior and the generators statefulness allowing what otherwise seems inconsistent. However, might it make sense to no-op instead of erroring? If generators allow rewind(), it would be unexpected to receive an error for calling it.

Nikita Popov

14 years ago
On Tue, Aug 14, 2012 at 7:59 PM, Aaron Holmes <aaron@aaronholmes.net> wrote:
> Thanks for clarifying. It makes sense now, considering foreach's behavior > and the generators statefulness allowing what otherwise seems inconsistent. > However, might it make sense to no-op instead of erroring? If generators > allow rewind(), it would be unexpected to receive an error for calling it.
A no-op is the current behavior. The issue with that is that you could accidentally reuse an already depleted (or partially depleted) generator. I.e. you run it through one loop and then through another one. With a no-op rewind it would be hard to figure out what the issue is. The error makes clear that you tried to iterate an already iterated generator. Furthermore, if you indeed *want* to partially traverse a generator in several loops, you can simply wrap it in a NoRewindIterator. This has the additional benefit of making it more clear what you want to do. Nikita

Derick Rethans

14 years ago
On Sat, 11 Aug 2012, Nikita Popov wrote:
> Hi internals! > > I think there already was a lot of discussion on the generators, so > it's time to move to the next step. I'd like to vote on the feature in > two weeks, so this the "announce[ment] on internals@, by the author, > with the intention of voting on it". > > https://wiki.php.net/rfc/generators > > If you have any further feedback, now would be a good time to raise > it. If there is something you previously posted, but which I didn't > yet address, please let me know. There were around 150 mails and I > sure missed some of them.
I've some comments how that I've read the RFC:
> Recognition of generator functions > > 1. Any function which contains a yield statement is automatically a > generator function. > > 2. The initial implementation required that generator functions are > marked with an asterix modifier (function*). This method has the > advantage that generators are more explicit and also allows for > yield-less coroutines. > > The automatic detection was chosen over the asterix modifier for the > following reasons:
I am against this. This is even more magic in PHP. Is it really that difficult to have to mark the function with a different keyword, such as "generator":
> generator function getLinesFromFile($fileName) { > if (!$fileHandle = fopen($fileName, 'r')) { > return; > }
> There is an existing generator implementation in HipHop PHP, which > uses automatic-detection. Using the asterix modifier would break > compatibility.
This should not be a concern, sure, it's annoying for the hiphop developers but they chose to copy and then *chance* the PHP language for their own effect.
> yield: Yields the value null with an auto-incrementing integer key.
What is the usecase for this? cheers, Derick
-- http://derickrethans.nl | http://xdebug.org Like Xdebug? Consider a donation: http://xdebug.org/donate.php twitter: @derickr and @xdebug

Nikita Popov

14 years ago
On Sat, Aug 18, 2012 at 2:21 PM, Derick Rethans <derick@php.net> wrote:
> I've some comments how that I've read the RFC: > >> Recognition of generator functions >> >> 1. Any function which contains a yield statement is automatically a >> generator function. >> >> 2. The initial implementation required that generator functions are >> marked with an asterix modifier (function*). This method has the >> advantage that generators are more explicit and also allows for >> yield-less coroutines. >> >> The automatic detection was chosen over the asterix modifier for the >> following reasons: > > I am against this. This is even more magic in PHP. Is it really that > difficult to have to mark the function with a different keyword, such as > "generator":
There was already a rather long discussion on this topic. Most of it is noise, but I filtered out a few mails from Sara and Rasmus as they probably are of most interest: http://markmail.org/message/xzhdhbjozb4yrhh3 http://markmail.org/message/ryhygtimpd7q2nok http://markmail.org/message/32fklwqykpk56iph http://markmail.org/message/w2kbh7psplnmctcr
>> yield: Yields the value null with an auto-incrementing integer key. > > What is the usecase for this?
Use case is `$data = yield;`. In that case you don't care about what you yield (so just the default value and key are yielded), you are only interested in what you get sent back. Nikita

Lester Caine

14 years ago
> On Sat, 11 Aug 2012, Nikita Popov wrote: > >> Hi internals! >> >> I think there already was a lot of discussion on the generators, so >> it's time to move to the next step. I'd like to vote on the feature in >> two weeks, so this the "announce[ment] on internals@, by the author, >> with the intention of voting on it". >> >> https://wiki.php.net/rfc/generators >> >> If you have any further feedback, now would be a good time to raise >> it. If there is something you previously posted, but which I didn't >> yet address, please let me know. There were around 150 mails and I >> sure missed some of them. > > I've some comments how that I've read the RFC: > >> Recognition of generator functions >> >> 1. Any function which contains a yield statement is automatically a >> generator function. >> >> 2. The initial implementation required that generator functions are >> marked with an asterix modifier (function*). This method has the >> advantage that generators are more explicit and also allows for >> yield-less coroutines. >> >> The automatic detection was chosen over the asterix modifier for the >> following reasons: > > I am against this. This is even more magic in PHP. Is it really that > difficult to have to mark the function with a different keyword, such as > "generator":
Since this is yet another area where 'one does not have to use it if one does not want to' ... FLAGGING to the other users that a function is a 'special one' rather than just a normal function with a generator function seems to me just a necessity? HAVING to work through a functions code simply to see if it contains a 'yeild' so as to understand that it's not going to give a normal return seems insane? Alright the function can be properly documented with a docblock, but if it isn't ...
>> generator function getLinesFromFile($fileName) { >> if (!$fileHandle = fopen($fileName, 'r')) { >> return; >> } > >> There is an existing generator implementation in HipHop PHP, which >> uses automatic-detection. Using the asterix modifier would break >> compatibility. > > This should not be a concern, sure, it's annoying for the hiphop > developers but they chose to copy and then *chance* the PHP language for > their own effect. > >> yield: Yields the value null with an auto-incrementing integer key. > What is the usecase for this?
I can see some interesting porting problems with this ... one of the stock changes when moving from a MySQL only setup to support other databases is to remove the auto-increment magic, and replace it with proper sequence code prior to inserting. I can see translating a 'MySQL' geared generator into a more general one as being impossible if some tricks like this are used. (Out in an exhibition hall this weekend so not able to follow properly)

Rasmus Lerdorf

14 years ago
On 08/18/2012 10:12 AM, lester@lsces.co.uk wrote:
> Since this is yet another area where 'one does not have to use it if one > does not want to' ... FLAGGING to the other users that a function is a > 'special one' rather than just a normal function with a generator function > seems to me just a necessity? HAVING to work through a functions code > simply to see if it contains a 'yeild' so as to understand that it's not > going to give a normal return seems insane? Alright the function can be > properly documented with a docblock, but if it isn't ...
I don't see how that is any different from having to look through a function to see if it is a void function or if it returns something, and if so, what it returns. Or checking to see if it outputs something. PHP is very dynamic. A function can be both void and non-void at the same time. It can return mixed types and do pretty much anything you can imagine. I don't really see what the generator keyword achieves. What happens if I have a function that I mark as a generator but then I don't put a yield in it? Is that a compile-level error? That might work, but then what if I put a yield in it but the logic of the function never gets to it: if(cond) return [1,2,3]; else if(other_cond) yield $val; else return; There is no way to know at compile-time whether the yield will ever be reached. To me it seems odd to have a generator keyword that cannot be enforced and doesn't mean anything other than stating a developer's intent. Whether the developer actually carried through and implemented the intended generator can only be verified by reading the code. This smells much more like a source comment to me. If we were going to go with a generator keyword, then I think the feature needs to be completely re-thought. I would suggest dropping the yield keyword entirely and having return be equivalent to a yield if the function is marked as a generator. Then the keyword actually does something. -Rasmus

Nikita Popov

14 years ago
On Sat, Aug 18, 2012 at 4:12 PM, <lester@lsces.co.uk> wrote:
> Since this is yet another area where 'one does not have to use it if one > does not want to' ... FLAGGING to the other users that a function is a > 'special one' rather than just a normal function with a generator function > seems to me just a necessity? HAVING to work through a functions code > simply to see if it contains a 'yeild' so as to understand that it's not > going to give a normal return seems insane? Alright the function can be > properly documented with a docblock, but if it isn't ...
I don't understand this argument. Generator functions are transparent to the user. You use a generator function just like you would use a function that returns an array. From a user point of view it does not matter whether getLinesFromFile() is just a function returning an array, or whether it is a generator (or maybe even returns a hand-implemented iterator). It's just all the same. The fact that the function uses `yield` internally is just an implementation detail, not something that has to be part of the public API. Actually you can swap between an array and generator implementation just by replacing one line in the function body (yield <=> $array[])...
>>> generator function getLinesFromFile($fileName) { >>> if (!$fileHandle = fopen($fileName, 'r')) { >>> return; >>> } >> >>> There is an existing generator implementation in HipHop PHP, which >>> uses automatic-detection. Using the asterix modifier would break >>> compatibility. >> >> This should not be a concern, sure, it's annoying for the hiphop >> developers but they chose to copy and then *chance* the PHP language for >> their own effect. >> >>> yield: Yields the value null with an auto-incrementing integer key. >> What is the usecase for this? > I can see some interesting porting problems with this ... one of the stock > changes when moving from a MySQL only setup to support other databases is > to remove the auto-increment magic, and replace it with proper sequence > code prior to inserting. I can see translating a 'MySQL' geared generator > into a more general one as being impossible if some tricks like this are > used.
Generators are supposed to act very similar to array-returning functions, so they also have similar key semantics. If you do $result[] = $foo; that will behave the same as doing yield $foo;. Both will use an auto-incrementing key. Nikita

Lester Caine

14 years ago
Nikita Popov wrote:
> I don't understand this argument. Generator functions are transparent > to the user. You use a generator function just like you would use a > function that returns an array. From a user point of view it does not > matter whether getLinesFromFile() is just a function returning an > array, or whether it is a generator (or maybe even returns a > hand-implemented iterator). It's just all the same. The fact that the > function uses `yield` internally is just an implementation detail, not > something that has to be part of the public API. Actually you can swap > between an array and generator implementation just by replacing one > line in the function body (yield <=> $array[])...
Then I am now totally confused ... I was under the impression that the IDEA was that EACH call to a generator would return the next value? So you do not get an array built. Just as one would with SUSPEND in an SQL query process. The only value available is the current one, and one processes that. So that the user of a generator needs to handle the results sequentially rather than simply accessing the array of results. There does not need to be an array of results, only the processed output of each cycle of the generator? This of cause may still be confused thinking, since I would never be working in the way that this stuff is supposed to simplify. I WOULD be handling each element as I read it and building the details required from each cycle of the process. getLinesFromFile() would only ever return an array if that is what was needed to build the resulting page. It would normally be processed into a database, and later the required information selected from that. But that is a specialist iterator process just built with simple PHP.
-- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk

Stas Malyshev

14 years ago
Hi!
> I am against this. This is even more magic in PHP. Is it really that > difficult to have to mark the function with a different keyword, such as > "generator":
You have a point here, but "public static final generator function foo()" sounds a bit long-winded to me... Also, we'd have then to decide which function can be marked generator and which can't (e.g., interface probably can't, abstract probably can't, anonymous probably can, etc.) which adds more complexity. Also, I think that people that complain about having to scan through huge functions to see if they're generators or not, forget one thing: documentation. Yes, there is a way to make the purpose of the function understandable to a human without having him to do computer's work. That's documentation. Undocumented code is broken code. Broken code is not a good example when we're talking about right design.
> This should not be a concern, sure, it's annoying for the hiphop > developers but they chose to copy and then *chance* the PHP language for > their own effect.
Here I tend to agree with you - we should base on what's right for PHP, not what HipHop or any other implementation is doing. If we can make their lives easier - fine, but we don't have to be bound by their decisions.
-- Stanislav Malyshev, Software Architect SugarCRM: http://www.sugarcrm.com/ (408)454-6900 ext. 227

Rasmus Lerdorf

14 years ago
On 08/19/2012 07:07 PM, Stas Malyshev wrote:
> Hi! > >> I am against this. This is even more magic in PHP. Is it really that >> difficult to have to mark the function with a different keyword, such as >> "generator": > > You have a point here, but "public static final generator function > foo()" sounds a bit long-winded to me... Also, we'd have then to decide > which function can be marked generator and which can't (e.g., interface > probably can't, abstract probably can't, anonymous probably can, etc.) > which adds more complexity. > > Also, I think that people that complain about having to scan through > huge functions to see if they're generators or not, forget one thing: > documentation. Yes, there is a way to make the purpose of the function > understandable to a human without having him to do computer's work. > That's documentation. Undocumented code is broken code. Broken code is > not a good example when we're talking about right design.
I would still like to understand what this generator keyword would actually do. I don't see how it would work. Would a function marked generator somehow not be allowed to return normally or to finish and not return anything? How could this be enforced? I am completely against any keyword that is essentially documentation-only. -Rasmus

Pierre Joye

14 years ago
hi Rasmus! On Mon, Aug 20, 2012 at 2:01 AM, Rasmus Lerdorf <rasmus@lerdorf.com> wrote:
> I would still like to understand what this generator keyword would > actually do. I don't see how it would work. Would a function marked > generator somehow not be allowed to return normally or to finish and not > return anything? How could this be enforced? I am completely against any > keyword that is essentially documentation-only.
It is not documentation only (or should not but I have to check the implementation). It allows the use of the generators features inside this function. Just like what we have for static methods and this (somehow). Also I would really not begin to totally rethink the way generators work or are defined. There are clean and known ways in other languages, the closer we get from them the easier it will be to learn how to use them in PHP. Cheers,
-- Pierre @pierrejoye | http://blog.thepimp.net | http://www.libgd.org

Keisial

14 years ago
On 20/08/12 02:01, Rasmus Lerdorf wrote:
> I would still like to understand what this generator keyword would > actually do. I don't see how it would work. Would a function marked > generator somehow not be allowed to return normally or to finish and not > return anything? How could this be enforced? I am completely against any > keyword that is essentially documentation-only. > > -Rasmus
Given that such function could "return several times", seems a different enough function type to have its keyword. You could not decorate it and rely instead on the presence of the yield keyword, but parsers will thank knowing about it from the start rather than realising at mid-parsing that the function is a completely different beast. Regards

Rasmus Lerdorf

14 years ago
On 08/20/2012 07:56 AM, Ángel González wrote:
> On 20/08/12 02:01, Rasmus Lerdorf wrote: >> I would still like to understand what this generator keyword would >> actually do. I don't see how it would work. Would a function marked >> generator somehow not be allowed to return normally or to finish and not >> return anything? How could this be enforced? I am completely against any >> keyword that is essentially documentation-only. >> >> -Rasmus > Given that such function could "return several times", seems a different > enough function type to have its keyword. > You could not decorate it and rely instead on the presence of the yield > keyword, but parsers will thank knowing about it from the start rather > than realising at mid-parsing that the function is a completely > different beast.
So how about something like this: generator function f() { echo "Hello World"; } generator function f() { return 1; } generator function f($arg) { if(f!$arg) yield 1; else if($arg<0) return 1; else return; } What does the generator keyword mean in each of these cases? Anything? Nothing? Would I see a difference either at compile-time or at execute time if I left it out? -Rasmus

Lester Caine

14 years ago
Rasmus Lerdorf wrote:
> So how about something like this: > > generator function f() { > echo "Hello World"; > } > > generator function f() { > return 1; > } > > generator function f($arg) { > if(f!$arg) yield 1; > else if($arg<0) return 1; > else return; > } > > What does the generator keyword mean in each of these cases? Anything? > Nothing? Would I see a difference either at compile-time or at execute > time if I left it out?
Well the same could be said of some of the error messages generated by 'static' if E_STRICT is enabled. However an error on the first two saying 'no yield point found' would not be unreasonable. In reality a lot of this should simply be handled by a good IDE leaving the runtime code as clean as possible? For the third one ... I'm still waiting for some clarification on how yield is SUPPOSED to work anyway? If you are using a 'generator' to return a sequence of data elements, then just what does happen between each call to the generator ... DOES the generator get called for each value until something flags there are no more? I still don't see the advantage of restructuring how things are called to get around a problem which can just as easily be solved by proper design in the first place.
-- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk

Morgan L. Owens

14 years ago
On 2012-08-21 01:10, Lester Caine wrote:
> > For the third one ... I'm still waiting for some clarification on how > yield is SUPPOSED to work anyway? If you are using a 'generator' to > return a sequence of data elements, then just what does happen between > each call to the generator ... DOES the generator get called for each > value until something flags there are no more? I still don't see the > advantage of restructuring how things are called to get around a problem > which can just as easily be solved by proper design in the first place. >
Have you even read the RFC yet? Calling a generator function returns an instance of a Generator object. That object has a method interface - described in the RFC. That interface is accessible to the foreach() construct so the loop can iterate over the successive values returned by the object. If by "proper design" you mean your own personal preference for using callbacks, that's also discussed in the RFC. You were told this last month: On Sat, Jul 28, 2012 as 05:34 AM, Nikita Popov <nikita.ppv@gmail.com> wrote
> On Wed, Jul 25, 2012 at 10:36 PM, Lester Caine <lester@lsces.co.uk>
wrote:
>> But WHY would you not simply put the function that is handling the
returned
>> data in place of the yield? >> Why do you want to keep exiting and re-entering the 'do loop' when
the data
>> can simply be forwarded direct to a function to handle it? > >This question has come up a few times now, i.e. why one can't just use >callbacks. So I added a section about this, explaining it with a few >examples:
https://wiki.php.net/rfc/generators#why_not_just_use_callback_functions
> >Hope it helps, >Nikita
But let's compare the two side-by-side. So there is a "producer" that has or generates the successive elements of a sequence, and a "consumer" that does something with each element in turn. One can make a choice about which of the two - producer or consumer - gets to drive the process. In one (basically the Observer pattern), the producer drives, and the consumer gets to observe by supplying a callback that the producer calls for each new element. Since the producer has no idea about the internal workings of the consumer, the consumer has to ensure that it can properly resume the next time it is called. In the other (basically the Iterator pattern), the consumer drives, calling the producer each time a new element is required. Since the consumer has no idea about the internal workings of the producer, the producer has to ensure that it can properly resume the next time it is called. There are a couple of differences between using an observer and using an iterator. The main difference is because at least one of the producer or consumer needs to maintain its own state between iterations. For the producer-driven observer approach, it is the consumer's job to maintain state. - the producer's state is safe on the call stack. For the consumer-driven iterator approach, it is the producer's job to maintain state - the consumer's state is safe on the call stack. In the observer approach, writing the state-maintenance code falls to whoever is writing the consumer; the producer is unable to help with that at all because it's different for each consumer. In the iterator approach, writing the state-maintenance code falls to whoever is writing the producer; that code is the same regardless of the consumer. As this proposal and existing implementations show, it is possible, given the kernel behaviour of the iteration, to mechanically generate all the state-handling and other boilerplate necessary to produce a Generator object. That's the main difference. The other difference is that in the case of the iterator, in part because the code generation can be mechanised, the iteration mechanism can be done in such a way that it fits in with the iteration mechanisms that PHP already has built into the language: the foreach() statement doesn't care whether it's iterating over an array or an iterator or a generator. Either could be supplied at runtime. In contrast, to achieve the analogous effect with an observer mechanism would involve (among other things) searching for occurrences of control flow statements that reference the sequence, excising those statements and using them as the kernel for a callback method to be supplied to the sequence producer ... but only if the sequence in question wants all that done in the first place.

Lester Caine

14 years ago
Morgan L. Owens wrote:
>> For the third one ... I'm still waiting for some clarification on how >> yield is SUPPOSED to work anyway? If you are using a 'generator' to >> return a sequence of data elements, then just what does happen between >> each call to the generator ... DOES the generator get called for each >> value until something flags there are no more? I still don't see the >> advantage of restructuring how things are called to get around a problem >> which can just as easily be solved by proper design in the first place. >> > Have you even read the RFC yet? Calling a generator function returns an instance > of a Generator object. That object has a method interface - described in the > RFC. That interface is accessible to the foreach() construct so the loop can > iterate over the successive values returned by the object. > > If by "proper design" you mean your own personal preference for using callbacks, > that's also discussed in the RFC. You were told this last month:
You see that is my problem ... WHY is using an existing object instance so wrong? $handle = fopen("../data/clientdatabase.csv", "r"); if ( $handle == FALSE) { $row = -999; } else { while (($data = fgetcsv($handle, 800, ",")) !== FALSE) { if ( $row ) $contact->ContactRecordLoad( $data ); $row++; } fclose($handle); } $contact->ContactRecordLoad( $data ); simply processes the data and then throws it away, so why is this so bad? I am just trying to understand why my long time method of working is so frond upon, and understand if generators will improve the performance of the above 'very heavy' code. ContactRecordLoad identifies the 'line type' and then extracts the data based on the line type. My NLPG data importer has a dozen processor functions selected by the base 'RecordLoad', normally the only problem area is translating different date formats into something generic, so the basic iteration loop isn't a problem. How will the use of 'yield' make this better? Or more important - based on the rfc - why is this so wrong?
-- Lester Caine - G8HFL ----------------------------- Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk

John LeSueur

14 years ago
On Tue, Aug 21, 2012 at 12:31 AM, Lester Caine <lester@lsces.co.uk> wrote:
> Morgan L. Owens wrote: > >> For the third one ... I'm still waiting for some clarification on how >>> yield is SUPPOSED to work anyway? If you are using a 'generator' to >>> return a sequence of data elements, then just what does happen between >>> each call to the generator ... DOES the generator get called for each >>> value until something flags there are no more? I still don't see the >>> advantage of restructuring how things are called to get around a problem >>> which can just as easily be solved by proper design in the first place. >>> >>> Have you even read the RFC yet? Calling a generator function returns an >> instance >> of a Generator object. That object has a method interface - described in >> the >> RFC. That interface is accessible to the foreach() construct so the loop >> can >> iterate over the successive values returned by the object. >> >> If by "proper design" you mean your own personal preference for using >> callbacks, >> that's also discussed in the RFC. You were told this last month: >> > > You see that is my problem ... WHY is using an existing object instance so > wrong? > > $handle = fopen("../data/clientdatabase.**csv", "r"); > if ( $handle == FALSE) { > $row = -999; > } else { > while (($data = fgetcsv($handle, 800, ",")) !== FALSE) { > if ( $row ) $contact->ContactRecordLoad( $data ); > $row++; > } > fclose($handle); > } > > $contact->ContactRecordLoad( $data ); simply processes the data and then > throws it away, so why is this so bad? I am just trying to understand why > my long time method of working is so frond upon, and understand if > generators will improve the performance of the above 'very heavy' code. > ContactRecordLoad identifies the 'line type' and then extracts the data > based on the line type. My NLPG data importer has a dozen processor > functions selected by the base 'RecordLoad', normally the only problem area > is translating different date formats into something generic, so the basic > iteration loop isn't a problem. How will the use of 'yield' make this > better? Or more important - based on the rfc - why is this so wrong? > > No one is frowning on this way of doing things. It's perfectly fine for
the circumstances. Imagine however, that you have a new client, who provides a different format in their clientdatabase.csv. There are definitely ways you can handle this in your current code. But with generators you might be able to do: function origFormat() { $handle = fopen("../data/clientdatabase.csv", "r"); if ( $handle == FALSE) { return false; } else { $row = fgetcsv($handle, 800, ","); if($row !== false) yield fgetcsv($handle, 800, ","); } } function strangeFormat() { $data = origFormat(); foreach($data as $row) { //move some fields around to match the original format yield $row; } } if($client == 'strange') $data = strangeFormat(); else $data = origFormat(); foreach($data as $row) { $contact->ContactRecordLoad($data); } Another possibility is that you are importing records from another database, and now you can create a third generator that pulls from the db. The point is that your data source(the producer) is abstracted from the consumer, so the consumer doesn't need to have the logic to handle the different cases inside its (possibly complicated) foreach() loop. And the producer can be a simple function, rather than an object that implements the Iterator interface. Again, the case you've cited is probably not a case where generators give much advantage. But anytime the logic to produce some data becomes more complicated (Imagine that you needed to skip some of the lines in the .csv file based on some kind of filter), then being able to extricate the producer logic from the consumer logic is helpful. Thanks, John

Yahav Gindi Bar

14 years ago
On Mon, Aug 20, 2012 at 3:48 PM, Rasmus Lerdorf <rasmus@lerdorf.com> wrote:
> On 08/20/2012 07:56 AM, Ángel González wrote: > > On 20/08/12 02:01, Rasmus Lerdorf wrote: > >> I would still like to understand what this generator keyword would > >> actually do. I don't see how it would work. Would a function marked > >> generator somehow not be allowed to return normally or to finish and not > >> return anything? How could this be enforced? I am completely against any > >> keyword that is essentially documentation-only. > >> > >> -Rasmus > > Given that such function could "return several times", seems a different > > enough function type to have its keyword. > > You could not decorate it and rely instead on the presence of the yield > > keyword, but parsers will thank knowing about it from the start rather > > than realising at mid-parsing that the function is a completely > > different beast. > > So how about something like this: > > generator function f() { > echo "Hello World"; > } > > generator function f() { > return 1; > } > > generator function f($arg) { > if(f!$arg) yield 1; > else if($arg<0) return 1; > else return; > } > > What does the generator keyword mean in each of these cases? Anything? > Nothing? Would I see a difference either at compile-time or at execute > time if I left it out? > > -Rasmus > > -- > PHP Internals - PHP Runtime Development Mailing List > To unsubscribe, visit: http://www.php.net/unsub.php > >
I still don't think that we shall separate it into two different keywords, and even if so - we shall still reserve the ability to use return. Think of this case: function keywordsInFile($file, $keyword) { if ( ($han = fopen($file, 'r') === FALSE ) { return; // <-- this is better than starting to wrap the function with if-else } while(!feof(...)) { foreach ($keywords as $k) { if ( strpos($k, $line) !== FALSE ) { yield $k; } } } } Though it's not the best example and I can think about more complicated - it can reveal my point - I wish to use the "return" keyword in order to stop the current function run. In many complicated sites you got many if-else cases in one function that when the statement evaluates to true/false you shall return from the function. Structure like if ( ... ) { if ( ! ... ) { if ( .... ) { if ( ... ) { yield $v; } } } } is ugly and make the program harder to read than if ( ! ... ) return; if ( ... ) return; if (! ... ) return; if ( ! ... ) return; yield $k;

(Wietse Venema)

14 years ago
Rasmus Lerdorf:
> So how about something like this: > > generator function f() { > echo "Hello World"; > } > > generator function f() { > return 1; > } > > generator function f($arg) { > if(f!$arg) yield 1; > else if($arg<0) return 1; > else return; > } > > What does the generator keyword mean in each of these cases? Anything? > Nothing? Would I see a difference either at compile-time or at execute > time if I left it out?
An excellent argument to use "yield" where it is needed, instead of using "generator" at the top. Wietse

Stas Malyshev

14 years ago
Hi!
> generator function f() { > echo "Hello World"; > }
even more interesting, $a = generator function() { echo "Hello World"; } or even: function foo() { return generator function() { echo "Hello World"; } } $a = foo();
-- Stanislav Malyshev, Software Architect SugarCRM: http://www.sugarcrm.com/ (408)454-6900 ext. 227

Nikita Popov

14 years ago
On Mon, Aug 20, 2012 at 1:56 PM, Ángel González <keisial@gmail.com> wrote:
> On 20/08/12 02:01, Rasmus Lerdorf wrote: >> I would still like to understand what this generator keyword would >> actually do. I don't see how it would work. Would a function marked >> generator somehow not be allowed to return normally or to finish and not >> return anything? How could this be enforced? I am completely against any >> keyword that is essentially documentation-only. >> >> -Rasmus > Given that such function could "return several times", seems a different > enough function type to have its keyword.
The method signature defines the API, so the caller knows what to use. Can we agree on that? In this case it makes absolutely no difference to the caller whether the function is implemented using a generator, or whether it returns a custom Iterator object. The "generator" keyword wouldn't document the API, it would document an implementation-detail. What would *actually* make sense here are return value typehints. E.g. one could have something like `public Iterator getIterator() { ... }`. This would provide the caller with information on what the function returns, but would leave out the implementation detail that the Iterator was implemented using a generator. Or, if the generator implementation is actually important (because it is used as a coroutine) one could also explicitly typehint against it: `public Generator getCoroutine() { ... }`. But return-value type hints are not directly related to generators. They are a more general concept. If that's what all of you want, then I'd recommend opening up a new thread for it.
> You could not decorate it and rely instead on the presence of the yield > keyword, but parsers will thank knowing about it from the start rather > than realising at mid-parsing that the function is a completely > different beast.
No, parsers don't care about this. It's trivial to detect in both cases. Nikita

Jared Williams

14 years ago
> -----Original Message----- > From: Nikita Popov [mailto:nikita.ppv@gmail.com] > Sent: 20 August 2012 14:12 > To: Ángel González > Cc: Rasmus Lerdorf; Stas Malyshev; Derick Rethans; PHP internals > Subject: Re: [PHP-DEV] [RFC] Generators > > On Mon, Aug 20, 2012 at 1:56 PM, Ángel González > <keisial@gmail.com> wrote: > > On 20/08/12 02:01, Rasmus Lerdorf wrote: > >> I would still like to understand what this generator keyword
would
> >> actually do. I don't see how it would work. Would a > function marked > >> generator somehow not be allowed to return normally or to > finish and > >> not return anything? How could this be enforced? I am completely > >> against any keyword that is essentially documentation-only. > >> > >> -Rasmus > > Given that such function could "return several times", seems a > > different enough function type to have its keyword. > The method signature defines the API, so the caller knows what to
use.
> Can we agree on that? In this case it makes absolutely no > difference to the caller whether the function is implemented > using a generator, or whether it returns a custom Iterator > object. The "generator" > keyword wouldn't document the API, it would document an > implementation-detail. > > What would *actually* make sense here are return value typehints.
E.g.
> one could have something like `public Iterator getIterator() { ...
}`.
> This would provide the caller with information on what the > function returns, but would leave out the implementation > detail that the Iterator was implemented using a generator. > Or, if the generator implementation is actually important > (because it is used as a > coroutine) one could also explicitly typehint against it: > `public Generator getCoroutine() { ... }`. >
Yes. Quick example... interface A { function gen(); } class AImplementation implements A { function gen() { for($i = 0; $i < 10; ++$i) yield $i; } } class ADecorator implements A { private $a; function __construct(A $a) { $this->a = $a; } function gen() { return $this->a->gen(); } } $a = new ADecorator(new AImplementation()); foreach($a->gen() as $v) echo $v, "\n"; Obviously, AImplementation::gen() is a generator, but ADecorator::gen() isn't, but would want both to adhere to the same interface. So only return type hinting makes sense.
> But return-value type hints are not directly related to generators. > They are a more general concept. If that's what all of you > want, then I'd recommend opening up a new thread for it. > > > You could not decorate it and rely instead on the presence of the > > yield keyword, but parsers will thank knowing about it from > the start > > rather than realising at mid-parsing that the function is a > completely > > different beast. > No, parsers don't care about this. It's trivial to detect in > both cases. > > Nikita > > -- > PHP Internals - PHP Runtime Development Mailing List To > unsubscribe, visit: http://www.php.net/unsub.php >
Jared

Stas Malyshev

14 years ago
Hi!
> What would *actually* make sense here are return value typehints. E.g. > one could have something like `public Iterator getIterator() { ... }`.
And again we're back to inventing syntax to do what documentation should be doing.
-- Stanislav Malyshev, Software Architect SugarCRM: http://www.sugarcrm.com/ (408)454-6900 ext. 227

Andrew Faulds

14 years ago
On 20/08/12 19:46, Stas Malyshev wrote:
> Hi! > >> What would *actually* make sense here are return value typehints. E.g. >> one could have something like `public Iterator getIterator() { ... }`. > And again we're back to inventing syntax to do what documentation should > be doing. >
For the large part, but consider type hints mean that if you return a completely wrong type, you can warn the programmer.
-- Andrew Faulds http://ajf.me/

Keisial

14 years ago
On 20/08/12 15:12, Nikita Popov wrote:
>> You could not decorate it and rely instead on the presence of the yield >> keyword, but parsers will thank knowing about it from the start rather >> than realising at mid-parsing that the function is a completely >> different beast. > No, parsers don't care about this. It's trivial to detect in both cases. > > Nikita
Yes, it's trivial to detect, but how much work is for it to do so? Suppose the parser reads php code and outputs machine instructions. When it encouters a function(), it adds the function prologue, goes reserving stack space for the variables it encounters, and so on. Then you find a yield keyword. Options: - Discard the generated code and go back to the function begin, using this time an storage into an object instead of the stack. - Copy all the state to an object and return it. - Add an initial pass checking which functions are generators. PHP being a dynamic language, it may be able to move the function variables to a class. But it's not trivial for any parser.

Alex Aulbach

14 years ago
I think this doesn't bring anything further to talk about how to implement this in the parser. After reading the thread again, I would like to say that - either this should be implemented as currently defined - or it is implemented with the generator-keyword. But then it isn't a function anymore, because that's is ridiculous. In that case it should look like this: generator g() { yield; return; } 2012/8/20 Ángel González <keisial@gmail.com>:
> On 20/08/12 15:12, Nikita Popov wrote: >>> You could not decorate it and rely instead on the presence of the yield >>> keyword, but parsers will thank knowing about it from the start rather >>> than realising at mid-parsing that the function is a completely >>> different beast. >> No, parsers don't care about this. It's trivial to detect in both cases. >> >> Nikita > Yes, it's trivial to detect, but how much work is for it to do so? > Suppose the parser reads php code and outputs machine instructions. When > it encouters a function(), it adds the function prologue, goes reserving > stack > space for the variables it encounters, and so on. Then you find a yield > keyword. > Options: > - Discard the generated code and go back to the function begin, using > this time > an storage into an object instead of the stack. > - Copy all the state to an object and return it. > - Add an initial pass checking which functions are generators. > > PHP being a dynamic language, it may be able to move the function variables > to a class. But it's not trivial for any parser. > > > > -- > PHP Internals - PHP Runtime Development Mailing List > To unsubscribe, visit: http://www.php.net/unsub.php >
-- Freundliche Grüße Alex Aulbach

Nikita Popov

14 years ago
On Mon, Aug 20, 2012 at 10:07 PM, Ángel González <keisial@gmail.com> wrote:
> On 20/08/12 15:12, Nikita Popov wrote: >>> You could not decorate it and rely instead on the presence of the yield >>> keyword, but parsers will thank knowing about it from the start rather >>> than realising at mid-parsing that the function is a completely >>> different beast. >> No, parsers don't care about this. It's trivial to detect in both cases. >> >> Nikita > Yes, it's trivial to detect, but how much work is for it to do so? > Suppose the parser reads php code and outputs machine instructions. When > it encouters a function(), it adds the function prologue, goes reserving > stack > space for the variables it encounters, and so on. Then you find a yield > keyword. > Options: > - Discard the generated code and go back to the function begin, using > this time > an storage into an object instead of the stack. > - Copy all the state to an object and return it. > - Add an initial pass checking which functions are generators. > > PHP being a dynamic language, it may be able to move the function variables > to a class. But it's not trivial for any parser.
Sorry, I don't understand what we are arguing about here. I implemented this. I did the parser changes for this feature. And I told you that they were simple either way (both with keyword, as in the initial implementation, and without keyword, as in the current implementation). For 3rd parties (like IDEs) it is even simpler because their parsers are AST-based (unlike PHP's own parser), so they shouldn't care at all. Honestly, I stopped understanding what this whole discussion is about around ~100 mails ago. Nikita

Jared Williams

14 years ago
> -----Original Message----- > From: Stas Malyshev [mailto:smalyshev@sugarcrm.com] > Sent: 20 August 2012 00:08 > To: Derick Rethans > Cc: Nikita Popov; PHP internals > Subject: Re: [PHP-DEV] [RFC] Generators > > Hi! > > > I am against this. This is even more magic in PHP. Is it > really that > > difficult to have to mark the function with a different > keyword, such > > as > > "generator": > > You have a point here, but "public static final generator > function foo()" sounds a bit long-winded to me... Also, we'd > have then to decide which function can be marked generator > and which can't (e.g., interface probably can't, abstract > probably can't, anonymous probably can, etc.) which adds more > complexity.
That'd be return type hinting would it not? Given that generator functions currently return an instance of class Generator. Also anonymous generators would be quite strange $f = Generator function() { yield 'a'; }; Jared

Andrew Faulds

14 years ago
On 18/08/12 13:21, Derick Rethans wrote:
> On Sat, 11 Aug 2012, Nikita Popov wrote: > >> Hi internals! >> >> I think there already was a lot of discussion on the generators, so >> it's time to move to the next step. I'd like to vote on the feature in >> two weeks, so this the "announce[ment] on internals@, by the author, >> with the intention of voting on it". >> >> https://wiki.php.net/rfc/generators >> >> If you have any further feedback, now would be a good time to raise >> it. If there is something you previously posted, but which I didn't >> yet address, please let me know. There were around 150 mails and I >> sure missed some of them. > I've some comments how that I've read the RFC: > >> Recognition of generator functions >> >> 1. Any function which contains a yield statement is automatically a >> generator function. >> >> 2. The initial implementation required that generator functions are >> marked with an asterix modifier (function*). This method has the >> advantage that generators are more explicit and also allows for >> yield-less coroutines. >> >> The automatic detection was chosen over the asterix modifier for the >> following reasons: > I am against this. This is even more magic in PHP. Is it really that > difficult to have to mark the function with a different keyword, such as > "generator":
You know, this conversation feels like it's going in circles. I thought we already agreed there wouldn't be one.
> >> generator function getLinesFromFile($fileName) { >> if (!$fileHandle = fopen($fileName, 'r')) { >> return; >> } >> There is an existing generator implementation in HipHop PHP, which >> uses automatic-detection. Using the asterix modifier would break >> compatibility. > This should not be a concern, sure, it's annoying for the hiphop > developers but they chose to copy and then *chance* the PHP language for > their own effect. > >> yield: Yields the value null with an auto-incrementing integer key. > What is the usecase for this? > > cheers, > Derick >
-- Andrew Faulds http://ajf.me/