Integrating SPL

php.internals

Sterling Hughes

23 years ago
Hey, I'd like to talk about merging SPL into the engine, or at least taking some of the core components of SPL and adapting them to the engine, so that the same functionality is available per-default in PHP5. SPL provides a standard set of interfaces, specifically: Iterator: Allows you to override the foreach construct to work on your object. interface spl_forward { function current(); function next(); function has_more(); } class foo implements spl_forward { } $f = new foo; foreach ($f as $e) { } Array spl_array_read allows you to overload reading elements from an array. interface spl_array_read { function exists($key); function get($key); } spl_array_access allows you to overload writes as well as reads: interface spl_array_access implements spl_array_read { function set($value, $index); } The way I look at it, SPL has to important concepts that we should look at integrating in to Zend Engine 2/PHP5: 1) Overloading objects as arrays. This is a crucial feature when interfacing with external sources, like java, or .net, or even a db extension. It allows you to lazy initialize all data coming from an external source, only taking what you need. Further, there is no need to drink the OO kool-aid, in order to take advantage of this feature. 2) The use of interfaces to ascertain whether or not an object supports a given feature set. The usage of interfaces is good for three reasons. First it is much faster than our current method of checking whether a method exists or not. Second it allows us to have coherent method names (no double underscore), without risking any backwards compatibility isues. Thirdly, it makes it more clear that a class is overloaded, and allows you to simply check for overloads. Therefore, when I talk about integrating SPL and PHP, I think the following steps should be taken: 1) "Fix" the object overloading model to allow you to handle array accesses internally. This includes the iterator concept and and array read and write. This portion of the code would be an integral part of the overloading mechanism (no interfaces required). 2) Export and revise the userspace implementation of overloading, using interfaces to define when and how a object is overloaded. I propose something along the lines of (this stuff isn't original, just formalized) :: // Base overload interface, to allow for checks on whether or not a // object is overloaded interface Overloaded { }; // Overloaded_Value overloads the object itself, allowing you to treat // the object as a base type interface Overloaded_Value implements Overloaded { // Called when the object is directly assigned to // @return null function setValue($value); // Called when the object is cast to type a certain type // or kind-of type // valid types are :: // 'boolean' // 'integer' // 'double' // 'number' // 'string' // 'array' // @return value A value of type $type function asType($type); }; // Overloaded_Iterator allows you to overload an object, to allow // foreach () to move over the object interface Overloaded_Iterator implements Overloaded { // Move forward one element in the object // @return null function next(); // Get the value of the current object // @return zval (a PHP value) function current(); // Tells whether or not the object has more data // @return bool function hasMore(); }; // Overloaded properties allows you to overload both property // *and* array accesses. interface Overloaded_Properties implements Overloaded { // Get $property // @return zval (a PHP value) function get($property); // Set $property // @return nothing function set($property, $value); // Check whether or not $property exists // @return bool function exists($property); } // Allow you to overload method calls interface Overloaded_Methods implements Overloaded { // Catch a method call, $name is the name of the method // @return zval (a PHP value) function call($name); }; Thoughts? This should be relatively easy to implement (although I'm not sure on the timing with regards to the beta), and after thinking about it, this just seems like the way to do it. -Sterling PS: 95% of this message is just building on prior art. Zeev and Andi's prior art, and marcus's prior art. All credit to them :)
-- Good judgement comes from experience, and experience comes from bad judgement. - Fred Brooks

Andrei Zmievski

23 years ago
On Wed, 25 Jun 2003, Sterling Hughes wrote:
> 1) "Fix" the object overloading model to allow you to handle array > accesses internally. This includes the iterator concept and and array > read and write. This portion of the code would be an integral part of > the overloading mechanism (no interfaces required).
Please - I've been asking for this feature for god knows how long now. I very much hope this is done soon, because PHP-GTK requires it. -Andrei * "I'll need daily status reports on why you're so behind." -- Dilbert's boss *

Andi Gutmans

23 years ago
Hey, In general I think the ideas behind SPL are interesting. However, my main problem with the whole array and iterator overloading is that it's not quite clear to me where this should have an effect. Will it only work in foreach()? Is it supposed to work in array_sort() and all other internal functions? Is it supposed to work with assignment of the array() construct to an object? You can see where I am heading... I don't have the answers and I think we might end up in one big mess... About the use of interfaces for implementing methods such as __call(). This might be sexier but in many cases it won't improve performance (I think). For example, __call() only gets called if the first function lookup failed. Once that happens we just check if ce has a __call function ptr. Searching the class tree for the Callable_Overload (or whatever you want to call it) interface will not be as quick, or in the least not quicker. A bit of a mess email but I hope I gave you guys some food for thought. Please think about these issues carefully before replying. In general, I don't think this should effect the release date of the beta. Got to run now. Andi At 05:59 PM 25/6/2003 -0400, Sterling Hughes wrote:

(Marcus Börger)

23 years ago
Hello Andi, Friday, June 27, 2003, 9:53:50 AM, you wrote: AG> Hey, AG> In general I think the ideas behind SPL are interesting. However, my main AG> problem with the whole array and iterator overloading is that it's not AG> quite clear to me where this should have an effect. Will it only work in AG> foreach()? Is it supposed to work in array_sort() and all other internal AG> functions? Is it supposed to work with assignment of the array() construct AG> to an object? AG> You can see where I am heading... I don't have the answers and I think we AG> might end up in one big mess... I can't see where you're heading because the engine structure makes anything besides my intention to use array syntax with objects impossible. According to iterators it is the same. They are intended to be used inside foreach but there is noone forcing you to do, it would be quite enough to use them in a for loop. However using them in a for loop wouldn't be that efficient. Anyway the idea behind iterators is to be possible to consistent way to work with that commonly used pattern. And in the future i plan to implement more higher level classes which will make use of the spl interfaces. And by the use of typehints this will becoming a really nice thing i guess. AG> About the use of interfaces for implementing methods such as __call(). This AG> might be sexier but in many cases it won't improve performance (I think). AG> For example, __call() only gets called if the first function lookup failed. AG> Once that happens we just check if ce has a __call function ptr. Searching AG> the class tree for the Callable_Overload (or whatever you want to call it) AG> interface will not be as quick, or in the least not quicker. To the above i can only agree, it will most likely not improve speed at all. However sometimes there are other things besides speed. And then, lemme think, checking for an interface (1) is much faster then search for a function in a function table and then it will be absolutly exceptional having __call so my idea is to remove the __call pointer and use interfaces instead. The result will be a little speed decrease for a vaste minority of classes but less memory. 1) No need to traverse the class hierarchy. AG> A bit of a mess email but I hope I gave you guys some food for thought. AG> Please think about these issues carefully before replying. Sure, i get what you maen. But maybe you should play a bit with the spl concepts already implemented. And by that get a feeling how sexy (sorry for borrowing the word) they really are. Maybe you could think about this. AG> In general, I don't think this should effect the release date of the beta. Again, i can't see the hurry. I mean we have really good concepts here and two of the ideas are ready to rock. So the only thing we ask is whether we may include foreach/array overloading before beta 1. AG> Got to run now. Yeah, me too, i spent already to much time with php. regards marcus
-- Best regards, Marcus mailto:helly@php.net

Sterling Hughes

23 years ago
On Fri, 2003-06-27 at 04:09, Marcus Börger wrote:
> Hello Andi, > > Friday, June 27, 2003, 9:53:50 AM, you wrote: > > AG> Hey, > > AG> In general I think the ideas behind SPL are interesting. However, my main > AG> problem with the whole array and iterator overloading is that it's not > AG> quite clear to me where this should have an effect. Will it only work in > AG> foreach()? Is it supposed to work in array_sort() and all other internal > AG> functions? Is it supposed to work with assignment of the array() construct > AG> to an object? > AG> You can see where I am heading... I don't have the answers and I think we > AG> might end up in one big mess... > > I can't see where you're heading because the engine structure makes anything > besides my intention to use array syntax with objects impossible. According to > iterators it is the same. They are intended to be used inside foreach but > there is noone forcing you to do, it would be quite enough to use them in a > for loop. However using them in a for loop wouldn't be that efficient. Anyway > the idea behind iterators is to be possible to consistent way to work with > that commonly used pattern. And in the future i plan to implement more higher > level classes which will make use of the spl interfaces. And by the use of > typehints this will becoming a really nice thing i guess. > > AG> About the use of interfaces for implementing methods such as __call(). This > AG> might be sexier but in many cases it won't improve performance (I think). > AG> For example, __call() only gets called if the first function lookup failed. > AG> Once that happens we just check if ce has a __call function ptr. Searching > AG> the class tree for the Callable_Overload (or whatever you want to call it) > AG> interface will not be as quick, or in the least not quicker. > > To the above i can only agree, it will most likely not improve speed at all. > However sometimes there are other things besides speed. And then, lemme think, > checking for an interface (1) is much faster then search for a function in a > function table and then it will be absolutly exceptional having __call so my > idea is to remove the __call pointer and use interfaces instead. The result > will be a little speed decrease for a vaste minority of classes but less > memory. >
It certainly doesn't have to hurt speed. Any cache that was maintained with the former implementation could be maintained with the current implementation. You also don't have to traverse the class tree. I think for builtin interfaces you can just use bitmask and check that. -Sterling
-- "First they ignore you, then they laugh at you, then they fight you, then you win." - Gandhi

(Marcus Börger)

23 years ago
Hello Sterling, Friday, June 27, 2003, 4:11:39 PM, you wrote: SH> On Fri, 2003-06-27 at 04:09, Marcus Börger wrote:
>> Hello Andi, >> >> Friday, June 27, 2003, 9:53:50 AM, you wrote: >> >> AG> Hey, >> >> AG> In general I think the ideas behind SPL are interesting. However, my main >> AG> problem with the whole array and iterator overloading is that it's not >> AG> quite clear to me where this should have an effect. Will it only work in >> AG> foreach()? Is it supposed to work in array_sort() and all other internal >> AG> functions? Is it supposed to work with assignment of the array() construct >> AG> to an object? >> AG> You can see where I am heading... I don't have the answers and I think we >> AG> might end up in one big mess... >> >> I can't see where you're heading because the engine structure makes anything >> besides my intention to use array syntax with objects impossible. According to >> iterators it is the same. They are intended to be used inside foreach but >> there is noone forcing you to do, it would be quite enough to use them in a >> for loop. However using them in a for loop wouldn't be that efficient. Anyway >> the idea behind iterators is to be possible to consistent way to work with >> that commonly used pattern. And in the future i plan to implement more higher >> level classes which will make use of the spl interfaces. And by the use of >> typehints this will becoming a really nice thing i guess. >> >> AG> About the use of interfaces for implementing methods such as __call(). This >> AG> might be sexier but in many cases it won't improve performance (I think). >> AG> For example, __call() only gets called if the first function lookup failed. >> AG> Once that happens we just check if ce has a __call function ptr. Searching >> AG> the class tree for the Callable_Overload (or whatever you want to call it) >> AG> interface will not be as quick, or in the least not quicker. >> >> To the above i can only agree, it will most likely not improve speed at all. >> However sometimes there are other things besides speed. And then, lemme think, >> checking for an interface (1) is much faster then search for a function in a >> function table and then it will be absolutly exceptional having __call so my >> idea is to remove the __call pointer and use interfaces instead. The result >> will be a little speed decrease for a vaste minority of classes but less >> memory. >>
SH> It certainly doesn't have to hurt speed. Any cache that was maintained SH> with the former implementation could be maintained with the current SH> implementation. You also don't have to traverse the class tree. I SH> think for builtin interfaces you can just use bitmask and check that. I don't see a reason to make it more compley....but if performance is everything then some bitmasks would help of course. Anyway inside foreach hook of SPL those things are used (did you get it from there?)
-- Best regards, Marcus mailto:helly@php.net

Andi Gutmans

23 years ago
At 10:09 AM 27/6/2003 +0200, Marcus Börger wrote:
>Hello Andi, > >Friday, June 27, 2003, 9:53:50 AM, you wrote: > >AG> Hey, > >AG> In general I think the ideas behind SPL are interesting. However, my main >AG> problem with the whole array and iterator overloading is that it's not >AG> quite clear to me where this should have an effect. Will it only work in >AG> foreach()? Is it supposed to work in array_sort() and all other internal >AG> functions? Is it supposed to work with assignment of the array() >construct >AG> to an object? >AG> You can see where I am heading... I don't have the answers and I think we >AG> might end up in one big mess... > >I can't see where you're heading because the engine structure makes anything >besides my intention to use array syntax with objects impossible. According to >iterators it is the same. They are intended to be used inside foreach but >there is noone forcing you to do, it would be quite enough to use them in a >for loop. However using them in a for loop wouldn't be that efficient. Anyway >the idea behind iterators is to be possible to consistent way to work with >that commonly used pattern. And in the future i plan to implement more higher >level classes which will make use of the spl interfaces. And by the use of >typehints this will becoming a really nice thing i guess.
Okay, so what I understand from your answer is that Iterators should only be a PHP language feature (i.e. usable inside scripts and automatically supported by foreach). Any other internal functions or code which iterates over arrays will not use this. That's an OK definition as far as I'm concerned because I don't think the latter is feasible, I just want to make sure we're on the same page. About array overloading I just want to make sure that $obj[1] is all you guys want. I hope you don't expect the engine to translate $obj = array(1,2,3) into some kind of $obj[0] = 1; $obj[1] = 2; kind of thingy. If all you guys need is the former I think that makes sense.
>AG> About the use of interfaces for implementing methods such as __call(). >This >AG> might be sexier but in many cases it won't improve performance (I think). >AG> For example, __call() only gets called if the first function lookup >failed. >AG> Once that happens we just check if ce has a __call function ptr. >Searching >AG> the class tree for the Callable_Overload (or whatever you want to call >it) >AG> interface will not be as quick, or in the least not quicker. > >To the above i can only agree, it will most likely not improve speed at all. >However sometimes there are other things besides speed. And then, lemme think, >checking for an interface (1) is much faster then search for a function in a >function table and then it will be absolutly exceptional having __call so my >idea is to remove the __call pointer and use interfaces instead. The result >will be a little speed decrease for a vaste minority of classes but less >memory. > >1) No need to traverse the class hierarchy.
I agree that there are things above speed. In any case, we could always make it an interface *and* for such important language level interfaces such as Callable_Overload support the ce->__call optimization. I think that over all supporting the __get/__set/__call via an interface might have its advantages of cleanliness although it's a very subtle issue.
>AG> A bit of a mess email but I hope I gave you guys some food for thought. >AG> Please think about these issues carefully before replying. > >Sure, i get what you maen. But maybe you should play a bit with the spl >concepts already implemented. And by that get a feeling how sexy (sorry for >borrowing the word) they really are. Maybe you could think about this.
I don't need to play with it. I've played enough with Java which does pretty much the same. AG> In general, I don't think this should effect the release date of the beta.
>Again, i can't see the hurry. I mean we have really good concepts here and >two of the ideas are ready to rock. So the only thing we ask is whether we may >include foreach/array overloading before beta 1.
The big hurry about beta 1 is that it has been lingering for too long and I think the only way to push PHP 5 is to finally get a beta out there. We can keep on compromising indefinitely because there'll always be that one last fix someone wants to do to the PHP 5 codebase and then we'd never get a beta out of the door. Personally, I don't think this code needs to be in the tree for beta 1. From past beta experience we'll probably have another one within a month (because ppl will finally be using/debugging our code), so we can put that in beta 2. So how do we get a formal spec for the changes (starting with nicer names, i.e. Iterator instead of spl_foreach)? :) It's probably best if you take Sterling's email and try and come to a final set of interfaces and a very very short explanation where relevant of what it'll do and then we can aye/nay each part. Does that make sense? Andi

Sterling Hughes

23 years ago
On Fri, 2003-06-27 at 09:22, Andi Gutmans wrote:
> At 10:09 AM 27/6/2003 +0200, Marcus Börger wrote: > >Hello Andi, > > > >Friday, June 27, 2003, 9:53:50 AM, you wrote: > > > >AG> Hey, > > > >AG> In general I think the ideas behind SPL are interesting. However, my main > >AG> problem with the whole array and iterator overloading is that it's not > >AG> quite clear to me where this should have an effect. Will it only work in > >AG> foreach()? Is it supposed to work in array_sort() and all other internal > >AG> functions? Is it supposed to work with assignment of the array() > >construct > >AG> to an object? > >AG> You can see where I am heading... I don't have the answers and I think we > >AG> might end up in one big mess... > > > >I can't see where you're heading because the engine structure makes anything > >besides my intention to use array syntax with objects impossible. According to > >iterators it is the same. They are intended to be used inside foreach but > >there is noone forcing you to do, it would be quite enough to use them in a > >for loop. However using them in a for loop wouldn't be that efficient. Anyway > >the idea behind iterators is to be possible to consistent way to work with > >that commonly used pattern. And in the future i plan to implement more higher > >level classes which will make use of the spl interfaces. And by the use of > >typehints this will becoming a really nice thing i guess. > > Okay, so what I understand from your answer is that Iterators should only > be a PHP language feature (i.e. usable inside scripts and automatically > supported by foreach). Any other internal functions or code which iterates > over arrays will not use this. That's an OK definition as far as I'm > concerned because I don't think the latter is feasible, I just want to make > sure we're on the same page.
Yep, this is also what I'm talking about.
> About array overloading I just want to make sure that $obj[1] is all you > guys want. > I hope you don't expect the engine to translate $obj = array(1,2,3) into > some kind of $obj[0] = 1; $obj[1] = 2; kind of thingy. If all you guys need > is the former I think that makes sense.
Yes. What I was talking about (as you'll see in my predefined interface list), is simply something that makes array accesses look like property accesses. The property name to get_property and set_property
> > >AG> About the use of interfaces for implementing methods such as __call(). > >This > >AG> might be sexier but in many cases it won't improve performance (I think). > >AG> For example, __call() only gets called if the first function lookup > >failed. > >AG> Once that happens we just check if ce has a __call function ptr. > >Searching > >AG> the class tree for the Callable_Overload (or whatever you want to call > >it) > >AG> interface will not be as quick, or in the least not quicker. > > > >To the above i can only agree, it will most likely not improve speed at all. > >However sometimes there are other things besides speed. And then, lemme think, > >checking for an interface (1) is much faster then search for a function in a > >function table and then it will be absolutly exceptional having __call so my > >idea is to remove the __call pointer and use interfaces instead. The result > >will be a little speed decrease for a vaste minority of classes but less > >memory. > > > >1) No need to traverse the class hierarchy. > > I agree that there are things above speed. In any case, we could always > make it an interface *and* for such important language level interfaces > such as Callable_Overload support the ce->__call optimization. > I think that over all supporting the __get/__set/__call via an interface > might have its advantages of cleanliness although it's a very subtle issue. >
Yeah. My main purpose was for cleanliness, not performance. Its very clear what you are doing when implement the interface. Further, you are 100% BC compliant, and you don't need those underscores before the function names (unless of course you think they look better ;-)
> AG> In general, I don't think this should effect the release date of the beta. > > >Again, i can't see the hurry. I mean we have really good concepts here and > >two of the ideas are ready to rock. So the only thing we ask is whether we may > >include foreach/array overloading before beta 1. > > The big hurry about beta 1 is that it has been lingering for too long and I > think the only way to push PHP 5 is to finally get a beta out there. We can > keep on compromising indefinitely because there'll always be that one last > fix someone wants to do to the PHP 5 codebase and then we'd never get a > beta out of the door. > Personally, I don't think this code needs to be in the tree for beta 1. > From past beta experience we'll probably have another one within a month > (because ppl will finally be using/debugging our code), so we can put that > in beta 2. >
I happen to agree. But I think we should set some sort-of "deadline" on this. Ie, if it doesn't hold up beta 1, then we should have it done by beta 2. How does that sound for a compromise?
> So how do we get a formal spec for the changes (starting with nicer names, > i.e. Iterator instead of spl_foreach)? :) It's probably best if you take > Sterling's email and try and come to a final set of interfaces and a very > very short explanation where relevant of what it'll do and then we can > aye/nay each part. >
What's wrong with my interfaces? ;-D -Sterling
-- "Backups are for wimps. Real men upload their data to an FTP site and have everyone else mirror it." - Linus Torvalds

Andi Gutmans

23 years ago
At 10:19 AM 27/6/2003 -0400, Sterling Hughes wrote:
> > The big hurry about beta 1 is that it has been lingering for too long > and I > > think the only way to push PHP 5 is to finally get a beta out there. We > can > > keep on compromising indefinitely because there'll always be that one last > > fix someone wants to do to the PHP 5 codebase and then we'd never get a > > beta out of the door. > > Personally, I don't think this code needs to be in the tree for beta 1. > > From past beta experience we'll probably have another one within a month > > (because ppl will finally be using/debugging our code), so we can put that > > in beta 2. > > > >I happen to agree. But I think we should set some sort-of "deadline" on >this. Ie, if it doesn't hold up beta 1, then we should have it done by >beta 2. How does that sound for a compromise?
We should decide and implement whatever is decided on by beta2.
> > So how do we get a formal spec for the changes (starting with nicer names, > > i.e. Iterator instead of spl_foreach)? :) It's probably best if you take > > Sterling's email and try and come to a final set of interfaces and a very > > very short explanation where relevant of what it'll do and then we can > > aye/nay each part. > > > >What's wrong with my interfaces? ;-D
Ugly names? :) I'm not very sure about bringing things like cast overloading into the language space but we can argue about that if/when we have the final list. If you say what you did is final we can start with that. Andi

Sterling Hughes

23 years ago
> We should decide and implement whatever is decided on by beta2. >
great.
> > > So how do we get a formal spec for the changes (starting with nicer names, > > > i.e. Iterator instead of spl_foreach)? :) It's probably best if you take > > > Sterling's email and try and come to a final set of interfaces and a very > > > very short explanation where relevant of what it'll do and then we can > > > aye/nay each part. > > > > > > >What's wrong with my interfaces? ;-D > > Ugly names? :) I'm not very sure about bringing things like cast > overloading into the language space but we can argue about that if/when we > have the final list. If you say what you did is final we can start with that. >
Ugly names - I'm wounded. ;-) Anyhow, I've repasted them here for your criticism ;-) // Base overload interface, to allow for checks on whether or not a // object is overloaded interface Overloaded { }; // Overloaded_Value overloads the object itself, allowing you to treat // the object as a base type interface Overloaded_Value implements Overloaded { // Called when the object is directly assigned to // @return null function setValue($value); // Called when the object is cast to type a certain type // or kind-of type // valid types are :: // 'boolean' // 'integer' // 'double' // 'number' // 'string' // 'array' // @return value A value of type $type function asType($type); }; // Overloaded_Iterator allows you to overload an object, to allow // foreach () to move over the object interface Overloaded_Iterator implements Overloaded { // Move forward one element in the object // @return null function next(); // Get the value of the current object // @return zval (a PHP value) function current(); // Tells whether or not the object has more data // @return bool function hasMore(); }; // Overloaded properties allows you to overload both property // *and* array accesses. interface Overloaded_Properties implements Overloaded { // Get $property // @return zval (a PHP value) function get($property); // Set $property // @return nothing function set($property, $value); // Check whether or not $property exists // @return bool function exists($property); } // Allow you to overload method calls interface Overloaded_Methods implements Overloaded { // Catch a method call, $name is the name of the method // @return zval (a PHP value) function call($name); }; -Sterling
> Andi
-- "Programming today is a race between software engineers stirring to build bigger and better idiot-proof programs, and the universe trying to produce bigger and better idiots. So far, the universe is winning." - Unknown