Everything
php
Learn Community Releases The PHP Project
Home » Project » Newsgroups » php.internals » serialize and cache handling »

serialize and cache handling

php.internals

Mathias Bank

19 years ago
Hallo, I use pear cache (light) to save calculated values. The package uses serialize to transform the content of a variable into a writable form. This transformed value will be saved to disk. This could be improved! Would it be possible to rewrite the serialize function to make it possible to write directly to disk? If you have a value, which needs 10MB you need also ca. 10MB to serialize and than you can write this data to disk. So, you needs 20MB. If serialize (and of course unserialize) would be able to write directly to disk (or read directly from disk), you only needs 10MB. I think, that could perform much better for every file cache than current implementation does. I have such a scenario, in which a variable needs 100MB. Because the calculation of this value needs much time, I wanted to save this value to cache (with the help of (un)serialize). But than, I need 200MB and that doesn't look nice, because it is not really necessary. As I understand the serialize structure, it would be no problem to extend it to directly save to disk. But my c knowledge is not as good as it should be. I would be very happy, if this idea could be realized. Mathias

Stanislav Malyshev

19 years ago
> can write this data to disk. So, you needs 20MB. If serialize (and of > course unserialize) would be able to write directly to disk (or read > directly from disk), you only needs 10MB.
Actually having serialize/unserialize be able to write directly to a stream and read directly from a stream might be interesting, would probably improve working with things like large sessions or caching large data substantially.
-- Stanislav Malyshev, Zend Products Engineer stas@zend.com http://www.zend.com/

Lukas Kahwe Smith

19 years ago
Stanislav Malyshev wrote:
>> can write this data to disk. So, you needs 20MB. If serialize (and of >> course unserialize) would be able to write directly to disk (or read >> directly from disk), you only needs 10MB. > > Actually having serialize/unserialize be able to write directly to a > stream and read directly from a stream might be interesting, would > probably improve working with things like large sessions or caching > large data substantially.
Indeed, especially since this is the most common use case. Maybe it should optionally also return an md5 of the written data. regards, Lukas

Derick Rethans

19 years ago
On Mon, 7 May 2007, Lukas Kahwe Smith wrote:
> Stanislav Malyshev wrote: > > > can write this data to disk. So, you needs 20MB. If serialize (and of > > > course unserialize) would be able to write directly to disk (or read > > > directly from disk), you only needs 10MB. > > > > Actually having serialize/unserialize be able to write directly to a stream > > and read directly from a stream might be interesting, would probably improve > > working with things like large sessions or caching large data substantially. > > Indeed, especially since this is the most common use case. Maybe it should > optionally also return an md5 of the written data.
If we're to add this, make sure writes to the files are atomic. regards, Derick
-- Derick Rethans http://derickrethans.nl | http://ez.no | http://xdebug.org

Stanislav Malyshev

19 years ago
> If we're to add this, make sure writes to the files are atomic.
Does PHP now ensure fwrite is atomic? If it doesn't than writing on serialize doesn't change a thing.
-- Stanislav Malyshev, Zend Products Engineer stas@zend.com http://www.zend.com/

Derick Rethans

19 years ago
On Tue, 8 May 2007, Stanislav Malyshev wrote:
> > If we're to add this, make sure writes to the files are atomic. > > Does PHP now ensure fwrite is atomic? If it doesn't than writing on serialize > doesn't change a thing.
Only "a" mode is atomic - per write call; normal fwrites are not. However, you'd need to write the *whole* file to disk atomically and not on every fwrite. And you can not first cache it in memory as you then lose the whole advantage of this idea. regards, Derick

Stanislav Malyshev

19 years ago
> on every fwrite. And you can not first cache it in memory as you then > lose the whole advantage of this idea.
IIRC sessions are locked by php anyway, and for other uses if locking is important it is already implemented anyway, so we shouldn't really try to solve all the world's problems with this one.
-- Stanislav Malyshev, Zend Products Engineer stas@zend.com http://www.zend.com/

Derick Rethans

19 years ago
On Tue, 8 May 2007, Stanislav Malyshev wrote:
> > on every fwrite. And you can not first cache it in memory as you then lose > > the whole advantage of this idea. > > IIRC sessions are locked by php anyway, and for other uses if locking is > important it is already implemented anyway, so we shouldn't really try to > solve all the world's problems with this one.
? Nobody is talking about sessions here, just about the serialize() function that is also used for a myriad of other things... regards, Derick

Stanislav Malyshev

19 years ago
> ? Nobody is talking about sessions here, just about the serialize()
You mean you are not talking about sessions. I, however, do. Sessions are one of the obvious examples where such functionality could improve performance.
> function that is also used for a myriad of other things...
Oh really? I guess that's why we talk about _adding_ stuff to it in order to make it more efficient *in certain scenarios* and not replacing it with the new one. Precisely the scenarios where external locking would be happening anyway - when either using sessions or caching.
-- Stanislav Malyshev, Zend Products Engineer stas@zend.com http://www.zend.com/

Lukas Kahwe Smith

19 years ago
Stanislav Malyshev wrote:
>> ? Nobody is talking about sessions here, just about the serialize() > > You mean you are not talking about sessions. I, however, do. Sessions > are one of the obvious examples where such functionality could improve > performance.
Well as the topic implies I am quite sure that the user request was about caching into a custom file and not inside the session. Both are frequent use cases. regards, Lukas

Richard Lynch

19 years ago
On Tue, May 8, 2007 1:17 am, Derick Rethans wrote:
> On Mon, 7 May 2007, Lukas Kahwe Smith wrote: > >> Stanislav Malyshev wrote: >> > > can write this data to disk. So, you needs 20MB. If serialize >> (and of >> > > course unserialize) would be able to write directly to disk (or >> read >> > > directly from disk), you only needs 10MB. >> > >> > Actually having serialize/unserialize be able to write directly to >> a stream >> > and read directly from a stream might be interesting, would >> probably improve >> > working with things like large sessions or caching large data >> substantially. >> >> Indeed, especially since this is the most common use case. Maybe it >> should >> optionally also return an md5 of the written data. > > If we're to add this, make sure writes to the files are atomic.
Is this suggesting that the entire 80M upload has to be done in a single operation?... Or is the md5/sha1 computed chunk by chunk, in parallel, with writing buffered data to the disk? Cuz if it's the former, I don't see that working out too well for ginormous uploaded files... Which people probably shouldn't be doing over HTTP anyway, but they do, and that's the reality one has to deal with... Apologies if I'm being alarmist and totally mis-reading this through my ignorance.
-- Some people have a "gift" link here. Know what I want? I want you to buy a CD from some indie artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So?

Derick Rethans

19 years ago
On Tue, 8 May 2007, Richard Lynch wrote:
> On Tue, May 8, 2007 1:17 am, Derick Rethans wrote: > > On Mon, 7 May 2007, Lukas Kahwe Smith wrote: > > > >> Stanislav Malyshev wrote: > >> > > can write this data to disk. So, you needs 20MB. If serialize > >> > > (and of course unserialize) would be able to write directly to > >> > > disk (or read directly from disk), you only needs 10MB. > >> > > >> > Actually having serialize/unserialize be able to write directly > >> > to a stream and read directly from a stream might be interesting, > >> > would probably improve working with things like large sessions or > >> > caching large data substantially. > >> > >> Indeed, especially since this is the most common use case. Maybe it > >> should optionally also return an md5 of the written data. > > > >>If we're to add this, make sure writes to the files are atomic. > > >s this suggesting that the entire 80M upload has to be done in a > >single operation?...
Wrong thread ;-) This is on serialize, not on hashes. regards, Derick
-- Derick Rethans http://derickrethans.nl | http://ez.no | http://xdebug.org