Everything

php

FastCGI limit memory

php.internals

Reinis Rozitis

19 years ago

Hello,
is there a chance that this will ever make in PHP5 (or othet future
releases) as base feature http://www.zend.com/zend/week/pat/pat48.txt
That is besides PHP_FCGI_MAX_REQUESTS to have also the posibility limit the
childs max memory usage/leak (PHP_FCGI_MAX_RAM_MB)

I'm asking about this just because yesterday while testing MagickWand (
http://www.magickwand.org  which according to Thomas Boutell
http://marc.theaimsgroup.com/?l=php-dev&m=115698025320619 in some point
should replace GD) just by accident discovered a huge leak in phpinfo()
output of the extension.

Just by running some 100-1000 requests each of the php childs (10 total)
memory usage grew up to ~1Gb but thats not the point (I'll try to valgrind
and isolate the problem and send the bug to the MW developers).

My point is - although 5.2.x core has become much much better in memory
management (4.4.x actually didnt really work without the patch) you can't
say that there are no leaks in longer time span in all the bundled and not
bundled extensions.
And although apache with its prefork is still a leader a lot of webservers
which run php in fastcgi mode are getting more and more popular.

For example lighttpd according to NetCraft:

01.2007 lighttpd 172819
02.2007 lighttpd 702712

Besides looking at http://survey.netcraft.com/Reports/0702/ those which I
know ( Zeus, lighttpd, nginx, LiteSpeed, Sun One) pretty all run PHP using
fastcgi.

So how about it?

rr

Pierre Joye

19 years ago

On 2/5/07, Reinis Rozitis <r@roze.lv> wrote:

> I'm asking about this just because yesterday while testing MagickWand (
> http://www.magickwand.org  which according to Thomas Boutell
> http://marc.theaimsgroup.com/?l=php-dev&m=115698025320619 in some point
> should replace GD) just by accident discovered a huge leak in phpinfo()
> output of the extension.

It will never replace the GD extension. Please read the whole thread
before making conclusions (btw, check out http://www.libgd.org).

> For example lighttpd according to NetCraft:
>
> 01.2007 lighttpd 172819
> 02.2007 lighttpd 702712

You should wait march, such a jump is suspect :)

--Pierre

Reinis Rozitis

19 years ago

> It will never replace the GD extension. Please read the whole thread
> before making conclusions (btw, check out http://www.libgd.org).

I didnt mean that way nor as a conclusion, GD is still much faster and so on
(but thanks for the link/hint).
It was just an excuse why did I play with magickwand (what Thomas proposed)
in case somebody would ask why there is a need for such feature and just
respond with "we dont support third-party extension.. if it breaks core then
its your own fault" ;)

Aye the lighty stats look "fishy" (lets wait march) but whatever its
something Netcraft provides..


Besides getting comments on all other parts but not the initial question? :)

rr

Pierre Joye

19 years ago

Hi,

On 2/5/07, Reinis Rozitis <r@roze.lv> wrote:

> > It will never replace the GD extension. Please read the whole thread
> > before making conclusions (btw, check out http://www.libgd.org).
>
> I didnt mean that way nor as a conclusion, GD is still much faster and so on
> (but thanks for the link/hint).

Keep an eye on the current 2.1.0 development, it will be really fast :)

> It was just an excuse why did I play with magickwand (what Thomas proposed)
> in case somebody would ask why there is a need for such feature and just
> respond with "we dont support third-party extension.. if it breaks core then
> its your own fault" ;)
>
> Aye the lighty stats look "fishy" (lets wait march) but whatever its
> something Netcraft provides..
>
>
> Besides getting comments on all other parts but not the initial question? :)

I'm not sure that you are looking at the right place to solve the
problem. If the leaks are in phpinfo (or in memory allocated by php),
then maybe (really not sure).

But if the leaks are in IM as their extension does not use php memory
manage, it is not something fixable by php or anything else but IM.

Note to anyone, please correct me if I'm mistaken :)

--Pierre

Reinis Rozitis

19 years ago

> I'm not sure that you are looking at the right place to solve the
> problem. If the leaks are in phpinfo (or in memory allocated by php),
> then maybe (really not sure).
>
> But if the leaks are in IM as their extension does not use php memory
> manage, it is not something fixable by php or anything else but IM.

Pierre you are getting me totally wrong.
I dont really care for this particular ImageMagick(Wand) or even phpinfo
leak at all :)

What I am talking about is a feature which similar as 'memory_limit' (inside
one script runtime) limits maximum memory usage for a long running php child
(if spawned with the -b option (external FASTCGI Server mode)).
At the moment the only option to avoid memory leaks is to set
PHP_FCGI_MAX_REQUESTS after which the child dies and a new is spawned. Still
the memory usage can grow (as example in this case) way too quick to
actually hit the request limit, which may bring the system in OOM state.

The patch was provided for 4.x branch
http://www.zend.com/zend/week/pat/pat48.txt
So I wanted to get some feebacks (without filling the feature request at
bugs.php.net which I apologise for) if its something worth to implement.
Probably to hear something from Dmitry.

> Keep an eye on the current 2.1.0 development, it will be really fast :)

Is the source allready available for testing?
Those 3 remaining tasks in roadmap didnt look problematic at this state
(though I suppose there are more issues)..

rr

Pierre Joye

19 years ago

On 2/5/07, Reinis Rozitis <r@roze.lv> wrote:

> > I'm not sure that you are looking at the right place to solve the
> > problem. If the leaks are in phpinfo (or in memory allocated by php),
> > then maybe (really not sure).
> >
> > But if the leaks are in IM as their extension does not use php memory
> > manage, it is not something fixable by php or anything else but IM.
>
> Pierre you are getting me totally wrong.
> I dont really care for this particular ImageMagick(Wand) or even phpinfo
> leak at all :)

Yes, that I got that, I only liked to say that this patch will not
help that much, only to be sure about what is your goal.

> What I am talking about is a feature which similar as 'memory_limit' (inside
> one script runtime) limits maximum memory usage for a long running php child
> (if spawned with the -b option (external FASTCGI Server mode)).
> At the moment the only option to avoid memory leaks is to set
> PHP_FCGI_MAX_REQUESTS after which the child dies and a new is spawned. Still
> the memory usage can grow (as example in this case) way too quick to
> actually hit the request limit, which may bring the system in OOM state.
>
> The patch was provided for 4.x branch
> http://www.zend.com/zend/week/pat/pat48.txt
> So I wanted to get some feebacks (without filling the feature request at
> bugs.php.net which I apologise for) if its something worth to implement.
> Probably to hear something from Dmitry.

The best place to have a definitive opinion would be to report a bug,
put a link to the patch and assign it to Dmitry. Having a bug report
will also help to do not get this patch lost somewhere in the www.

Please note that 4.4 is frozen, there is no chance
to get such patches in it, but maybe in 5.2.2.

> > Keep an eye on the current 2.1.0 development, it will be really fast :)
>
> Is the source allready available for testing?
> Those 3 remaining tasks in roadmap didnt look problematic at this state
> (though I suppose there are more issues)..

All sources are in CVS (see the Downloads page), I'm just beginning
the 2.1.0 changes in CVS (2.0.34-final is due for Wednesday) :).

--Pierre

Andi Gutmans

19 years ago

Don't the FastCGI processes inherit memory limits from their parent? (assuming you're not running standalone FastCGI which almost
noone does).

Andi

Reinis Rozitis

19 years ago

Andi Gutmans wrote:

> Don't the FastCGI processes inherit memory limits from their parent? (assuming you're not running standalone FastCGI which almost
> noone does).
>

Nope, the parent (master) process only keeps the track of active childs
(usually I spawn like 250 of them) and distributes the incoming
requests. As far as I understand each child operates on its own.

rr

Andi Gutmans

19 years ago

Yeah but process limits are inherited after fork().

Reinis Rozitis

19 years ago

Andi Gutmans wrote:

> Yeah but process limits are inherited after fork().
>

Well probably. But does there exists a configurable limit in php for that
(which was implemented by previos mentioned patch in 4.x branch)?  As if the
child exceeds the limit it handles the last request and shuts down to be
respawned by master. Killing the process from outside doesnt sound like a
nice solution.

rr

Reinis Rozitis

19 years ago

>> For example lighttpd according to NetCraft:
>>
>> 01.2007 lighttpd 172819
>> 02.2007 lighttpd 702712
>
> You should wait march, such a jump is suspect :)
>
> --Pierre

Hello Pierre,

03.2007 lighttpd 1.399.786

http://survey.netcraft.com/Reports/0703/

This gives place 4, right behind Sun (if you skip 'unknown'). Still quite a
small percentage from the overal pool but the growth remains ;)

What brings back my question about the possibility to enforce php fastcgi
childs max allowed memory (on the process itself).

rr