increase the default memory limit in 5.2.0+

bertrand Gugger

20 years ago

Bonjour,
Pierre wrote:

> Hello,
>
> After having tested 5.2 and the new memory manager for a couple of
> weeks, I notice a significant increase of the reported memory usage. As
> a side effect, many applications reach the 8M memory limit. It can be
> a problem.
>
> A good example is to compare the usage of a pear/pecl command between
> 5.1 and 5.2.
>
> I would suggest to increase the default 8M to 12M. I discussed this
> issue with Dmitry and he agrees on the principle. Any other opinions?
> Or is it fine to increase it?

I know a few cms they would be pleased with 12M.
That looks quite fair to me.

But , why the hell is this not a configurable limit by build ?
Why do we need to --enable-memory-limit to be able to get a custom value
but then at run time only ?

In clear, I would like to be able to generate php with a limit given in
./configure.
This limit may be different from an install to another , depending on
the applications.

I hope my poor english is understandable.

Regards.

Antony Dovgal

20 years ago

On 16.08.2006 15:13, bertrand Gugger wrote:

> Bonjour,
> Pierre wrote:
>> Hello,
>>
>> After having tested 5.2 and the new memory manager for a couple of
>> weeks, I notice a significant increase of the reported memory usage. As
>> a side effect, many applications reach the 8M memory limit. It can be
>> a problem.
>>
>> A good example is to compare the usage of a pear/pecl command between
>> 5.1 and 5.2.
>>
>> I would suggest to increase the default 8M to 12M. I discussed this
>> issue with Dmitry and he agrees on the principle. Any other opinions?
>> Or is it fine to increase it?
>
> I know a few cms they would be pleased with 12M.
> That looks quite fair to me.
>
> But , why the hell is this not a configurable limit by build ?

JFYI: Its value is configurable in php.ini, configure option just enables or disables the limit itself.

--
Wbr,
Antony Dovgal

bertrand Gugger

20 years ago

Antony Dovgal wrote:

> On 16.08.2006 15:13, bertrand Gugger wrote:
>
>> Bonjour,
>> Pierre wrote:
>>
>>> Hello,
>>>
>>> After having tested 5.2 and the new memory manager for a couple of
>>> weeks, I notice a significant increase of the reported memory usage. As
>>> a side effect, many applications reach the 8M memory limit. It can be
>>> a problem.
>>>
>>> A good example is to compare the usage of a pear/pecl command between
>>> 5.1 and 5.2.
>>>
>>> I would suggest to increase the default 8M to 12M. I discussed this
>>> issue with Dmitry and he agrees on the principle. Any other opinions?
>>> Or is it fine to increase it?
>>
>>
>> I know a few cms they would be pleased with 12M.
>> That looks quite fair to me.
>>
>> But , why the hell is this not a configurable limit by build ?
>
>
> JFYI: Its value is configurable in php.ini, configure option just
> enables or disables the limit itself.
>

Exactly my question , why is it configurable *only* at run time ?

I want to generate a php with a custom *fixed* limit , but not enable
memory limit setting at run time.

Sorry if my english is bad , but please , try to understand my question.

Regards

Pierre Joye

20 years ago

Hello,

On 8/16/06, bertrand Gugger <toggg@php.net> wrote:

> Antony Dovgal wrote:
> > On 16.08.2006 15:13, bertrand Gugger wrote:
> >
> >> Bonjour,
> >> Pierre wrote:
> >>
> >>> Hello,
> >>>
> >>> After having tested 5.2 and the new memory manager for a couple of
> >>> weeks, I notice a significant increase of the reported memory usage. As
> >>> a side effect, many applications reach the 8M memory limit. It can be
> >>> a problem.
> >>>
> >>> A good example is to compare the usage of a pear/pecl command between
> >>> 5.1 and 5.2.
> >>>
> >>> I would suggest to increase the default 8M to 12M. I discussed this
> >>> issue with Dmitry and he agrees on the principle. Any other opinions?
> >>> Or is it fine to increase it?
> >>
> >>
> >> I know a few cms they would be pleased with 12M.
> >> That looks quite fair to me.
> >>
> >> But , why the hell is this not a configurable limit by build ?
> >
> >
> > JFYI: Its value is configurable in php.ini, configure option just
> > enables or disables the limit itself.
> >
> Exactly my question , why is it configurable *only* at run time ?
>
> I want to generate a php with a custom *fixed* limit , but not enable
> memory limit setting at run time.

It is not at runtime, it is a php.ini definition, which is parsed
before the execution. But this is not the subject of this discussion.

--Pierre

Derick Rethans

20 years ago

On Wed, 16 Aug 2006, Pierre wrote:

> I would suggest to increase the default 8M to 12M. I discussed this
> issue with Dmitry and he agrees on the principle. Any other opinions?
> Or is it fine to increase it?

I actually thought we increased it to 16M already... but apparently not.

regards,
Derick

Pierre Joye

20 years ago

Hello,

On 8/16/06, Derick Rethans <derick@php.net> wrote:

> On Wed, 16 Aug 2006, Pierre wrote:
>
> > I would suggest to increase the default 8M to 12M. I discussed this
> > issue with Dmitry and he agrees on the principle. Any other opinions?
> > Or is it fine to increase it?
>
> I actually thought we increased it to 16M already... but apparently not.

I was trying to find a minimum value to reduce the troubles., but 16M
would be perfect :)

--Pierre

Mark Evans

20 years ago

>
>
> I would suggest to increase the default 8M to 12M. I discussed this
> issue with Dmitry and he agrees on the principle. Any other opinions?
> Or is it fine to increase it?

Could we not increase it to 16MB? Almost every server I have used has
changes the php.ini from 8MB to 16MB anyway.

Regards

Marco

Stefan Esser

20 years ago

Hello,

> After having tested 5.2 and the new memory manager for a couple of
> weeks, I notice a significant increase of the reported memory usage. As
> a side effect, many applications reach the 8M memory limit. It can be
> a problem.
>
> A good example is to compare the usage of a pear/pecl command between
> 5.1 and 5.2.
>

The question is why do we want a new memory manager that seems to
allocate 1.5 times more memory than the old one.
I haven't looked into it yet, but for me this looks like a waste of
ressources. What are the advantages of it. I really doubt,
that a significant speed increase can be measured, and I wonder if it
worth the waste of memory.

Yours,
Stefan Esser

Matt W

20 years ago

----- Original Message -----
From: "Stefan Esser"
Sent: Wednesday, August 16, 2006

> Hello,
> > After having tested 5.2 and the new memory manager for a couple of
> > weeks, I notice a significant increase of the reported memory usage. As
> > a side effect, many applications reach the 8M memory limit. It can be
> > a problem.
> >
> > A good example is to compare the usage of a pear/pecl command between
> > 5.1 and 5.2.
> >
> The question is why do we want a new memory manager that seems to
> allocate 1.5 times more memory than the old one.
> I haven't looked into it yet, but for me this looks like a waste of
> ressources. What are the advantages of it. I really doubt,
> that a significant speed increase can be measured, and I wonder if it
> worth the waste of memory.

I found there was about a 10% difference overall (Windows) before/after when
running the Zend/bench.php script -- some parts are ever-so-slightly slower,
but things like string concatenation (erealloc()...) are massively faster.
And I know Andi has said it's supposed to perform better under load and over
time.

About the memory usage amount, I had mentioned the following in a previous
discussion, but I guess I'm wrong, so I don't want to be annoying. :-)  I
found that emalloc()'ing big chunks anyway (many MB+) on Windows (assume
others are the same), *doesn't actually* take "real" memory from the system
(only VM) unless or until it's actually filled with something ("used"),
which makes sense.  So I was wondering why the overhead (where the "higher"
usage comes from) even needed to be counted with the new manager...?  But,
maybe that's only with larger chunks that it's not "really" taken, or it
changes over time with fragmentation or something...

Actually, after typing that, it probably is wrong for the way I think the
Memory Manager handles its small blocks!  (efree()'d space isn't returned?
But dunno when/if the blocks are truely free()'d.)

> Yours,
> Stefan Esser

Matt

Dmitry Stogov

20 years ago

> -----Original Message-----
> From: Stefan Esser [mailto:sesser@php.net]
> Sent: Wednesday, August 16, 2006 4:40 PM
> To: pierre.php@gmail.com
> Cc: internals@lists.php.net
> Subject: Re: [PHP-DEV] increase the default memory limit in 5.2.0+
>
>
> Hello,
> > After having tested 5.2 and the new memory manager for a couple of
> > weeks, I notice a significant increase of the reported
> memory usage.
> > As a side effect, many applications reach the 8M memory
> limit. It can
> > be a problem.
> >
> > A good example is to compare the usage of a pear/pecl
> command between
> > 5.1 and 5.2.
> >
> The question is why do we want a new memory manager that
> seems to allocate 1.5 times more memory than the old one. I
> haven't looked into it yet, but for me this looks like a
> waste of ressources. What are the advantages of it. I really
> doubt, that a significant speed increase can be measured, and
> I wonder if it worth the waste of memory.

The bew memory manager allocates LESS memory and works faster.
It allocates memory from system by large blocks and then manage heap by
itself.
The memory_limit is checked not for each emalloc() call (as befor), but for
real blocks that requested from system.
So memory_limit is more accurate in new memory manager.
Old memory manager didn't calculate all memory overheads that were used by
malloc library.

Dmitry.

Andi Gutmans

20 years ago

The new memory manager actually wastes less memory than the old one, scales
better and most important to this discussion, the memory consumption is far
more accurate as it also takes into account the memory manager overhead.
Therefore, we are not consuming 1.5x memory, but we are reporting a more
accurate number which is about 1.5x of what the old one reported.
My suggestion is to raise the limit to 16M both in the php.ini* files that
come with PHP and in the source.

Andi

Ilia A.

20 years ago

I've bumped the memory limit to 16 megs in RC2 and PHP6.

On 16-Aug-06, at 8:43 PM, Andi Gutmans wrote:

> The new memory manager actually wastes less memory than the old
> one, scales
> better and most important to this discussion, the memory
> consumption is far
> more accurate as it also takes into account the memory manager
> overhead.
> Therefore, we are not consuming 1.5x memory, but we are reporting a
> more
> accurate number which is about 1.5x of what the old one reported.
> My suggestion is to raise the limit to 16M both in the php.ini*
> files that
> come with PHP and in the source.
>
> Andi
>
>> -----Original Message-----
>> From: Stefan Esser [mailto:sesser@php.net]
>> Sent: Wednesday, August 16, 2006 5:40 AM
>> To: pierre.php@gmail.com
>> Cc: internals@lists.php.net
>> Subject: Re: [PHP-DEV] increase the default memory limit in 5.2.0+
>>
>> Hello,
>>> After having tested 5.2 and the new memory manager for a couple of
>>> weeks, I notice a significant increase of the reported
>> memory usage.
>>> As a side effect, many applications reach the 8M memory
>> limit. It can
>>> be a problem.
>>>
>>> A good example is to compare the usage of a pear/pecl
>> command between
>>> 5.1 and 5.2.
>>>
>> The question is why do we want a new memory manager that
>> seems to allocate 1.5 times more memory than the old one.
>> I haven't looked into it yet, but for me this looks like a
>> waste of ressources. What are the advantages of it. I really
>> doubt, that a significant speed increase can be measured, and
>> I wonder if it worth the waste of memory.
>>
>> Yours,
>> Stefan Esser
>>
>> --
>> PHP Internals - PHP Runtime Development Mailing List To
>> unsubscribe, visit: http://www.php.net/unsub.php
>>
>
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>

Ilia Alshanetsky

Ilia A.

20 years ago

On 16-Aug-06, at 6:30 AM, Pierre wrote:

> Hello,
>
> After having tested 5.2 and the new memory manager for a couple of
> weeks, I notice a significant increase of the reported memory
> usage. As
> a side effect, many applications reach the 8M memory limit. It can be
> a problem.
>

Is this caused by the increased memory usage during input processing
caused by the filter extension? I would be very interested in hearing
what has caused PHP memory utilization to go up, since that will
likely result in reduced performance as well.

Ilia Alshanetsky

Pierre Joye

20 years ago

Hello,

On 8/16/06, Ilia Alshanetsky <ilia@prohost.org> wrote:

> On 16-Aug-06, at 6:30 AM, Pierre wrote:
>
> > Hello,
> >
> > After having tested 5.2 and the new memory manager for a couple of
> > weeks, I notice a significant increase of the reported memory
> > usage. As
> > a side effect, many applications reach the 8M memory limit. It can be
> > a problem.
> >
>
> Is this caused by the increased memory usage during input processing
> caused by the filter extension? I would be very interested in hearing
> what has caused PHP memory utilization to go up, since that will
> likely result in reduced performance as well.

No, it is not filter extension. As I said, a good example is to use
the pear command (list-all, package or install).

Also I do not think (neither said) that there is a 1.5x increase
factor. I was talking about reported memory usage. My point is that
the new system seems more accurate.

In general, with or without the new memory manager, I do think we
should increase this limit. Many common applications/usages reach it
already.

--Pierre

Dmitry Stogov

20 years ago

PHP 5.2 and PHP 4.4 use the same default memory limit - 8M, but near each
internal structure in php 5.2 is bigger (zval, zend_class_entry,
zend_op_array, zend_opcode, ...).this is one of the reasin to incrise
default memory limit.

The second reason is new memory manager that uses sliht different symantick
for memory_limit. (Memeory taken from system, instead of memory requested
from malloc() library).

I think we should set default memory_limit to 16M.

Thanks. Dmitry.

Peter Brodersen

20 years ago

On Wed, 16 Aug 2006 12:30:06 +0200, pierre.php@gmail.com (Pierre)
wrote:

>I would suggest to increase the default 8M to 12M. I discussed this
>issue with Dmitry and he agrees on the principle. Any other opinions?
>Or is it fine to increase it?

Just out of curiousity regarding default values and slightly
off-memory_limit-topic:

session.gc_maxlifetime has a default value of 1440 (as of revision 1.2
of php.ini-dist where it was introduced). Obviously some value has to
be the default value, but 1440 seconds seem as a peculiar value. 24
minutes is not a rounded value such as 1 day (86400 secounds) would
be.

I'm only guessing, but is it possible that this default value was
added with the idea that the value was specified in minutes, not
seconds (as 1440 minutes equals 1 day)? The current php.ini-dist
mentions correctly that it equals 24 minutes (per 1.162).


I would like this default value to be globally changed from 1440 to
86400. I think that the 24 minute "timeout" limit would and does
confuse developers and users with spurious "suddently my session
disappeared, but I didn't close my browser" issues.

I believe that it is more easy and nice for a web developer to discard
a session in a system with a high gc_maxlifetime than to keep a
session alive (e.g. having the page access a php resource every couple
of minutes using javascript).

--
- Peter Brodersen

Robert Cummings

20 years ago

On Tue, 2006-08-22 at 04:56 +0200, Peter Brodersen wrote:

> On Wed, 16 Aug 2006 12:30:06 +0200, pierre.php@gmail.com (Pierre)
> wrote:
>
> >I would suggest to increase the default 8M to 12M. I discussed this
> >issue with Dmitry and he agrees on the principle. Any other opinions?
> >Or is it fine to increase it?
>
> Just out of curiousity regarding default values and slightly
> off-memory_limit-topic:
>
> session.gc_maxlifetime has a default value of 1440 (as of revision 1.2
> of php.ini-dist where it was introduced). Obviously some value has to
> be the default value, but 1440 seconds seem as a peculiar value. 24
> minutes is not a rounded value such as 1 day (86400 secounds) would
> be.
>
> I'm only guessing, but is it possible that this default value was
> added with the idea that the value was specified in minutes, not
> seconds (as 1440 minutes equals 1 day)? The current php.ini-dist
> mentions correctly that it equals 24 minutes (per 1.162).
>
>
> I would like this default value to be globally changed from 1440 to
> 86400. I think that the 24 minute "timeout" limit would and does
> confuse developers and users with spurious "suddently my session
> disappeared, but I didn't close my browser" issues.
>
> I believe that it is more easy and nice for a web developer to discard
> a session in a system with a high gc_maxlifetime than to keep a
> session alive (e.g. having the page access a php resource every couple
> of minutes using javascript).

The short duration is to help ensure better privacy. It means that if
you leave a computer and forget to close down the browser then snoopy
people only have 24 minutes to access anything you left behind.

24 minutes is probably some value that lies between irritatingly quick
session expiry and overly long session lifespans -- at least for a
default value.

Cheers,
Rob.

--
.------------------------------------------------------------.
| InterJinn Application Framework - http://www.interjinn.com |
:------------------------------------------------------------:
| An application and templating framework for PHP. Boasting  |
| a powerful, scalable system for accessing system services  |
| such as forms, properties, sessions, and caches. InterJinn |
| also provides an extremely flexible architecture for       |
| creating re-usable components quickly and easily.          |
`------------------------------------------------------------'

Peter Brodersen

20 years ago

Hi,

On Mon, 21 Aug 2006 23:26:18 -0400, in php.internals
robert@interjinn.com (Robert Cummings) wrote:

>> I believe that it is more easy and nice for a web developer to discard
>> a session in a system with a high gc_maxlifetime than to keep a
>> session alive (e.g. having the page access a php resource every couple
>> of minutes using javascript).
>
>The short duration is to help ensure better privacy. It means that if
>you leave a computer and forget to close down the browser then snoopy
>people only have 24 minutes to access anything you left behind.

That value still seem pretty arbitrary. As mentioned, it would be much
more easy for a developer to discard specific session variables after
a shorter period of time than by extending this period.

Far from all session data is private data and don't have to be nuked
after 24 minutes. The developer could also distinguish between
sensitive- and non-sensitive-data and discard a few session variables
instead of nuking every data in the current session.

The method of extending the timeout is also pretty cumbersome and
pretty much a hack. One would have the pages to perform background
requests as a form of keepalive requests.

>24 minutes is probably some value that lies between irritatingly quick
>session expiry and overly long session lifespans -- at least for a
>default value.

I would disagree. I don't think that the default value of 1440 seconds
was set because of this - I simply think it was meant to be one day in
the first place. I believe that a value of half a day or a day is more
sensible. At least I have experienced a couple of incidents, where I
or some of my users have used more than 24 minutes between pages.

As well as other parts of PHP the application should handle security,
including reasonable timeout values for different tasks. The garbage
collection is a safety valve (as with any other garbage collection).

Summarized:
A higher default gc_maxlifetime would benefit:
+ Fewer strange session expiration incidents (with non-private data)
+ Easier for the administrator to maintain his own timeout value
(above 24 minutes) without resolving to keepalive-hacks
! Security handling should, where possible, be performed by the code.

--
- Peter Brodersen

Robert Cummings

20 years ago

On Thu, 2006-08-31 at 03:21 +0200, Peter Brodersen wrote:

> Hi,
>
> On Mon, 21 Aug 2006 23:26:18 -0400, in php.internals
> robert@interjinn.com (Robert Cummings) wrote:
>
> [-SNIP-]
>
> ! Security handling should, where possible, be performed by the code.

Yes, security should be handled by the code, but erring on the side of
caution is a good practice when setting defaults.

Cheers,
Rob.

--
.------------------------------------------------------------.
| InterJinn Application Framework - http://www.interjinn.com |
:------------------------------------------------------------:
| An application and templating framework for PHP. Boasting  |
| a powerful, scalable system for accessing system services  |
| such as forms, properties, sessions, and caches. InterJinn |
| also provides an extremely flexible architecture for       |
| creating re-usable components quickly and easily.          |
`------------------------------------------------------------'