[PATCH] Disable the Zend memory cache

php.internals

Vesselin Atanasov

22 years ago
Hello. The memory cache code in zend_alloc.c has serious problems with memory fragmentation. When a big script is run which allocates both lots of small objects and a few big objects, the memory cache grows until the allowed limits (11 buckets x 256 blocks each) keeping all of the cached memory occupied although the script uses much less memory. This is caused by the cache code which caches small objects and does not allow the low level memory manager (either the zend_mm.c or the C runtime malloc) to merge small free blocks into bigger free blocks. From the point of view of the low level memory manager there are lots of occupied memory blocks interspersed among the allocated memory segment. the occupied memory blocks are actually cached by the high level cache code. So when the script is run a new segment is allocated although there is free memory. With each call new segment(s) are allocated until the memory cache is full. This scenario happens when a script first creates lots of small objects and then a few large objects. Even if the small objects are freed, the memory is fragmented and a new block is occupied for the big object(s). So with each request the lost memory increases, effectively this is like a memory leak. I attach a patch which adds --disable-memory-cache option which disables the memory cache. vesselin diff -ruN php5-200309020130.orig/Zend/Zend.m4 php5-200309020130/Zend/Zend.m4 --- php5-200309020130.orig/Zend/Zend.m4 2003-08-11 06:07:39.000000000 +0000 +++ php5-200309020130/Zend/Zend.m4 2003-09-02 09:28:46.000000000 +0000 @@ -122,6 +122,13 @@ ZEND_INLINE_OPTIMIZATION=yes ]) +AC_ARG_ENABLE(memory-cache, +[ --disable-memory-cache Disable the Zend memory cache. ], [ + ZEND_MEMORY_CACHE=$enableval +],[ + ZEND_MEMORY_CACHE=yes +]) + AC_ARG_ENABLE(memory-limit, [ --enable-memory-limit Compile with memory limit support. ], [ ZEND_MEMORY_LIMIT=$enableval @@ -142,6 +149,9 @@ AC_MSG_CHECKING(whether to enable inline optimization for GCC) AC_MSG_RESULT($ZEND_INLINE_OPTIMIZATION) +AC_MSG_CHECKING(whether to enable Zend memory cache) +AC_MSG_RESULT($ZEND_MEMORY_CACHE) + AC_MSG_CHECKING(whether to enable a memory limit) AC_MSG_RESULT($ZEND_MEMORY_LIMIT) @@ -172,6 +182,12 @@ LIBZEND_CPLUSPLUS_CHECKS fi +if test "$ZEND_MEMORY_CACHE" = "yes"; then + AC_DEFINE(ZEND_DISABLE_MEMORY_CACHE, 0, [Memory cache]) +else + AC_DEFINE(ZEND_DISABLE_MEMORY_CACHE, 1, [Memory cache]) +fi + if test "$ZEND_MEMORY_LIMIT" = "yes"; then AC_DEFINE(MEMORY_LIMIT, 1, [Memory limit]) else diff -ruN php5-200309020130.orig/Zend/zend_alloc.c php5-200309020130/Zend/zend_alloc.c --- php5-200309020130.orig/Zend/zend_alloc.c 2003-08-28 17:06:54.000000000 +0000 +++ php5-200309020130/Zend/zend_alloc.c 2003-09-02 09:28:46.000000000 +0000 @@ -38,12 +38,6 @@ #ifdef ZEND_MM -#define ZEND_DISABLE_MEMORY_CACHE 0 -#else -#define ZEND_DISABLE_MEMORY_CACHE 0 -#endif - -#ifdef ZEND_MM #define ZEND_DO_MALLOC(size) zend_mm_alloc(&AG(mm_heap), size) #define ZEND_DO_FREE(ptr) zend_mm_free(&AG(mm_heap), ptr) #define ZEND_DO_REALLOC(ptr, size) zend_mm_realloc(&AG(mm_heap), ptr, size)

Zeev Suraski

22 years ago
At 12:46 02/09/2003, Vesselin Atanasov wrote:
>Hello. >The memory cache code in zend_alloc.c has serious problems with memory >fragmentation. >When a big script is run which allocates both lots of small objects and >a few big objects, the memory cache grows until the allowed limits >(11 buckets x 256 blocks each) keeping all of the cached memory occupied >although >the script uses much less memory. This is caused by the cache code which >caches small >objects and does not allow the low level memory manager (either the >zend_mm.c or the >C runtime malloc) to merge small free blocks into bigger free blocks. From >the point >of view of the low level memory manager there are lots of occupied memory >blocks >interspersed among the allocated memory segment. the occupied memory >blocks are >actually cached by the high level cache code. So when the script is run a >new segment >is allocated although there is free memory. With each call new segment(s) >are allocated >until the memory cache is full. This scenario happens when a script first >creates lots >of small objects and then a few large objects. Even if the small objects >are freed, >the memory is fragmented and a new block is occupied for the big >object(s). So with >each request the lost memory increases, effectively this is like a memory >leak. >I attach a patch which adds --disable-memory-cache option which disables >the memory >cache.
Vesselin, The memory fragmentation issues were reported by Sterling a few weeks ago - we know they're there. The simplest solution appears to be decreasing the zend_mm block size to 16KB instead of 256KB. Due to bugs elsewhere (libxml2), making this change appeared to cause problems, but with these bugs now fixed - it appears to work fine. After we perform some more benchmarks we'll commit this change. Disabling memory caching altogether is probably not a good idea since PHP allocates and frees small blocks VERY often. Zeev

Andi Gutmans

22 years ago
Let's hold on this patch until we finish benchmarking. We might just "do the right thing" automagically. I prefer the user not to have to be smart about such internals. Andi At 12:46 PM 9/2/2003 +0300, Vesselin Atanasov wrote: