Everything

php

Home » Project » Newsgroups » php.internals » zval "passthrough"? »

zval "passthrough"?

php.internals

David Olsson

20 years ago

Hello

I'm writing a PHP extension in C++ which need to provide a function
which will take as a parameter a zval *. This zval * should, when the
function has completed its task, simply be returned unmodified. How do I
accomplish this? I've tried something like this:

PHP_FUNCTION(my_func) {
zval *v;
if(zend_parse_parameters(ZEND_NUM_ARGS() TSRMLS_CC, "z", &v) == FAILURE) {
WRONG_PARAM_COUNT;
}

.
.
.

return_value = v;
}

The above doesn't seem to be working at all though. Any ideas? I guess
it is a really simple task once you known how do actually do it... :-)

Thankful for all help!
David Olsson

Sara Golemon

20 years ago

> I'm writing a PHP extension in C++ which need to provide a function which
> will take as a parameter a zval *. This zval * should, when the function
> has completed its task, simply be returned unmodified. How do I accomplish
> this? I've tried something like this:
>
> return_value = v;
>

There are two problems here:

#1 return_value is just a parameter in the prototype:

void zif_my_func(int ht, zval *return_value, zval **return_value_ptr, zval
*this_ptr, int return_value_used TSRMLS_DC)

So when you set:   return_value = v;   You are in fact changing where
return_value points, but you're not doing anything to the contents of the
return_value zval that the calling scope (the engine) knows about.  You're
just changing the local pointer to look at another address.

Using return_value_ptr you can both clean up the original memory allocated
for return_value, and change what the engine knows about return_value since
you have the pointer to the pointer which the engine uses:

/* Destroy old return_value (prevents leaks) */
zval_ptr_dtor(return_value_ptr);

/* Point return_value at the passed in val */
*return_value_ptr = v;

This brings us to the second problem:

#2 The refcount of v will have been temporarily increased by the engine when
it was placed on the parameter stack, however when the function exits, that
refcount will be decremented back down and there will be no refcount
associated with the copy stored in return_value.  Long story short you need
to add ref the value when you assign it to a new container:

ZVAL_ADDREF(v);
zval_ptr_dtor(return_value_ptr);
*return_value_ptr = v;


Now for problem number 3 (What, you thought I'd stop at two?)

If v was an is_ref value, with a refcount of only 2 at the time of leaving
your internal function, and you're using PHP 5.0, the function return will
break the reference set.  This is a known bug and is fixed in the 5.1
branch.

And problem #3a (PHP4 compatability)

PHP4 doesn't even have the return_value_ptr parameter so you can't even do
that stuff with it.  What you can do is perform a deep copy on the value:

*return_value = *v;
zval_copy_ctor(return_value);

While it's less memory/cpu efficient, it is usable across all versions.

-Sara

David Olsson

20 years ago

Thanks! That was very useful information. Had to go with the #3a
solution as I need to be compatible with PHP4.

I still have some problems with my extension though. The extension
maintains a module global C++ std::map allocated in my MINIT function
and deallocated in my MSHUTDOWN function. This map stores some zval *
added to the map like this:

int nameLen;
char *name;
zval *v;
zend_parse_parameters(ZEND_NUM_ARGS() TSRMLS_CC, "sz!", &v);
zval *newv = new zval[1];
*newv = *v;
zval_copy_ctor(newv);
(*reinterpret_cast<map<string, zval *> *>(PSV_G(psvs)))[name] = newv;

where PSV_G(psvs) is a pointer to the map. The problem is that the
content of the map may change or dissapear without my extension
accessing it at all. Does the Zend engine do anything with the memory I
have allocated? Currently i allocate memory for the map with new instead
of emalloc but should that make any difference (except for larger risks
for memory leaks)?

Thanks in advance!
David Olsson

Sara Golemon wrote:

Sara Golemon

20 years ago

> Thanks! That was very useful information. Had to go with the #3a solution
> as I need to be compatible with PHP4.
>
> I still have some problems with my extension though. The extension
> maintains a module global C++ std::map allocated in my MINIT function and
> deallocated in my MSHUTDOWN function. This map stores some zval * added to
> the map like this:
>

Normal userspace variables (zvals) are allocated "non-persistently".  This
means that, for example, creating a string variable consists of emalloc()
calls:

foo = emalloc(sizeof(zval));
foo->value.str.val = emalloc(sizeof("bar"));

emalloc()'d pointers are forcibly freed between requests (as part of Zend's
garbage collection), so even though you've taken care of all the other bits
or placing the zval* into your map in regards to reference counting and
creating a true copy, the zval itself is allocated non-persistently and will
be quietly killed by the engine at the end of your request. (Well, no so
quietly if you have --enable-debug turned on).

In order to hold a zval between requests you'll have to make a persistent
copy of it (using pemalloc() which, I'm afraid, is a bit more complicated
than it sounds).  Take a look at apc_store() and apc_fetch() which
do...well....pretty much what you're talking about here.

> Currently i allocate memory for the map with new instead of emalloc but
> should that make any difference (except for larger risks for memory
> leaks)?
>

new will end up using malloc() which is what a persistent allocation would
use anyway.  You actually *can't* use emalloc() in your MINIT code because
emalloc() is only to be used within a request (which you're not inside of at
that point).

-Sara

David Olsson

20 years ago

> Normal userspace variables (zvals) are allocated "non-persistently".
> This means that, for example, creating a string variable consists of
> emalloc() calls:
>
> foo = emalloc(sizeof(zval));
> foo->value.str.val = emalloc(sizeof("bar"));
>
> emalloc()'d pointers are forcibly freed between requests (as part of
> Zend's garbage collection), so even though you've taken care of all
> the other bits or placing the zval* into your map in regards to
> reference counting and creating a true copy, the zval itself is
> allocated non-persistently and will be quietly killed by the engine at
> the end of your request. (Well, no so quietly if you have
> --enable-debug turned on).

So that means that all the zval * I store in my map simply point to
invalid memory once the request is completed? Well, that would explain
some of the strange behaviour I have observed (such as string changing
content between requests). The solution to this would be to allocate the
zval struct using pemalloc and allocate memory for the data the zval
(whatever type it is) uses with pemalloc as well?

>
> In order to hold a zval between requests you'll have to make a
> persistent copy of it (using pemalloc() which, I'm afraid, is a bit
> more complicated than it sounds).  Take a look at apc_store() and
> apc_fetch() which do...well....pretty much what you're talking about
> here.
>

As far as I can tell, the APC extension doesn't use pemalloc but rather
malloc. Furthermore, pemalloc simply seems to be a macro which uses
malloc to allocate memory. This means that my memory allocations using
c++ new should allocate persistent memory chunks. So the
persistent/non-persistent memory issue doesn't explain why my std::map
simply is empty. The fact that it contain zval pointers pointing to
invalid memory I understand but I don't get why it gets emptied. Any ideas?

Best regards
David Olsson

Ondrej Ivanič

20 years ago

David Olsson wrote:

> malloc to allocate memory. This means that my memory allocations using
> c++ new should allocate persistent memory chunks. So the
> persistent/non-persistent memory issue doesn't explain why my std::map
> simply is empty. The fact that it contain zval pointers pointing to
> invalid memory I understand but I don't get why it gets emptied. Any ideas?

Overload new and delete operators?

--
Ondrej Ivanic
(ondrej@kmit.sk)

Sara Golemon

20 years ago

> So that means that all the zval * I store in my map simply point to
> invalid memory once the request is completed? Well, that would explain
> some of the strange behaviour I have observed (such as string changing
> content between requests).
>

Yep.

> The solution to this would be to allocate the
> zval struct using pemalloc and allocate memory for the data the zval
> (whatever type it is) uses with pemalloc as well?
>

When you get the zval* in your strore_var_in_map() function it's
non-persistent.  You need to copy all of its data (and dependent data) into
persistent storage when you store it in your map.   Then when you take it
out of map to return it back to userspace (possibly in another request), you
need to copy it again to place it in non-persistent storage.

> As far as I can tell, the APC extension doesn't use pemalloc but rather
> malloc. Furthermore, pemalloc simply seems to be a macro which uses
> malloc to allocate memory. This means that my memory allocations using
> c++ new should allocate persistent memory chunks.
>

That's correct.  It's not allocating the persistent memory chunks that's
hard, it's duplicating a generic zval (in all the various forms that a zval
can take) that's tricky (particularly in the case of IS_OBJECT and
IS_RESOURCE).

> S the persistent/non-persistent memory issue doesn't explain why my
> std::map
> simply is empty. The fact that it contain zval pointers pointing to
> invalid memory I understand but I don't get why it gets emptied. Any
> ideas?
>

Post the code you have so far.  I'm working with a pretty limited pinhole
view here....

-Sara

Marcus Börger

20 years ago

Hello David,

Thursday, October 6, 2005, 9:49:35 AM, you wrote:

> Thanks! That was very useful information. Had to go with the #3a
> solution as I need to be compatible with PHP4.

> I still have some problems with my extension though. The extension
> maintains a module global C++ std::map allocated in my MINIT function
> and deallocated in my MSHUTDOWN function. This map stores some zval *
> added to the map like this:

>         int nameLen;
>         char *name;
>         zval *v;
>         zend_parse_parameters(ZEND_NUM_ARGS() TSRMLS_CC, "sz!", &v);
>         zval *newv = new zval[1];
>         *newv = *v;
>         zval_copy_ctor(newv);
>         (*reinterpret_cast<map<string, zval *> *>(PSV_G(psvs)))[name] = newv;

do you have any idea what a reinterpret_cast<>() does? So probably change
your code so that you are actually use what you think you do.

apart from that you should consider storing the zval's rather than pointers.

> where PSV_G(psvs) is a pointer to the map. The problem is that the
> content of the map may change or dissapear without my extension
> accessing it at all. Does the Zend engine do anything with the memory I
> have allocated? Currently i allocate memory for the map with new instead
> of emalloc but should that make any difference (except for larger risks
> for memory leaks)?

> Thanks in advance!
> David Olsson

> Sara Golemon wrote:
>>> I'm writing a PHP extension in C++ which need to provide a function
>>> which will take as a parameter a zval *. This zval * should, when the
>>> function has completed its task, simply be returned unmodified. How do
>>> I accomplish this? I've tried something like this:
>>>
>>> return_value = v;
>>>
>> There are two problems here:
>>
>> #1 return_value is just a parameter in the prototype:
>>
>> void zif_my_func(int ht, zval *return_value, zval **return_value_ptr,
>> zval *this_ptr, int return_value_used TSRMLS_DC)
>>
>> So when you set:   return_value = v;   You are in fact changing where
>> return_value points, but you're not doing anything to the contents of
>> the return_value zval that the calling scope (the engine) knows about.
>> You're just changing the local pointer to look at another address.
>>
>> Using return_value_ptr you can both clean up the original memory
>> allocated for return_value, and change what the engine knows about
>> return_value since you have the pointer to the pointer which the engine
>> uses:
>>
>> /* Destroy old return_value (prevents leaks) */
>> zval_ptr_dtor(return_value_ptr);
>>
>> /* Point return_value at the passed in val */
>> *return_value_ptr = v;
>>
>> This brings us to the second problem:
>>
>> #2 The refcount of v will have been temporarily increased by the engine
>> when it was placed on the parameter stack, however when the function
>> exits, that refcount will be decremented back down and there will be no
>> refcount associated with the copy stored in return_value.  Long story
>> short you need to add ref the value when you assign it to a new container:
>>
>> ZVAL_ADDREF(v);
>> zval_ptr_dtor(return_value_ptr);
>> *return_value_ptr = v;
>>
>>
>> Now for problem number 3 (What, you thought I'd stop at two?)
>>
>> If v was an is_ref value, with a refcount of only 2 at the time of
>> leaving your internal function, and you're using PHP 5.0, the function
>> return will break the reference set.  This is a known bug and is fixed
>> in the 5.1 branch.
>>
>> And problem #3a (PHP4 compatability)
>>
>> PHP4 doesn't even have the return_value_ptr parameter so you can't even
>> do that stuff with it.  What you can do is perform a deep copy on the
>> value:
>>
>> *return_value = *v;
>> zval_copy_ctor(return_value);
>>
>> While it's less memory/cpu efficient, it is usable across all versions.
>>
>> -Sara

Best regards,
Marcus