TSRM changes broke windows compile

php.internals

Stanislav Malyshev

19 years ago
Recent TSRM changes seem to break the windows compilation for me. I'm getting: tsrm_virtual_cwd.c C:\Program Files\Microsoft Visual Studio 8\VC\PlatformSDK\include\winsock2.h(112 ) : error C2011: 'fd_set' : 'struct' type redefinition C:\Program Files\Microsoft Visual Studio 8\VC\PlatformSDK\include\winsoc k.h(54) : see declaration of 'fd_set' C:\Program Files\Microsoft Visual Studio 8\VC\PlatformSDK\include\winsock2.h(147 ) : warning C4005: 'FD_SET' : macro redefinition and so on many times. PHP compiled just fine for a long time with the same setup for me, broke just very recently. Any ideas what happened?
-- Stanislav Malyshev, Zend Products Engineer stas@zend.com http://www.zend.com/

Stanislav Malyshev

19 years ago
I tried removing recently added include to SAPI.h and it compiles just fine now. In general, I think linking TSRM to SAPI is not a good idea. It means, among other things, that TSRM can never be used in a context that not links it to PHP. This is something that wasn't before and I don't think that introducing such change in a minore bugfix version worth it, yet more - for a miniscule performance optimization. Stanislav Malyshev wrote:
> Recent TSRM changes seem to break the windows compilation for me. I'm > getting: > > tsrm_virtual_cwd.c > C:\Program Files\Microsoft Visual Studio > 8\VC\PlatformSDK\include\winsock2.h(112 > ) : error C2011: 'fd_set' : 'struct' type redefinition > C:\Program Files\Microsoft Visual Studio > 8\VC\PlatformSDK\include\winsoc > k.h(54) : see declaration of 'fd_set' > C:\Program Files\Microsoft Visual Studio > 8\VC\PlatformSDK\include\winsock2.h(147 > ) : warning C4005: 'FD_SET' : macro redefinition > > and so on many times. PHP compiled just fine for a long time with the > same setup for me, broke just very recently. Any ideas what happened? >
-- Stanislav Malyshev, Zend Products Engineer stas@zend.com http://www.zend.com/

Rasmus Lerdorf

19 years ago
Stanislav Malyshev wrote:
> I tried removing recently added include to SAPI.h and it compiles just > fine now. In general, I think linking TSRM to SAPI is not a good idea. > It means, among other things, that TSRM can never be used in a context > that not links it to PHP. This is something that wasn't before and I > don't think that introducing such change in a minore bugfix version > worth it, yet more - for a miniscule performance optimization.
An extra syscall on every file open isn't exactly miniscule. Edin also just built Windows binaries without problems. Why did it work for him? -Rasmus

Stanislav Malyshev

19 years ago
> An extra syscall on every file open isn't exactly miniscule.
It's a syscall not related to any filesystem or I/O so it can't be that bad. And we managed to live with it so far.
> Edin also just built Windows binaries without problems. Why did it work > for him?
I have no idea. But it is obvious that it doesn't work in all environments, and it worked before just fine. Anyway, both breaking build _and_ intoducing dependency of TSRM on PHP and both in a minor version _and_ both without discussing it doesn't look like very good idea to me. I would propose: 1. Revert it for 5.2.3 2. Discuss if we want TSRM/Zend be now usable only with PHP 3. Discuss if we really need this patch and if so can we do it without breaking (2)
-- Stanislav Malyshev, Zend Products Engineer stas@zend.com http://www.zend.com/

Uwe Schindler

19 years ago
> > An extra syscall on every file open isn't exactly miniscule. > > It's a syscall not related to any filesystem or I/O so it can't be that > bad. And we managed to live with it so far. > > > Edin also just built Windows binaries without problems. Why did it work > > for him? > > I have no idea. But it is obvious that it doesn't work in all > environments, and it worked before just fine. Anyway, both breaking > build _and_ intoducing dependency of TSRM on PHP and both in a minor > version _and_ both without discussing it doesn't look like very good > idea to me. I would propose: > 1. Revert it for 5.2.3 > 2. Discuss if we want TSRM/Zend be now usable only with PHP > 3. Discuss if we really need this patch and if so can we do it without > breaking (2)
As I was involved today this morning also with startup failures because of this change, I came to the following: It would be perhaps OK to make TSRM dependent on some global PHP functions, but to make it depend on SAPI code that is only for webserver interaction (!) and not related to TSRM (yes SAPI uses TSRM but not the other way round) is not so good. And that only because of a not IO-related syscall! A solution would be to make a function pointer inside TSRM that normally maps to time() or a TSRM-local wrapper of it. This would make TSRM not dependent on PHP. On the PHP part in SAPI startup there could optionally be a redirect of the pointer to SAPI's get_request_time() implementation. Uwe

Rasmus Lerdorf

19 years ago
Uwe Schindler wrote:
>>> An extra syscall on every file open isn't exactly miniscule. >> It's a syscall not related to any filesystem or I/O so it can't be that >> bad. And we managed to live with it so far. >> >>> Edin also just built Windows binaries without problems. Why did it work >>> for him? >> I have no idea. But it is obvious that it doesn't work in all >> environments, and it worked before just fine. Anyway, both breaking >> build _and_ intoducing dependency of TSRM on PHP and both in a minor >> version _and_ both without discussing it doesn't look like very good >> idea to me. I would propose: >> 1. Revert it for 5.2.3 >> 2. Discuss if we want TSRM/Zend be now usable only with PHP >> 3. Discuss if we really need this patch and if so can we do it without >> breaking (2) > > As I was involved today this morning also with startup failures because of > this change, I came to the following: > > It would be perhaps OK to make TSRM dependent on some global PHP functions, > but to make it depend on SAPI code that is only for webserver interaction > (!) and not related to TSRM (yes SAPI uses TSRM but not the other way round) > is not so good. And that only because of a not IO-related syscall!
It may not be IO, but on some systems fetching the time is actually quite expensive. And SAPI isn't only for web server interaction. SAPI is a general-purpose abstraction API that sits between PHP and anything PHP talks to. CGI, CLI, and even embedded PHP are all SAPI-driven. There is no way to use PHP without at least a stub SAPI layer.
> A solution would be to make a function pointer inside TSRM that normally > maps to time() or a TSRM-local wrapper of it. This would make TSRM not > dependent on PHP. On the PHP part in SAPI startup there could optionally be > a redirect of the pointer to SAPI's get_request_time() implementation.
That could be done, yes. I'm just not sure it is worth the complexity. TSRM is obviously a PHP-specific thing and it is already somewhat SAPI-dependant even without this. The THREAD_T and MUTEX_T definitions change based on the chosen SAPI. It doesn't explicitly include SAPI.h because it relies on the side-effect of TSRM.h being included after the SAPI stuff. I don't see an explicit include as really changing things very much. -Rasmus

Stanislav Malyshev

19 years ago
> quite expensive. And SAPI isn't only for web server interaction. SAPI > is a general-purpose abstraction API that sits between PHP and anything > PHP talks to. CGI, CLI, and even embedded PHP are all SAPI-driven. > There is no way to use PHP without at least a stub SAPI layer.
There is, however, a way to use Zend Engine/TSRM. With this change, Zend Engine/TSRM could no longer be used without PHP. And all this for some time() patch...
> That could be done, yes. I'm just not sure it is worth the complexity.
I, for one, don't really understand what all this filesystem stuff does in TSRM anyway. It's not like it has anything to do with thread safety. Maybe it should just be moved to PHP and Zend Engine should talk to it via handlers?
> TSRM is obviously a PHP-specific thing and it is already somewhat > SAPI-dependant even without this. The THREAD_T and MUTEX_T definitions > change based on the chosen SAPI. It doesn't explicitly include SAPI.h
No, it's not SAPI-dependent. It's environment-dependent, which is entirely different thing. It can be configured, built and used without PHP.
> because it relies on the side-effect of TSRM.h being included after the > SAPI stuff. I don't see an explicit include as really changing things > very much.
It does. Now it's impossible to use TSRM without PHP and it also doesn't build on Windows.
-- Stanislav Malyshev, Zend Products Engineer stas@zend.com http://www.zend.com/

Rasmus Lerdorf

19 years ago
Stanislav Malyshev wrote:
>> quite expensive. And SAPI isn't only for web server interaction. SAPI >> is a general-purpose abstraction API that sits between PHP and anything >> PHP talks to. CGI, CLI, and even embedded PHP are all SAPI-driven. >> There is no way to use PHP without at least a stub SAPI layer. > > There is, however, a way to use Zend Engine/TSRM. With this change, Zend > Engine/TSRM could no longer be used without PHP. And all this for some > time() patch... > >> That could be done, yes. I'm just not sure it is worth the complexity. > > I, for one, don't really understand what all this filesystem stuff does > in TSRM anyway. It's not like it has anything to do with thread safety. > Maybe it should just be moved to PHP and Zend Engine should talk to it > via handlers? > >> TSRM is obviously a PHP-specific thing and it is already somewhat >> SAPI-dependant even without this. The THREAD_T and MUTEX_T definitions >> change based on the chosen SAPI. It doesn't explicitly include SAPI.h > > No, it's not SAPI-dependent. It's environment-dependent, which is > entirely different thing. It can be configured, built and used without PHP.
Well, by that logic you would be fine with: #define TIME_CALL sapi_get_request_time(TSRMLS_C) and in tsrm_virtual_cwd.c #ifndef TIME_CALL define TIME_CALL time(0) #endif ... t = CWDG(realpath_cache_ttl)?TIME_CALL:0; that makes it environment-dependent instead of explicitly-dependent, and also way harder to debug. Environmental side-effects like that are nasty. -Rasmus

Stanislav Malyshev

19 years ago
> Well, by that logic you would be fine with: > > #define TIME_CALL sapi_get_request_time(TSRMLS_C) > > and in tsrm_virtual_cwd.c > > #ifndef TIME_CALL > define TIME_CALL time(0) > #endif > ... > t = CWDG(realpath_cache_ttl)?TIME_CALL:0;
Theoretically yes, that could work, but:
> > that makes it environment-dependent instead of explicitly-dependent, and > also way harder to debug. > > Environmental side-effects like that are nasty.
Exactly. So I'd propose to think how to do it right - not excluding taking the whole cache & system-dependant filesystem things out of TSRM. They do not really related to the rest of TSRM and could be abstracted same way as open/output/error functions are abstracted.
-- Stanislav Malyshev, Zend Products Engineer stas@zend.com http://www.zend.com/

Rasmus Lerdorf

19 years ago
Stanislav Malyshev wrote:
>> Well, by that logic you would be fine with: >> >> #define TIME_CALL sapi_get_request_time(TSRMLS_C) >> >> and in tsrm_virtual_cwd.c >> >> #ifndef TIME_CALL >> define TIME_CALL time(0) >> #endif >> ... >> t = CWDG(realpath_cache_ttl)?TIME_CALL:0; > > Theoretically yes, that could work, but: > >> >> that makes it environment-dependent instead of explicitly-dependent, and >> also way harder to debug. >> >> Environmental side-effects like that are nasty. > > Exactly. So I'd propose to think how to do it right - not excluding > taking the whole cache & system-dependant filesystem things out of TSRM. > They do not really related to the rest of TSRM and could be abstracted > same way as open/output/error functions are abstracted.
But that doesn't fix the broken side-effects we already have in the locking code depending on defines in the individuals SAPIs which Uwe showed isn't working either. And if we exclude locking from TSRM as well, what is left? We either need to tie TSRM to SAPI or we need to add the ability for the SAPIs to register their various semantics with TSRM. The filesystem code is in TSRM because of the virtual working directory stuff. That is thread-related in that each thread needs it own working directory. -Rasmus

Uwe Schindler

19 years ago
> > It would be perhaps OK to make TSRM dependent on some global PHP > functions, > > but to make it depend on SAPI code that is only for webserver > interaction > > (!) and not related to TSRM (yes SAPI uses TSRM but not the other way > round) > > is not so good. And that only because of a not IO-related syscall! > > It may not be IO, but on some systems fetching the time is actually > quite expensive. And SAPI isn't only for web server interaction. SAPI > is a general-purpose abstraction API that sits between PHP and anything > PHP talks to. CGI, CLI, and even embedded PHP are all SAPI-driven. > There is no way to use PHP without at least a stub SAPI layer.
I know, I wanted only to put more pressure on that.
> > A solution would be to make a function pointer inside TSRM that normally > > maps to time() or a TSRM-local wrapper of it. This would make TSRM not > > dependent on PHP. On the PHP part in SAPI startup there could optionally > be > > a redirect of the pointer to SAPI's get_request_time() implementation. > > That could be done, yes. I'm just not sure it is worth the complexity. > TSRM is obviously a PHP-specific thing and it is already somewhat > SAPI-dependant even without this. The THREAD_T and MUTEX_T definitions > change based on the chosen SAPI.
This is no longer the case. E.g. when compiling the module for the NSAPI webserver (Sun Java System Webserver) there should be a global define like -DNSAPI in the makefiles (not only for nsapi.c) which is not. Because of that all SAPIs do not have an effect on the thread implementation. When compiling PHP with ZTS you always get an .o file that links to pthreads. This must be the case, because without that it would not be possible to link a CLI PHP in parallel to the webserver specific SAPI. Because of that limitation for example the Sun Webservers on (SJSWS) on AIX crash on startup (there are a lot of bug reports about that, but I cannot help because I have no access to such a system). I only know that SJSWS is implemented on AIX not with POSIX threads. And on Solaris the webservers also crash when you switch to an alternative "kernel" (do not know the exact name) thread implementation in the webserver. This is because the ifdefs in TSRM.h never apply specific to the SAPI implementation.
> It doesn't explicitly include SAPI.h > because it relies on the side-effect of TSRM.h being included after the > SAPI stuff. I don't see an explicit include as really changing things > very much.
It was just a suggestion. But it should not hard to implement. Uwe

Rasmus Lerdorf

19 years ago
Uwe Schindler wrote:
>>> It would be perhaps OK to make TSRM dependent on some global PHP >> functions, >>> but to make it depend on SAPI code that is only for webserver >> interaction >>> (!) and not related to TSRM (yes SAPI uses TSRM but not the other way >> round) >>> is not so good. And that only because of a not IO-related syscall! >> It may not be IO, but on some systems fetching the time is actually >> quite expensive. And SAPI isn't only for web server interaction. SAPI >> is a general-purpose abstraction API that sits between PHP and anything >> PHP talks to. CGI, CLI, and even embedded PHP are all SAPI-driven. >> There is no way to use PHP without at least a stub SAPI layer. > > I know, I wanted only to put more pressure on that. > >>> A solution would be to make a function pointer inside TSRM that normally >>> maps to time() or a TSRM-local wrapper of it. This would make TSRM not >>> dependent on PHP. On the PHP part in SAPI startup there could optionally >> be >>> a redirect of the pointer to SAPI's get_request_time() implementation. >> That could be done, yes. I'm just not sure it is worth the complexity. >> TSRM is obviously a PHP-specific thing and it is already somewhat >> SAPI-dependant even without this. The THREAD_T and MUTEX_T definitions >> change based on the chosen SAPI. > > This is no longer the case. E.g. when compiling the module for the NSAPI > webserver (Sun Java System Webserver) there should be a global define like > -DNSAPI in the makefiles (not only for nsapi.c) which is not. Because of > that all SAPIs do not have an effect on the thread implementation. When > compiling PHP with ZTS you always get an .o file that links to pthreads. > This must be the case, because without that it would not be possible to link > a CLI PHP in parallel to the webserver specific SAPI.
That's simply not the case. Look at the nsapi code: #define NSAPI 1 #ifdef HAVE_CONFIG_H #include "config.h" #endif #include "php.h" #include "php_variables.h" #include "ext/standard/info.h" #include "php_ini.h" #include "php_globals.h" #include "SAPI.h" #include "php_main.h" #include "php_version.h" #include "TSRM.h" #include "ext/standard/php_standard.h" #include <sys/types.h> #include <sys/stat.h> It defines NSAPI before including TSRM.h and in TSRM.h we have: #elif defined(NSAPI) # define THREAD_T SYS_THREAD # define MUTEX_T CRITICAL So I don't see how this is no longer the case. -Rasmus

Uwe Schindler

19 years ago
Hi Rasmus,
> > This is no longer the case. E.g. when compiling the module for the NSAPI > > webserver (Sun Java System Webserver) there should be a global define > like > > -DNSAPI in the makefiles (not only for nsapi.c) which is not. Because of > > that all SAPIs do not have an effect on the thread implementation. When > > compiling PHP with ZTS you always get an .o file that links to pthreads. > > This must be the case, because without that it would not be possible to > link > > a CLI PHP in parallel to the webserver specific SAPI. > > That's simply not the case. Look at the nsapi code: > > #define NSAPI 1 > > #ifdef HAVE_CONFIG_H > #include "config.h" > #endif > > #include "php.h" > #include "php_variables.h" > #include "ext/standard/info.h" > #include "php_ini.h" > #include "php_globals.h" > #include "SAPI.h" > #include "php_main.h" > #include "php_version.h" > #include "TSRM.h" > #include "ext/standard/php_standard.h" > #include <sys/types.h> > #include <sys/stat.h> > > It defines NSAPI before including TSRM.h and in TSRM.h we have: > > #elif defined(NSAPI) > # define THREAD_T SYS_THREAD > # define MUTEX_T CRITICAL
Butt his is only for nsapi.c. All other PHP modules and the core itself do not know anything about NSAPI. This leads to the fact that if the webservers thread implementation is incompatible to the system default the build .o of nsapi.c is not compatible to the other .o files. This happens here: When compiling TSRM.c with the lines in it: TSRM_API THREAD_T tsrm_thread_id(void) { #ifdef TSRM_WIN32 return GetCurrentThreadId(); #elif defined(GNUPTH) return pth_self(); #elif defined(PTHREADS) return pthread_self(); #elif defined(NSAPI) return systhread_current(); #elif defined(PI3WEB) return PIThread_getCurrent(); #elif defined(TSRM_ST) return st_thread_self(); #elif defined(BETHREADS) return find_thread(NULL); #endif } NSAPI is *not* defined. This is not a problem because the systhread_current()-NSAPI function internally calls pthread_self() but only if the webserver runs in pthreads. On AIX (or was it HP-UX??? I should look into my bug reports) the webserver runs in another thread implementation. This leads to crashes in the webserver because PHP uses pthreads (because of TSRM.c) and all the webserver another one. In the worst case #define THREAD_T SYS_THREAD is also incompatible in size and structure and misbehaves between SAPI code (with define) and PHP (without define). Uwe

Rasmus Lerdorf

19 years ago
Uwe Schindler wrote:
> Hi Rasmus, > >>> This is no longer the case. E.g. when compiling the module for the NSAPI >>> webserver (Sun Java System Webserver) there should be a global define >> like >>> -DNSAPI in the makefiles (not only for nsapi.c) which is not. Because of >>> that all SAPIs do not have an effect on the thread implementation. When >>> compiling PHP with ZTS you always get an .o file that links to pthreads. >>> This must be the case, because without that it would not be possible to >> link >>> a CLI PHP in parallel to the webserver specific SAPI. >> That's simply not the case. Look at the nsapi code: >> >> #define NSAPI 1 >> >> #ifdef HAVE_CONFIG_H >> #include "config.h" >> #endif >> >> #include "php.h" >> #include "php_variables.h" >> #include "ext/standard/info.h" >> #include "php_ini.h" >> #include "php_globals.h" >> #include "SAPI.h" >> #include "php_main.h" >> #include "php_version.h" >> #include "TSRM.h" >> #include "ext/standard/php_standard.h" >> #include <sys/types.h> >> #include <sys/stat.h> >> >> It defines NSAPI before including TSRM.h and in TSRM.h we have: >> >> #elif defined(NSAPI) >> # define THREAD_T SYS_THREAD >> # define MUTEX_T CRITICAL > > Butt his is only for nsapi.c. All other PHP modules and the core itself do > not know anything about NSAPI. This leads to the fact that if the webservers > thread implementation is incompatible to the system default the build .o of > nsapi.c is not compatible to the other .o files. > > This happens here: > When compiling TSRM.c with the lines in it: > > TSRM_API THREAD_T tsrm_thread_id(void) > { > #ifdef TSRM_WIN32 > return GetCurrentThreadId(); > #elif defined(GNUPTH) > return pth_self(); > #elif defined(PTHREADS) > return pthread_self(); > #elif defined(NSAPI) > return systhread_current(); > #elif defined(PI3WEB) > return PIThread_getCurrent(); > #elif defined(TSRM_ST) > return st_thread_self(); > #elif defined(BETHREADS) > return find_thread(NULL); > #endif > } > > NSAPI is *not* defined. This is not a problem because the > systhread_current()-NSAPI function internally calls pthread_self() but only > if the webserver runs in pthreads. > On AIX (or was it HP-UX??? I should look into my bug reports) the webserver > runs in another thread implementation. This leads to crashes in the > webserver because PHP uses pthreads (because of TSRM.c) and all the > webserver another one. In the worst case > #define THREAD_T SYS_THREAD > is also incompatible in size and structure and misbehaves between SAPI code > (with define) and PHP (without define).
Right I see what you mean. But to me this only illustrates why environmental side-effects like this are an incredibly bad idea. They tend to be forgotten, which appears to be the case here, and they are hard to debug. If the SAPI affects how TSRM does things, this should be an explicit dependency either by making TSRM always include the SAPI environment so no matter where TSRM is called from it behaves the same, or we move to a system where on sapi startup we register function pointers with TSRM both for the locking. Either solution will solve the time() thing as well. I don't mind removing the time() call for this release, but I think we clearly need to fix this loose broken coupling we have now. -Rasmus