issues with gettext on multithreaded servers

php.internals

Nathan Fredrickson

23 years ago
Hi all, I have found some problems (and possible solutions) concerning use of the gettext extension on multithreaded servers. Usually gettext determines the locale to use for translations from one of the following environment variables: LC_MESSAGES, LANG, LANGUAGE, LC_ALL. These environment variables are usually set with calls to setlocale(). In multithreaded server environments, such as IIS on win32, this setup works fine IF you only want to use a single locale across all php requests. All threads/request call setlocale() with the same locale. However a typical reason to internationalize a php web application is to allow users to choose their own locale. Currently gettext on a multithreaded server will not support this scenario. As each thread calls setlocale() they will be setting the current locale for ALL threads. A solution on win32 is to use the thread-aware NLS win32 API [1]. The current win32 build of gettext (libintl-1.dll) will call GetThreadLocale() to determine locale if none of the environment variables listed above are set. For this to work though, SetThreadLocale() must be called previously. My first thought was that the win32 implementation of php setlocale() should call SetThreadLocale() instead. The benefit of this option would be transparency to the user. But I don't like this option since it mixes the old NLS API with the new one. Also setlocale() by definition sets the environment variables, which will cause gettext to never call GetThreadLocale(). The alternative then is to call SetThreadLocale() through a new php extension. I think this should be seperate from the gettext extension since locales are used more widely than just gettext. My current idea is to create an extension named w32nls. This would be a win32 only extension that would provide access to a collection of the functions in the win32 NLS API [1] including Get/SetThreadLocale(). Other usful functions include replacements for php functions that are dependent on any of the LC_* environment variables. These functions would suffer from the same problems as gettext in a multithreaded environment where each thread sets different locales. For example the strftime() function relies on the LC_TIME environment variable. The alternative functions, GetDateFormat() and GetTimeFormat(), could be provided in the w32nls extension. The downside however will be increased incompatibility between win32 and unix based PHP applications if such an extension is ever widely used. I welcome any feedback on these ideas before I proceed. Thanks, Nathan [1] http://msdn.microsoft.com/library/default.asp?url=/library/en-us/intl/nls_52 lh.asp

Wez Furlong

23 years ago
Hi Nathan, In short, don't rely on the system locale support then :/ (You can't rely on it even between different versions of glibc, let alone difference flavours of UNIX). A win32nls extension is a good idea and a nice candidate for PECL. Perhaps it would be a good idea to develop a PEAR package that sits on top of the win32nls and/or UNIX nls functions to provide a portable API for PHP scripts? --Wez. On Fri, 21 Mar 2003, Nathan Fredrickson wrote:
> Hi all, > > I have found some problems (and possible solutions) concerning use of the > gettext extension on multithreaded servers.
...
> The alternative then is to call SetThreadLocale() through a new php > extension. I think this should be seperate from the gettext extension since > locales are used more widely than just gettext. My current idea is to > create an extension named w32nls. This would be a win32 only extension that > would provide access to a collection of the functions in the win32 NLS API > [1] including Get/SetThreadLocale().
...

David Sklar

23 years ago
On Friday, March 21, 2003 1:11 PM, Wez Furlong wrote:
> Perhaps it would be a good idea to develop a PEAR package that sits on > top of the win32nls and/or UNIX nls functions to provide a portable > API for PHP scripts?
This is a good idea. The extension can be told whether thread-specific settings or environment variables should take priority and then do the right think based on the OS. David

David Hill (php.net)

23 years ago
> I have found some problems (and possible solutions) concerning use
of the
> gettext extension on multithreaded servers.
The gettext library caches information in static variables..... This is true in gettext 0.10.40 at least, which is that last that I looked in. This has a number of "features" that are less than desirable with PHP. First - thread safety.... what thread saftey ? Second - as the locale is cached in static data when gettext is first initialized, I have yet to figure out how you could use this to support multiple locales in series (ie Apache 1.3) much less in parallel (Apache 2.0 with worker). I found that after changing the local variable, and then reopening the cat file that I was getting the old local, which is why I went looking into this. This of course was made more fun by that fact that I had 20 Apache procs running - each with a different cached locale as I was trying to figure out my semi random results. Maybe I missed something when I was looking into this (like some other way to call gettext). If so I would love to hear about it. dave