Everything

php

Native SSL support in Phar extension

php.internals

Peter Kokot

2 years ago

Hello,

There came up another idea/issue about the Phar extension and its
native SSL support.

As you might know or not, when building PHP:

./configure --with-openssl --enable-phar

the Phar extension will get so-called native SSL enabled through
OpenSSL directly. However, when built like this:

./configure --with-openssl=shared --enable-phar=shared
or
./configure --with-openssl=shared --enable-phar

the SSL support will depend on whether the ext/openssl is loaded and
enabled during PHP runtime. SSL support in both cases is done with
different pieces of code (one is using ext/openssl functions and the
other one - native SSL - is using OpenSSL calls directly). Also in
phpinfo output there is a bit of different info given based on the SSL
type.

On Windows there is also a slight inconsistency at the moment where
there is obsolete --enable-phar-native-ssl configure option available
which is using missing libeay32 library.

My first understanding was that native SSL is "better" as it uses
OpenSSL directly but it turned out that the code is outdated:
https://github.com/php/php-src/pull/14578

So, a new suggestion here is to remove the native SSL support in phar
extension in favor of using a PHP openssl extension's API (or better
call it function calls for the time being):
https://github.com/php/php-src/pull/15574

This would make the SSL support in phar more consistent because now
people aren't even aware of this. For example, on Alpine Linux default
packages there is openssl extension used in phar extension and on
Ubuntu's default packages there is this native SSL used.

Does anyone see any issues with removing the native SSL support in
Phar extension? Let me know. Ideally we would remove it in PHP 8.4
otherwise in the version after that...

Calvin Buckley

2 years ago

On Aug 26, 2024, at 1:23 PM, Peter Kokot <petk@php.net> wrote:

>
> Hello,
>
> There came up another idea/issue about the Phar extension and its
> native SSL support.
>
> As you might know or not, when building PHP:
>
> ./configure --with-openssl --enable-phar
>
> the Phar extension will get so-called native SSL enabled through
> OpenSSL directly. However, when built like this:
>
> ./configure --with-openssl=shared --enable-phar=shared
> or
> ./configure --with-openssl=shared --enable-phar
>
> the SSL support will depend on whether the ext/openssl is loaded and
> enabled during PHP runtime. SSL support in both cases is done with
> different pieces of code (one is using ext/openssl functions and the
> other one - native SSL - is using OpenSSL calls directly). Also in
> phpinfo output there is a bit of different info given based on the SSL
> type.
>
> On Windows there is also a slight inconsistency at the moment where
> there is obsolete --enable-phar-native-ssl configure option available
> which is using missing libeay32 library.
>
> My first understanding was that native SSL is "better" as it uses
> OpenSSL directly but it turned out that the code is outdated:
> https://github.com/php/php-src/pull/14578
>
> So, a new suggestion here is to remove the native SSL support in phar
> extension in favor of using a PHP openssl extension's API (or better
> call it function calls for the time being):
> https://github.com/php/php-src/pull/15574
>
> This would make the SSL support in phar more consistent because now
> people aren't even aware of this. For example, on Alpine Linux default
> packages there is openssl extension used in phar extension and on
> Ubuntu's default packages there is this native SSL used.
>
> Does anyone see any issues with removing the native SSL support in
> Phar extension? Let me know. Ideally we would remove it in PHP 8.4
> otherwise in the version after that...

Extensions referencing other extensions' functions (beyond things mediated
by the PHP runtime like streams) can be tricky depending on the dynamic
linking story.

Dynamic linkers that pull from a general symbol slurry (your typical ld.so
for instance) don't have a problem with it, but some platforms have more
strict linkage, like Windows. IIRC, things like PDO are built into the PHP
executable/library for this reason. I know AIX can actually do it both ways;
I used to build the ld.so style way (they call it "runtime linking"), but it
turns out that causes problems with different versions of symbols (i.e.
OpenSSL 1.1 and 3.0 in same address space), so I had to switch to building it
with the other dynamic linking option, which means I have to make the same
build compromises as on Windows.

As such, it might be a bit tricky for people on Windows/AIX; the easiest
solution if PHAR is using the openssl extension's symbols would be to not
build the openssl extension as shared.

(Sorry for the impromptu lesson on dynamic linking; it's complex, ugly, and I
still don't fully understand the vagaries myself. If you want more details, I
can bury my head in the documentation again. Korrine Adler has an decent
summary of the situation here [1].)

[1]: https://kadler.io/blog/2024-08-12-openssl3-migration/

Peter Kokot

2 years ago

On Mon, 26 Aug 2024 at 20:05, Calvin Buckley <calvin@cmpct.info> wrote:

> As such, it might be a bit tricky for people on Windows/AIX; the easiest
> solution if PHAR is using the openssl extension's symbols would be to not
> build the openssl extension as shared.

I've just checked Windows build and the PHP downloaded from PHP.net
and it seems to already use the openssl and phar extensions like this.
The ext/openssl is built as shared (the php_openssl.dll) and phar is
built into the php binary (which doesn't matter much here I think).
So I think this means one platform to worry about this issue here less.

About the AIX it's totally possible that this is still an issue. I'll
need to check it out in more detail at some point. If I understand the
issue mentioned in the blog post it is something else - two different
OpenSSL versions being linked. Which is of course bad. And this
removal is moving towards this direction also. OpenSSL being linked
only in ext/openssl. And in PHP different OpenSSL versions linked
could happen in a few other places as well - the curl, ftp and mysqlnd
extensions.

Well, this issue sounds complicated for PHP 8.4, so we can do it in
the next version then.

Christoph Becker

2 years ago

On 26.08.2024 at 20:05, Calvin Buckley wrote:

> Dynamic linkers that pull from a general symbol slurry (your typical ld.so
> for instance) don't have a problem with it, but some platforms have more
> strict linkage, like Windows. IIRC, things like PDO are built into the PHP
> executable/library for this reason. I know AIX can actually do it both ways;
> I used to build the ld.so style way (they call it "runtime linking"), but it
> turns out that causes problems with different versions of symbols (i.e.
> OpenSSL 1.1 and 3.0 in same address space), so I had to switch to building it
> with the other dynamic linking option, which means I have to make the same
> build compromises as on Windows.

Interesting!  Just a the other day I (re-?)learned that there are
usually no "import libraries" (Windows terminology) on Linux, which are
*very* common on Windows.  Such import libraries define stubs (i.e.
wrappers) for all exported symbols; for functions, basically (pseudo-code):

the_function(a, b, c) {
module = dl_load("library.dll")
proc = dl_fetch_symbol(module, "__imp_the_function")
return proc(a, b, c)
}

Might roughly be similar on AIX with "the other dynamic linking option".

> As such, it might be a bit tricky for people on Windows/AIX; the easiest
> solution if PHAR is using the openssl extension's symbols would be to not
> build the openssl extension as shared.

Well, that is not an option for any binaries we build and ship, since
the core is not supposed to rely on any external dependencies (that's
the reason why we still bundle libpcre2 although it is supposed to be
available almost everywhere).

Christoph

Calvin Buckley

2 years ago

On Aug 27, 2024, at 8:52 AM, Christoph M. Becker <cmbecker69@gmx.de> wrote:

>
> Interesting!  Just a the other day I (re-?)learned that there are
> usually no "import libraries" (Windows terminology) on Linux, which are
> *very* common on Windows.  Such import libraries define stubs (i.e.
> wrappers) for all exported symbols; for functions, basically (pseudo-code):
>
>  the_function(a, b, c) {
>      module = dl_load("library.dll")
>      proc = dl_fetch_symbol(module, "__imp_the_function")
>      return proc(a, b, c)
>  }
>
> Might roughly be similar on AIX with "the other dynamic linking option".

There isn't something like import libraries, but rather, the shared libs
are actually .a archive files that contain the .so to load at runtime
(of both bitnesses, as well as being able to do things like pack multiple
SOVERs), plus an import file that specifies the name of the library, the
symbols contained within, etc.

My impression is the runtime loader/linker option basically ignores all
of that infrastructure in favour of something that works like what
commodity Unix has, trading AIX's problems with those problems (i.e. lack
of symbol namespacing). Unfortunately, I'm in a situation where I do need
symbol namespacing, so I'm starting to deal with AIX's problems instead.
Unfortunately, I'm not David Edelsohn.

(I wish ELF-world would get symbol namespacing - it would make shipping
binaries far less spooky to debug. Windows gets this right.)

> Well, that is not an option for any binaries we build and ship, since
> the core is not supposed to rely on any external dependencies (that's
> the reason why we still bundle libpcre2 although it is supposed to be
> available almost everywhere).

I didn't know that; it's kinda confusing considering they're all in ext/,
and some vendor (i.e. gd, pcre), but others do not.

Christoph Becker

2 years ago

On 27.08.2024 at 16:42, Calvin Buckley wrote:

> On Aug 27, 2024, at 8:52 AM, Christoph M. Becker <cmbecker69@gmx.de> wrote:
>>
>> Might roughly be similar on AIX with "the other dynamic linking option".
>
> There isn't something like import libraries, but rather, the shared libs
> are actually .a archive files that contain the .so to load at runtime
> (of both bitnesses, as well as being able to do things like pack multiple
> SOVERs), plus an import file that specifies the name of the library, the
> symbols contained within, etc.

Ah, thanks!  Then this is completely different to Windows.

> My impression is the runtime loader/linker option basically ignores all
> of that infrastructure in favour of something that works like what
> commodity Unix has, trading AIX's problems with those problems (i.e. lack
> of symbol namespacing). Unfortunately, I'm in a situation where I do need
> symbol namespacing, so I'm starting to deal with AIX's problems instead.
> Unfortunately, I'm not David Edelsohn.

:)

>> Well, that is not an option for any binaries we build and ship, since
>> the core is not supposed to rely on any external dependencies (that's
>> the reason why we still bundle libpcre2 although it is supposed to be
>> available almost everywhere).
>
> I didn't know that; it's kinda confusing considering they're all in ext/,
> and some vendor (i.e. gd, pcre), but others do not.

It's only about the mandatory extensions, i.e. those you get with
./configure --disable-all (date, hash, json, pcre, random, reflection,
spl and standard).  The idea is to be able to have a minimal PHP build
without the need to install any libraries (except, of course, libc and
such).

ext/gd is a different thing; the bundled libgd is only there for BC
reasons (and should have been unbundled years ago, but hasn't due to
missing ZMM support in libgd); on most systems you usually build against
a system libgd, though.

Christoph