Everything
php
Learn Community Releases The PHP Project
Home » Project » Newsgroups » php.internals » bitwise operations and Unicode strings »

bitwise operations and Unicode strings

php.internals

Antony Dovgal

19 years ago
Hello. Do you think the engine should support bitwise operators and Unicode strings? If yes, how do you think it should work? Example: <?php $a = "1"; $a|="2"; var_dump($a); ?> This code outputs "3" in native mode and "Fatal error: Unsupported operand types" in Unicode mode. I believe this is an inconsistency and it should be possible to use Unicode strings there. There are several possible ways to implement it: 1) the same as with native strings - apply the operator to each element of the string separately; 2) convert the string to binary (using say iso-8859-1) and then see 1); We can also leave it as is (since it doesn't seem very useful) or even drop the native strings support (it doesn't seem very useful to me either). Opinions?
-- Wbr, Antony Dovgal

David Coallier

19 years ago
On 5/29/07, Antony Dovgal <antony@zend.com> wrote:
> Hello. > > Do you think the engine should support bitwise operators and Unicode strings? > If yes, how do you think it should work? > > Example: > <?php > $a = "1"; > $a|="2"; > var_dump($a); > ?> > > This code outputs "3" in native mode and "Fatal error: Unsupported operand types" in Unicode mode. > I believe this is an inconsistency and it should be possible to use Unicode strings there.
Definitely
> > There are several possible ways to implement it: > 1) the same as with native strings - apply the operator to each element of the string separately; > 2) convert the string to binary (using say iso-8859-1) and then see 1);
I would probably be for the solution #2 and of course using the unicode.runtime_encoding ini setting to convert it to binary.
> > We can also leave it as is (since it doesn't seem very useful) or even drop the native strings support (it doesn't seem very useful to me either).
I think this is a bad choice.. might not be useful to you, but might be to some other people
> Opinions? > > -- > Wbr, > Antony Dovgal > > -- > PHP Internals - PHP Runtime Development Mailing List > To unsubscribe, visit: http://www.php.net/unsub.php > >
-- David Coallier

Pierre Joye

19 years ago
Hi, On 5/29/07, David Coallier <davidc@php.net> wrote:
> On 5/29/07, Antony Dovgal <antony@zend.com> wrote: > > Hello. > > > > Do you think the engine should support bitwise operators and Unicode strings? > > If yes, how do you think it should work? > > > > Example: > > <?php > > $a = "1"; > > $a|="2"; > > var_dump($a); > > ?> > > > > This code outputs "3" in native mode and "Fatal error: Unsupported operand types" in Unicode mode. > > I believe this is an inconsistency and it should be possible to use Unicode strings there. > > Definitely > > > > > There are several possible ways to implement it: > > 1) the same as with native strings - apply the operator to each element of the string separately; > > 2) convert the string to binary (using say iso-8859-1) and then see 1); > > I would probably be for the solution #2 and of course using the > unicode.runtime_encoding ini setting to convert it to binary. > > > > > > We can also leave it as is (since it doesn't seem very useful) or even drop the native strings support (it doesn't seem very useful to me either). > > I think this is a bad choice.. might not be useful to you, but might > be to some other people
I fail to see or imagine any useful usages for such hacks. Laziness justifies it with binary strings (as in 5.x), but as long as an unicode string is given, one should really first cast to integer before using it with a bitwise operator (or any other non string operation). I'm in favour of keeping the current behaviors and drop the native support as well for consistency (and documentation/wtf headaches). --Pierre

Richard Quadling

19 years ago
As PHP is loosely typed, "1"==1 is fine. Is it that with Unicode, this looseness is gone? On 29/05/07, Pierre <pierre.php@gmail.com> wrote:
> Hi, > > On 5/29/07, David Coallier <davidc@php.net> wrote: > > On 5/29/07, Antony Dovgal <antony@zend.com> wrote: > > > Hello. > > > > > > Do you think the engine should support bitwise operators and Unicode strings? > > > If yes, how do you think it should work? > > > > > > Example: > > > <?php > > > $a = "1"; > > > $a|="2"; > > > var_dump($a); > > > ?> > > > > > > This code outputs "3" in native mode and "Fatal error: Unsupported operand types" in Unicode mode. > > > I believe this is an inconsistency and it should be possible to use Unicode strings there. > > > > Definitely > > > > > > > > There are several possible ways to implement it: > > > 1) the same as with native strings - apply the operator to each element of the string separately; > > > 2) convert the string to binary (using say iso-8859-1) and then see 1); > > > > I would probably be for the solution #2 and of course using the > > unicode.runtime_encoding ini setting to convert it to binary. > > > > > > > > > > We can also leave it as is (since it doesn't seem very useful) or even drop the native strings support (it doesn't seem very useful to me either). > > > > I think this is a bad choice.. might not be useful to you, but might > > be to some other people > > I fail to see or imagine any useful usages for such hacks. Laziness > justifies it with binary strings (as in 5.x), but as long as an > unicode string is given, one should really first cast to integer > before using it with a bitwise operator (or any other non string > operation). > > I'm in favour of keeping the current behaviors and drop the native > support as well for consistency (and documentation/wtf headaches). > > --Pierre > > -- > PHP Internals - PHP Runtime Development Mailing List > To unsubscribe, visit: http://www.php.net/unsub.php > >
-- ----- Richard Quadling Zend Certified Engineer : http://zend.com/zce.php?c=ZEND002498&r=213474731 "Standing on the shoulders of some very clever giants!"

Antony Dovgal

19 years ago
On 29.05.2007 19:30, Richard Quadling wrote:
> As PHP is loosely typed, "1"==1 is fine.
Right, but that's not true for bitwise operators. Try this: <?php $a = 111; $a |= 50; var_dump($a); ?> and this <?php $a = "111"; $a |= "50"; var_dump($a); ?> OR is applied to _characters_ of the string. Hence I can see no reason to support it at all.
> Is it that with Unicode, this looseness is gone?
Nope.
-- Wbr, Antony Dovgal

Marcus Börger

19 years ago
Hello Antony, so depending on the operator I either have no types or strict typing even though I cannot see/know what I have? That is far from KISS. best regards marcus Tuesday, May 29, 2007, 5:37:55 PM, you wrote:
> On 29.05.2007 19:30, Richard Quadling wrote: >> As PHP is loosely typed, "1"==1 is fine.
> Right, but that's not true for bitwise operators.
> Try this: > <?php > $a = 111; > $a |= 50; > var_dump($a);
?>>
> and this > <?php > $a = "111"; > $a |= "50"; > var_dump($a);
?>>
> OR is applied to _characters_ of the string. > Hence I can see no reason to support it at all.
>> Is it that with Unicode, this looseness is gone?
> Nope.
> -- > Wbr, > Antony Dovgal
Best regards, Marcus

Stanislav Malyshev

19 years ago
> so depending on the operator I either have no types or strict typing even > though I cannot see/know what I have? That is far from KISS.
Using bitwise operators on string is far from anything, so I see no real way to make it obvious, since the operation itself is very non-obvious.
-- Stanislav Malyshev, Zend Products Engineer stas@zend.com http://www.zend.com/

Antony Dovgal

19 years ago
On 29.05.2007 23:04, Marcus Boerger wrote:
> Hello Antony, > > so depending on the operator I either have no types or strict typing even > though I cannot see/know what I have? That is far from KISS.
You should have told that to the person who wrote this code, not me. And I believe you're at least 7 years late..
-- Wbr, Antony Dovgal

Richard Lynch

19 years ago
On Tue, May 29, 2007 4:25 am, Antony Dovgal wrote:
> Hello. > > Do you think the engine should support bitwise operators and Unicode > strings? > If yes, how do you think it should work? > > Example: > <?php > $a = "1"; > $a|="2"; > var_dump($a); > ?> > > This code outputs "3" in native mode and "Fatal error: Unsupported > operand types" in Unicode mode. > I believe this is an inconsistency and it should be possible to use > Unicode strings there.
Given that there are probably a bazillion PHP scripts written by newbies just like that, I'd have to say it's crucial for wide-spread Unicode adoption for it to "just work"
> There are several possible ways to implement it: > 1) the same as with native strings - apply the operator to each > element of the string separately; > 2) convert the string to binary (using say iso-8859-1) and then see > 1);
How do you type-juggle Unicode strings now when they are used as (int), regardless of the bit-wise operator or not? Seems to me that you'd want the exact same operations as: <?php $a = '1'; $a += '2'; var_dump($a); ?> Doesn't seem like the bitwise operator is relevant, really...
> We can also leave it as is (since it doesn't seem very useful) or even > drop the native strings support (it doesn't seem very useful to me > either). > Opinions?
Dropping support for the bazillion scripts that expected PHP to type-juggle '2' into 2 and do math with them is probably a Bad Idea... It's so bad, I must be missing something here, because I don't think you'd suggest it... :-) Ultimately, though, it seems like it should just do what PHP has always done with that code, because if it doesn't, here's what'll happen: ISP turns on Unicode support. Client scripts break. ISP reverts to PHP 5, or PHP 4 even, or turns off Unicode support. I am almost certain that I have code that does something not unlike: <?php $value = 0; //read checkboxes foreach($_GET['flag'] as $flag){ $value |= $flag; } //store $flag in DB as int ?> I'm not claiming it's the Best Code Ever, but it's not exactly Horrible either... Seems like with Unicode turned "on" I'd still expect this to "work" without throwing an (int) in there.
-- Some people have a "gift" link here. Know what I want? I want you to buy a CD from some indie artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So?

Antony Dovgal

19 years ago
On 30.05.2007 04:15, Richard Lynch wrote:
>> This code outputs "3" in native mode and "Fatal error: Unsupported >> operand types" in Unicode mode. >> I believe this is an inconsistency and it should be possible to use >> Unicode strings there. > > Given that there are probably a bazillion PHP scripts written by > newbies just like that, I'd have to say it's crucial for wide-spread > Unicode adoption for it to "just work"
No, I'd say there might be like ten scripts on the planet relying on bitwise operations with strings, since numeric strings behaviour is different here.
>> There are several possible ways to implement it: >> 1) the same as with native strings - apply the operator to each >> element of the string separately; >> 2) convert the string to binary (using say iso-8859-1) and then see >> 1); > > How do you type-juggle Unicode strings now when they are used as > (int), regardless of the bit-wise operator or not? > > Seems to me that you'd want the exact same operations as: > <?php > $a = '1'; > $a += '2'; > var_dump($a); > ?>
Well, it already works in a different way with native strings, so we can't change it.
> Doesn't seem like the bitwise operator is relevant, really... > >> We can also leave it as is (since it doesn't seem very useful) or even >> drop the native strings support (it doesn't seem very useful to me >> either). >> Opinions? > > Dropping support for the bazillion scripts that expected PHP to > type-juggle '2' into 2 and do math with them is probably a Bad Idea...
It doesn't cast '2' into 2 in this case. '22' is not 22, it's '2' (chr(50)) and '2' (chr(50)).
> It's so bad, I must be missing something here, because I don't think > you'd suggest it... :-) > > Ultimately, though, it seems like it should just do what PHP has > always done with that code, because if it doesn't, here's what'll > happen: > > ISP turns on Unicode support. > Client scripts break. > ISP reverts to PHP 5, or PHP 4 even, or turns off Unicode support. > > I am almost certain that I have code that does something not unlike: > > <?php > $value = 0; > //read checkboxes > foreach($_GET['flag'] as $flag){ > $value |= $flag; > } > //store $flag in DB as int > ?> > > I'm not claiming it's the Best Code Ever, but it's not exactly > Horrible either... > > Seems like with Unicode turned "on" I'd still expect this to "work" > without throwing an (int) in there. >
-- Wbr, Antony Dovgal

Jared Williams

19 years ago
> -----Original Message----- > From: Antony Dovgal [mailto:antony@zend.com] > Sent: 30 May 2007 09:21 > To: ceo@l-i-e.com > Cc: php-dev > Subject: Re: [PHP-DEV] bitwise operations and Unicode strings > > On 30.05.2007 04:15, Richard Lynch wrote: > >> This code outputs "3" in native mode and "Fatal error: Unsupported > >> operand types" in Unicode mode. > >> I believe this is an inconsistency and it should be > possible to use > >> Unicode strings there. > > > > Given that there are probably a bazillion PHP scripts written by > > newbies just like that, I'd have to say it's crucial for > wide-spread > > Unicode adoption for it to "just work" > > No, I'd say there might be like ten scripts on the planet > relying on bitwise operations with strings, since numeric > strings behaviour is different here.
I'd say more. Pear::Crypt_HMAC, for instance, does. /* Calculate the padded keys and save them */ $this->_ipad = (substr($key, 0, 64) ^ str_repeat(chr(0x36), 64)); $this->_opad = (substr($key, 0, 64) ^ str_repeat(chr(0x5C), 64)); Jared

Andi Gutmans

19 years ago
I'd go with (2) for BC sake and as breaking BC here would have very little value in the first place. Andi

Andrei Zmievski

19 years ago
I don't think we should support bitwise ops on Unicode strings. I am not even sure why we support them for normal strings. -Andrei On May 29, 2007, at 2:25 AM, Antony Dovgal wrote: