Does not work with non-ascii ouput

[bleargh45]

Migrated from rt.cpan.org #112446 (status was 'open')

Requestors:

• [email protected]

Attachments:

• unicode.patch

From [email protected] on 2016-02-26 08:05:24
:

i.e. when tests have non-ascii messages like:

ok 1, "п�иве�"; # Russian message

attaching patch (for version 0.11).

[bleargh45]

From [email protected] (@eserte) on 2016-02-28 18:50:13
:

On 2016-02-26 03:05:24, vsespb wrote:

i.e. when tests have non-ascii messages like:

ok 1, "п�иве�"; # Russian message

attaching patch (for version 0.11).

I wonder if a better solution would be to use an encoding layer on stdout.

Also, what if the current terminal is not a utf8 one? Probably one should look at the current locale to decide whether to output utf8 or something different.

Also this removal does not look right:

• $string =~ s/([\x7f-\xff])/'[\x'.sprintf('%02x',ord($1)).']'/ge;

There are control characters in the range \x80-\x9f which should be handled.

[bleargh45]

From [email protected] on 2016-02-28 19:02:27
:

I wonder if a better solution would be to use an encoding layer on stdout.

Okay. The bug persists (and fix fixes it) for any valid test, which prints UTF-8 to console.

i.e. for this too:

===
use utf8;
use Test::More;
use Test::Builder ();
binmode $, ':encoding(UTF-8)' for map { Test::Builder->new->$ } qw(output failure_output);
ok 1, "п�иве�"; # Russian message
done_testing;

Also, what if the current terminal is not a utf8 one?
Yes, I agree here. This only for UTF-8. (except, maybe, one can think that XML with UTF-8 header is not a text, but binary data, and should not be printed as text at all - i.e. should be printed to UTF8 console only).

There are control characters in the range \x80-\x9f which should be handled.
Indeed!

Ok, let's think of this patch as a proof-of bug and example of fix.

On Sun Feb 28 21:50:13 2016, SREZIC wrote:

On 2016-02-26 03:05:24, vsespb wrote:

i.e. when tests have non-ascii messages like:

ok 1, "п�иве�"; # Russian message

attaching patch (for version 0.11).

I wonder if a better solution would be to use an encoding layer on
stdout.

Also, what if the current terminal is not a utf8 one? Probably one
should look at the current locale to decide whether to output utf8 or
something different.

Also this removal does not look right:

• $string =~ s/([\x7f-\xff])/'[\x'.sprintf('%02x',ord($1)).']'/ge;

There are control characters in the range \x80-\x9f which should be
handled.

[bleargh45]

From [email protected] on 2016-02-28 19:11:11
:

I wonder if a better solution would be to use an encoding layer on stdout.

If you meant

use binmode STDOUT, ":utf8", instead of

"print { $self->stdout } Encode::encode("UTF-8", $self->xml->testsuites( @suites ));"

I think no, it's better when only main program (.pl script; not .pm) changes STDOUT layers, otherwise it's a mess.

And btw, even if we do (like in patch)

print { $self->stdout } Encode::encode("UTF-8", $self->xml->testsuites( @suites ));

this won't work if main program already did

===
"use binmode STDOUT, ":utf8""

If main program is "prove" it does not do that, so we assume no one do that. And we should not change layer, as it could break "prove".

[bleargh45]

From [email protected] (@eserte) on 2016-02-28 19:21:39
:

On 2016-02-28 14:02:27, vsespb wrote:

I wonder if a better solution would be to use an encoding layer on
stdout.

Okay. The bug persists (and fix fixes it) for any valid test, which
prints UTF-8 to console.

i.e. for this too:

===
use utf8;
use Test::More;
use Test::Builder ();
binmode $, ':encoding(UTF-8)' for map { Test::Builder->new->$ }
qw(output failure_output);
ok 1, "п�иве�"; # Russian message
done_testing;

Maybe this kind of problems will be addressed in the refactored Test::More/Test::Builder --- see https://metacpan.org/release/Test2