The test runner hides errors in non-Unicode locales

[mgedmin]

A build failed on winbot recently, but we can't see why, because the error reporting failed with a UnicodeError:

Traceback (most recent call last):
  File "c:\buildslave\z3c.form\build\bin\test-script.py", line 68, in <module>
    '--test-path', 'c:\\buildslave\\z3c.form\\build\\src',
  File "d:\eggs\zope.testrunner-4.4.4-py2.6.egg\zope\testrunner\__init__.py", line 27, in run
    failed = run_internal(defaults, args, script_parts=script_parts, cwd=cwd)
  File "d:\eggs\zope.testrunner-4.4.4-py2.6.egg\zope\testrunner\__init__.py", line 44, in run_internal
    runner.run()
  File "d:\eggs\zope.testrunner-4.4.4-py2.6.egg\zope\testrunner\runner.py", line 160, in run
    self.run_tests()
  File "d:\eggs\zope.testrunner-4.4.4-py2.6.egg\zope\testrunner\runner.py", line 242, in run_tests
    self.skipped, self.import_errors)
  File "d:\eggs\zope.testrunner-4.4.4-py2.6.egg\zope\testrunner\runner.py", line 411, in run_layer
    import_errors)
  File "d:\eggs\zope.testrunner-4.4.4-py2.6.egg\zope\testrunner\runner.py", line 331, in run_tests
    test(result)
  File "c:\Python26_32\lib\unittest.py", line 300, in __call__
    return self.run(*args, **kwds)
  File "c:\Python26_32\lib\unittest.py", line 282, in run
    result.addFailure(self, self._exc_info())
  File "d:\eggs\zope.testrunner-4.4.4-py2.6.egg\zope\testrunner\runner.py", line 804, in addFailure
    exc_info)
  File "d:\eggs\zope.testrunner-4.4.4-py2.6.egg\zope\testrunner\formatter.py", line 348, in test_failure
    self.print_traceback("Failure in test %s" % test, exc_info)
  File "d:\eggs\zope.testrunner-4.4.4-py2.6.egg\zope\testrunner\formatter.py", line 355, in print_traceback
    print(self.format_traceback(exc_info))
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2192' in position 789: ordinal not in range(128)

I think zope.testrunner should be prepared to deal with non-UTF-8 locales, by using .encode(sys.stdout.encoding, 'backslashreplace') or something like that.

[mgedmin]

This happened again: http://winbot.zope.org/builders/z3c.coverage_py_330_win32/builds/7/steps/test/logs/stdio

Failure in test doctest_syntax_highlight_without_enscript (z3c.coverage.tests)
Traceback (most recent call last):
  File "c:\buildslave\z3c.coverage_py_330_win32\build\bin\test-script.py", line 24, in <module>
    '--test-path', 'c:\\buildslave\\z3c.coverage_py_330_win32\\build\\src',
  File "d:\eggs\zope.testrunner-4.5.0-py3.3.egg\zope\testrunner\__init__.py", line 27, in run
    failed = run_internal(defaults, args, script_parts=script_parts, cwd=cwd)
  File "d:\eggs\zope.testrunner-4.5.0-py3.3.egg\zope\testrunner\__init__.py", line 44, in run_internal
    runner.run()
  File "d:\eggs\zope.testrunner-4.5.0-py3.3.egg\zope\testrunner\runner.py", line 171, in run
    self.run_tests()
  File "d:\eggs\zope.testrunner-4.5.0-py3.3.egg\zope\testrunner\runner.py", line 253, in run_tests
    self.skipped, self.import_errors)
  File "d:\eggs\zope.testrunner-4.5.0-py3.3.egg\zope\testrunner\runner.py", line 425, in run_layer
    import_errors)
  File "d:\eggs\zope.testrunner-4.5.0-py3.3.egg\zope\testrunner\runner.py", line 345, in run_tests
    test(result)
  File "c:\Python33_32\lib\unittest\case.py", line 491, in __call__
    return self.run(*args, **kwds)
  File "c:\Python33_32\lib\unittest\case.py", line 451, in run
    result.addFailure(self, exc_info)
  File "d:\eggs\zope.testrunner-4.5.0-py3.3.egg\zope\testrunner\runner.py", line 818, in addFailure
    exc_info)
  File "d:\eggs\zope.testrunner-4.5.0-py3.3.egg\zope\testrunner\formatter.py", line 348, in test_failure
    self.print_traceback("Failure in test %s" % test, exc_info)
  File "d:\eggs\zope.testrunner-4.5.0-py3.3.egg\zope\testrunner\formatter.py", line 355, in print_traceback
    print(self.format_traceback(exc_info))
  File "c:\Python33_32\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u0bf2' in position 652: character maps to <undefined>

[mgedmin]

A couple of recent examples:

• https://github.com/buildout/buildout/pull/534/checks?check_run_id=1130262907
• https://travis-ci.org/github/buildout/buildout/jobs/728115069