colorama breaks randomly on Windows 10 and Python 3.6

[segevfiner]

By the very same issue that causes pytest-dev/py#103 the handles used by colorama are getting closed by Pytest's FDCapture. Though it only manifests randomly.

colorama ignores the errors though and all colorama features just randomly stop working (e.g. Colors stop working). Adding error handling to colorama.winterm reveals the very same exception. Essentially colorama caches the std handles in colorama.win32.handles. Which then get closed by Pytest's FDCapture calling dup2.

This probably affects other Python modules or anything else loaded into Pytest's process (Native libraries) that caches the stdio handles. And the underlying issue might have other affects besides this. I can think of pyreadline, win-unicode-console and click for example being broken by this but I haven't tested.

I am running this with pytest -v and randomly the green PASSED text stops being colored:

import time
import random
import pytest


@pytest.mark.parametrize("i", range(1000))
def test_random_sleep(i, capfd):
    time.sleep(random.random() * 0.2)

EDIT: You might not be able to reproduce this because Pytest will break by pytest-dev/py#103 first. It still happens with PYTHONLEGACYWINDOWSSTDIO since it's unrelated to WindowsConsoleIO breaking, but is still caused by the same cause. It might also happen in older Python 3 versions. Might be related to the VS/CRT 2015 switch.

EDIT: Possible Fixes (At least ones I can come up with)

1. colorama can stop caching the console handles. Not sure if this has performance implications.
2. colorama can DuplicateHandle the console handles.
3. We can overwrite colorama's handles with duplicates our self.

See

• [WIP] A workaround for Python 3.6 WindowsConsoleIO breaking with FDCapture #2462
• Error with console output in Python 3.6 on Windows py#103
• EDIT: https://bugs.python.org/issue30555 - A similar issue with WindowsConsoleIO in CPython
• Issue on windows 10 Python 3.6 tartley/colorama#119

System Info

Package    Version                         Location
---------- ------------------------------- -------------------------
certifi    2017.4.17
chardet    3.0.3
colorama   0.3.9
hypothesis 3.11.1
idna       2.5
mock       2.0.0
nose       1.3.7
pbr        3.0.1
pip        9.0.1
py         1.4.33
pytest     3.1.2.dev14+g05796be2.d20170602 c:\users\segev\prj\pytest
requests   2.17.3
setuptools 36.0.1
six        1.10.0
urllib3    1.21.1
wheel      0.29.0

Windows 10.0.14393 x64 with Python 3.6.1

EDIT: Also reproduces for me on Python 3.5.3.

Thanks for submitting an issue!

Here's a quick checklist in what to include:

• Include a detailed description of the bug or suggestion
• pip list of the virtual environment you are using
• pytest and operating system versions
• Minimal example if possible

[nicoddemus]

Fixed in #2462

[segevfiner]

@nicoddemus Actually #2462 fixes pytest-dev/py#103. This one is a separate issue because colorama is managing it's own Windows console handles, and gets broken independently of what #2462 fixes. I submitted a PR to colorama which should fix this one too: tartley/colorama#131.

This one even happens on Python 3.5, which doesn't even have WindowsConsoleIO. Probably since the console handle closing is a change in the crt.

[nicoddemus]

Oops sorry, my mistake! 😅

[nicoddemus]

@segevfiner actually is there anything else to do on pytest's side? Seems like the ball is in colorama's court here.

[segevfiner]

@nicoddemus Not unless someone can come up with a workaround that avoids this issue in the first place. This sort of issue will repeat itself in many Python libraries which cache the std handles.

[nicoddemus]

OK thanks, let's keep it open to track it at least.