[mutmut3] Forced fail does not fail.

[awgymer]

Tried updating to mutmut 3.0.5 but I the forced fail test seems to pass and thus (ironically) fail.

running forced fail test========================================================================================== test session starts ===========================================================================================

...

========================================================================================== 1440 passed in 8.16s ==========================================================================================


FAILED

[boxed]

Could you supply the project so I can test?

[awgymer]

I cannot as this is not an open-source library it's a proprietary one for work.

I looked at the code for run_forced_fail though and I couldn't figure out why there is an assumption it would fail? AFAICT it runs pytest like normal and just doesn't supply any test args?

[boxed]

It sets the environment variable that makes all trampolines fail immediately. Think of it like turning on an exception being thrown in all functions in your code base.

[awgymer]

I tried again with the latest version of my codebase.

Now I am getting an error on the stats collection:

mutmut run
generating mutants
    done in 77ms
⠏ running stats
    done
failed to collect stats, no active tests found

I see a mutants folder and inside that I see htmlcov and tests (which appears to contain a copy of my tests dir).

My pkg structure is:

pkg-name
    - pkg_name/<source code here>
    - tests/<test code and resources here>
    - pyproject.toml

[boxed]

Did you try with 3.1.0? I just released it with some fixes for stuff found by other users.

[awgymer]

Yes it ran on 3.1.0

[awgymer]

Ok. Turns out I left an empty src dir from some previous testing so I solved that issue.

However during stats I get:

FAILED tests/test_main.py::test_rich_force_colours[env_var2-True] - KeyError: 'MUTANT_UNDER_TEST'

Which I think might be due to the test itself mocking os.environ?

[boxed]

Ah, nice that you solved it.

mocking os.environ

That's an interesting situation yea. Hmm.. actually.. I don't think I need to use os.environ at all now that I think of it. I think that's a leftover from some earlier idea.

[boxed]

I realized I do need to use os.environ for multiprocessing support later.

Could you make your mock at least fall back to reading the real environ if there's a keyerror?

[awgymer]

I'm not sure how that would work? Would a patch rather than mock fall back to the real object except for the value i am patching?

I probably can actually avoid mocking/patching it anyway. The method under test is simply checking an ENV var and returning some bool based on the value. I can probably just directly set the os.environ and call it good.

[boxed]

You don't have to use Mock to do mocking :)

But yea, just setting it seems more sane.