-
Notifications
You must be signed in to change notification settings - Fork 0
/
Notes.txt
134 lines (92 loc) · 5.89 KB
/
Notes.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
Advanced PyTest Notes
# conftest.py exercise # - tested value declared in fixture across multiple tests in different functions
- conftest.py file makes the fixtures in it available to all tests within that folder.
- in the exercise is a fixture function with name 'input_value()' that, when invoked, returns integer 10.
# test_retval1 exercise # - using conftest.py fixture for assert test
- 'input_value()' fixture is used as a parameter in the test .
- the return value of 'input_value()' is assigned to a variable of the same name (in 'test_func_retval()' ).
- 'input_value' variable is compared for equality to the return value from invoking 'func_retval()'.
- ran pytest and it PASSES, as 10==10.
# parametrization exercises # - run tests multiple times with different inputs
- two types: test parametrization and fixture parametrization.
I. test parametrization
- used the 'pytest.mark.parametrize' decorator.
- contains two arguments: str variables "numerator, operator" and a list.
- first number in list will be used for numerator value.
- second number will be used as the operator value.
- '%' modulo operator used (divides left by right, then returns the remainder).
- assert statement checks if a numerator divided by 5 has a remainder of 0.
- 'test_divisible_by_five()' test runs four times, iterating once per item on the list.
- only the third test should fail, the rest pass
- ran 'pytest test_param.py -v' (-v flag for verbose output)
II. fixture parametrization
- class Numerator initializes single instance attribute, 'num_list', with default value [0,0].
- fixture decorator has a parameter list: 'params = [[...]]'.
- 'params' keyword required, outer data structure is a list but inside values can be other data type.
- fixture function 'divisible_by_two' defined and has parameter 'request'.
- when this fixture function runs, pytest will run once per item on list. -- three times.
- each iteration, pytest passes in request object.
- pytest ensures that request object will have a param attribute with the current item from the 'params' list.
- the function body is able to access and test each parameter.
- 'divisible_by_two' instantiates an object of the Numerator class and returns it.
- object can now be accessed in a test function by using 'divisible_by_two' as a parameter.
- 'test_divisible_by_two' instantiates an object of the Numerator class, with different values for the 'num.list' attribute.
- test runs three times, each time looping through one of the three lists.
- ran 'pytest test_param.py -v' verbose flag.
- as expected, two tests fail and the last one passes.
# command-line options for pytest exercise #
- a list of these command-line options can be seen using - pytest -h
- or online at https://docs.pytest.org/en/6.2.x/reference.html#command-line-flags
I. Stopping test suite after a certain number of test failures
- '--maxfail=num' option to stop a test suite after 'num' number of times
- ran 'pytest test_nfailsuite.py -v --maxfail=1'
- change the value for 'num' (--maxfail=2 and --maxfail=3)
- all tests expected to fail.
II. Run multiple test in parallel -- using pytest plugin pytest-xdist
- useful when test suites are very large
- make sure your Python testing environment is activated
- python -m pip install pytest-xdist
- '-n' option allows to specify number of tests to run in parallel
- using test_param.py file
- ran 'pytest test_param.py -v -n 3'
-three forked processes were spawned to run three tests in parallel
# Running tests without specifying a file #
- run 'pytest' on terminal (make sure testenv is activated)
- it will run every test in the current folder and output a summary
- green dot means passed, red F means test failed
# Using the pytest -k flag #
- flag '-k' in pytest will only run tests functions that match the following substring expression
- test_retval2.py file used in exercise
- ran 'pytest -k func_retval -v' on terminal
- no filename specified so pytest ran all test-*.py files
- test_retval1.py, test_func_retval passed
- test_retval2.py, test_func_retval failed
- All other tests, including the test_simple_pass test in test_retval2.py, were deselected and not run
- ran 'pytest test_retval2.py -k simple -v'
- provided filename, pytest only collects tests in that file
- -k flag narrows down test functions, 'simple' passed as the argument
- only test_simple_pass() test ran and it PASSES
# Using xfail or skipping tests #
- useful decorators: pytest.mark.xfail and pytest.mark.skip
- when test is decorated with @pytest.mark.xfail, it is expected to fail
- testing bugs, functions not implemented yet
- test failing will result in yellow 'XFAIL'
- when test is decorated with @pytest.mark.skip, it will not run the test at all
- ran 'pytest test_xfail_skip.py -v'
test_xfail_skip.py::test_func_one XFAIL [ 50%]
test_xfail_skip.py::test_func_two SKIPPED (unconditional skip) [100%]
# Using custom marks #
- run tests based on that custom mark
- @pytest.mark.markone decorator to test_func_two(), which already has the @pytest.mark.skip decorator
- more than one decorator can exist on the same function
- test_func_three() and test_func_four(), each decorated with markone and marktwo
- ran 'pytest -m markone'
- two marked functions @pytest.mark.markone and @pytest.mark.marktwo
- warnings summary include 11 warnings, this is due to not registering custom marks
- to register: create pytest.ini file and update your marks, mark name descriptions are optional
- ran 'pytest -m markone' again
- same results as before but with no warnings
- ran 'pytest -m "markone or marktwo" -v'
- "markone or marktwo" argument for -m flag caused all tests to run in either markone OR marktwo
- ran 'pytest -m "not markone" -v'
- results in all tests, except for markone, to run