Test collection¶
Before HypoFuzz starts fuzzing by assigning tests to workers, it first needs to know what tests are available. HypoFuzz does so by running a collection step.
How HypoFuzz collects tests¶
HypoFuzz uses pytest to collect test functions, with an almost identical command-line interface.
If you’re using pytest ...
to run your tests, hypothesis fuzz -- ...
will fuzz them. Everything after --
is passed to a call to pytest --collect-only
in HypoFuzz.
Among others, this means you can use the standard pytest -k
selector to configure which tests to run, with hypothesis fuzz -- -k ...
. See pytest’s docs on Specifying which tests to run and Conventions for Python test discovery for more details.
Concretely, if your source layout looks like this:
src/
...
tests/
test_a.py
test_b.py
and you normally run your tests with pytest
, then:
hypothesis fuzz
will fuzz all your testshypothesis fuzz -- tests/test_a.py
will fuzz the tests intest_a.py
hypothesis fuzz -- -k selector_string
will fuzz the tests matchingselector_string
(see Specifying which tests to run).
Pytest plugins during collection¶
hypothesis fuzz
does not run pytest plugins during collection. If your standard pytest command includes plugin parameters like pytest -n auto
(from e.g. pytest-xdist), do not pass these parameters to hypothesis fuzz
.
Skipping tests under HypoFuzz¶
If you would like to skip a test under HypoFuzz while keeping it part of your normal test suite, you can use pytest.mark.skipif in combination with in_hypofuzz_run
:
import pytest
from hypothesis import given, strategies as st
from hypofuzz.detection import in_hypofuzz_run
@pytest.mark.skipif(in_hypofuzz_run)
@given(st.integers())
def test_will_not_be_fuzzed(n):
pass
in_hypofuzz_run
can also be used to mark tests which should run only under HypoFuzz, by inverting the conditional above.