Cheuk Ting Ho: I Hate Writing Tests, That’s Why I Use Hypothesis | PyData Tel Aviv 2022 - 2023

Title : Cheuk Ting Ho: I Hate Writing Tests, That’s Why I Use Hypothesis | PyData Tel Aviv 2022 Author(s): PyData Link(s) : https://www.youtube.com/watch?v=__DfM6R4nVs

The difficult parts of writing tests include:

• Importing the code.
• Organizing the tests.
• Thinking of the test cases.

Hypothesis helps with the last 2.

Property based testing: Easiest to see by difference again testing by example.

Hypothesis does this by using Python decorators to modify your test so that it will take an input and it will run a defined strategy to be used for that input. A strategy generates test data. For e.g. if we have some encode and decode functions for strings of some format:

from hypothesis import given
from hypothesis.strategies import text

@given(text())
def test_decode_inverts_encode(s):
    assert decode(encode(s)) == s

For scientific computing, there are Numpy related methods, which require separate installation via pip install hypothesis[numpy]. The relevant functionality is located at hypothesis.extra.numpy, which will have strategies for scalar and array dtypes.

There is also Pandas functionality, which is also installed when the numpy functionality is installed. The pandas strategies are located in hypothesis.extra.pandas.

If you lazy, there is a ghostwriter that will write tests for you, using your typing as a hint. The ghostwriter will also find the right strategy for you, and also includes a CLI tool like hypothesis write gzip. It will also automatically format the code using Black.

Ghostwriters do the following among many things:

• Fuzz i.e. checking valid input only leads to expected exceptions.
• Idempotency i.e. result does not change when using the function on its own output.
• Roundtrip i.e. calling the 2nd function to the result of the 1st one will will go back to the input.
• Equivalence i.e. check the 1st function has the same effect as the 2nd function, helpful when reinventing the function.
• Binary operations which test for binary operators.
• Ufunc for testing numpy array ufunc.

Some points to consider:

• Tests can run slower, generating strategies is expensive.
• Tests can be harder to understand.
• If no types are there, nothing much can be done.
• Cannot test whether a machine learning model is doing what it is supposed to do.