Carl Meyer - Type-checked Python in the Real World - PyCon 2018 - 2018

Consider the following function:

def process(self, items):
    for item in items:
        self.append(item.value.id)

If someone asks what items is, from the above code we can only say that it is a collection, where each object in the collection has a value attribute, which has an id attribute. Even though we can infer this by looking at the code above, it can break in other parts of the code base, e.g. somewhere there might be an object where the value attribute could be None, or if we want to extend the process function by accessing a new attribute of item , we need to know whether this new attribute exists everywhere else the process function is used.

With type type annotations we get more context:

from typing import Sequence
from .models import Item

def proces(self, items: Sequence[Iterm]) -> None:
    for item in items:
        self.append(item.value.id)

People do put this information in docstrings, but often the function could be updated without the docstring being updated.

Consider the following code:

def square(x: int) -> int:
    return x**2

square(3)
square('foo')
square(4) + 'foo'

Type checking this code involves running

pip install mypy #if mypy not installed
mypy square.py

Type inference:

from typing import Tuple

class Photo:
    def __init__(self, width:int, height:int) -> None:
        self.width = width
        self.height = height

    def get_dimensions(self) -> Tuple[str, str]
        return (self.width, self.height)

In the above code, the type checker will catch that self.width,self.height are integers and hence get_dimensions should return Tuple[int, int].

When creating empty containers, the type checker would want type annotations.

tags = [] # will return error
tags: List[str] = []

If the function can return more than 1 possible type, we can use Union. (#NOTE PEP604 allows use of "|").

from typing import Union, Optional

def foo(id: int) -> Union[Foo, Bar]

def foo(id: int) -> Union[Foo, None]

def foo(id: int) -> Optional[Foo] #Same as Union[Foo, None]

Ideally, using overload is better than Union and Optional.

from typing import Optional, overload

@overload
def get_foo(foo_id: None) -> None:
    pass

@overload
def get_foo(foo_id: int) -> Foo:
    pass

def get_foo(foo_id: Optional[int]) -> Optional[Foo]:
    is foo_id is None:
        return None
    return Foo(foo_id)

The type checker can use the functions with the @overload decorator to know the return types.

Generic functions can also be implemented by creating type variables.

from typing import TypeVar

AnyStr = TypeVar('AnyStr', str, bytes)

def concat(a: AnyStr, b: AnyStr) -> AnyStr:
    return a + b

The above function will return an error if given a type that is neither str not bytes, and also if it is given both a str and byte at the same time.

If you really want duck typing, we can use Any.

from typing import Any

# Will not type check since object has no method render()
def render(obj: object) -> str:
    return obj.render()

# Type checks but can fail in runtime if obj has no method render()
def render(obj: Any) -> str:
    return obj.render()

The type system solution for the above is called Protocol, called structural sub-typing (in comparison to nominal sub-typing which is in typical inheritance).

from typing_extensions import Protocol

class Renderable(Protocol): #Just an interface
    def render(self) -> str: ...

def render(obj: Renderable) -> str:
    return obj.render()

class Foo:
#No relation the Renderable but has a render method with correct signature
    def render(self) -> str:
        return "Foo!"

render(Foo()) # will type check and work
render(3) # will not type check

In practice however, we may need some "escape hatches" when changing large code bases, for e.g.

• Any.
• cast which lets you lie to the type checker.
• type: ignore - nuclear option.
• stub (pyi) files.

This means type checking programs even if not all expressions are typed, e.g. using Any. In Python, gradual typing is implemented by only checking functions with type annotations, and other functions are assumed to take Any and return Any.

The package MonkeyType can help when rewriting untyped to typed code by running monkeytype run mytests.py. This takes the untyped code and gives a stub file with function signatures containing the types which were instantiated in run time. These types can then by applied via monkeytype apply some.module.