Carl Meyer - Type-checked Python in the Real World - PyCon 2018 - 2018
Details
Title : Carl Meyer - Type-checked Python in the Real World - PyCon 2018 Author(s): PyCon 2018 Link(s) : https://www.youtube.com/watch?v=pMgmKJyWKn8
Rough Notes
Why type Python code?
Consider the following function:
def process(self, items): for item in items: self.append(item.value.id)
If someone asks what items
is, from the above code we can only say
that it is a collection, where each object in the collection has a
value
attribute, which has an id
attribute. Even though we can infer
this by looking at the code above, it can break in other parts of the
code base, e.g. somewhere there might be an object where the value
attribute could be None
, or if we want to extend the process
function
by accessing a new attribute of item
, we need to know whether this
new attribute exists everywhere else the process
function is used.
With type type annotations we get more context:
from typing import Sequence from .models import Item def proces(self, items: Sequence[Iterm]) -> None: for item in items: self.append(item.value.id)
People do put this information in docstrings, but often the function could be updated without the docstring being updated.
How to type in Python?
Consider the following code:
def square(x: int) -> int: return x**2 square(3) square('foo') square(4) + 'foo'
Type checking this code involves running
pip install mypy #if mypy not installed mypy square.py
Type inference:
from typing import Tuple class Photo: def __init__(self, width:int, height:int) -> None: self.width = width self.height = height def get_dimensions(self) -> Tuple[str, str] return (self.width, self.height)
In the above code, the type checker will catch that
self.width,self.height
are integers and hence get_dimensions
should
return Tuple[int, int]
.
When creating empty containers, the type checker would want type annotations.
tags = [] # will return error tags: List[str] = []
If the function can return more than 1 possible type, we can use
Union
. (#NOTE PEP604 allows use of "|").
from typing import Union, Optional def foo(id: int) -> Union[Foo, Bar] def foo(id: int) -> Union[Foo, None] def foo(id: int) -> Optional[Foo] #Same as Union[Foo, None]
Ideally, using overload
is better than Union
and Optional
.
from typing import Optional, overload @overload def get_foo(foo_id: None) -> None: pass @overload def get_foo(foo_id: int) -> Foo: pass def get_foo(foo_id: Optional[int]) -> Optional[Foo]: is foo_id is None: return None return Foo(foo_id)
The type checker can use the functions with the @overload
decorator to
know the return types.
Generic functions can also be implemented by creating type variables.
from typing import TypeVar AnyStr = TypeVar('AnyStr', str, bytes) def concat(a: AnyStr, b: AnyStr) -> AnyStr: return a + b
The above function will return an error if given a type that is
neither str
not bytes
, and also if it is given both a str
and byte
at
the same time.
If you really want duck typing, we can use Any
.
from typing import Any # Will not type check since object has no method render() def render(obj: object) -> str: return obj.render() # Type checks but can fail in runtime if obj has no method render() def render(obj: Any) -> str: return obj.render()
The type system solution for the above is called Protocol
, called
structural sub-typing (in comparison to nominal sub-typing which is in
typical inheritance).
from typing_extensions import Protocol class Renderable(Protocol): #Just an interface def render(self) -> str: ... def render(obj: Renderable) -> str: return obj.render() class Foo: #No relation the Renderable but has a render method with correct signature def render(self) -> str: return "Foo!" render(Foo()) # will type check and work render(3) # will not type check
In practice however, we may need some "escape hatches" when changing large code bases, for e.g.
Any.
cast
which lets you lie to the type checker.type: ignore
- nuclear option.stub (pyi) files
.
Gradual typing
This means type checking programs even if not all expressions are
typed, e.g. using Any
. In Python, gradual typing is implemented by
only checking functions with type annotations, and other functions are
assumed to take Any
and return Any
.
The package MonkeyType
can help when rewriting untyped to typed code
by running monkeytype run mytests.py
. This takes the untyped code and
gives a stub file with function signatures containing the types which
were instantiated in run time. These types can then by applied via
monkeytype apply some.module
.