Introduction to mypy. Static type checking for Python | by Oliver S | Apr, 2023
Static type checking for Python
We mentioned mypy as a must-have in a previous post about Python best practices — here, we want to introduce it with more details.
mypy, as the docs explain, is a “static type checker for Python”. This means, it adds type annotations and checks to the language Python, which is dynamically typed by design (types are inferred at runtime, as opposed to, e.g. C++). Doing so lets you find bugs in your code during compile-time, which is a great help — and a must for any semi-professional Python project, as explained in my previous post.
In this post we will introduce mypy using several examples. Disclaimer: this post won’t introduce every mypy feature (not even near it). Instead, I’ll try to find a good balance between sufficient details to let you write nearly all the code you want — and generating a steep learning curve from zero to solid mypy understanding. For more details, I’d like to refer to the official docs or any other great tutorial out there.
To install mypy, simply run: pip install mypy
However, I would recommend to use some form of dependency management system, such as poetry. How to include this and mypy in a larger software project, is explained here.
Let’s motivate the usage of mypy with a first example. Consider the following code:
def multiply(num_1, num_2):
return num_1 * num_2print(multiply(3, 10))
print(multiply("a", "b"))
multiply expects two numbers and returns their product. Thus, multiply(3, 10) works well and returns the desired result. But the second statement fails and crashes the execution, as we can’t multiply strings. Due to Python being dynamically typed, nothing stopped us from coding / executing that statement, and we only found the issue at run time — which is problematic.
Here, mypy comes to the rescue. We can now annotate the arguments, and also the return type of the function, as such:
def multiply(num_1: int, num_2: int) -> int:
return num_1 * num_2print(multiply(3, 10))
print(multiply("a", "b"))
This annotation won’t change the execution in anyway, in particular, you can still run this faulty program. However, before doing so and shipping our program, we can now run mypy and check for any possible errors via: mypy .
Running this command will fail, and correctly point out that we can’t pass strings to multiply. Above command is meant to be executed from the main folder of the application, . will check every file in the current folder and subdirectories. But you can also check specific files via mypy file_to_check.py.
This hopefully motivated the need and usage of mypy, now let’s dive deeper.
mypy can be configured in many different ways — without going into details, it just needs to find one config file (such as mypy.ini, pyproject.toml, …) with a “mypy” section in it. Here, we will create the default file mypy.ini, which should live in the project’s main folder.
Now, let’s come to possible configuration options. For this, we go back to our initial example:
def multiply(num_1, num_2):
return num_1 * num_2print(multiply(3, 10))
print(multiply("a", "b"))
Simply running mypy actually yields no errors! That is, because type hints are optional by default — and mypy only checks types where an annotation is given. We can disable this via the flag — disallow-untyped-defs. Furthermore, there is a multitude of other flags one can use (see here). However, in line with the general format of this post, we won’t go into detail of all these — and instead just present the strict mode. This mode turns on basically all optional checks. And in my experience, the best way of using mypy is to simply ask for the strictest checking possible — and then fix (or selectively ignore) any upbrought issues.
To do so, let’s fill the mypy.ini file like this:
[mypy]
strict = true
The section header [mypy] is needed for any mypy related configuration, and the next line is pretty self-explanatory.
When we now run mypy as usual, we are getting errors complaining about the missing type annotations — which only go away once everything is typed and we remove the faulty string call.
Now let’s have a closer look at how to annotate with mypy.
In this section we’ll describe the most common type annotations and mypy keywords.
We can annotate primitive types by simply using their Python type, i.e. bool, int, float, str, …:
def negate(value: bool) -> bool:
return not valuedef multiply(multiplicand: int, multiplier: int) -> int:
return multiplicand * multiplier
def divide(dividend: float, divisor: float) -> float:
return dividend / divisor
def concat(str_a: str, str_b: str) -> str:
return str_a + " " + str_b
print(negate(True))
print(multiply(3, 10))
print(divide(10, 3))
print(concat("Hello", "world"))
Starting from Python 3.9 upwards, also the built-in collection types can be used as type annotations. That is list, set, dict, …:
def add_numbers(numbers: list[int]) -> int:
return sum(numbers)def cardinality(numbers: set[int]) -> int:
return len(numbers)
def concat_values(value_dict: dict[str, float]) -> list[float]:
return [val for _, val in value_dict.items()]
print(add_numbers([1, 2, 3, 4]))
print(cardinality({1, 2, 3}))
print(concat_values({"a": 1.5, "b": 10}))
As we can see, we have to specify the contents of the containers (e.g. int). For mixed types, see below.
Earlier Python Versions
For earlier Python versions, one had to use legacy types from the typing module:
from typing import Dict, List, Setdef add_numbers(numbers: List[int]) -> int:
return sum(numbers)
def cardinality(numbers: Set[int]) -> int:
return len(numbers)
def concat_values(value_dict: Dict[str, float]) -> list[float]:
return [val for _, val in value_dict.items()]
print(add_numbers([1, 2, 3, 4]))
print(cardinality({1, 2, 3}))
print(concat_values({"a": 1.5, "b": 10}))
Mixing Contents
As teased above, we might want to create containers holding different data types. To do so, we can use the Union keyword — which allows us to annotate a type as a union of types:
from typing import Uniondef scan_list(elements: list[Union[str | int]]) -> None:
for el in elements:
if isinstance(el, str):
print(f"I got a string! ({el})")
elif isinstance(el, int):
print(f"I got an int! ({el})")
else:
# NOTE: we don't reach here because of mypy!
raise ValueError(f"Unexpected element type {el}")
scan_list([1, "hello", "world", 100])
Similar to the simplifications done in Python 3.9, Python 3.10 (specifically PEP 604) introduces an abbreviated notation of the Union type using the logical or operator (|):
def scan_list(elements: list[str | int]) -> None:
for el in elements:
if isinstance(el, str):
print(f"I got a string! ({el})")
elif isinstance(el, int):
print(f"I got an int! ({el})")
else:
# NOTE: we don't reach here because of mypy!
raise ValueError(f"Unexpected element type {el}")scan_list([1, "hello", "world", 100])
In this section we’ll introduce more essential types and keywords.
None
None, just as in “normal” Python, denotes a None value — most commonly used for annotating functions without return type:
def print_foo() -> None:
print("Foo")print_foo()
Optional
Often, we might come across situations where we want to implement branching code based on whether we passed a value for a parameter or not — and often, we use None to indicate the absence of it. For this, we can use typing.Optional[X] — which denotes exactly this: it annotates type X, but also allows None:
from typing import Optionaldef square_number(x: Optional[int]) -> Optional[int]:
return x**2 if x is not None else None
print(square_number(14))
Following Python 3.10 and above introduced PEP 604, Optional can again be shorted to X | None:
def square_number(x: int | None) -> int | None:
return x**2 if x is not None else Noneprint(square_number(14))
Note that this does not correspond to required or optional parameters, which is often confused! An optional parameter is one which we do not have to specify when calling a function — whereas mypy’s Optional indicates a parameter which can be of some type, but also None. A possible source of confusion could be that a common default value for optional parameters is None.
Any
Any, as the name suggests (I feel I keep repeating this sentence …) simply allows every type — thus effectively turning off any kinds of type checking. Thus, try to avoid this whenever you can.
from typing import Anydef print_everything(val: Any) -> None:
print(val)
print_everything(0)
print_everything(True)
print_everything("hello")
Annotating Variables
So far, we have used mypy to only annotate function parameters and return types. It is just natural to extend this to any kind of variables:
int_var: int = 0
float_var: float = 1.5
str_var: str = "hello"
However, this is somewhat lesser used (and also not enforced by the strict version of mypy), as the types of variables are mostly clear from the context. Usually, you would only do this when the code is relatively ambiguous and hard to read.
In this section we’ll discuss annotating classes, but also annotating with your own and other complex classes.
Annotating Classes
Annotating classes can be handled pretty quickly: just annotate class functions as any other function, but don’t annotate the self argument in the constructor:
class SampleClass:
def __init__(self, x: int) -> None:
self.x = xdef get_x(self) -> int:
return self.x
sample_class = SampleClass(5)
print(sample_class.get_x())
Annotating with Custom / Complex Classes
With our class defined, we can now use its name as any other type annotation:
sample_class: SampleClass = SampleClass(5)
In fact, mypy works with most classes and types out of the box, e.g.:
import pathlibdef read_file(path: pathlib.Path) -> str:
with open(path, "r") as file:
return file.read()
print(read_file(pathlib.Path("mypy.ini")))
In this section we’ll see how to deal with external libraries which do not support typing and selectively disable type checking for certain lines which cause issues — on the basis of a slightly more complex example involving numpy and matplotlib.
Let’s begin with a first version of the code:
import matplotlib.pyplot as plt
import numpy as np
import numpy.typing as nptdef calc_np_sin(x: npt.NDArray[np.float32]) -> npt.NDArray[np.float32]:
return np.sin(x)
x = np.linspace(0, 10, 100)
y = calc_np_sin(x)
plt.plot(x, y)
plt.savefig("plot.png")
We define a simple function computing the sinus of a numpy array, and apply it to the input values x, which span the space [0, 10]. Then, we plot the sinus curve using matplotlib.
In this code, we also see the correct typing of numpy arrays using numpy.typing.
However, if we run mypy on this, we’ll get two errors. The first one is:
error: Returning Any from function declared to return “ndarray[Any, dtype[floating[_32Bit]]]”
This is a relatively common pattern in mypy. We actually did not do anything wrong, but mypy would like it somewhat more explicit — and here — as well as in other situations — we have to “force” mypy to accept our code. We can do this for example by introducing a proxy variable of the correct type:
def calc_np_sin(x: npt.NDArray[np.float32]) -> npt.NDArray[np.float32]:
y: npt.NDArray[np.float32] = np.sin(x)
return y
The next error is:
error: Skipping analyzing “matplotlib”: found module but no type hints or library stubs
This is because matplotlib is not typed (yet). Thus, we need to let mypy know to exclude it from checking. We do this by adding the following to our mypy.ini file:
[mypy-matplotlib.*]
ignore_missing_imports = True
ignore_errors = True
Lastly, note that you can also selectively ignore any lines of code by appending # type: ignore to it. Do this, if there really is an unsolvable issue with mypy, or you want to silence some known but irrelevant warnings / errors. We could have also hidden our first error above via this:
def calc_np_sin(x: npt.NDArray[np.float32]) -> npt.NDArray[np.float32]:
return np.sin(x) # type: ignore
In this post we introduced mypy, which is a static type checker for Python. Using mypy, we can (and should) annotate types of variables, parameters and return values — giving us a way of sanity checking our program at compile time. mypy is very wide-spread, and recommended for any semi-large software project.
We started by installing and configuring mypy. Then, we introduced how to annotate primitive and complex types, such as lists, dicts or sets. Next, we discussed other important annotators, such as Union, Optional, None, or Any. Eventually, we showed that mypy supports a wide range of complex types, such as custom classes. We finished the tutorial by showing how to debug and fix mypy errors.
That’s it for mypy — I hope, you liked this post, thanks for reading!
Static type checking for Python
We mentioned mypy as a must-have in a previous post about Python best practices — here, we want to introduce it with more details.
mypy, as the docs explain, is a “static type checker for Python”. This means, it adds type annotations and checks to the language Python, which is dynamically typed by design (types are inferred at runtime, as opposed to, e.g. C++). Doing so lets you find bugs in your code during compile-time, which is a great help — and a must for any semi-professional Python project, as explained in my previous post.
In this post we will introduce mypy using several examples. Disclaimer: this post won’t introduce every mypy feature (not even near it). Instead, I’ll try to find a good balance between sufficient details to let you write nearly all the code you want — and generating a steep learning curve from zero to solid mypy understanding. For more details, I’d like to refer to the official docs or any other great tutorial out there.
To install mypy, simply run: pip install mypy
However, I would recommend to use some form of dependency management system, such as poetry. How to include this and mypy in a larger software project, is explained here.
Let’s motivate the usage of mypy with a first example. Consider the following code:
def multiply(num_1, num_2):
return num_1 * num_2print(multiply(3, 10))
print(multiply("a", "b"))
multiply expects two numbers and returns their product. Thus, multiply(3, 10) works well and returns the desired result. But the second statement fails and crashes the execution, as we can’t multiply strings. Due to Python being dynamically typed, nothing stopped us from coding / executing that statement, and we only found the issue at run time — which is problematic.
Here, mypy comes to the rescue. We can now annotate the arguments, and also the return type of the function, as such:
def multiply(num_1: int, num_2: int) -> int:
return num_1 * num_2print(multiply(3, 10))
print(multiply("a", "b"))
This annotation won’t change the execution in anyway, in particular, you can still run this faulty program. However, before doing so and shipping our program, we can now run mypy and check for any possible errors via: mypy .
Running this command will fail, and correctly point out that we can’t pass strings to multiply. Above command is meant to be executed from the main folder of the application, . will check every file in the current folder and subdirectories. But you can also check specific files via mypy file_to_check.py.
This hopefully motivated the need and usage of mypy, now let’s dive deeper.
mypy can be configured in many different ways — without going into details, it just needs to find one config file (such as mypy.ini, pyproject.toml, …) with a “mypy” section in it. Here, we will create the default file mypy.ini, which should live in the project’s main folder.
Now, let’s come to possible configuration options. For this, we go back to our initial example:
def multiply(num_1, num_2):
return num_1 * num_2print(multiply(3, 10))
print(multiply("a", "b"))
Simply running mypy actually yields no errors! That is, because type hints are optional by default — and mypy only checks types where an annotation is given. We can disable this via the flag — disallow-untyped-defs. Furthermore, there is a multitude of other flags one can use (see here). However, in line with the general format of this post, we won’t go into detail of all these — and instead just present the strict mode. This mode turns on basically all optional checks. And in my experience, the best way of using mypy is to simply ask for the strictest checking possible — and then fix (or selectively ignore) any upbrought issues.
To do so, let’s fill the mypy.ini file like this:
[mypy]
strict = true
The section header [mypy] is needed for any mypy related configuration, and the next line is pretty self-explanatory.
When we now run mypy as usual, we are getting errors complaining about the missing type annotations — which only go away once everything is typed and we remove the faulty string call.
Now let’s have a closer look at how to annotate with mypy.
In this section we’ll describe the most common type annotations and mypy keywords.
We can annotate primitive types by simply using their Python type, i.e. bool, int, float, str, …:
def negate(value: bool) -> bool:
return not valuedef multiply(multiplicand: int, multiplier: int) -> int:
return multiplicand * multiplier
def divide(dividend: float, divisor: float) -> float:
return dividend / divisor
def concat(str_a: str, str_b: str) -> str:
return str_a + " " + str_b
print(negate(True))
print(multiply(3, 10))
print(divide(10, 3))
print(concat("Hello", "world"))
Starting from Python 3.9 upwards, also the built-in collection types can be used as type annotations. That is list, set, dict, …:
def add_numbers(numbers: list[int]) -> int:
return sum(numbers)def cardinality(numbers: set[int]) -> int:
return len(numbers)
def concat_values(value_dict: dict[str, float]) -> list[float]:
return [val for _, val in value_dict.items()]
print(add_numbers([1, 2, 3, 4]))
print(cardinality({1, 2, 3}))
print(concat_values({"a": 1.5, "b": 10}))
As we can see, we have to specify the contents of the containers (e.g. int). For mixed types, see below.
Earlier Python Versions
For earlier Python versions, one had to use legacy types from the typing module:
from typing import Dict, List, Setdef add_numbers(numbers: List[int]) -> int:
return sum(numbers)
def cardinality(numbers: Set[int]) -> int:
return len(numbers)
def concat_values(value_dict: Dict[str, float]) -> list[float]:
return [val for _, val in value_dict.items()]
print(add_numbers([1, 2, 3, 4]))
print(cardinality({1, 2, 3}))
print(concat_values({"a": 1.5, "b": 10}))
Mixing Contents
As teased above, we might want to create containers holding different data types. To do so, we can use the Union keyword — which allows us to annotate a type as a union of types:
from typing import Uniondef scan_list(elements: list[Union[str | int]]) -> None:
for el in elements:
if isinstance(el, str):
print(f"I got a string! ({el})")
elif isinstance(el, int):
print(f"I got an int! ({el})")
else:
# NOTE: we don't reach here because of mypy!
raise ValueError(f"Unexpected element type {el}")
scan_list([1, "hello", "world", 100])
Similar to the simplifications done in Python 3.9, Python 3.10 (specifically PEP 604) introduces an abbreviated notation of the Union type using the logical or operator (|):
def scan_list(elements: list[str | int]) -> None:
for el in elements:
if isinstance(el, str):
print(f"I got a string! ({el})")
elif isinstance(el, int):
print(f"I got an int! ({el})")
else:
# NOTE: we don't reach here because of mypy!
raise ValueError(f"Unexpected element type {el}")scan_list([1, "hello", "world", 100])
In this section we’ll introduce more essential types and keywords.
None
None, just as in “normal” Python, denotes a None value — most commonly used for annotating functions without return type:
def print_foo() -> None:
print("Foo")print_foo()
Optional
Often, we might come across situations where we want to implement branching code based on whether we passed a value for a parameter or not — and often, we use None to indicate the absence of it. For this, we can use typing.Optional[X] — which denotes exactly this: it annotates type X, but also allows None:
from typing import Optionaldef square_number(x: Optional[int]) -> Optional[int]:
return x**2 if x is not None else None
print(square_number(14))
Following Python 3.10 and above introduced PEP 604, Optional can again be shorted to X | None:
def square_number(x: int | None) -> int | None:
return x**2 if x is not None else Noneprint(square_number(14))
Note that this does not correspond to required or optional parameters, which is often confused! An optional parameter is one which we do not have to specify when calling a function — whereas mypy’s Optional indicates a parameter which can be of some type, but also None. A possible source of confusion could be that a common default value for optional parameters is None.
Any
Any, as the name suggests (I feel I keep repeating this sentence …) simply allows every type — thus effectively turning off any kinds of type checking. Thus, try to avoid this whenever you can.
from typing import Anydef print_everything(val: Any) -> None:
print(val)
print_everything(0)
print_everything(True)
print_everything("hello")
Annotating Variables
So far, we have used mypy to only annotate function parameters and return types. It is just natural to extend this to any kind of variables:
int_var: int = 0
float_var: float = 1.5
str_var: str = "hello"
However, this is somewhat lesser used (and also not enforced by the strict version of mypy), as the types of variables are mostly clear from the context. Usually, you would only do this when the code is relatively ambiguous and hard to read.
In this section we’ll discuss annotating classes, but also annotating with your own and other complex classes.
Annotating Classes
Annotating classes can be handled pretty quickly: just annotate class functions as any other function, but don’t annotate the self argument in the constructor:
class SampleClass:
def __init__(self, x: int) -> None:
self.x = xdef get_x(self) -> int:
return self.x
sample_class = SampleClass(5)
print(sample_class.get_x())
Annotating with Custom / Complex Classes
With our class defined, we can now use its name as any other type annotation:
sample_class: SampleClass = SampleClass(5)
In fact, mypy works with most classes and types out of the box, e.g.:
import pathlibdef read_file(path: pathlib.Path) -> str:
with open(path, "r") as file:
return file.read()
print(read_file(pathlib.Path("mypy.ini")))
In this section we’ll see how to deal with external libraries which do not support typing and selectively disable type checking for certain lines which cause issues — on the basis of a slightly more complex example involving numpy and matplotlib.
Let’s begin with a first version of the code:
import matplotlib.pyplot as plt
import numpy as np
import numpy.typing as nptdef calc_np_sin(x: npt.NDArray[np.float32]) -> npt.NDArray[np.float32]:
return np.sin(x)
x = np.linspace(0, 10, 100)
y = calc_np_sin(x)
plt.plot(x, y)
plt.savefig("plot.png")
We define a simple function computing the sinus of a numpy array, and apply it to the input values x, which span the space [0, 10]. Then, we plot the sinus curve using matplotlib.
In this code, we also see the correct typing of numpy arrays using numpy.typing.
However, if we run mypy on this, we’ll get two errors. The first one is:
error: Returning Any from function declared to return “ndarray[Any, dtype[floating[_32Bit]]]”
This is a relatively common pattern in mypy. We actually did not do anything wrong, but mypy would like it somewhat more explicit — and here — as well as in other situations — we have to “force” mypy to accept our code. We can do this for example by introducing a proxy variable of the correct type:
def calc_np_sin(x: npt.NDArray[np.float32]) -> npt.NDArray[np.float32]:
y: npt.NDArray[np.float32] = np.sin(x)
return y
The next error is:
error: Skipping analyzing “matplotlib”: found module but no type hints or library stubs
This is because matplotlib is not typed (yet). Thus, we need to let mypy know to exclude it from checking. We do this by adding the following to our mypy.ini file:
[mypy-matplotlib.*]
ignore_missing_imports = True
ignore_errors = True
Lastly, note that you can also selectively ignore any lines of code by appending # type: ignore to it. Do this, if there really is an unsolvable issue with mypy, or you want to silence some known but irrelevant warnings / errors. We could have also hidden our first error above via this:
def calc_np_sin(x: npt.NDArray[np.float32]) -> npt.NDArray[np.float32]:
return np.sin(x) # type: ignore
In this post we introduced mypy, which is a static type checker for Python. Using mypy, we can (and should) annotate types of variables, parameters and return values — giving us a way of sanity checking our program at compile time. mypy is very wide-spread, and recommended for any semi-large software project.
We started by installing and configuring mypy. Then, we introduced how to annotate primitive and complex types, such as lists, dicts or sets. Next, we discussed other important annotators, such as Union, Optional, None, or Any. Eventually, we showed that mypy supports a wide range of complex types, such as custom classes. We finished the tutorial by showing how to debug and fix mypy errors.
That’s it for mypy — I hope, you liked this post, thanks for reading!
Denial of responsibility! Techno Blender is an automatic aggregator of the all world’s media. In each content, the hyperlink to the primary source is specified. All trademarks belong to their rightful owners, all materials to their authors. If you are the owner of the content and do not want us to publish your materials, please contact us by email – [email protected]. The content will be deleted within 24 hours.