Setting up Python Projects: Part III | by Johannes Schmidt

By Jessie Hobb On Apr 27, 2023

Mastering the Art of Python Project Setup: A Step-by-Step Guide

Whether you’re a seasoned developer or just getting started with 🐍 Python, it’s important to know how to build robust and maintainable projects. This tutorial will guide you through the process of setting up a Python project using some of the most popular and effective tools in the industry. You will learn how to use GitHub and GitHub Actions for version control and continuous integration, as well as other tools for testing, documentation, packaging and distribution. The tutorial is inspired by resources such as Hypermodern Python and Best Practices for a new Python project. However, this is not the only way to do things and you might have different preferences or opinions. The tutorial is intended to be beginner-friendly but also cover some advanced topics. In each section, you will automate some tasks and add badges to your project to show your progress and achievements.

The repository for this series can be found at github.com/johschmidt42/python-project-johannes

OS: Linux, Unix, macOS, Windows (WSL2 with e.g. Ubuntu 20.04 LTS)
Tools: python3.10, bash, git, tree
Version Control System (VCS) Host: GitHub
Continuous Integration (CI) Tool: GitHub Actions

It is expected that you are familiar with the versioning control system (VCS) git. If not, here’s a refresher for you: Introduction to Git

Commits will be based on best practices for git commits & Conventional commits. There is the conventional commit plugin for PyCharm or a VSCode Extension that help you to write commits in this format.

Overview

Structure

Testing framework (pytest)
Pytest configuration (pytest.ini_options)
Testing the application (fastAPI, httpx)
Coverage (pytest-coverage)
Coverage configuration (coverage.report)
CI (test.yml)
Badge (Testing)
Bonus (Report coverage in README.md)

Testing your code is a vital part of software development. It helps you ensure that your code works as expected. You can test your code or application manually or use a testing framework to automate the process. Automated tests can be of different types, such as unit tests, integration tests, end-to-end tests, penetration tests, etc. In this tutorial, we will focus on writing a simple unit test for our single function in our project. This will demonstrate that our codebase is well tested and reliable, which is a basic requirement for any proper project.

Python has some testing frameworks to choose from, such as the built-in standard library unittest. However, this module has some drawbacks, such as requiring boilerplate code, class-based tests and specific assert methods. A better alternative is pytest, which is a popular and powerful testing framework with many plugins. If you are not familiar with pytest, you should read this introductory tutorial before you continue, because we will write a simple test without explaining much of the basics.

So let’s get started by creating a new branch: feat/unit-tests

In our app src/example_app we only have two files that can be tested: __init__.py and app.py . The __init__ file contains just the version and the app.py our fastAPI application and the GET pokemon endpoint. We don’t need to test the __init__.py file because it only contains the version and it will be executed when we import app.py or any other file from our app.

We can create a tests folder in the project’s root and add the test file test_app.py so that it looks like this:

.
...
├── src
│   └── example_app
│       ├── __init__.py
│       └── app.py
└── tests
└── test_app.py

Before we add a test function with pytest, we need to install the testing framework first and add some configurations to make our lives a little easier:

Because the default visual output in the terminal leaves some room for improvement, I like to use the plugin pytest-sugar. This is completely optional, but if you like the visuals, give it a try. We install these dependencies to a new group that we call test. Again, as explained in the last part (part II), this is to separate app and dev dependencies.

Because pytest might not know where our tests are located, we can add this information to the pyproject.toml:

# pyproject.toml
...[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "-p no:cacheprovider" # deactivating pytest caching.

Where addopts stands for “add options” or “additional options” and the value -p no:cacheprovider tells pytest to not cache runs. Alternatively, we can create a pytest.ini and add these lines there.

Let’s continue with adding a test to the fastAPI endpoint that we created in app.py. Because we use httpx, we need to mock the response from the HTTP call (https://pokeapi.co/api). We could use monkeypatch or unittest.mock to change the behaviour of some functions or classes in httpx but there already exists a plugin that we can use: respx

Mock HTTPX with awesome request patterns and response side effects.

Furthermore, because fastAPI is an ASGI and not a WSGI, we need to write an async test, for which we can use the pytest plugin: pytest-asyncio together with trio . Don’t worry if these are new to you, they are just libraries for async Python and you don’t need to understand what they do.

> poetry add --group test respx pytest-asyncio trio

Let’s create our test in the test_app.py:

I won’t go into the details of how to create unit-tests with pytest, because this topic could cover a whole series of tutorials! But to summarise, I created an async test called test_get_pokemon in which the response will be the expected_response because we are using the respx_mock library. The endpoint of our fastAPI application is called and the result is compared to the expected result. If you want to find more about how to test with fastAPI and httpx, check out the official documentation: Testing in fastAPI

And if you have async functions, and don’t know how to deal with them, take a look at: Testing with async functions in fastAPI

Assuming that you installed your application with poetry install we now can run pytest with

> pytest

and pytest knows in which directory it needs to look for test files!

To make our linters happy, we should also run them on the newly created file. For this, we need to modify the command lint-mypy so that mypy also covers files in the tests directory (previously only src):

# Makefile...lint-mypy:
@mypy ....

At last, we can now run our formatters and linters before committing:

> make format
> make lint

Running formatters and linters — Image by author

The code coverage in a project is a good indicator of how much of the code is covered by unit tests. Hence, code coverage is a good metric (not always) to check if a particular codebase is well tested and reliable.

We can check our code coverage with the coverage module. It creates a coverage report and gives information about the lines that we missed with our unit-tests. We can install it via a pytest plugin pytest-cov:

> poetry add --group test pytest-cov

We can run the coverage module through pytest:

> pytest --cov=src --cov-report term-missing --cov-report=html

To only check the coverage for the src directory we add the flag --cov=src . We want the report to be displayed in the terminal --cov-report term-missing and stored in a html file with --cov-report html

Coverage report terminal — Image by author

We see that a coverage HTML report has been created in the directory htmlcov in which we find an index.html.

.
...
├── index.html
├── keybd_closed.png
├── keybd_open.png
├── status.json
└── style.css

Opening it in a browser allows us to visually see the lines that we covered with our tests:

Coverage report HTML (overview) — Image by author

Clicking on the link src/example_app/app.py we see a detailed view of what our unit-tests covered in the file and more importantly which lines they missed:

Coverage report HTML (detailed) — Image by author

We notice that the code under the if __name__ == "main": line is included in our coverage report. We can exclude this by setting the correct flag when running pytest, or better, add this configuration in our pyproject.toml:

# pyproject.toml
...[tool.coverage.report]
exclude_lines = [
'if __name__ == "__main__":'
]

The lines after the if __name__==”__main__" are now excluded*.

*It probably makes sense to include other common lines such as

def __repr__
def __str__
raise NotImplementedError
…

If we run pytest with the coverage module again

> pytest --cov=src --cov-report term-missing --cov-report=html

Coverage report HTML (excluded lines) — Image by author

the last line is not excluded as expected.

We have covered the basics of the coverage module, but there are more features that you can explore. You can read the official documentation to learn more about the options.

Let’s add these commands (pytest, coverage) to our Makefile, the same way we did in Part II, so that we don’t have to remember them. Additionally we add a command that uses the --cov-fail-under=80 flag. This signals pytest to fail if the total coverage is lower than 80 %. We will use this later in the CI part of this tutorial. Because the coverage report creates some files and directories within the project, we should also add a command that removes these for us (clean-up):

# Makefileunit-tests:
@pytestunit-tests-cov:
@pytest --cov=src --cov-report term-missing --cov-report=htmlunit-tests-cov-fail:
@pytest --cov=src --cov-report term-missing --cov-report=html --cov-fail-under=80clean-cov:
@rm -rf .coverage
@rm -rf htmlcov...

And now we can invoke these with

> make unit-tests
> make unit-tests-cov

and clean up the created files with

> make clean-cov

Once again, we use the software development practice CI to make sure that nothing is broken every time we commit to our default branch main.

Up until now, we were able to run our tests locally. So let us create our second workflow that will run on a server from GitHub! We have the option of using codecov.io in combination with the codecov-action OR we can create the report in the Pull Request (PR) itself with a pytest-comment action. I will choose the second option for simplicity.

We can either create a new workflow that runs parallel to our linter lint.yml (faster) or have one workflow that runs the linters first and then the testing job (more efficient). This is a design choice that depends on the project’s needs. Both options have pros and cons. For this tutorial, I will create a separate workflow (test.yml). But before we do that, we need to update our command in the Makefile, so that we create a pytest.xml and a pytest-coverage.txt, which are needed for the pytest-comment action:

# Makefile...unit-tests-cov-fail:
@pytest --cov=src --cov-report term-missing --cov-report=html --cov-fail-under=80 --junitxml=pytest.xml | tee pytest-coverage.txtclean-cov:
@rm -rf .coverage
@rm -rf htmlcov
   @rm -rf pytest.xml
@rm -rf pytest-coverage.txt...

Now we can write our workflow test.yml:

Let’s break it down to make sure we understand each part. GitHub action workflows must be created in the .github/workflows directory of the repository in the format of .yaml or .yml files. If you’re seeing these for the first time, you can check them out here to better understand them. In the upper part of the file, we give the workflow a name name: Testing and define on which signals/events, this workflow should be started: on: ... . Here, we want that it runs when new commits come into a PullRequest targeting the main branch or commits go the main branch directly. The job runs in an ubuntu-latest (runs-on) environment and executes the following steps:

checkout the repository using the branch name that is stored in the default environment variable ${{ github.head_ref }} . GitHub action: checkout@v3
install Poetry with pipx because it’s pre-installed on all GitHub runners. If you have a self-hosted runner in e.g. Azure, you’d need to install it yourself or use an existing GitHub action that does it for you.
Setup the python environment and caching the virtualenv based on the content in the poetry.lock file. GitHub action: setup-python@v4
Install the application & its requirements together with the test dependencies that are needed to run the tests with pytest: poetry install --with test
Running the tests with the make command: poetry run make unit-tests-cov-vail Please note, that running the tools is only possible in the virtualenv, which we can access through poetry run.
We use a GitHub action that allows us to automatically create a comment in the PR with the coverage report. GitHub action: pytest-coverage-comment@main

When we open a PR targeting the main branch, the CI pipeline will run and we will see a comment like this in our PR:

Pytest coverage report in PR comment — Image by author

It created a small badge with the total coverage percentage (81%) and has linked the tested files with URLs. With another commit in the same feature branch (PR), the same comment for the coverage report is overwritten by default.

To display the status of our new CI pipeline on the homepage of our repository, we can add a badge to the README.md file.

We can retrieve the badge when we click on a workflow run:

Create a status badge from workflow file on GitHub — Image by author

Copy the badge markdown — Image by author

and select the main branch. The badge markdown can be copied and added to the README.md:

Our landing page of the GitHub now looks like this ❤:

Second badge in README.md: Testing — Image by author

If you are curious about how this badge reflects the latest status of the pipeline run in the main branch, you can check out the statuses API on GitHub.