Understand Context Managers in Python and Learn to Use Them in Unit Tests | by Lynn Kwong | Jul, 2022

By Jessie Hobb On Jul 5, 2022

Use context managers to make your code more robust and more Pythonic

Image by BUMIPUTRA in Pixabay

We have been using the with keyword with the open function to create a context that will close the file after we have finished reading/writing. However, context managers in Python are not limited to managing external resources like file descriptors, database connections, threading locks, etc, they can be used to provide all kinds of contexts which require some setup and teardown actions before and after the main code block is executed. In this post, we will demystify the context manager in Python by explaining all the technical mechanisms with simple code snippets. We will also introduce how to use mocks/patches in Python unit tests where context managers are heavily used.

Compare context manager with try/finally statement

When used to manage external resources, the with context manager can be seen as the syntax sugar for the try/finally statement. Both will close the external resource at the end no matter whether the main code is successful or not. The following code snippets using the with context manager and the try/finally statement are equivalent:

As we see, with the context manager, the code can be more concise.

Create a custom context manager

Let’s now create a custom context manager in order to understand the technical mechanism behind the scene.

To create a context manager, we can create a class that implements the __enter__() and __exit__() magic methods. __enter__() includes the setup code for the context and will be executed when the context is created. It can return a value that will be assigned to the variable in the with … as <var> statement. On the other hand, __exit__() includes the teardown code that will be executed when the context is exited, regardless of whether the main code block raises an exception or not. If an exception is raised in the main code block, the exception type, exception value, and traceback will be passed to exc_type, exc_value and exc_tb, respectively.

Now the custom context manager should work in the same way as the built-in open function:

Create a custom context manager using a generator function

Above we created a custom context manager using the classical class-based protocol which implements the __enter__() and __exit__() method. Similar to iterators, we can create a context manager using a generator function in a more concise way. To do this, we just need to decorate the generator function with the contextmanager decorator from the contextlib library:

Now we can use the generator function as a context manager:

As we see, the code before and after the yield statement corresponds to the __enter__() and __exit__() methods, respectively. The yielded value, if any, is bound to the variable in the with … as <var> statement. Note that if nothing is yielded, the yield statement should also be present otherwise it would not be a generator function. There will be an example for this case soon.

Mock a function with a context manager

With a context manager, we can change the behavior of a function in the context and reset it when it exits the context. This is actually the mechanism of how patching works in unit tests. Let’s mock the datetime.now() function with a context manager:

Keynotes:

We must assign the datetime.now function object to a new variable, otherwise, it cannot be assigned a new value to change its behavior in the context manager. Later you will see that we can use the special patch function from unittest.mock to mock any function/method very conveniently, with no need to create new intermediate variables.
A generator-based context manager is created here, you can also create a class-based one, which should be very straightforward. Just copy the code before and after the yield statement to the __enter__() and __exit__() methods, respectively.
In the generator function, we must use the global keyword to declare that now is a global variable, otherwise, it cannot be accessed there. It is saved to a new variable saved_now so it can be reset later.
The now variable is then assigned a new lambda function which always returns the same datetime object. In this way, the behavior of the now() function is changed in the context.
In this context manager, nothing is yielded but we still need to yield explicitly to make it a valid generator function.
As we already know, the code after the yield statement is the teardown part. We should reset now() back to the original datetime.now().

Let’s use this context manager to check the result of the now() function inside and outside the context:

We can see that the behavior of the now() function is mocked inside the context and reset back to normal when it exits the context.

Mock datetime.now() in unit tests

Most functions can be mocked directly. However, we cannot mock datetime.now() directly, otherwise, we will see the following TypeError:

We need to mock the datetime module instead. Be extremely careful of the code below, it’s very easy to make mistakes here:

Keynotes:

You need to install the pytest module to run the unit tests:

Since the datetime.now() function cannot be patched directly, we need to patch the datetime module instead. Note that we need to patch the datetime module imported in the current test file, not the one in the built-in datetime library. If you change test_mock_now.datetime to datetime.datetime, the test will fail. This is because, with this change, it is datetime.datetime that is mocked, not the datetime module imported in the current test file. Therefore, the assertions on lines 10 and 11 would fail. Do try it out by ourself so you can understand it better.

Mock requests.get() in unit tests

Actually, mocking is pretty simple in Python unit tests. It is just that the datetime.now() function from the datetime library is a bit special. Let’s patch the requests.get() function which is more commonly used in unit tests to mock the response from some HTTP requests:

This time the requests.get() function can be patched directly and the code is much simpler.

Mocking is very important in unit tests which can make your test code work independent of external resources and let you focus on the logic and robustness of the code. Almost everything can be mocked in unit tests. We have just introduced the mocking of functions in this post. More examples will be introduced in a more dedicated post for unit tests soon.

In this post, we have introduced the fundamentals of context managers and different ways to implement them in Python. A context manager can be implemented with either the class-based or generator-based protocol, with the latter being more concise. Context managers can make your code more robust because the resources are guaranteed to be closed. Besides, context managers can be used to change the behavior of some functions in the context which makes them widely used in unit tests because they can let your test code work independent of external resources.

Use context managers to make your code more robust and more Pythonic

Compare context manager with try/finally statement

As we see, with the context manager, the code can be more concise.

Create a custom context manager

Let’s now create a custom context manager in order to understand the technical mechanism behind the scene.

Now the custom context manager should work in the same way as the built-in open function:

Create a custom context manager using a generator function

Now we can use the generator function as a context manager:

Mock a function with a context manager

Keynotes:

We must assign the datetime.now function object to a new variable, otherwise, it cannot be assigned a new value to change its behavior in the context manager. Later you will see that we can use the special patch function from unittest.mock to mock any function/method very conveniently, with no need to create new intermediate variables.
A generator-based context manager is created here, you can also create a class-based one, which should be very straightforward. Just copy the code before and after the yield statement to the __enter__() and __exit__() methods, respectively.
In the generator function, we must use the global keyword to declare that now is a global variable, otherwise, it cannot be accessed there. It is saved to a new variable saved_now so it can be reset later.
The now variable is then assigned a new lambda function which always returns the same datetime object. In this way, the behavior of the now() function is changed in the context.
In this context manager, nothing is yielded but we still need to yield explicitly to make it a valid generator function.
As we already know, the code after the yield statement is the teardown part. We should reset now() back to the original datetime.now().

Let’s use this context manager to check the result of the now() function inside and outside the context:

We can see that the behavior of the now() function is mocked inside the context and reset back to normal when it exits the context.

Mock datetime.now() in unit tests

Most functions can be mocked directly. However, we cannot mock datetime.now() directly, otherwise, we will see the following TypeError:

We need to mock the datetime module instead. Be extremely careful of the code below, it’s very easy to make mistakes here:

Keynotes:

You need to install the pytest module to run the unit tests:

Since the datetime.now() function cannot be patched directly, we need to patch the datetime module instead. Note that we need to patch the datetime module imported in the current test file, not the one in the built-in datetime library. If you change test_mock_now.datetime to datetime.datetime, the test will fail. This is because, with this change, it is datetime.datetime that is mocked, not the datetime module imported in the current test file. Therefore, the assertions on lines 10 and 11 would fail. Do try it out by ourself so you can understand it better.

Mock requests.get() in unit tests

This time the requests.get() function can be patched directly and the code is much simpler.

Read original article here

Denial of responsibility! Techno Blender is an automatic aggregator of the all world’s media. In each content, the hyperlink to the primary source is specified. All trademarks belong to their rightful owners, all materials to their authors. If you are the owner of the content and do not want us to publish your materials, please contact us by email – [email protected]. The content will be deleted within 24 hours.