Why Taskgroup and timeout Are so Crucial in Python 3.11 Asyncio | Structured Concurrency
New features of the Python 3.11 asyncio package
1. Introduction
For every data scientist, improving Python code’s efficiency is essential. Two ideas can help us achieve this goal:
Due to the existence of GIL in Python, multi-threading has never been as efficient as expected, and each thread switch needs to compete for GIL locks leading to a severe waste of resources.
So, starting with version 3.4, Python introduced a new approach to concurrent programming: coroutine. Without GIL, the coroutine puts the timing of background task switches back into the hands of the programmer, and is, therefore, very efficient.
However, this mechanism also introduces new problems:
- Many newcomers don’t know they need to use create_task to convert async methods into background execution and often directly await func() to turn a concurrent process into synchronous execution.
- It isn’t easy to manage started tasks. Worse, the lack of a valid reference to the background task often results in the task being garbage collected before it has finished executing.
- Breaks the abstraction of codes. Programmers need to know which tasks are started by a particular package they call, which tasks have been processed within the package, and which tasks need to be processed manually after the call.
- The execution of tasks is asynchronous, resulting in the inability to catch and handle exceptions thrown by tasks.
- Not knowing when background tasks will finish executing often leads to timing issues.
In last year’s Python 3.11 release, the asyncio package added the TaskGroup and timeout APIs. These two APIs introduced the official Structured Concurrency feature to help us better manage the life cycle of concurrent tasks. Today, I’ll introduce you to using these two APIs and the significant improvements Python has brought to our concurrent programming with the introduction of Structured Concurrency.
2. TaskGroup
TaskGroup is created using an asynchronous context manager, and concurrent tasks can be added to the group by the method create_task, with the following code example:
When the context manager exits, it waits for all tasks in the group to complete. While waiting, we can still add new tasks to TaskGroup.
Note that assuming that a task in the group throws an exception other than asyncio.CancelledError while waiting, all other tasks in the group will be canceled.
Also, all exceptions were thrown except for asyncio.CanceledError will be combined and thrown in the ExceptionGroup.
3. timeout
asyncio.timeout is also created using the asynchronous context manager. It limits the execution time of concurrent code in a context.
Let’s assume that if we need to set a timeout to a single function call, it is sufficient to call asyncio.wait_for:
But when it is necessary to set a uniform timeout for multiple concurrent calls, things will become problematic. Let’s assume we have two concurrent tasks and want them to run to completion in 8 seconds. Let’s try to assign an average timeout of 4 seconds to each task, with code like the following:
You can see that although we set an average timeout for each concurrent method, such a setting may cause uncontrollable situations since each call to the IO-bound task is not guaranteed to return simultaneously and we still got a TimeoutError.
At this point, we use the asyncio.timeout block to ensure that we set an overall timeout for all concurrent tasks:
4. What is Structured Concurrency
TaskGroup and asyncio.timeout above uses the async with feature. Just like the with struct block can manage the life cycle of resources uniformly, like this:
But calling concurrent tasks inside with block does not work, because the concurrent task will continue executing in the background while the with block has already exited, which will lead to improper closure of the resource:
Therefore, we introduced the async with feature here. As with, async with and TaskGroup is used to manage the life cycle of concurrent code uniformly, thus making the code clear and saving development time. We call this feature our main character today: Structured Concurrency.
Why Structured Concurrency is so important
1. History of concurrent programming
Before the advent of concurrent programming, We executed our code serially. Code would execute for_loop loops, if_else conditional branches, and function calls sequentially, depending on the order in the call stack.
However, as the speed of code execution became more and more demanding in terms of computational efficiency, and as computer hardware developed significantly, parallel programming (CPU bound) and concurrent programming (IO bound) gradually emerged.
Before coroutine emerged, Python programmers used threading to implement concurrent programming. But Python’s threads have a problem, that is, GIL (Global Interpreter Lock), the existence of GIL makes the thread-based Concurrency has been unable to achieve the desired performance.
So asyncio coroutine emerged. Without GIL and inter-thread switching, concurrent execution is much more efficient. If threads are time-slice-based task switching controlled by the CPU, then coroutine is the creation and switching of subtasks back into the hands of the programmer himself. While programmers enjoy convenience, they also encounter a new set of problems.
2. Problems with the Concurrent Programming Model
As detailed in this article, concurrent programming raises several issues regarding control flow.
Concurrent programming is opening up multiple branch processes in our main thread. These branch tasks silently perform network requests, file accesses, database queries, and other duties in the background.
Concurrent programming will change the flow of our code from this to this:
According to the “low coupling, high cohesion” rule of programming, we all want to join all the background tasks in a module together after execution like this:
But the fact is that since multiple members develop our application or call numerous third-party components, we need to know which tasks are still executing in the background and which tasks are finished. It’s more likely that one background task will branch into several other branch tasks.
Ultimately, these branching tasks need to be found by the caller and wait for their execution to complete, so it becomes like this:
Although this is not Marvel’s multiverse, the situation is now just like the multiverse, bringing absolute chaos to our natural world.
Some readers might say that asyncio.gather could be responsible for joining all the background tasks. But asyncio.gather has its problems:
- It cannot centrally manage backend tasks in a unified way. Often creating backend tasks in one place and calling asyncio.gather in another.
- The argument aws received by asyncio.gather is a fixed list, which means that we have set the number of background tasks when asyncio.gather is called, and cannot be added randomly on the way to waiting.
- When a task is waiting in asyncio.gather throws an exception, it cannot cancel other tasks that are executing, which may cause some tasks to run indefinitely in the background and the program to falsely die.
Therefore, the Structured Concurrency feature introduced in Python 3.11 is an excellent solution to our concurrency problems. It allows the related asynchronous code to all finish executing in the same place, and at the same time, it will enable tg instances to be passed as arguments to background tasks, so that new background tasks created in the background tasks will not jump out of the current life cycle management of the asynchronous context.
Thus, Structured Concurrency is a revolutionary improvement to Python asyncio.
Comparison with other libraries that implement Structured Concurrency
Structured Concurrency is not the first of its kind in Python 3.11; we had several concurrency-based packages that implemented this feature nicely before 3.11.
1. Nurseries in Trio
Trio was the first library to propose Structure Concurrency in the Python world, and in Trio, the API open_nursery is used to achieve the goal:
2. create_task_group in Anyio
But with the advent of the official Python asyncio package, more and more third-party packages are using asyncio to implement concurrent programming. At this point, using Trio will inevitably run into compatibility problems.
At this point, Anyio, which claims to be compatible with both asyncio and Trio, emerged. It can also implement Structured Concurrency through the create_task_group API:
3. Using quattro in low Python versions
If you want to keep your code native to Python to easily enjoy the convenience of Python 3.11 asyncio in the future, there is a good alternative, quattro, which has fewer stars and is risk-averse.
Conclusion
The TaskGroup and timeout APIs introduced in Python 3.11 bring us the official Structured Concurrency feature.
With Structured Concurrency, we can make concurrent programming code better abstracted, and programmers can more easily control the life cycle of background tasks, thus improving programming efficiency and avoiding errors.
Because of limited experience, if there are any omissions in this article about concurrent programming or Structured Concurrency, or if you have better suggestions, please leave a comment. I will be grateful to answer you.
New features of the Python 3.11 asyncio package
1. Introduction
For every data scientist, improving Python code’s efficiency is essential. Two ideas can help us achieve this goal:
Due to the existence of GIL in Python, multi-threading has never been as efficient as expected, and each thread switch needs to compete for GIL locks leading to a severe waste of resources.
So, starting with version 3.4, Python introduced a new approach to concurrent programming: coroutine. Without GIL, the coroutine puts the timing of background task switches back into the hands of the programmer, and is, therefore, very efficient.
However, this mechanism also introduces new problems:
- Many newcomers don’t know they need to use create_task to convert async methods into background execution and often directly await func() to turn a concurrent process into synchronous execution.
- It isn’t easy to manage started tasks. Worse, the lack of a valid reference to the background task often results in the task being garbage collected before it has finished executing.
- Breaks the abstraction of codes. Programmers need to know which tasks are started by a particular package they call, which tasks have been processed within the package, and which tasks need to be processed manually after the call.
- The execution of tasks is asynchronous, resulting in the inability to catch and handle exceptions thrown by tasks.
- Not knowing when background tasks will finish executing often leads to timing issues.
In last year’s Python 3.11 release, the asyncio package added the TaskGroup and timeout APIs. These two APIs introduced the official Structured Concurrency feature to help us better manage the life cycle of concurrent tasks. Today, I’ll introduce you to using these two APIs and the significant improvements Python has brought to our concurrent programming with the introduction of Structured Concurrency.
2. TaskGroup
TaskGroup is created using an asynchronous context manager, and concurrent tasks can be added to the group by the method create_task, with the following code example:
When the context manager exits, it waits for all tasks in the group to complete. While waiting, we can still add new tasks to TaskGroup.
Note that assuming that a task in the group throws an exception other than asyncio.CancelledError while waiting, all other tasks in the group will be canceled.
Also, all exceptions were thrown except for asyncio.CanceledError will be combined and thrown in the ExceptionGroup.
3. timeout
asyncio.timeout is also created using the asynchronous context manager. It limits the execution time of concurrent code in a context.
Let’s assume that if we need to set a timeout to a single function call, it is sufficient to call asyncio.wait_for:
But when it is necessary to set a uniform timeout for multiple concurrent calls, things will become problematic. Let’s assume we have two concurrent tasks and want them to run to completion in 8 seconds. Let’s try to assign an average timeout of 4 seconds to each task, with code like the following:
You can see that although we set an average timeout for each concurrent method, such a setting may cause uncontrollable situations since each call to the IO-bound task is not guaranteed to return simultaneously and we still got a TimeoutError.
At this point, we use the asyncio.timeout block to ensure that we set an overall timeout for all concurrent tasks:
4. What is Structured Concurrency
TaskGroup and asyncio.timeout above uses the async with feature. Just like the with struct block can manage the life cycle of resources uniformly, like this:
But calling concurrent tasks inside with block does not work, because the concurrent task will continue executing in the background while the with block has already exited, which will lead to improper closure of the resource:
Therefore, we introduced the async with feature here. As with, async with and TaskGroup is used to manage the life cycle of concurrent code uniformly, thus making the code clear and saving development time. We call this feature our main character today: Structured Concurrency.
Why Structured Concurrency is so important
1. History of concurrent programming
Before the advent of concurrent programming, We executed our code serially. Code would execute for_loop loops, if_else conditional branches, and function calls sequentially, depending on the order in the call stack.
However, as the speed of code execution became more and more demanding in terms of computational efficiency, and as computer hardware developed significantly, parallel programming (CPU bound) and concurrent programming (IO bound) gradually emerged.
Before coroutine emerged, Python programmers used threading to implement concurrent programming. But Python’s threads have a problem, that is, GIL (Global Interpreter Lock), the existence of GIL makes the thread-based Concurrency has been unable to achieve the desired performance.
So asyncio coroutine emerged. Without GIL and inter-thread switching, concurrent execution is much more efficient. If threads are time-slice-based task switching controlled by the CPU, then coroutine is the creation and switching of subtasks back into the hands of the programmer himself. While programmers enjoy convenience, they also encounter a new set of problems.
2. Problems with the Concurrent Programming Model
As detailed in this article, concurrent programming raises several issues regarding control flow.
Concurrent programming is opening up multiple branch processes in our main thread. These branch tasks silently perform network requests, file accesses, database queries, and other duties in the background.
Concurrent programming will change the flow of our code from this to this:
According to the “low coupling, high cohesion” rule of programming, we all want to join all the background tasks in a module together after execution like this:
But the fact is that since multiple members develop our application or call numerous third-party components, we need to know which tasks are still executing in the background and which tasks are finished. It’s more likely that one background task will branch into several other branch tasks.
Ultimately, these branching tasks need to be found by the caller and wait for their execution to complete, so it becomes like this:
Although this is not Marvel’s multiverse, the situation is now just like the multiverse, bringing absolute chaos to our natural world.
Some readers might say that asyncio.gather could be responsible for joining all the background tasks. But asyncio.gather has its problems:
- It cannot centrally manage backend tasks in a unified way. Often creating backend tasks in one place and calling asyncio.gather in another.
- The argument aws received by asyncio.gather is a fixed list, which means that we have set the number of background tasks when asyncio.gather is called, and cannot be added randomly on the way to waiting.
- When a task is waiting in asyncio.gather throws an exception, it cannot cancel other tasks that are executing, which may cause some tasks to run indefinitely in the background and the program to falsely die.
Therefore, the Structured Concurrency feature introduced in Python 3.11 is an excellent solution to our concurrency problems. It allows the related asynchronous code to all finish executing in the same place, and at the same time, it will enable tg instances to be passed as arguments to background tasks, so that new background tasks created in the background tasks will not jump out of the current life cycle management of the asynchronous context.
Thus, Structured Concurrency is a revolutionary improvement to Python asyncio.
Comparison with other libraries that implement Structured Concurrency
Structured Concurrency is not the first of its kind in Python 3.11; we had several concurrency-based packages that implemented this feature nicely before 3.11.
1. Nurseries in Trio
Trio was the first library to propose Structure Concurrency in the Python world, and in Trio, the API open_nursery is used to achieve the goal:
2. create_task_group in Anyio
But with the advent of the official Python asyncio package, more and more third-party packages are using asyncio to implement concurrent programming. At this point, using Trio will inevitably run into compatibility problems.
At this point, Anyio, which claims to be compatible with both asyncio and Trio, emerged. It can also implement Structured Concurrency through the create_task_group API:
3. Using quattro in low Python versions
If you want to keep your code native to Python to easily enjoy the convenience of Python 3.11 asyncio in the future, there is a good alternative, quattro, which has fewer stars and is risk-averse.
Conclusion
The TaskGroup and timeout APIs introduced in Python 3.11 bring us the official Structured Concurrency feature.
With Structured Concurrency, we can make concurrent programming code better abstracted, and programmers can more easily control the life cycle of background tasks, thus improving programming efficiency and avoiding errors.
Because of limited experience, if there are any omissions in this article about concurrent programming or Structured Concurrency, or if you have better suggestions, please leave a comment. I will be grateful to answer you.
Denial of responsibility! Techno Blender is an automatic aggregator of the all world’s media. In each content, the hyperlink to the primary source is specified. All trademarks belong to their rightful owners, all materials to their authors. If you are the owner of the content and do not want us to publish your materials, please contact us by email – [email protected]. The content will be deleted within 24 hours.