Python AsyncIO Explained in 9 Minutes

Key Concepts

• Asynchronous Programming (Async IO): A concurrent programming paradigm where a single thread manages multiple tasks by switching between them during idle periods (e.g., waiting for I/O).
• Co-routine: A program component that can be suspended and resumed, allowing for non-blocking execution. Defined using async def.
• Event Loop: The core of Async IO, responsible for scheduling and executing co-routines. It switches between tasks when they are waiting (e.g., for I/O).
• await: A keyword used to yield control back to the event loop, allowing other co-routines to execute while the current one is waiting.
• Concurrency vs. Parallelism: Concurrency is managing multiple tasks at the same time, while parallelism is executing multiple tasks simultaneously (typically using multiple cores/processes).
• asyncio.gather(): A function that runs multiple co-routines concurrently and returns their results as a list.
• asyncio.create_task(): A function that schedules a co-routine to run in the background as a task.
• asyncio.wait(): A function that waits for multiple tasks to complete, with the option to specify a return condition (e.g., asyncio.FIRST_COMPLETED).
• asyncio.wait_for(): A function that waits for a single task to complete, with a timeout.
• Global Interpreter Lock (GIL): A mechanism in CPython that allows only one thread to hold control of the Python interpreter at any one time.

1. Introduction to Asynchronous Programming

• Asynchronous programming is a form of concurrent programming in Python, alongside multi-threading and multipprocessing.
• Multipprocessing is suitable for CPU-bound tasks (heavy computations) to leverage multiple CPU cores.
• Multi-threading is used when manual control over thread switching is not desired, or when working with libraries that don't support asynchronous programming.
• The video focuses on asynchronous programming using Async IO.

2. Core Concepts and Implementation

• Co-routines: Defined using async def, they can be suspended and resumed.
  • Example:

    async def io_task(name, delay, iterations):
        for i in range(iterations):
            print(f"Task {name}, iteration {i}")
            await asyncio.sleep(delay)
    

  • The await asyncio.sleep(delay) line is crucial; it yields control to the event loop.
• Event Loop: Manages the execution of co-routines within a single thread.
  • It switches between co-routines when they are waiting (e.g., for I/O).
  • No parallel execution; concurrency is achieved by rapidly switching between tasks.
• asyncio.gather(): Executes multiple co-routines concurrently.
  • Example:

    async def main():
        start = time.time()
        await asyncio.gather(
            io_task("A", 1.5, 3),
            io_task("B", 1, 3),
            io_task("C", 0.5, 3)
        )
        end = time.time()
        print(f"Total time: {end - start:.2f} seconds")
    

  • Tasks A, B, and C run concurrently, taking approximately 4.5 seconds.
• Serial Execution: Awaiting tasks sequentially results in longer execution times.
  • Example:

    async def main():
        start = time.time()
        await io_task("A", 1.5, 3)
        await io_task("B", 1, 3)
        await io_task("C", 0.5, 3)
        end = time.time()
        print(f"Total time: {end - start:.2f} seconds")
    

  • Tasks A, B, and C run one after another, taking approximately 11 seconds.
• Importance of await: Using time.sleep() instead of await asyncio.sleep() blocks the thread, preventing the event loop from switching to other tasks.
  • await is essential for yielding control back to the event loop.

3. Background Tasks and asyncio.create_task()

• asyncio.create_task(): Runs a co-routine in the background.
  • Example:

    async def background_task():
        print("Running")
        await asyncio.sleep(5)
        print("Finishing")
        return "Done"
    
    async def main():
        task = asyncio.create_task(background_task())
        print("Continuing immediately")
        await task
        print("But for this we need to wait")
    

  • The "Continuing immediately" message is printed before "Running" because the task is started in the background.
  • await task ensures the main function waits for the background task to complete before proceeding.

4. asyncio.wait() and Return Conditions

• asyncio.wait(): Waits for multiple tasks to complete, with the option to specify a return condition.
  • Example:

    async def task_one():
        print("One start")
        await asyncio.sleep(2)
        print("One end")
        return 1
    
    async def task_two():
        print("Two start")
        await asyncio.sleep(5)
        print("Two end")
        return 2
    
    async def main():
        task1 = asyncio.create_task(task_one())
        task2 = asyncio.create_task(task_two())
    
        done, pending = await asyncio.wait(
            [task1, task2],
            return_when=asyncio.FIRST_COMPLETED
        )
    
        print("Finished tasks:", done)
        print("Pending tasks:", pending)
    
        await asyncio.wait(pending)
    

  • return_when=asyncio.FIRST_COMPLETED makes the wait function return when the first task completes.
  • The function returns two sets: done (completed tasks) and pending (incomplete tasks).
  • The remaining pending tasks can be awaited later.

5. asyncio.wait_for() and Timeouts

• asyncio.wait_for(): Waits for a single task to complete, with a timeout.
  • Example:

    async def long_operation():
        print("One")
        await asyncio.sleep(5)
        print("Two")
    
    async def main():
        try:
            await asyncio.wait_for(long_operation(), timeout=2)
        except asyncio.TimeoutError:
            print("Took too long")
    

  • If the task takes longer than the specified timeout, an asyncio.TimeoutError is raised.
  • The try...except block handles the timeout error.

6. Running the Event Loop

• asyncio.run(main()): Starts the event loop and runs the main() co-routine.
  • The main function is called directly within asyncio.run().

7. Conclusion

• Asynchronous programming with Async IO allows for concurrent execution of tasks within a single thread.
• The await keyword is crucial for yielding control to the event loop.
• asyncio.gather(), asyncio.create_task(), asyncio.wait(), and asyncio.wait_for() provide different ways to manage and control asynchronous tasks.
• The video provides a quick introduction, and further exploration of topics like manual event loop control, task groups, and shielding is recommended.