🌿 Exploring different concurrency paradigms in Python

Description

This repository contains links to some awesome blog posts, books, talks, and docs to get you up and running with Python's asynchronous paradigm. It tries to avoid resources that are outdated or contain deprecated APIs that make the async subspace of Python seem overly complicated, especially for beginners.

Concepts & philosophy

• Sync vs. async Python: what is the difference? -> Differences between sync and async Python, and when async is the right tool to solve your concurrency problems

• Why do we need coroutines in Python?

• Generators & coroutines - David Beazly -> This is outdated and uses python 2x but the underlying concept is still relevant

• Concurrency for people in a hurry - FastAPI doc -> Simple high-level overview of concurrency in general

• How the heck does async/await work in Python 3.5? -> Good intro to async but slightly outdated

• Unyielding - why threads are bad? -> Glyph, the creator of the Twisted projects argues why writing multi-threaded code correctly is hard

• Nathaniel J. Smith - Notes on structured concurrency, or: go statement considered harmful -> Nataniel Smith argues that Dijkstra's strurctured programming philosophy can show us the correct way of doing concurrent I/O programming

• Nathaniel J. Smith - Timeouts and cancellation for humans -> Nathaniel Smith ponders on the ways timeout and cancellation can be handled across different concurrency primitives

• Some thoughts on asynchronous API design in a post-async/await world -> Conundrums of crafting causality-compliant, coroutine-chauffeured concurrency-contraptions—catering callback compliance

Asyncio overview

• AsyncIO in Python: A complete walkthrough -> Start grasping the bare-minimum concepts from here.

• Asyncio for the working Python developer -> Getting your hands dirty with the async features.

• Calling sync from async & vice versa -> Fantastic article on the gradual adoption of Asyncio.

Asyncio API

• Guide to concurrency in Python with asyncio -> Asyncio's create_task, ensure_future, wait, wait_for, gather APIs.

• Waiting in asyncio -> Difference between gather, wait, wait_for, create_task, as_completed, etc.

• Latency in asynchronous Python -> Using asyncio.Queue(maxsize=1) to run blocking code.

• Difference between asyncio gather & wait

• Differences among asyncio ensure_future, create_task & simple coroutines

• Does asyncio gather guarantees execution & result order?

Examples

• Asynchronous I/O examples -> Self contained examples of how to perform a few common asynchronous I/O tasks

• Asyncio by examples -> A dense example-driven overview of the Asyncio APIs

• Simple IPv4 TCP scanner leveraging asyncio -> A complete example of using asynchronous producer-consumer pattern to create a simple TCP scanner

Books

• Using asyncio in Python - Caleb Hattingh

Tutorials

• Python threading tutorial: Run code concurrently using the 'threading' module -> Corey Schafer explains how to use the built-in threading module in Python to achieve concurrency

• Python multiprocessing tutorial: Run code parallelly using the 'multiprocessing' module -> Corey Schafer explains how to use the built-in multiprocessing module in Python to achieve parallelism

Talks

• Understanding async and await in Python - Jonathan Slenders -> A succinct introduction to asyncio in Python.

• Raymond Hettinger - Keynote on concurrency - PyBay 2017 -> Raymond Rettinger mostly talking about threads, queues, and processes.

• Robert Smallshire - Coroutine concurrency in Python 3 with asyncio -> Gradually building up the intuition for coroutine driven I/O.

• David Beazley - Fear and awaiting in async: A savage journey to the heart of the coroutine dream -> Reasoning asynchronous programming from first principles.

• Import asyncio - YouTube series -> Łukasz Langa's asyncio videos—from basics to application.

• Yury Selivanov - High performance networking in Python -> The talk covers the async/await syntax in Python, asyncio library and ecosystem around it, and ways to use them for creating high performance servers.

• Yury Selivanov - Async/await in Python 3.5 and why it is awesome -> An overview of asyncio's high-level APIs

• Yury Selivanov - Asyncio: what's next - PyBay2018 -> Another walkthrough of asyncio APIs and speculations on future improvements.

• Yury Selivanov - Asyncio in Python 3.7 and 3.8 || A guide to asynchronous programming in Python with asyncio -> Asyncio's past, present and future.

• Build your own async - YouTube screencast -> David Beazley's takes you through building your own asyncio-like library.

• John Reese - Thinking outside the GIL with asyncio and multiprocessing - PyCon 2018 -> Superpowered asyncio with multiprocessing.

• Nathaniel J. Smith - Trio: Async concurrency for mere mortals - PyCon 2018 -> Structured concurrency with trio.

• Kavya Joshi - A tale of concurrency through creativity in Python: A deep dive into how gevent works -> Implicit cooperative multitasking via Gevent.

• Understanding the Python GIL -> David Beazly explains the behavior of threads in Python why it yields concurrency rather than parallelism.

• The other async (threads + async = ❤️) -> David Beazley builds a thread and async compatible queue from scratch.

• Łukasz Langa – Thinking in coroutines -> Understanding event loop, task cancellation, and exception handling in Asyncio.

Podcasts

• John Reese - Asyncio all the things with Omnilib -> Making the fundamental toolsets async in Python.

• David Beazley - Python concurrency with Curio -> David Beazley explains the problems with Asyncio and how he built Curio by reasoning from the first principle.

• Nathaniel Smith - Simplifying Python's async with Trio -> Nathaniel Smith talks about why cancellation and exception handling can be difficult in background tasks, and how Trio attempts to solve that.

• Lukasz Langa - AsyncIO + music, origins of Black, and managing Python releases -> Lukasz Langa talks about analog synthesizers, AsyncIO and the Black code formatter.

Banters

• I don't understand asyncio -> Armin Ronacher complains about baffling complexities in the design decisions of the asyncio module.

• Hackernews discourse - I don't understand asyncio -> People Complaining About the Design Decisions behind the Asyncio API.

• Reddit discourse - I don't understand asyncio -> Armin Ronacher's conversation with a Python core developer.

• I'm not feeling the async pressure -> Armin Ronacher points out how Asyncio API overlooks the complexities imposed by backpressure.

• Hackernews discourse - I'm not feeling the async pressure -> Discussion on whether language implementation should deal with backpressure or not.

• Hackernews discourse - People comparing Python's coroutine driven concurrency with Golang's actor pattern -> People seems to hate asyncio's Twisted inspired design philosophy in general.

• Hackernews discourse - The bare minimum asyncio -> One Guy's attempt to cover the entire high-level API surface of asyncio in a single Hackernews comment.

Reactive programming & microservices

• Redis Streams Featuring Salvatore Sanfilippo - Redis Labs -> Introduction to the Redis streams data structure and how can it be used as a safer alternative to Pub-Sub.

Python enhancement proposals (PEPs)

• PEP 255 -- Simple generators
• PEP 342 -- Coroutines via enhanced generators
• PEP 380 -- Syntax for delegating to a subgenerator
• PEP 3156 -- Asynchronous i/o support rebooted: the "asyncio" module
• PEP 492 -- Coroutines with async and await syntax
• PEP 654 -- Exception groups and except*