Concurrency & parallelism — the event loop you already live in

01Concurrency ≠ parallelism

Concurrency = dealing with many things at once — structuring a program so multiple tasks are in flight, interleaved. One barista juggling 5 orders. Parallelism = doing many things at once — literally simultaneous execution on multiple cores. Five baristas.

 Concurrency (1 core, interleaved):     Parallelism (2 cores, simultaneous):
 Task A: ██──██────██                   Core 1: ████████ (Task A)
 Task B: ──██──████──                   Core 2: ████████ (Task B)
   time ─────────────►                    time ─────────►

Rob Pike's line: "Concurrency is about structure, parallelism is about execution." An asyncio server on one core is highly concurrent, zero parallel. A NumPy matrix multiply on 8 cores is parallel with no concurrency structure. Your prod stack does both: N uvicorn worker processes (parallelism), each running an event loop juggling hundreds of requests (concurrency).

02Race conditions

A race condition: correctness depends on the timing/interleaving of concurrent operations. The canonical bug:

counter += 1     # looks atomic. Is actually: LOAD counter → ADD 1 → STORE

 Thread A              Thread B            counter
 LOAD  (reads 5)                              5
                       LOAD  (reads 5)        5
 ADD → 6                                      5
                       ADD → 6                5
 STORE 6                                      6
                       STORE 6                6   ← two increments, +1. Lost update.

The bad interleaving might hit 1-in-a-million runs — which at 2M req/month means it's guaranteed in production and unreproducible on your laptop. That's what makes races evil: they're timing-dependent Heisenbugs.

The general term is a critical section: a chunk of code that must not be interleaved with itself. Fixes: make it atomic (hardware atomics, INCR in Redis, UPDATE ... SET x = x + 1 in Postgres) or mutually exclude (locks).

03Locks, mutexes, semaphores

You use the semaphore shape constantly: a DB connection pool of size 20 is semantically a semaphore(20). asyncio.Semaphore(10) to cap concurrent calls to a rate-limited LLM API is the textbook use.

Honorable mentions: RWLock (many readers OR one writer), spinlock (busy-wait instead of sleeping — kernels use them for tiny sections), atomic ops / CAS (lock-free single-variable updates). Under the hood on Linux, a contended lock parks the thread via the futex syscall; an uncontended acquire is just an atomic compare-and-swap — cheap.

04Deadlock

Two or more tasks each holding a resource the other needs. Nobody moves, forever.

 Thread A: holds L1 ── wants L2 ──┐
                                  ▼
 Thread B: holds L2 ── wants L1 ──┘   (cycle → deadlock)

The four Coffman conditions (ALL required — memorize)

Prevention = break any one condition

• Break circular wait (the workhorse): impose a global lock ordering — always acquire L1 before L2, everywhere. No ordering violation → no cycle → no deadlock.
• Break hold and wait: acquire everything atomically up front, or release what you hold before waiting (try_lock, back off, retry).
• Break no preemption: timeouts. lock.acquire(timeout=5), statement_timeout / lock_timeout in Postgres.
• Detection + recovery: Postgres does this for you — it walks the waits-for graph, and on a cycle kills one transaction with deadlock detected. Classic prod cause: two transactions updating the same rows in opposite orders. Fix: touch rows in a consistent order (e.g., always by ascending id) — lock ordering, in SQL clothing.

05async/await and the event loop — deep, because FastAPI

Instead of one thread per request (blocked threads just burning stack + context switches while waiting on I/O), keep one thread and a loop:

                ┌─────────────────────────────┐
                │         EVENT LOOP          │
                │  ready queue: [t3, t7, ...] │
                └──────┬────────────▲─────────┘
    run task until it  │            │  I/O ready? put its
    awaits (yields)    ▼            │  waiting task back on queue
                ┌──────────────┐    │
                │ current task │    │
                └──────┬───────┘    │
                await db.fetch()    │
                       ▼            │
                ┌────────────────────────────┐
                │ epoll_wait(all sockets)    │  ← the OS multiplexer
                └────────────────────────────┘

One iteration of the loop:

1. Pop a ready task, run it. It executes real Python until it hits an await on something not yet finished.
2. The coroutine suspends — its state (locals, instruction pointer) is saved in the coroutine object itself. Coroutines are resumable stack frames; no OS involvement.
3. The loop registers "wake this task when FD X is readable / timer fires / that future resolves."
4. Nothing ready to run? Call epoll_wait() — sleep in the kernel until any watched socket has data.
5. I/O readiness arrives → the corresponding tasks go back on the ready queue. Repeat.

Key properties to say out loud:

• Cooperative scheduling. Tasks are never preempted; they run until they voluntarily await.
• await is the yield point. async def functions return coroutine objects; the loop drives them via the generator protocol (send/throw — async/await is literally built on generator machinery).
• Switching cost is ~a function call — nanoseconds. That's why one process can hold 10k+ concurrent requests.

One-thread concurrency still races (subtle, impressive point)

No parallelism ≠ no races. Interleaving happens at every await:

balance = await get_balance(user)      # task A reads 100... suspends
# ← another request for the same user runs here, also reads 100
await set_balance(user, balance - 10)  # both write 90. Lost update.

Anything spanning an await over shared state is a critical section → asyncio.Lock, or push atomicity to the DB. Between awaits, though, code is atomic w.r.t. other coroutines — a plain counter += 1 with no await inside is safe in asyncio (unlike threads).

06FastAPI specifics — the def vs async def rule

Tools worth naming: asyncio.gather() (fan out DB + cache + external API concurrently instead of sequential awaits — easy latency win), asyncio.Semaphore (cap concurrency), asyncio.wait_for (timeouts), TaskGroups (structured concurrency, 3.11+). Also uvloop: a libuv-based drop-in event loop (same engine family as Node's) that uvicorn uses for a meaningful speedup.

07Threads vs processes vs coroutines in Python

08Classic interview scenarios

8a · Thread-safe counter

import threading

class Counter:
    def __init__(self):
        self._value = 0
        self._lock = threading.Lock()

    def increment(self):
        with self._lock:          # context manager: releases even on exception
            self._value += 1      # read-modify-write, now atomic

Talking points: why += isn't atomic (3 bytecode ops, the GIL can switch between them); with guarantees release; alternatives — the itertools.count trick, atomics, or sharding the counter per-thread and summing to cut contention. Distributed version: Redis INCR or Postgres UPDATE ... SET n = n + 1 — atomic at the data store, no app lock at all.

8b · Producer–consumer

Producers generate work, consumers process it, a bounded queue decouples them and provides backpressure (a full queue blocks producers — the system self-regulates instead of OOMing).

 producers ──► [ bounded queue (size N) ] ──► consumers
   block when full ▲                ▲ block when empty
                   └── backpressure ┘

import threading, queue

q = queue.Queue(maxsize=100)          # thread-safe, condition-vars inside

def producer():
    for item in source():
        q.put(item)                   # blocks when full  → backpressure

def consumer():
    while True:
        item = q.get()                # blocks when empty
        process(item)
        q.task_done()

If asked to build it without queue.Queue, that's a condition-variable exercise:

class BoundedQueue:
    def __init__(self, cap):
        self.buf, self.cap = [], cap
        self.lock = threading.Lock()
        self.not_full  = threading.Condition(self.lock)
        self.not_empty = threading.Condition(self.lock)

    def put(self, item):
        with self.not_full:
            while len(self.buf) >= self.cap:   # WHILE, not if → spurious wakeups
                self.not_full.wait()
            self.buf.append(item)
            self.not_empty.notify()

    def get(self):
        with self.not_empty:
            while not self.buf:
                self.not_empty.wait()
            item = self.buf.pop(0)
            self.not_full.notify()
            return item

8c · Quick-fire scenarios to have an answer for

09Interview questions you should be able to answer

10Self-test

← Operating systems Networking →