How code actually runs — Python, V8, and the profiling mindset

01Compilers vs interpreters — it's a spectrum, not a binary

Compiler: translate the whole program to a lower-level form ahead of time (AOT), run the result. Interpreter: read program forms and execute their effects now. Every real system sits somewhere between these poles.

The classic pipeline is the same either way — only the target differs:

 source ─► lexer ─► parser ─► AST ─► [optimizer] ─► target
 (text)   (tokens)          (tree)                  │
                                        ┌───────────┼─────────────┐
                                        ▼           ▼             ▼
                                   machine code  bytecode     walk the AST
                                   (C, Rust, Go  (Python,     (simple interps,
                                    — AOT)        Java→JVM)    early Ruby)
                                                  for a VM

Tradeoffs to articulate: AOT buys peak speed + no warmup + errors at build time. Interpretation buys instant iteration, portability, and runtime dynamism (eval, monkeypatching, REPLs). JIT buys near-native speed for dynamic languages at the cost of warmup and memory.

02What Python actually does

python app.py, step by step:

 app.py ──parse──► AST ──compile──► bytecode (co_code) ──cached──► __pycache__/app.cpython-312.pyc
                                        │
                                        ▼
                     CPython VM: a loop over bytecode instructions
                     (stack machine: push/pop operands; one giant
                      switch / computed-goto in ceval.c)

You can watch the bytecode yourself with dis:

import dis
def add(a, b): return a + b
dis.dis(add)

#   LOAD_FAST     a
#   LOAD_FAST     b
#   BINARY_OP     +
#   RETURN_VALUE

Walking it line by line, the way the VM does:

1. LOAD_FAST a — push the local variable a onto the value stack. "Fast" because locals live in a fixed-size array indexed by position, not a dict lookup.
2. LOAD_FAST b — push b. The stack now holds two PyObject pointers.
3. BINARY_OP + — pop both operands, check their types at runtime, find __add__ via the type object, call it, box the result as a new PyObject, adjust refcounts, push the result.
4. RETURN_VALUE — pop the result and hand it back to the caller's frame.

Worth knowing exists: .pyc files skip re-parsing (not re-executing — the bytecode still runs on the VM). Python 3.11+ added the specializing adaptive interpreter (bytecode rewrites itself for observed types — 25%+ faster); 3.13+ ships an experimental JIT and a free-threaded build. PyPy is the long-standing JIT'd alternative Python.

03Garbage collection — Python's two-part scheme

Part 1: reference counting (the workhorse)

Every PyObject carries a refcount. Bindings increment; unbinding / going out of scope decrements; hit zero → freed immediately.

a = [1, 2]     # list refcount = 1
b = a          # 2
del a          # 1
b = None       # 0 → freed RIGHT NOW (deterministic!)

Pros: prompt, deterministic reclamation; memory freed the instant it's unused; no long GC pauses. Cons: constant refcount churn on every assignment — this is the reason the GIL exists (refcount updates must be thread-safe; see the concurrency page) — and it has a fatal blind spot:

Part 2: the cycle collector (the safety net)

 a.ref = b ──►┐        Even with no outside references,
 b.ref = a ◄──┘        each keeps the other's refcount ≥ 1.
                       Refcounting alone leaks this forever.

The gc module's generational collector runs periodically, finds groups of objects only reachable from each other, and frees them. Generational = built on the empirical rule that most objects die young: new objects sit in gen0 (collected often); survivors promote to gen1, then gen2 (collected rarely). Collection triggers on allocation-count thresholds.

Practical angles: cycles are common in real code (parent↔child pointers; exceptions holding tracebacks holding frames); __del__ + cycles is historically cursed; weakref breaks cycles by design (caches, observer patterns). Long-lived "leaks" in Python services are usually unbounded containers — module-level dict caches, growing lists — rather than GC failures: the GC can't free what you still reference. Tools: gc.collect(), tracemalloc, heap profilers.

04The JIT idea — V8, for your Node work

A JIT (just-in-time) compiler observes the program running, then compiles the hot parts to optimized machine code using facts an AOT compiler could never assume.

 JS source ─► parse ─► Ignition (bytecode interpreter) — everything starts here
                          │  profiling: which functions are hot?
                          │  what SHAPES/types actually flow through?
                          ▼
              hot code ─► optimizing JIT (TurboFan; Sparkplug/Maglev mid-tiers)
                          compiles assuming observed types:
                          "x was always a small int → emit raw machine add"
                          + guard checks
                          │
                          ▼ guard fails (a string shows up!)
                        DEOPTIMIZE → fall back to bytecode, maybe re-optimize

V8's tiered pipeline, tier by tier:

Two ideas carry the whole thing:

• Speculative optimization + deopt: compile for what you've seen, guard the assumption, bail out if the world disagrees. This is how a dynamic language reaches near-native speed.
• Hidden classes (shapes) + inline caches: objects created with the same property layout share a "shape," so obj.x becomes a fixed-offset load (like a C struct) instead of a hash lookup — if call sites stay monomorphic (one shape). Polymorphic sites get slower; megamorphic ones fall back to dictionary mode.

05Linking & loading — intuition level

How does source become a running process?

• Compile each translation unit to an object file: machine code + a symbol table + "holes" for symbols defined elsewhere.
• Link: resolve symbols across object files and libraries; complain about undefined reference. Two flavors:
  • Static — copy library code into the binary. Big, self-contained (Go's default; why Go containers can be FROM scratch).
  • Dynamic — record "needs libssl.so.3"; resolution deferred to load time. Small binaries, shared pages of library code across processes, system-wide security patching — at the cost of runtime dependencies.
• Load (execve): kernel + dynamic loader (ld.so) mmap the executable and its .so dependencies into the address space (the OS page's virtual memory), fix up addresses (relocations; lazy PLT binding), jump to _start → main.

Where you actually feel this:

• "Works on my machine" / glibc-version crashes in containers = dynamic linking: the binary expected a shared library the image lacks. Alpine (musl) vs Debian (glibc) wheels; ldd <binary> to inspect.
• Python import is the same idea at runtime: locate module → execute it once → cache in sys.modules. Native extensions (.so files — numpy, psycopg2's C parts) load via dlopen. Import-time work is real startup cost (cold starts, worker boot).
• Node's require/ESM: same resolve-execute-cache pattern; native addons are .node dynamic libraries.

06"It's slow" usually isn't the language — the profiling mindset

The reasoning chain to internalize (and recite):

The narrowing toolkit, by layer:

The mature closing take: language speed does matter for tight CPU loops at scale — that's why the ecosystem's hot paths (tokenizers, JSON parsers, numpy, uvloop, pydantic-core) are C/Rust under Python APIs. The skill isn't "use a fast language," it's knowing which 2% of the system is compute-bound and giving only that the fast-path treatment.

Test yourself

07Interview questions you should be able to answer