GIL and Subinterpreters
Most Python concurrency arguments eventually collapse into this topic. Threads, async tasks, processes, subinterpreters, and free-threaded builds are not one-dimensional alternatives; they make different tradeoffs around shared state, isolation, and parallel bytecode execution.
Quick takeaway: in traditional CPython, the GIL limits parallel execution of Python bytecode inside one interpreter. Threads can still help I/O-bound workloads, but CPU-bound parallelism usually pushes you toward processes or newer interpreter-level options.
Map the Concurrency Choices
What the GIL Actually Restricts
- multiple threads running Python bytecode in parallel inside one interpreter
- many I/O operations and some C extensions can still release the GIL temporarily
- so the useful rule is not "threads are useless," but rather "pure-Python CPU parallelism is limited"
Why InterpreterPoolExecutor Matters
from concurrent.futures import InterpreterPoolExecutor
def square(value: int) -> int:
return value * value
with InterpreterPoolExecutor(max_workers=2) as pool:
print(list(pool.map(square, [1, 2, 3, 4])))Subinterpreters give you separate interpreter state within one process. They are more isolated than threads and can be lighter than full processes, but they are not a model for free shared mutable state.
When Each Tool Fits
| Tool | Good fit | Main caution |
|---|---|---|
threading | I/O-heavy work with shared state | pure-Python CPU scaling is limited |
asyncio | I/O multiplexing and structured concurrency | CPU work still needs offload |
multiprocessing | strong isolation and CPU parallelism | process and IPC overhead |
| subinterpreters | lighter-than-process isolation inside one process | sharing model and library support still matter |
How to Read Free-Threaded Builds
- they are not the default distribution model today
- ecosystem compatibility and performance tradeoffs still matter
- the important signal is that CPython's concurrency model is evolving
Practical Connections
- when threads are fine
- when processes are better
- what to expect from
InterpreterPoolExecutor
Checklist
Do you need shared state?
Threads are easiest for sharing. Processes and subinterpreters trade sharing for stronger isolation.
Is the workload CPU-bound or I/O-bound?
This should be your first cut before picking a concurrency strategy.
Did you check library compatibility?
Especially for subinterpreters and free-threaded builds, third-party library support matters a lot.
Did you price the boundary cost?
Processes and interpreter boundaries add serialization, communication, and state-management costs.