Onboarder¶
This guide walks through tox’s internal architecture — how the pieces fit together, what each subsystem does, and where to look when contributing. It is aimed at first-time contributors and new team members.
How to use this guide:
Part 1 gives you the big picture and tells you where to start reading code.
Part 2 follows tox’s execution flow from startup to completion.
Part 3 covers supporting systems you’ll need when working on specific features.
Part 4 contains reference materials for looking up details.
Part 1: Get Oriented¶
Quick facts¶
Repository layout¶
tox/
├── src/tox/ # main package
│ ├── run.py # entry point
│ ├── provision.py # self-provisioning logic
│ ├── report.py # logging / colored output
│ ├── config/ # configuration subsystem
│ │ ├── cli/ # CLI parser (two-phase)
│ │ ├── loader/ # INI / TOML / memory loaders
│ │ └── source/ # config file discovery
│ ├── session/ # runtime state + subcommands
│ │ └── cmd/ # built-in subcommands
│ ├── tox_env/ # environment classes
│ │ └── python/ # Python-specific envs
│ │ └── virtual_env/ # virtualenv-based concrete envs
│ ├── execute/ # subprocess execution
│ ├── plugin/ # pluggy integration
│ ├── journal/ # JSON result journal
│ └── util/ # helpers (graph, cpu, path, …)
├── tests/ # mirrors src/tox/ structure
└── docs/ # Sphinx documentation
Start here: Recommended reading order¶
For a first-time contributor, read the code in this order:
src/tox/run.py — entry point (~50 lines)
src/tox/session/state.py — state construction
src/tox/provision.py — provisioning logic
src/tox/session/env_select.py — env discovery
src/tox/tox_env/api.py — base environment
src/tox/execute/api.py — execution framework
src/tox/config/main.py — configuration
The big picture: End-to-end flow¶
This traces tox run -e py311 from start to finish, showing the six main phases:
flowchart LR
subgraph ENTRY["1️⃣ ENTRY POINT"]
direction TB
A[tox run -e py311]
B[Parse CLI]
C[Load config]
D[Build State]
A --> B --> C --> D
end
subgraph PROV["2️⃣ PROVISIONING"]
direction TB
E{Requirements<br/>satisfied?}
F[Create provision env]
G[Re-invoke tox]
E -->|No| F --> G
end
subgraph DISC["3️⃣ DISCOVERY"]
direction TB
H[Dispatch subcommand]
I[Discover envs]
J[Resolve dependencies]
H --> I --> J
end
subgraph SETUP["4️⃣ ENVIRONMENT SETUP"]
direction TB
K{Virtualenv<br/>exists?}
L[Create virtualenv]
M[Install dependencies]
N[Build & install package]
O[Run extra_setup_commands]
K -->|No| L --> M
K -->|Yes| M
M --> N --> O
end
subgraph EXEC["5️⃣ EXECUTION"]
direction TB
P[Run commands_pre]
Q[Run commands]
R[Run commands_post]
P --> Q --> R
end
subgraph RESULT["6️⃣ RESULTS"]
direction TB
S[Collect outcomes]
T[Write journal]
U[Report summary]
S --> T --> U
end
ENTRY --> PROV
G -.Re-invoke.-> ENTRY
PROV -->|Yes| DISC
DISC --> SETUP
SETUP --> EXEC
EXEC --> RESULT
style ENTRY fill:#E8F5E9,stroke:#4CAF50,stroke-width:2px
style PROV fill:#FFF3E0,stroke:#FF9800,stroke-width:2px
style DISC fill:#E3F2FD,stroke:#2196F3,stroke-width:2px
style SETUP fill:#F3E5F5,stroke:#9C27B0,stroke-width:2px
style EXEC fill:#E1F5FE,stroke:#03A9F4,stroke-width:2px
style RESULT fill:#FFEBEE,stroke:#F44336,stroke-width:2px
style A fill:#5CB85C,stroke:#3D8B40,color:#fff
style B fill:#5CB85C,stroke:#3D8B40,color:#fff
style C fill:#5CB85C,stroke:#3D8B40,color:#fff
style D fill:#5CB85C,stroke:#3D8B40,color:#fff
style E fill:#F0AD4E,stroke:#C88A3C,color:#fff
style F fill:#F0AD4E,stroke:#C88A3C,color:#fff
style G fill:#F0AD4E,stroke:#C88A3C,color:#fff
style H fill:#4A90E2,stroke:#2E5C8A,color:#fff
style I fill:#4A90E2,stroke:#2E5C8A,color:#fff
style J fill:#4A90E2,stroke:#2E5C8A,color:#fff
style K fill:#9B59B6,stroke:#6D3F8C,color:#fff
style L fill:#9B59B6,stroke:#6D3F8C,color:#fff
style M fill:#9B59B6,stroke:#6D3F8C,color:#fff
style N fill:#9B59B6,stroke:#6D3F8C,color:#fff
style O fill:#9B59B6,stroke:#6D3F8C,color:#fff
style P fill:#3498DB,stroke:#2574A9,color:#fff
style Q fill:#3498DB,stroke:#2574A9,color:#fff
style R fill:#3498DB,stroke:#2574A9,color:#fff
style S fill:#E74C3C,stroke:#B23A2F,color:#fff
style T fill:#E74C3C,stroke:#B23A2F,color:#fff
style U fill:#E74C3C,stroke:#B23A2F,color:#fff
High-level architecture¶
The following diagram shows the main flow through tox’s subsystems:
flowchart TD
A[CLI / argv]
B[Two-Phase Parser<br/>cli/]
C[Config<br/>config/main.py<br/>Sources → Loaders → Types]
D[State<br/>session/state.py<br/>owns Config + EnvSelector]
E[Provisioning<br/>provision.py<br/>ensures requires/min_ver]
F[Subcommand Handler<br/>run / list / config / …]
G[EnvSelector<br/>env_select.py<br/>discovers + builds ToxEnv instances]
H[RunToxEnv<br/>commands, deps]
I[PackageToxEnv<br/>wheel / sdist]
J[Execute<br/>subprocess mgmt]
A --> B --> C --> D --> E --> F --> G
G --> H
G --> I
H --> J
style A fill:#E74C3C,stroke:#B23A2F,color:#fff
style B fill:#5CB85C,stroke:#3D8B40,color:#fff
style C fill:#4A90E2,stroke:#2E5C8A,color:#fff
style D fill:#9B59B6,stroke:#6D3F8C,color:#fff
style E fill:#F0AD4E,stroke:#C88A3C,color:#fff
style F fill:#E67E22,stroke:#B85E18,color:#fff
style G fill:#3498DB,stroke:#2574A9,color:#fff
style H fill:#27AE60,stroke:#1E8449,color:#fff
style I fill:#16A085,stroke:#117864,color:#fff
style J fill:#C0392B,stroke:#943126,color:#fff
Part 2: Main Execution Flow¶
This section follows the chronological execution path through tox, from startup to completion.
Boot sequence¶
All roads lead to tox.run.run():
flowchart TD
A[run]
B[main]
C[setup_report]
D[get_options]
E[setup_state]
F{provision check?}
G[Create provision env]
H[handler]
A --> B --> C --> D --> E --> F
F -->|Missing| G
F -->|OK| H
G -.Re-invoke.-> A
style A fill:#5CB85C,stroke:#3D8B40,color:#fff
style B fill:#5CB85C,stroke:#3D8B40,color:#fff
style C fill:#5CB85C,stroke:#3D8B40,color:#fff
style D fill:#4A90E2,stroke:#2E5C8A,color:#fff
style E fill:#9B59B6,stroke:#6D3F8C,color:#fff
style F fill:#F0AD4E,stroke:#C88A3C,color:#fff
style G fill:#E67E22,stroke:#B85E18,color:#fff
style H fill:#3498DB,stroke:#2574A9,color:#fff
Entry points & CLI¶
The CLI parsing and command dispatch system handles argument processing, plugin loading, and routing to subcommands.
Two-phase CLI parsing¶
Phase 1 — bootstrap (before plugins load):
It parses only
-c/--conf(config path) and-v/-q(verbosity).It sets up logging (see Reporting & logging).
It discovers configuration source (see Configuration system).
It loads plugins (see Plugin system).
Phase 2 — full parse (after plugins):
Plugins register their arguments via the
tox_add_optionhook (see Plugin system).It performs a full
argparseparse over all arguments.Subcommand handlers are stored in
Parsed.cmd_handlers.
Key classes¶
ToxParser — subclass of
argparse.ArgumentParser. Supportsadd_command()which creates a sub-parser and registers the handler.Parsed — the result of Phase 2. Holds
options,pos_args,cmd_handlers,env(aCliEnv), andoverride(from-x).Options — the
argparse.Namespace. Notably hascommand(the chosen subcommand name).
Built-in subcommands¶
Command |
Alias |
Module |
Purpose |
|---|---|---|---|
|
|
Run environments sequentially (default) |
|
|
|
Run environments in parallel |
|
|
|
List available environments |
|
|
|
Show materialised configuration |
|
|
|
Create a development environment |
|
|
|
Run an arbitrary command in an env |
|
|
|
Visualize environment dependency graph |
|
|
|
Generate a starter tox config |
|
|
|
Backward-compatible tox 3 entry point |
Session & state management¶
State¶
State is the top-level runtime container:
Attribute |
Type |
Purpose |
|---|---|---|
|
Fully materialised tox configuration |
|
|
CLI parse result |
|
|
|
Original CLI args (re-invocation during provisioning) |
|
Run journal for |
|
|
Lazily created; single entry point for all env discovery |
The envs property creates the EnvSelector on first access — typically when the
subcommand handler starts iterating environments.
CliEnv¶
CliEnv captures the user’s -e flag in three
modes:
Specific list (
-e py39,py310) — run exactly these envs.ALL (
-e ALL) — run every defined env.Default (no
-e) — useenv_listfrom config.
EnvSelector¶
The core environment discovery engine EnvSelector:
Collect names from config, CLI
-e, labels (-m), factors (-f).Phase 1: For each name, build a
ToxEnvvia_build_run_env(), then query its packaging needs and build package envs.Phase 2: Reorder to match original definition order, apply label/factor filtering.
Iterate:
iter()yields envs filtered by active status, run-env-only, and--skip-envregex.
Subcommand dispatch¶
After provisioning, the handler is dispatched (see Entry points & CLI for built-in subcommands):
handler = state._options.cmd_handlers[state.conf.options.command]
return handler(state)
Run execution uses ready_to_run_envs() for topological batch scheduling — environments whose depends are all
satisfied are yielded first.
Environment lifecycle¶
flowchart TD
A["1. SETUP<br/>• Platform check (skip if no match)<br/>• Recreate check → clean()<br/>• Create virtualenv<br/>• Install deps<br/>• Install package (build + install)<br/>• extra_setup_commands"]
B["2. EXECUTE<br/>• commands_pre<br/>• commands<br/>• commands_post (always runs)"]
C["3. TEARDOWN<br/>• Detach from package envs<br/>• Close PEP-517 backend<br/>• Delete built packages"]
A --> B --> C
style A fill:#5CB85C,stroke:#3D8B40,color:#fff
style B fill:#F0AD4E,stroke:#C88A3C,color:#fff
style C fill:#E74C3C,stroke:#B23A2F,color:#fff
If a Recreate exception is raised during setup (e.g., Python version changed), tox automatically cleans and retries.
Execution system¶
flowchart TD
A["Execute (ABC)<br/>execute/api.py"]
B["LocalSubProcessExecutor<br/>execute/local_sub_process/"]
A --> B
style A fill:#9B59B6,stroke:#6D3F8C,color:#fff
style B fill:#4A90E2,stroke:#2E5C8A,color:#fff
Command flow¶
ToxEnv.execute()builds an ExecuteRequest.Execute.call() — context manager — creates SyncWrite streams for stdout/stderr.
ExecuteInstance.start() spawns a
subprocess.Popensubprocess and starts drain threads.ExecuteStatus.wait() blocks until completion.
An Outcome is constructed with exit code, captured output, and timing.
Command resolution¶
Before execution, tox performs the following steps:
It resolves the executable via
shutil.which()using the env’sPATH.It checks against
allowlist_externalsglob patterns.On Unix, it checks for
TOX_LIMITED_SHEBANGand prepends the shebang interpreter.
Output capture¶
The SyncWrite class accumulates all bytes in a
io.BytesIO buffer (thread-safe):
If
show=True, it forwards to real stdout/stderr in real-time.It uses a 0.1s timer to flush partial lines.
Stderr gets colored red when color is enabled.
Interrupt mechanism¶
Escalating stop when tox receives a signal:
Wait
cmd_kill_delay(default 0.0s) for voluntary exit.Send
SIGINT(Unix) /Ctrl+C(Windows), wait 0.3s.Send
SIGTERM, wait 0.2s.Send
SIGKILL(Unix only), unconditional wait.
Part 3: Supporting Systems¶
These subsystems support the main execution flow. Read these when working on specific features.
Configuration system¶
The configuration system converts raw text (from INI/TOML files, CLI flags, and environment variables) into strongly-typed Python objects.
Layer diagram¶
flowchart TD
A[CLI Options<br/>Parsed]
B[Source<br/>tox.ini / pyproject.toml / tox.toml / setup.cfg<br/>provides Loaders per section]
C[Config<br/>main.py<br/>central registry of all ConfigSets]
D[ConfigSet<br/>sets.py<br/>group of named config keys for one env/core]
E[ConfigDefinition<br/>of_type.py<br/>single key with type, default, post-process]
F[Loader<br/>loader/api.py<br/>reads raw values from the source]
G[Convert / StrConvert<br/>loader/convert.py, loader/str_convert.py]
A --> B
B --> C
C --> D
D --> E
E --> F
F --> G
style A fill:#4A90E2,stroke:#2E5C8A,color:#fff
style B fill:#5CB85C,stroke:#3D8B40,color:#fff
style C fill:#F0AD4E,stroke:#C88A3C,color:#fff
style D fill:#9B59B6,stroke:#6D3F8C,color:#fff
style E fill:#E67E22,stroke:#B85E18,color:#fff
style F fill:#3498DB,stroke:#2574A9,color:#fff
style G fill:#E74C3C,stroke:#B23A2F,color:#fff
Config¶
The central Config object is created once per session:
The
coreproperty lazily creates CoreConfigSet and attaches loaders.The
get_env(name)method returns EnvConfigSet for a tox environment and resolves sections and base inheritance.The
memory_seed_loadersis adefaultdict(list[MemoryLoader])that injects config values programmatically.The
env_names()method iterates environment names from source and plugin-extended envs.
Source discovery¶
The discover_source() function searches for configuration in this order:
If
--confis specified, it uses that file directly.Otherwise, it walks up from CWD looking for:
tox.tomlwhich uses TomlSource.pyproject.toml(with[tool.tox]) which uses PyProjectTomlSource.tox.iniwhich uses IniSource.setup.cfg(with[tox:tox]) which uses SetupCfgSource.
If nothing is found, it creates an empty IniSource at CWD.
Source hierarchy:
flowchart TD
A["Source (ABC)<br/>source/api.py"]
B["IniSource<br/>source/ini.py"]
C["SetupCfgSource<br/>source/setup_cfg.py"]
D["TomlSource<br/>source/toml_.py"]
E["PyProjectTomlSource<br/>source/pyproject.py"]
A --> B
A --> D
B --> C
D --> E
style A fill:#9B59B6,stroke:#6D3F8C,color:#fff
style B fill:#4A90E2,stroke:#2E5C8A,color:#fff
style C fill:#3498DB,stroke:#2574A9,color:#fff
style D fill:#5CB85C,stroke:#3D8B40,color:#fff
style E fill:#27AE60,stroke:#1E8449,color:#fff
Loaders¶
Loader |
Reads from |
|---|---|
INI |
|
TOML |
|
In-memory dict (provisioning, plugins) |
How a value is resolved¶
When you access env_config["deps"]:
ConfigDynamicDefinition.__call__() is invoked.
It checks its cache.
Iterates through the attached loaders (INI loader, TOML loader, memory loader).
The first loader that has the key returns the raw value.
The raw value is converted to the target type (via Convert).
Overrides from
--overrideare applied.post_processcallback runs (if registered).The result is cached.
Plugin system¶
Tox uses pluggy with the "tox" namespace.
Hooks¶
Hook |
When called |
|---|---|
|
Register custom run/package env types |
|
Declare additional env names dynamically |
|
Add CLI arguments (after logging + config discovery) |
|
Core configuration is being built |
|
Per-environment configuration is being built |
|
Just before commands execute in an env |
|
Just after commands execute |
|
Before executing an install command |
|
After an environment is torn down |
Plugin loading sequence¶
flowchart TD
A["setup_report()<br/>configure logging first"]
B["import plugin.manager<br/>triggers MANAGER = ToxPluginManager()"]
C["MANAGER._setup()"]
D["1. Load inline plugin<br/>toxfile.py / ☣.py"]
E["2. Load external plugins<br/>entry points"]
F["Respect TOX_DISABLED_EXTERNAL_PLUGINS<br/>env var"]
G["3. Load internal plugins<br/>16 hardcoded modules"]
H["4. check_pending()<br/>validate all hooks implemented"]
A --> B --> C
C --> D
C --> E
E --> F
C --> G
C --> H
style A fill:#5CB85C,stroke:#3D8B40,color:#fff
style B fill:#4A90E2,stroke:#2E5C8A,color:#fff
style C fill:#9B59B6,stroke:#6D3F8C,color:#fff
style D fill:#F0AD4E,stroke:#C88A3C,color:#fff
style E fill:#E67E22,stroke:#B85E18,color:#fff
style F fill:#D35400,stroke:#A04000,color:#fff
style G fill:#3498DB,stroke:#2574A9,color:#fff
style H fill:#E74C3C,stroke:#B23A2F,color:#fff
Inline plugins¶
Tox searches the parent directory of the config file for toxfile.py or ☣.py. The first match wins. Example:
from tox.plugin import impl
@impl
def tox_add_option(parser):
parser.add_argument("--my-flag", action="store_true")
@impl
def tox_add_env_config(env_conf, state):
env_conf.add_config("my_key", of_type=str, default="", desc="My custom key")
External plugins¶
Any installed package declaring a tox entry point group is discovered:
# In the plugin's pyproject.toml
[project.entry-points.tox]
my_plugin = "my_tox_plugin"
Set TOX_DISABLED_EXTERNAL_PLUGINS to a comma-separated list of plugin names to block specific plugins.
Packaging details¶
REGISTER singleton¶
The REGISTER singleton maintains two registries:
_run_envsstores runner types keyed byid()string._package_envsstores packager types keyed byid()string.default_env_runnerholds the first registered runner.
Plugins register new env types via the tox_register_tox_env hook (see Plugin system).
Package environment¶
A RunToxEnv determines if packaging is needed via package_envs:
It checks
no_package/skip_installconfig.It returns
(package_env_name, package_tox_env_type)tuples.
The Pep517VenvPackager uses pyproject_api to communicate with the build backend (setuptools, flit, etc.) as a subprocess. It supports sdist, wheel, editable (PEP-660), and editable-legacy builds.
The VenvCmdBuilder runs user-defined commands to build packages.
Package envs are thread-safe, meaning a single package env can serve multiple runners in parallel, guarded by
filelock.
Pip installer¶
The concrete installer (tox_env/python/pip/) has the following features:
Incremental installs compare new vs cached requirements and only install changes.
It supports
pip_pre, constraints, anduse_frozen_constraints.The
{packages}placeholder ininstall_commandis replaced with actual arguments.See Execution system for how installation commands are executed.
Reporting & logging¶
Verbosity levels¶
Verbosity |
Flags |
Log level |
Output |
|---|---|---|---|
0 |
|
CRITICAL |
Almost nothing |
1 |
|
ERROR |
Errors only |
2 |
(default) |
WARNING |
Normal output |
3 |
|
INFO |
Detailed progress |
4 |
|
DEBUG |
Debug messages + timestamps |
5 |
|
NOTSET |
Everything |
ToxHandler¶
The central logging handler ToxHandler is added to the root logger and provides:
Thread-local output where each thread gets its own stdout/stderr via
_LogThreadLocal, enabling parallel execution to capture output separately.Colored formatters where Error = red, Warning = cyan, Info = white.
Per-environment context where
with_context(name)sets the current env name for the log prefix (magenta-coloured).Command-echo pattern where messages matching
"%s%s> %s"get special color treatment (magenta env name, green>, reset for command text).
Journal system¶
Optional structured JSON output is enabled by --result-json <path>:
Journal (session level) collects
toxversion,platform,host, and per-environment results.EnvJournal (per-environment) records metadata and execution outcomes (command, stdout, stderr, exit code, elapsed time).
Part 4: Reference Materials¶
Class hierarchies¶
ToxEnv hierarchy¶
flowchart TD
A["ToxEnv (ABC)<br/>tox_env/api.py"]
B["RunToxEnv (ABC)<br/>tox_env/runner.py"]
C["PythonRun<br/>Python + RunToxEnv (ABC)<br/>tox_env/python/runner.py"]
D["VirtualEnvRunner<br/>VirtualEnv + PythonRun<br/>virtual_env/runner.py<br/>id = virtualenv"]
E["PackageToxEnv (ABC)<br/>tox_env/package.py"]
F["PythonPackageToxEnv<br/>Python + PackageToxEnv (ABC)"]
G["Pep517VirtualEnvPackager<br/>id = virtualenv-pep-517"]
H["VirtualEnvCmdBuilder<br/>id = virtualenv-cmd-builder"]
I["Python (ABC, mixin)<br/>tox_env/python/api.py"]
J["VirtualEnv (ABC, mixin)<br/>virtual_env/api.py"]
A --> B
A --> E
A --> I
B --> C
C --> D
E --> F
F --> G
F --> H
I --> J
style A fill:#9B59B6,stroke:#6D3F8C,color:#fff
style B fill:#4A90E2,stroke:#2E5C8A,color:#fff
style C fill:#3498DB,stroke:#2574A9,color:#fff
style D fill:#27AE60,stroke:#1E8449,color:#fff
style E fill:#F0AD4E,stroke:#C88A3C,color:#fff
style F fill:#E67E22,stroke:#B85E18,color:#fff
style G fill:#5CB85C,stroke:#3D8B40,color:#fff
style H fill:#16A085,stroke:#117864,color:#fff
style I fill:#E74C3C,stroke:#B23A2F,color:#fff
style J fill:#C0392B,stroke:#943126,color:#fff
The diamond pattern means that leaf classes combine a role (runner or packager) with an implementation strategy (VirtualEnv). For example, VirtualEnvRunner inherits from both VirtualEnv and PythonRun.
Design patterns¶
Pattern |
Where |
|---|---|
Lazy properties |
|
Two-phase init |
CLI parsing (bootstrap then full), env build (run then package) |
Plugin hooks at every seam |
Config, env setup, install, commands, teardown |
Abstract base + mixins |
|
Registry / singleton |
|
Thread-local state |
ToxHandler’s |
Incremental caching |
CacheToxInfo ( |
Key decision points¶
Point |
Decision |
Outcome |
|---|---|---|
Provisioning |
Are |
Continue or re-invoke in venv |
Platform check |
Does |
Run or skip env |
Recreate |
Config changed since last run? |
Wipe and rebuild, or reuse |
Package |
|
Build and install, or skip |
Command prefix |
|
Invert or ignore exit code |
Interrupt |
Signal received? |
Escalate: INT → TERM → KILL |
Testing guide¶
The tests/ directory mirrors src/tox/:
tests/
├── conftest.py # Root fixtures
├── config/ # ← mirrors src/tox/config/
│ ├── cli/
│ ├── loader/
│ └── source/
├── execute/ # ← mirrors src/tox/execute/
├── journal/
├── plugin/
├── session/
│ └── cmd/ # Subcommand tests
├── tox_env/
│ └── python/
├── util/
├── demo_pkg_inline/ # Minimal PEP-517 test package
└── demo_pkg_setuptools/ # Setuptools test package
Key fixtures¶
tox_project creates a temporary project with a tox config and runs tox against it. This is the most important integration test fixture and uses the full flow described in Part 1.
demo_pkg_inline is a minimal PEP-517 package with a custom build backend.
demo_pkg_setuptools is a setuptools-based package for setuptools-specific tests.