Onboarder

This guide walks through tox’s internal architecture — how the pieces fit together, what each subsystem does, and where to look when contributing. It is aimed at first-time contributors and new team members.

How to use this guide:

  • Part 1 gives you the big picture and tells you where to start reading code.

  • Part 2 follows tox’s execution flow from startup to completion.

  • Part 3 covers supporting systems you’ll need when working on specific features.

  • Part 4 contains reference materials for looking up details.

Part 1: Get Oriented

Quick facts

Language

Python ≥ 3.10

Build system

hatchling (PEP 517 / PEP 660)

Plugin framework

pluggy

Test framework

pytest (+ xdist, pytest-mock)

Docs

Sphinx + Furo theme

License

MIT

Repository layout

tox/
├── src/tox/                   # main package
│   ├── run.py                 # entry point
│   ├── provision.py           # self-provisioning logic
│   ├── report.py              # logging / colored output
│   ├── config/                # configuration subsystem
│   │   ├── cli/               #   CLI parser (two-phase)
│   │   ├── loader/            #   INI / TOML / memory loaders
│   │   └── source/            #   config file discovery
│   ├── session/               # runtime state + subcommands
│   │   └── cmd/               #   built-in subcommands
│   ├── tox_env/               # environment classes
│   │   └── python/            #   Python-specific envs
│   │       └── virtual_env/   #     virtualenv-based concrete envs
│   ├── execute/               # subprocess execution
│   ├── plugin/                # pluggy integration
│   ├── journal/               # JSON result journal
│   └── util/                  # helpers (graph, cpu, path, …)
├── tests/                     # mirrors src/tox/ structure
└── docs/                      # Sphinx documentation

The big picture: End-to-end flow

This traces tox run -e py311 from start to finish, showing the six main phases:

        flowchart LR
    subgraph ENTRY["1️⃣ ENTRY POINT"]
        direction TB
        A[tox run -e py311]
        B[Parse CLI]
        C[Load config]
        D[Build State]
        A --> B --> C --> D
    end

    subgraph PROV["2️⃣ PROVISIONING"]
        direction TB
        E{Requirements<br/>satisfied?}
        F[Create provision env]
        G[Re-invoke tox]
        E -->|No| F --> G
    end

    subgraph DISC["3️⃣ DISCOVERY"]
        direction TB
        H[Dispatch subcommand]
        I[Discover envs]
        J[Resolve dependencies]
        H --> I --> J
    end

    subgraph SETUP["4️⃣ ENVIRONMENT SETUP"]
        direction TB
        K{Virtualenv<br/>exists?}
        L[Create virtualenv]
        M[Install dependencies]
        N[Build & install package]
        O[Run extra_setup_commands]
        K -->|No| L --> M
        K -->|Yes| M
        M --> N --> O
    end

    subgraph EXEC["5️⃣ EXECUTION"]
        direction TB
        P[Run commands_pre]
        Q[Run commands]
        R[Run commands_post]
        P --> Q --> R
    end

    subgraph RESULT["6️⃣ RESULTS"]
        direction TB
        S[Collect outcomes]
        T[Write journal]
        U[Report summary]
        S --> T --> U
    end

    ENTRY --> PROV
    G -.Re-invoke.-> ENTRY
    PROV -->|Yes| DISC
    DISC --> SETUP
    SETUP --> EXEC
    EXEC --> RESULT

    style ENTRY fill:#E8F5E9,stroke:#4CAF50,stroke-width:2px
    style PROV fill:#FFF3E0,stroke:#FF9800,stroke-width:2px
    style DISC fill:#E3F2FD,stroke:#2196F3,stroke-width:2px
    style SETUP fill:#F3E5F5,stroke:#9C27B0,stroke-width:2px
    style EXEC fill:#E1F5FE,stroke:#03A9F4,stroke-width:2px
    style RESULT fill:#FFEBEE,stroke:#F44336,stroke-width:2px

    style A fill:#5CB85C,stroke:#3D8B40,color:#fff
    style B fill:#5CB85C,stroke:#3D8B40,color:#fff
    style C fill:#5CB85C,stroke:#3D8B40,color:#fff
    style D fill:#5CB85C,stroke:#3D8B40,color:#fff
    style E fill:#F0AD4E,stroke:#C88A3C,color:#fff
    style F fill:#F0AD4E,stroke:#C88A3C,color:#fff
    style G fill:#F0AD4E,stroke:#C88A3C,color:#fff
    style H fill:#4A90E2,stroke:#2E5C8A,color:#fff
    style I fill:#4A90E2,stroke:#2E5C8A,color:#fff
    style J fill:#4A90E2,stroke:#2E5C8A,color:#fff
    style K fill:#9B59B6,stroke:#6D3F8C,color:#fff
    style L fill:#9B59B6,stroke:#6D3F8C,color:#fff
    style M fill:#9B59B6,stroke:#6D3F8C,color:#fff
    style N fill:#9B59B6,stroke:#6D3F8C,color:#fff
    style O fill:#9B59B6,stroke:#6D3F8C,color:#fff
    style P fill:#3498DB,stroke:#2574A9,color:#fff
    style Q fill:#3498DB,stroke:#2574A9,color:#fff
    style R fill:#3498DB,stroke:#2574A9,color:#fff
    style S fill:#E74C3C,stroke:#B23A2F,color:#fff
    style T fill:#E74C3C,stroke:#B23A2F,color:#fff
    style U fill:#E74C3C,stroke:#B23A2F,color:#fff
    

High-level architecture

The following diagram shows the main flow through tox’s subsystems:

        flowchart TD
    A[CLI / argv]
    B[Two-Phase Parser<br/>cli/]
    C[Config<br/>config/main.py<br/>Sources → Loaders → Types]
    D[State<br/>session/state.py<br/>owns Config + EnvSelector]
    E[Provisioning<br/>provision.py<br/>ensures requires/min_ver]
    F[Subcommand Handler<br/>run / list / config / …]
    G[EnvSelector<br/>env_select.py<br/>discovers + builds ToxEnv instances]
    H[RunToxEnv<br/>commands, deps]
    I[PackageToxEnv<br/>wheel / sdist]
    J[Execute<br/>subprocess mgmt]

    A --> B --> C --> D --> E --> F --> G
    G --> H
    G --> I
    H --> J

    style A fill:#E74C3C,stroke:#B23A2F,color:#fff
    style B fill:#5CB85C,stroke:#3D8B40,color:#fff
    style C fill:#4A90E2,stroke:#2E5C8A,color:#fff
    style D fill:#9B59B6,stroke:#6D3F8C,color:#fff
    style E fill:#F0AD4E,stroke:#C88A3C,color:#fff
    style F fill:#E67E22,stroke:#B85E18,color:#fff
    style G fill:#3498DB,stroke:#2574A9,color:#fff
    style H fill:#27AE60,stroke:#1E8449,color:#fff
    style I fill:#16A085,stroke:#117864,color:#fff
    style J fill:#C0392B,stroke:#943126,color:#fff
    

Part 2: Main Execution Flow

This section follows the chronological execution path through tox, from startup to completion.

Boot sequence

All roads lead to tox.run.run():

        flowchart TD
    A[run]
    B[main]
    C[setup_report]
    D[get_options]
    E[setup_state]
    F{provision check?}
    G[Create provision env]
    H[handler]

    A --> B --> C --> D --> E --> F
    F -->|Missing| G
    F -->|OK| H
    G -.Re-invoke.-> A

    style A fill:#5CB85C,stroke:#3D8B40,color:#fff
    style B fill:#5CB85C,stroke:#3D8B40,color:#fff
    style C fill:#5CB85C,stroke:#3D8B40,color:#fff
    style D fill:#4A90E2,stroke:#2E5C8A,color:#fff
    style E fill:#9B59B6,stroke:#6D3F8C,color:#fff
    style F fill:#F0AD4E,stroke:#C88A3C,color:#fff
    style G fill:#E67E22,stroke:#B85E18,color:#fff
    style H fill:#3498DB,stroke:#2574A9,color:#fff
    

Entry points & CLI

The CLI parsing and command dispatch system handles argument processing, plugin loading, and routing to subcommands.

Two-phase CLI parsing

Phase 1 — bootstrap (before plugins load):

Phase 2 — full parse (after plugins):

  • Plugins register their arguments via the tox_add_option hook (see Plugin system).

  • It performs a full argparse parse over all arguments.

  • Subcommand handlers are stored in Parsed.cmd_handlers.

Key classes

  • ToxParser — subclass of argparse.ArgumentParser. Supports add_command() which creates a sub-parser and registers the handler.

  • Parsed — the result of Phase 2. Holds options, pos_args, cmd_handlers, env (a CliEnv), and override (from -x).

  • Options — the argparse.Namespace. Notably has command (the chosen subcommand name).

Built-in subcommands

Command

Alias

Module

Purpose

run

r

session/cmd/run/sequential.py

Run environments sequentially (default)

run-parallel

p

session/cmd/run/parallel.py

Run environments in parallel

list

l

session/cmd/list_envs.py

List available environments

config

c

session/cmd/show_config.py

Show materialised configuration

devenv

d

session/cmd/devenv.py

Create a development environment

exec

e

session/cmd/exec_.py

Run an arbitrary command in an env

depends

de

session/cmd/depends.py

Visualize environment dependency graph

quickstart

q

session/cmd/quickstart.py

Generate a starter tox config

legacy

le

session/cmd/legacy.py

Backward-compatible tox 3 entry point

Session & state management

State

State is the top-level runtime container:

Attribute

Type

Purpose

conf

Config

Fully materialised tox configuration

_options

Parsed

CLI parse result

_args

list[str]

Original CLI args (re-invocation during provisioning)

journal

Journal

Run journal for --result-json

envs

EnvSelector

Lazily created; single entry point for all env discovery

The envs property creates the EnvSelector on first access — typically when the subcommand handler starts iterating environments.

CliEnv

CliEnv captures the user’s -e flag in three modes:

  • Specific list (-e py39,py310) — run exactly these envs.

  • ALL (-e ALL) — run every defined env.

  • Default (no -e) — use env_list from config.

EnvSelector

The core environment discovery engine EnvSelector:

  1. Collect names from config, CLI -e, labels (-m), factors (-f).

  2. Phase 1: For each name, build a ToxEnv via _build_run_env(), then query its packaging needs and build package envs.

  3. Phase 2: Reorder to match original definition order, apply label/factor filtering.

  4. Iterate: iter() yields envs filtered by active status, run-env-only, and --skip-env regex.

Subcommand dispatch

After provisioning, the handler is dispatched (see Entry points & CLI for built-in subcommands):

handler = state._options.cmd_handlers[state.conf.options.command]
return handler(state)

Run execution uses ready_to_run_envs() for topological batch scheduling — environments whose depends are all satisfied are yielded first.

Environment lifecycle

        flowchart TD
    A["1. SETUP<br/>• Platform check (skip if no match)<br/>• Recreate check → clean()<br/>• Create virtualenv<br/>• Install deps<br/>• Install package (build + install)<br/>• extra_setup_commands"]
    B["2. EXECUTE<br/>• commands_pre<br/>• commands<br/>• commands_post (always runs)"]
    C["3. TEARDOWN<br/>• Detach from package envs<br/>• Close PEP-517 backend<br/>• Delete built packages"]

    A --> B --> C

    style A fill:#5CB85C,stroke:#3D8B40,color:#fff
    style B fill:#F0AD4E,stroke:#C88A3C,color:#fff
    style C fill:#E74C3C,stroke:#B23A2F,color:#fff
    

If a Recreate exception is raised during setup (e.g., Python version changed), tox automatically cleans and retries.

Execution system

        flowchart TD
    A["Execute (ABC)<br/>execute/api.py"]
    B["LocalSubProcessExecutor<br/>execute/local_sub_process/"]

    A --> B

    style A fill:#9B59B6,stroke:#6D3F8C,color:#fff
    style B fill:#4A90E2,stroke:#2E5C8A,color:#fff
    

Command flow

  1. ToxEnv.execute() builds an ExecuteRequest.

  2. Execute.call() — context manager — creates SyncWrite streams for stdout/stderr.

  3. ExecuteInstance.start() spawns a subprocess.Popen subprocess and starts drain threads.

  4. ExecuteStatus.wait() blocks until completion.

  5. An Outcome is constructed with exit code, captured output, and timing.

Command resolution

Before execution, tox performs the following steps:

  • It resolves the executable via shutil.which() using the env’s PATH.

  • It checks against allowlist_externals glob patterns.

  • On Unix, it checks for TOX_LIMITED_SHEBANG and prepends the shebang interpreter.

Output capture

The SyncWrite class accumulates all bytes in a io.BytesIO buffer (thread-safe):

  • If show=True, it forwards to real stdout/stderr in real-time.

  • It uses a 0.1s timer to flush partial lines.

  • Stderr gets colored red when color is enabled.

Interrupt mechanism

Escalating stop when tox receives a signal:

  1. Wait cmd_kill_delay (default 0.0s) for voluntary exit.

  2. Send SIGINT (Unix) / Ctrl+C (Windows), wait 0.3s.

  3. Send SIGTERM, wait 0.2s.

  4. Send SIGKILL (Unix only), unconditional wait.

Part 3: Supporting Systems

These subsystems support the main execution flow. Read these when working on specific features.

Configuration system

The configuration system converts raw text (from INI/TOML files, CLI flags, and environment variables) into strongly-typed Python objects.

Layer diagram

        flowchart TD
    A[CLI Options<br/>Parsed]
    B[Source<br/>tox.ini / pyproject.toml / tox.toml / setup.cfg<br/>provides Loaders per section]
    C[Config<br/>main.py<br/>central registry of all ConfigSets]
    D[ConfigSet<br/>sets.py<br/>group of named config keys for one env/core]
    E[ConfigDefinition<br/>of_type.py<br/>single key with type, default, post-process]
    F[Loader<br/>loader/api.py<br/>reads raw values from the source]
    G[Convert / StrConvert<br/>loader/convert.py, loader/str_convert.py]

    A --> B
    B --> C
    C --> D
    D --> E
    E --> F
    F --> G

    style A fill:#4A90E2,stroke:#2E5C8A,color:#fff
    style B fill:#5CB85C,stroke:#3D8B40,color:#fff
    style C fill:#F0AD4E,stroke:#C88A3C,color:#fff
    style D fill:#9B59B6,stroke:#6D3F8C,color:#fff
    style E fill:#E67E22,stroke:#B85E18,color:#fff
    style F fill:#3498DB,stroke:#2574A9,color:#fff
    style G fill:#E74C3C,stroke:#B23A2F,color:#fff
    

Config

The central Config object is created once per session:

  • The core property lazily creates CoreConfigSet and attaches loaders.

  • The get_env(name) method returns EnvConfigSet for a tox environment and resolves sections and base inheritance.

  • The memory_seed_loaders is a defaultdict(list[MemoryLoader]) that injects config values programmatically.

  • The env_names() method iterates environment names from source and plugin-extended envs.

Source discovery

The discover_source() function searches for configuration in this order:

  1. If --conf is specified, it uses that file directly.

  2. Otherwise, it walks up from CWD looking for:

  3. If nothing is found, it creates an empty IniSource at CWD.

Source hierarchy:

        flowchart TD
    A["Source (ABC)<br/>source/api.py"]
    B["IniSource<br/>source/ini.py"]
    C["SetupCfgSource<br/>source/setup_cfg.py"]
    D["TomlSource<br/>source/toml_.py"]
    E["PyProjectTomlSource<br/>source/pyproject.py"]

    A --> B
    A --> D
    B --> C
    D --> E

    style A fill:#9B59B6,stroke:#6D3F8C,color:#fff
    style B fill:#4A90E2,stroke:#2E5C8A,color:#fff
    style C fill:#3498DB,stroke:#2574A9,color:#fff
    style D fill:#5CB85C,stroke:#3D8B40,color:#fff
    style E fill:#27AE60,stroke:#1E8449,color:#fff
    

Loaders

Loader

Reads from

IniLoader

INI [section] key=value pairs

TomlLoader

TOML [table] key-value pairs

MemoryLoader

In-memory dict (provisioning, plugins)

How a value is resolved

When you access env_config["deps"]:

  1. ConfigDynamicDefinition.__call__() is invoked.

  2. It checks its cache.

  3. Iterates through the attached loaders (INI loader, TOML loader, memory loader).

  4. The first loader that has the key returns the raw value.

  5. The raw value is converted to the target type (via Convert).

  6. Overrides from --override are applied.

  7. post_process callback runs (if registered).

  8. The result is cached.

Plugin system

Tox uses pluggy with the "tox" namespace.

Hooks

Hook

When called

tox_register_tox_env

Register custom run/package env types

tox_extend_env_list

Declare additional env names dynamically

tox_add_option

Add CLI arguments (after logging + config discovery)

tox_add_core_config

Core configuration is being built

tox_add_env_config

Per-environment configuration is being built

tox_before_run_commands

Just before commands execute in an env

tox_after_run_commands

Just after commands execute

tox_on_install

Before executing an install command

tox_env_teardown

After an environment is torn down

Plugin loading sequence

        flowchart TD
    A["setup_report()<br/>configure logging first"]
    B["import plugin.manager<br/>triggers MANAGER = ToxPluginManager()"]
    C["MANAGER._setup()"]
    D["1. Load inline plugin<br/>toxfile.py / ☣.py"]
    E["2. Load external plugins<br/>entry points"]
    F["Respect TOX_DISABLED_EXTERNAL_PLUGINS<br/>env var"]
    G["3. Load internal plugins<br/>16 hardcoded modules"]
    H["4. check_pending()<br/>validate all hooks implemented"]

    A --> B --> C
    C --> D
    C --> E
    E --> F
    C --> G
    C --> H

    style A fill:#5CB85C,stroke:#3D8B40,color:#fff
    style B fill:#4A90E2,stroke:#2E5C8A,color:#fff
    style C fill:#9B59B6,stroke:#6D3F8C,color:#fff
    style D fill:#F0AD4E,stroke:#C88A3C,color:#fff
    style E fill:#E67E22,stroke:#B85E18,color:#fff
    style F fill:#D35400,stroke:#A04000,color:#fff
    style G fill:#3498DB,stroke:#2574A9,color:#fff
    style H fill:#E74C3C,stroke:#B23A2F,color:#fff
    

Inline plugins

Tox searches the parent directory of the config file for toxfile.py or ☣.py. The first match wins. Example:

from tox.plugin import impl


@impl
def tox_add_option(parser):
    parser.add_argument("--my-flag", action="store_true")


@impl
def tox_add_env_config(env_conf, state):
    env_conf.add_config("my_key", of_type=str, default="", desc="My custom key")

External plugins

Any installed package declaring a tox entry point group is discovered:

# In the plugin's pyproject.toml
[project.entry-points.tox]
my_plugin = "my_tox_plugin"

Set TOX_DISABLED_EXTERNAL_PLUGINS to a comma-separated list of plugin names to block specific plugins.

Packaging details

REGISTER singleton

The REGISTER singleton maintains two registries:

  • _run_envs stores runner types keyed by id() string.

  • _package_envs stores packager types keyed by id() string.

  • default_env_runner holds the first registered runner.

Plugins register new env types via the tox_register_tox_env hook (see Plugin system).

Package environment

A RunToxEnv determines if packaging is needed via package_envs:

  • It checks no_package / skip_install config.

  • It returns (package_env_name, package_tox_env_type) tuples.

The Pep517VenvPackager uses pyproject_api to communicate with the build backend (setuptools, flit, etc.) as a subprocess. It supports sdist, wheel, editable (PEP-660), and editable-legacy builds.

The VenvCmdBuilder runs user-defined commands to build packages.

Package envs are thread-safe, meaning a single package env can serve multiple runners in parallel, guarded by filelock.

Pip installer

The concrete installer (tox_env/python/pip/) has the following features:

  • Incremental installs compare new vs cached requirements and only install changes.

  • It supports pip_pre, constraints, and use_frozen_constraints.

  • The {packages} placeholder in install_command is replaced with actual arguments.

  • See Execution system for how installation commands are executed.

Reporting & logging

Verbosity levels

Verbosity

Flags

Log level

Output

0

-qq

CRITICAL

Almost nothing

1

-q

ERROR

Errors only

2

(default)

WARNING

Normal output

3

-v

INFO

Detailed progress

4

-vv

DEBUG

Debug messages + timestamps

5

-vvv

NOTSET

Everything

ToxHandler

The central logging handler ToxHandler is added to the root logger and provides:

  • Thread-local output where each thread gets its own stdout/stderr via _LogThreadLocal, enabling parallel execution to capture output separately.

  • Colored formatters where Error = red, Warning = cyan, Info = white.

  • Per-environment context where with_context(name) sets the current env name for the log prefix (magenta-coloured).

  • Command-echo pattern where messages matching "%s%s> %s" get special color treatment (magenta env name, green >, reset for command text).

Journal system

Optional structured JSON output is enabled by --result-json <path>:

  • Journal (session level) collects toxversion, platform, host, and per-environment results.

  • EnvJournal (per-environment) records metadata and execution outcomes (command, stdout, stderr, exit code, elapsed time).

Part 4: Reference Materials

Class hierarchies

ToxEnv hierarchy

        flowchart TD
    A["ToxEnv (ABC)<br/>tox_env/api.py"]
    B["RunToxEnv (ABC)<br/>tox_env/runner.py"]
    C["PythonRun<br/>Python + RunToxEnv (ABC)<br/>tox_env/python/runner.py"]
    D["VirtualEnvRunner<br/>VirtualEnv + PythonRun<br/>virtual_env/runner.py<br/>id = virtualenv"]
    E["PackageToxEnv (ABC)<br/>tox_env/package.py"]
    F["PythonPackageToxEnv<br/>Python + PackageToxEnv (ABC)"]
    G["Pep517VirtualEnvPackager<br/>id = virtualenv-pep-517"]
    H["VirtualEnvCmdBuilder<br/>id = virtualenv-cmd-builder"]
    I["Python (ABC, mixin)<br/>tox_env/python/api.py"]
    J["VirtualEnv (ABC, mixin)<br/>virtual_env/api.py"]

    A --> B
    A --> E
    A --> I
    B --> C
    C --> D
    E --> F
    F --> G
    F --> H
    I --> J

    style A fill:#9B59B6,stroke:#6D3F8C,color:#fff
    style B fill:#4A90E2,stroke:#2E5C8A,color:#fff
    style C fill:#3498DB,stroke:#2574A9,color:#fff
    style D fill:#27AE60,stroke:#1E8449,color:#fff
    style E fill:#F0AD4E,stroke:#C88A3C,color:#fff
    style F fill:#E67E22,stroke:#B85E18,color:#fff
    style G fill:#5CB85C,stroke:#3D8B40,color:#fff
    style H fill:#16A085,stroke:#117864,color:#fff
    style I fill:#E74C3C,stroke:#B23A2F,color:#fff
    style J fill:#C0392B,stroke:#943126,color:#fff
    

The diamond pattern means that leaf classes combine a role (runner or packager) with an implementation strategy (VirtualEnv). For example, VirtualEnvRunner inherits from both VirtualEnv and PythonRun.

Design patterns

Pattern

Where

Lazy properties

State.envs, Config.core, EnvSelector._defined_envs

Two-phase init

CLI parsing (bootstrap then full), env build (run then package)

Plugin hooks at every seam

Config, env setup, install, commands, teardown

Abstract base + mixins

ToxEnv hierarchy (diamond MRO)

Registry / singleton

REGISTER for env types, MANAGER for plugins

Thread-local state

ToxHandler’s _LogThreadLocal for per-env output

Incremental caching

CacheToxInfo (.tox-info.json), pip installer diff

Key decision points

Point

Decision

Outcome

Provisioning

Are requires satisfied?

Continue or re-invoke in venv

Platform check

Does platform regex match?

Run or skip env

Recreate

Config changed since last run?

Wipe and rebuild, or reuse

Package

skip_install / no_package?

Build and install, or skip

Command prefix

! or - prefix?

Invert or ignore exit code

Interrupt

Signal received?

Escalate: INT → TERM → KILL

Testing guide

The tests/ directory mirrors src/tox/:

tests/
├── conftest.py                    # Root fixtures
├── config/                        # ← mirrors src/tox/config/
│   ├── cli/
│   ├── loader/
│   └── source/
├── execute/                       # ← mirrors src/tox/execute/
├── journal/
├── plugin/
├── session/
│   └── cmd/                       # Subcommand tests
├── tox_env/
│   └── python/
├── util/
├── demo_pkg_inline/               # Minimal PEP-517 test package
└── demo_pkg_setuptools/           # Setuptools test package

Key fixtures

  • tox_project creates a temporary project with a tox config and runs tox against it. This is the most important integration test fixture and uses the full flow described in Part 1.

  • demo_pkg_inline is a minimal PEP-517 package with a custom build backend.

  • demo_pkg_setuptools is a setuptools-based package for setuptools-specific tests.