Structured Parameter Sweep

Runs a multi-stage parameter sweep where each stage tests a category of parameters and the winner’s values carry forward to subsequent stages. This is more efficient than a full re-grid when only a few parameters need adjustment.

This implements Lesson L7 (structured sweep > full re-grid): test by category (entry, behavioral, initial, credit) rather than all at once.

Stage Syntax

Each stage is defined as LABEL:param1=v1,v2 param2=v3,v4:

--stages "entry:new_firm_size_factor=0.1,0.2 new_firm_price_markup=0.05,0.1" \
         "behavioral:beta=0.5,1.0,2.5 max_M=2,4"

The label is used for reporting. Parameters within a stage form a grid; stages execute sequentially with winners carried forward.

CLI Usage

# Two-stage sweep with cross-scenario check
python -m calibration --phase sweep --scenario growth_plus \
  --base output/growth_plus_stability.json \
  --stages "A:beta=0.5,1.0,2.5" "B:max_leverage=5,10,20" \
  --cross-scenario baseline

Required flags:

• --base: Path to stability result JSON or YAML with base config

• --stages: One or more stage definitions

Optional:

• --cross-scenario: Cross-evaluate against this scenario at each stage

Python API

from calibration.sweep import run_sweep, parse_stages

stages = parse_stages(
    [
        "entry:new_firm_size_factor=0.1,0.2",
        "behavioral:beta=0.5,1.0,2.5 max_M=2,4",
    ]
)

results = run_sweep(
    base_params={"beta": 5.0, "max_M": 4},
    stages=stages,
    scenario="baseline",
    n_workers=10,
)

for stage in results:
    print(f"Stage {stage.label}: {'improved' if stage.improved else 'no change'}")
    print(f"  Winner: {stage.winner_params}")

API Reference

Structured parameter sweep by category, carrying forward winners.

Each stage runs a grid of its parameters while holding everything else fixed from the base config (plus winners from prior stages). Optionally cross-evaluates against other scenarios at each stage.

calibration.sweep.parse_stage(stage_str)

Parse a single stage definition.

Format: “LABEL:param1=v1,v2,v3 param2=v4,v5”

Parameters:

stage_str (str) – Stage definition string.

Returns:

(label, param_grid)

Return type:

tuple[str, dict]

calibration.sweep.parse_stages(stage_args)

Parse multiple stage definitions.

Parameters:

stage_args (list[str]) – List of stage definition strings.

Returns:

List of (label, param_grid) tuples.

Return type:

list[tuple[str, dict]]

class calibration.sweep.StageResult(label, winner_params, combined_score, mean_score, pass_rate, n_candidates)

Result of a single sweep stage.

Variables:

• label (str) – Stage label.

• winner_params (dict[str, Any]) – Winner’s parameters.

• combined_score (float) – Winner’s combined score.

• mean_score (float) – Winner’s mean score.

• pass_rate (float) – Winner’s pass rate.

• n_candidates (int) – Number of grid combinations tested.

label

winner_params

combined_score

mean_score

pass_rate

n_candidates

calibration.sweep.run_sweep(base_params, stages, scenario, n_workers=10, n_periods=1000, stability_tiers=None, rank_by='combined', k_factor=1.0, cross_scenario=None)

Run structured multi-stage parameter sweep.

Parameters:

• base_params (dict) – Starting configuration.

• stages (list[tuple[str, dict]]) – List of (label, param_grid) stages to run in order.

• scenario (str) – Scenario name.

• n_workers (int) – Parallel workers.

• n_periods (int) – Simulation periods.

• stability_tiers (list[tuple[int, int]], optional) – Tiers for stability testing. Defaults to [(100, 10), (50, 20), (10, 100)].

• rank_by (str) – Ranking strategy for stability.

• k_factor (float) – k in combined score formula.

• cross_scenario (str, optional) – If set, cross-evaluate the stage winner against this scenario.

Returns:

Per-stage results with winner params and scores.

Return type:

list[StageResult]

calibration.sweep.run_sweep_phase(args, run_dir=None)

CLI entry point for sweep phase.