Targeted Cost Analysis

Quantifies the score cost of substituting preferred parameter values into an optimized base configuration. Each swap is classified by impact:

Classification	Absolute Delta	Meaning
FREE	< 0.002	No measurable impact
CHEAP	< 0.005	Negligible impact
MODERATE	< 0.010	Small but detectable
EXPENSIVE	>= 0.010	Significant impact

This implements Lesson L5 (targeted cost analysis for endgame): most parameter preferences turn out to be FREE or CHEAP.

CLI Usage

# Test swapping preferred values
python -m calibration --phase cost --scenario baseline \
  --base output/baseline_stability.json \
  --swaps "price_init=2.0,1.5" "min_wage_ratio=0.3" --seeds 20

# Run combo grid of all cheap swaps
python -m calibration --phase cost --scenario baseline \
  --base output/baseline_stability.json \
  --swaps "price_init=2.0" "beta=2.5" --combo-grid

Required flags:

• --base: Path to stability result JSON or YAML with base config

• --swaps: One or more param=v1,v2 specifications

Python API

from calibration.cost import run_cost_analysis, classify_cost

results = run_cost_analysis(
    base_params={"beta": 5.0, "max_M": 4},
    swaps={"beta": [2.5], "price_init": [2.0, 1.5]},
    scenario="baseline",
    n_seeds=20,
)

for r in results:
    print(f"{r.param}={r.value}: {r.classification} (delta={r.delta:+.4f})")

API Reference

Targeted cost analysis – measure the cost of swapping values into a base config.

Evaluates the impact of substituting preferred parameter values into an optimized base configuration. Classifies each swap by cost: FREE (<0.002), CHEAP (<0.005), MODERATE (<0.010), EXPENSIVE (>=0.010).

class calibration.cost.SwapResult(param, value, base_combined, swap_combined, delta, classification, pass_rate)

Result of swapping a single parameter value into the base config.

Variables:

• param (str) – Parameter name.

• value (Any) – Swapped value.

• base_combined (float) – Base config’s combined score.

• swap_combined (float) – Combined score with this value swapped in.

• delta (float) – Score change (swap - base). Negative = worse.

• classification (str) – Cost classification: FREE, CHEAP, MODERATE, or EXPENSIVE.

• pass_rate (float) – Pass rate with swapped value.

param

value

base_combined

swap_combined

delta

classification

pass_rate

calibration.cost.classify_cost(delta_abs)

Classify the absolute cost of a swap.

Parameters:

delta_abs (float) – Absolute combined score difference.

Returns:

“FREE”, “CHEAP”, “MODERATE”, or “EXPENSIVE”.

Return type:

str

calibration.cost.parse_swaps(swap_args)

Parse swap arguments from CLI.

Parameters:

swap_args (list[str]) – List of “param=v1,v2,v3” strings.

Returns:

Parameter -> list of values to try.

Return type:

dict[str, list]

calibration.cost.run_cost_analysis(base_params, swaps, scenario, n_seeds=20, n_periods=1000, n_workers=10, base_combined=None)

Run targeted cost analysis for parameter swaps.

Parameters:

• base_params (dict) – Base configuration (the stability winner).

• swaps (dict[str, list]) – Parameters to swap and their candidate values.

• scenario (str) – Scenario name.

• n_seeds (int) – Seeds per evaluation.

• n_periods (int) – Simulation periods.

• n_workers (int) – Parallel workers.

• base_combined (float, optional) – Pre-computed base combined score. If None, evaluates the base.

Returns:

Results for each swap, sorted by absolute delta.

Return type:

list[SwapResult]

calibration.cost.save_cost_results(results, scenario, path)

Save cost analysis results to JSON.

calibration.cost.run_cost_phase(args, run_dir=None)

CLI entry point for cost phase.