Sensitivity Analysis#

Section 3.10.1, Part 2

Univariate sensitivity analysis varies one parameter at a time while holding all others at baseline values. For each parameter value, it runs multiple simulations and computes the same statistics as the internal validity analysis.

The Five Experiments#

#	Experiment	Parameter	Values	Baseline
i	Credit market	`max_H`	1, 2, 3, 4, 6	2
ii	Goods market	`max_Z`	2, 3, 4, 5, 6	2
iii	Labor applications	`max_M`	2, 3, 4, 5, 6	4
iv	Contract length	`theta`	1, 4, 6, 8, 10, 12, 14	8
v	Economy size	multi-param	7 configurations	100/500/10

Key Findings from the Book#

Experiment	Finding
Credit market (H)	General properties stable. As H increases, price index becomes coincident with output; net worth distribution becomes more Pareto-like.
Goods market (Z)	As Z increases, competition rises, production smooths, firm size kurtosis decreases. Real wages become lagging.
Labor (M)	As M decreases, prices become pro-cyclical/lagging, instability rises. As M increases, wages are pushed above productivity.
Contract length (\(\theta\))	Extreme values cause degenerate dynamics. \(\theta=1\): collapse. \(\theta \geq 12\): supply-side breakdown. \(\theta=6\)–10: stable.
Economy size	Proportional scaling preserves co-movements but smooths fluctuations.

Usage#

from validation.robustness import (
    run_sensitivity_analysis,
    print_sensitivity_report,
    plot_sensitivity_comovements,
)

sa = run_sensitivity_analysis(
    experiments=["credit_market", "contract_length"],
    n_seeds=20,
    n_periods=1000,
)

print_sensitivity_report(sa)
for exp in sa.experiments.values():
    plot_sensitivity_comovements(exp, show=True)

Custom Experiments#

Define custom experiments using the Experiment dataclass:

from validation.robustness.experiments import Experiment, EXPERIMENTS

my_experiment = Experiment(
    name="custom_delta",
    description="Sensitivity to depreciation rate",
    param="delta",
    values=[0.05, 0.10, 0.15, 0.20],
    baseline_value=0.10,
)

EXPERIMENTS["custom_delta"] = my_experiment
result = run_sensitivity_analysis(experiments=["custom_delta"])

API Reference#

Univariate sensitivity analysis (Section 3.10.1, Part 2).

Varies one parameter at a time while holding all others at baseline values. For each parameter value, runs multiple simulations with different seeds and computes the same statistics as the internal validity analysis.

The five parameter groups from the book are:

H — local credit markets
Z — local consumption goods markets
M — local labour markets (applications)
theta — employment contracts duration
Economy size and composition

class validation.robustness.sensitivity.ValueResult(label, config_overrides, n_seeds, n_collapsed, mean_comovements, std_comovements, mean_ar_coeffs, mean_ar_r_squared, mean_irf, stats=<factory>, mean_peak_lags=<factory>, n_degenerate=0)[source]#

Aggregated results for one parameter value across multiple seeds.

label#

config_overrides#

n_seeds#

n_collapsed#

mean_comovements#

std_comovements#

mean_ar_coeffs#

mean_ar_r_squared#

mean_irf#

stats#

mean_peak_lags#

n_degenerate = 0#

property collapse_rate#

property degenerate_rate#

class validation.robustness.sensitivity.ExperimentResult(experiment, value_results, baseline_idx)[source]#

Results for one sensitivity experiment (all parameter values).

experiment#

value_results#

baseline_idx#

property baseline#

get_stat_table(stat_name)[source]#: Get a statistic across all values as (label, mean, std) tuples.

class validation.robustness.sensitivity.SensitivityResult(experiments, n_seeds_per_value, n_periods, burn_in)[source]#

Full sensitivity analysis result across all experiments.

experiments#

n_seeds_per_value#

n_periods#

burn_in#

validation.robustness.sensitivity.run_sensitivity_analysis(experiments=None, n_seeds=20, n_periods=1000, burn_in=500, n_workers=10, max_lag=4, ar_order=2, irf_periods=20, verbose=True, setup_hook=None, collect_config=None, **config_overrides)[source]#

Run univariate sensitivity analysis (Section 3.10.1, Part 2).

For each experiment, varies one parameter while holding all others at baseline values. Runs n_seeds simulations per parameter value.

Parameters:

experiments (list[str] or None) – Experiment names to run. None means all experiments.
n_seeds (int) – Number of random seeds per parameter value.
n_periods (int) – Simulation periods per seed.
burn_in (int) – Burn-in periods to discard.
n_workers (int) – Parallel workers for simulation execution.
max_lag (int) – Maximum lead/lag for cross-correlation.
ar_order (int) – AR order for GDP cycle fitting.
irf_periods (int) – Impulse-response function horizon.
verbose (bool) – Print progress messages.
setup_hook (callable or None) – Optional function (sim) -> None called after Simulation.init() to attach extension roles, events, and config. Must be a module-level function for ProcessPoolExecutor pickling.
collect_config (dict or None) – Custom collection configuration. When None, uses the default ROBUSTNESS_COLLECT_CONFIG.
**config_overrides – Additional simulation config overrides applied to all runs.

Returns:

Results for all requested experiments.

Return type:

SensitivityResult

Experiment definitions for robustness analysis (Sections 3.10.1 & 3.10.2).

Defines the five parameter groups from the book’s univariate sensitivity analysis (Section 3.10.1) and the two structural experiments (Section 3.10.2): preferential attachment (PA) toggle and entry neutrality via taxation.

class validation.robustness.experiments.Experiment(name, description, param, values, labels=None, baseline_value=None, setup_fn=None)[source]#

A single sensitivity experiment definition.

Parameters:

name (str) – Short identifier for the experiment.
description (str) – Human-readable description.
param (str | None) – Config parameter name to vary. None for multi-param experiments.
values (list) – Parameter values to test. For multi-param experiments, each value is a dict of config overrides.
labels (list[str] | None) – Display labels for each value. Defaults to str(value).
baseline_value (Any) – The default/baseline value (for highlighting in reports).
setup_fn (callable or None) – Optional function (sim) -> None called after Simulation.init() to attach extension roles, events, and config. Must be a module-level function for ProcessPoolExecutor pickling.

name#

description#

param#

values#

labels = None#

baseline_value = None#

setup_fn = None#

get_labels()[source]#: Return display labels for each parameter value.

get_config(idx)[source]#: Return config overrides for the i-th parameter value.

validation.robustness.experiments.setup_taxation(sim)[source]#

Attach taxation extension to a simulation.

Module-level function so it can be pickled by ProcessPoolExecutor.