Internal Validity#

Section 3.10.1, Part 1

Internal validity tests whether the model produces qualitatively similar results regardless of the random seed. It runs multiple simulations (default 20) with default parameters and performs five checks.

Checks#

Check	Description
Cross-simulation variance	Key macro statistics should have small variance across seeds
Co-movement structure	Cross-correlations between HP-filtered GDP and other variables at leads/lags should be consistent (Figure 3.9)
AR model fit	Individual seeds fit AR(2); cross-seed average fits AR(1) with parameter ~0.8
Firm size distributions	Should remain non-normal and positively skewed across all seeds
Empirical curves	Phillips, Okun, and Beveridge curves should emerge from each simulation

Usage#

from validation.robustness import (
    run_internal_validity,
    print_internal_validity_report,
    plot_comovements,
    plot_irf,
)

result = run_internal_validity(
    n_seeds=20,
    n_periods=1000,
    burn_in=500,
    n_workers=10,
)

print_internal_validity_report(result)
plot_comovements(result, show=True)
plot_irf(result, show=True)

Parameters#

Parameter	Default	Description
`n_seeds`	20	Number of random seeds to test
`n_periods`	1000	Simulation periods per seed
`burn_in`	500	Burn-in periods to discard
`n_workers`	10	Parallel workers
`max_lag`	4	Maximum lead/lag for cross-correlations
`ar_order_single`	2	AR order for individual seeds
`ar_order_mean`	1	AR order for cross-seed average
`setup_hook`	None	Extension setup callable (e.g., `setup_growth_plus`)

Result Structure#

InternalValidityResult contains:

seed_analyses: Per-seed results (co-movements, AR coefficients, statistics)
mean_comovements / std_comovements: Cross-seed aggregates
mean_ar_coeffs, mean_ar_r_squared: Mean AR parameters
mean_irf: Mean impulse-response function
cross_sim_stats: Summary statistics with CV across seeds
n_collapsed, n_degenerate: Problematic simulation counts

Each SeedAnalysis contains co-movement correlations (5 variables × 9 lags), AR coefficients, IRF, summary statistics, empirical curve correlations, and firm size distribution metrics.

API Reference#

Internal validity analysis (Section 3.10.1, Part 1).

Runs multiple simulations with different random seeds using default parameters, then computes cross-simulation statistics to verify that the model’s qualitative results are robust to stochastic variation.

Key outputs: - Cross-simulation variance of macroeconomic variables - Co-movement analysis at leads and lags (Figure 3.9) - AR model fitting and impulse-response functions - Firm size distribution invariance - Empirical curve persistence (Phillips, Okun, Beveridge)

validation.robustness.internal_validity.setup_growth_plus(sim)[source]#

Attach R&D extension to a simulation (for Growth+ robustness).

Module-level function so it can be pickled by ProcessPoolExecutor.

class validation.robustness.internal_validity.SeedTimeSeries(seed, collapsed, n_periods_actual, gdp, log_gdp, unemployment, avg_productivity, avg_price, real_interest_rate, real_wage, inflation, vacancy_rate, n_firm_bankruptcies, wage_inflation, gdp_growth, unemployment_growth, final_production, final_net_worth)[source]#

Extracted time series from a single simulation run.

All arrays have length n_periods (full series, before burn-in).

seed#

collapsed#

n_periods_actual#

gdp#

log_gdp#

unemployment#

avg_productivity#

avg_price#

real_interest_rate#

real_wage#

inflation#

vacancy_rate#

n_firm_bankruptcies#

wage_inflation#

gdp_growth#

unemployment_growth#

final_production#

final_net_worth#

class validation.robustness.internal_validity.SeedAnalysis(seed, collapsed, comovements, ar_coeffs, ar_order, ar_r_squared, irf, unemployment_mean, unemployment_std, inflation_mean, inflation_std, gdp_growth_mean, gdp_growth_std, real_wage_mean, productivity_mean, phillips_corr, okun_corr, beveridge_corr, firm_size_skewness_sales, firm_size_skewness_net_worth, normality_pvalue_sales, normality_pvalue_net_worth, firm_size_kurtosis_sales, firm_size_kurtosis_net_worth, firm_size_tail_index, peak_lags, wage_productivity_ratio, log_gdp, hp_gdp_cycle, degenerate=False, degenerate_reasons=<factory>, firm_size_shape='')[source]#

Analysis results for a single seed.

seed#

collapsed#

comovements#

ar_coeffs#

ar_order#

ar_r_squared#

irf#

unemployment_mean#

unemployment_std#

inflation_mean#

inflation_std#

gdp_growth_mean#

gdp_growth_std#

real_wage_mean#

productivity_mean#

phillips_corr#

okun_corr#

beveridge_corr#

firm_size_skewness_sales#

firm_size_skewness_net_worth#

normality_pvalue_sales#

normality_pvalue_net_worth#

firm_size_kurtosis_sales#

firm_size_kurtosis_net_worth#

firm_size_tail_index#

peak_lags#

wage_productivity_ratio#

log_gdp#

hp_gdp_cycle#

degenerate = False#

degenerate_reasons#

firm_size_shape = ''#

class validation.robustness.internal_validity.InternalValidityResult(n_seeds, n_periods, burn_in, seed_analyses, baseline_comovements, mean_comovements, std_comovements, mean_ar_coeffs, mean_ar_order, mean_ar_r_squared, mean_irf, cross_sim_stats=<factory>, n_collapsed=0, n_degenerate=0)[source]#

Full internal validity analysis result.

Contains per-seed results and cross-simulation aggregates.

n_seeds#

n_periods#

burn_in#

seed_analyses#

baseline_comovements#

mean_comovements#

std_comovements#

mean_ar_coeffs#

mean_ar_order#

mean_ar_r_squared#

mean_irf#

cross_sim_stats#

n_collapsed = 0#

n_degenerate = 0#

property collapse_rate#: Fraction of seeds that led to economic collapse.

property degenerate_rate#: Fraction of seeds with degenerate dynamics.

validation.robustness.internal_validity.run_internal_validity(n_seeds=20, n_periods=1000, burn_in=500, n_workers=10, max_lag=4, ar_order_single=2, ar_order_mean=1, irf_periods=20, baseline_seed=0, verbose=True, setup_hook=None, collect_config=None, **config_overrides)[source]#

Run internal validity analysis (Section 3.10.1, Part 1).

Runs n_seeds simulations with different random seeds using default parameters, collecting time series for co-movement analysis, AR model fitting, and distribution checks.

Parameters:

n_seeds (int) – Number of random seeds to test.
n_periods (int) – Simulation periods per seed.
burn_in (int) – Burn-in periods to discard from analysis.
n_workers (int) – Parallel workers for simulation execution.
max_lag (int) – Maximum lead/lag for cross-correlation computation.
ar_order_single (int) – AR order for single-seed GDP cycle (book uses AR(2)).
ar_order_mean (int) – AR order for cross-seed average GDP cycle (book uses AR(1)).
irf_periods (int) – Number of periods for impulse-response function.
baseline_seed (int) – Seed to use as the ‘baseline’ reference in co-movement plots.
verbose (bool) – Print progress messages.
setup_hook (callable or None) – Optional function (sim) -> None called after Simulation.init() to attach extension roles, events, and config (e.g. R&D for Growth+). Must be a module-level function for ProcessPoolExecutor pickling — lambdas and closures will fail.
collect_config (dict or None) – Custom collection configuration. When None, uses the default ROBUSTNESS_COLLECT_CONFIG.
**config_overrides – Additional simulation config overrides.

Returns:

Complete analysis with per-seed results and aggregates.

Return type:

InternalValidityResult