BIF versus PMCMC Benchmarks

Report generated on: 2026-06-05 01:22:22.808227

This report summarizes two preliminary BIF versus PMCMC benchmarks for pypomp. The calculations use GPU-enabled JAX runs and keep the time horizon small enough to test the workflow before scaling to larger Dacca experiments.

The comparison is deliberately narrow: both methods use J=1000 particles, PMCMC uses four chains with 5000 iterations after a 1000-iteration scale sweep, and BIF uses a small tuning grid. The PMCMC runtime shown below is the selected final PMCMC run, not the total PMCMC tuning sweep.

Configuration

	example	T	active parameters	J	PMCMC chains	PMCMC sweep	PMCMC final	PMCMC scale
0	SIR four-parameter	100	gamma, beta1, beta2, beta3	1000	4	1000	5000	2.0
1	Dacca four-parameter	100	gamma, m, epsilon, sigma	1000	4	1000	5000	6.0

Runtime

The speedup column compares the selected BIF configuration against the selected final PMCMC run. It does not count the whole BIF tuning grid or the PMCMC scale sweep.

	example	PMCMC final runtime	BIF selected runtime	PMCMC total runtime	BIF grid total runtime	selected-run speedup	tuning-included speedup
0	SIR four-parameter	511.47	7.25	618.23	43.41	70.52	14.24
1	Dacca four-parameter	156.74	5.44	220.52	13.43	28.83	16.42

Posterior Comparison

The figures compare PMCMC marginal posteriors against the selected BIF importance-weighted approximation. These plots should be read as evidence for proposal quality and posterior shape agreement in these small test cases, not as a final large-scale validation.

Interval Comparison

PMCMC Diagnostics

Dacca Data

BIF Selection

The selected BIF configuration is currently chosen by internal ESS stability and then compared against PMCMC. The full tuning grid is stored in bif_metrics.csv and the derived ranking table.

	example	selected M	selected perturb	ESS fraction	score	grid rows
0	SIR four-parameter	100	0.03	0.302	0.585	4
1	Dacca four-parameter	150	0.10	0.601	0.330	2

Interpretation

For these two small examples, BIF produces posterior marginals that are close enough to the PMCMC reference to justify larger experiments. The measured speedup is substantial because BIF reuses a short IF2-style particle-cloud run and then applies a deconvolution weighting step, while PMCMC repeatedly evaluates the likelihood along an autocorrelated chain.

The comparison is not yet the final paper-scale benchmark. The next checks should include larger Dacca horizons, repeated seeds, and a clearer accounting of PMCMC tuning cost versus selected-run cost.

Source Files

bif_pmcmc_test.py contains the Great Lakes job configuration and delegates to the BIF paper repository scripts. sync_results.py copies compact summaries and figures into this quant report after the raw runs finish.