IF2 and particle filter throughput vs. particles in flight on GPU, all models

Published

October 22, 2026

Show Code 
import os

import pandas as pd
import report_utils as ru  # puts tests/ on sys.path
import scaling
from IPython.display import HTML, display
from plotnine import aes, facet_wrap, geom_line, geom_point, ggplot, labs, scale_x_log10, scale_y_log10

ru.nav_bar("scaling")
Likelihood Evaluation  |  Timing & Throughput  |  Scaling  |  All reports

Introduction

This benchmark finds how many concurrent particles it takes to saturate the GPU for each model. The number of particles in flight during IF2 is \(J \times \text{chains}\). We grow it two ways: by raising \(J\) at a fixed number of chains, and by adding chains at a fixed \(J\).

  • SPX: the stochastic volatility model of Sun (2024).
  • Dhaka cholera: nstep=20 Euler steps per monthly observation.
  • Measles: the discrete model (001b), London.
  • Panel measles: the 4-unit panel (001b; London, Halesworth, Hastings, Cardiff) under block IF2. Units are filtered one after another inside each iteration, so \(J\) is particles per unit and one iteration does roughly four single-unit filter passes. \(M\) and reps are smaller than in the single-unit tests to keep the run near 20 minutes.

Iteration counts are kept small on purpose. Each grid point runs mif twice with the same \(M\). The first call pays for JIT compilation, and the second is timed to give the steady-state cost per iteration. The key quantity is IF2 ns / particle / iter. Each grid point runs in its own subprocess so that peak VRAM is measured per configuration.

Show Code 
MODELS = ru.models("scaling")
runs = {m: ru.load_runs(m, "scaling") for m in MODELS}
SWEEP = {"particles": "Varying J (fixed chains)", "chains": "Varying chains (fixed J)"}

frames = {}
for m in MODELS:
    csv_path = os.path.join(ru.results_dir(ru.MODELS[m]["scaling"]["dir"]), "scaling.csv")
    label = ru.MODELS[m]["label"]
    if not os.path.exists(csv_path):
        display(HTML(
            f"<div class='alert alert-info'><strong>{label} not run yet:</strong> "
            f"<code>python scripts/run_tests.py run tests/{m}/scaling/run.py --run-level 4</code></div>"
        ))
        continue
    df = pd.read_csv(csv_path)
    if "error" in df.columns and df["error"].notna().any():
        items = "".join(
            f"<li>{r.scaling_type} J={r.J} chains={r.chains}: <pre>{r.error}</pre></li>"
            for r in df[df["error"].notna()].itertuples()
        )
        display(HTML(
            f"<div class='alert alert-warning'><strong>{label}, failed grid points:</strong><ul>{items}</ul></div>"
        ))
    frames[m] = df.dropna(subset=["mif_per_iter_seconds"]).assign(Model=label)

df = pd.concat(frames.values(), ignore_index=True)
df["Model"] = pd.Categorical(df["Model"], categories=df["Model"].unique())
df["Sweep"] = df["scaling_type"].map(SWEEP)
df["ns_per_particle"] = df["mif_per_iter_seconds"] / df["total_particles_mif"] * 1e9
df["pf_ns_per_particle"] = df["pfilter_warm_seconds"] / df["total_particles_pfilter"] * 1e9

Visual review

  • Where the GPU saturates: for each model, IF2 ns / particle / iter should fall as the number of particles in flight grows, then level off.
  • No grid point should be listed as failed above.

Where the GPU saturates

Show Code 
def sweep_plot(y, x, title, ylab, xlab="Particles in flight (J × chains)"):
    return (
        ggplot(df, aes(x, y, color="Model"))
        + geom_line(size=1.2)
        + geom_point(size=2.5)
        + facet_wrap("~Sweep")
        + scale_x_log10()
        + scale_y_log10()
        + labs(title=title, x=xlab, y=ylab, color="Model")
        + ru.scale_color_premium()
        + ru.theme_premium
    )


display(sweep_plot("mif_per_iter_seconds", "total_particles_mif",
                   "IF2 time per iteration vs. particles in flight", "IF2 seconds / iteration"))
display(sweep_plot("ns_per_particle", "total_particles_mif",
                   "IF2 cost per particle per iteration", "ns / particle / iteration"))
display(sweep_plot("pf_ns_per_particle", "total_particles_pfilter",
                   "Particle filter cost per particle (warm)", "ns / particle / filter pass",
                   xlab="Total pfilter particles (J × chains × reps)"))

VRAM

Show Code 
if df["peak_vram_mb"].notna().any():
    display(sweep_plot("peak_vram_mb", "total_particles_mif",
                       "Peak VRAM vs. particles in flight", "Peak VRAM (MB)"))

Results by model

Show Code 
for m, mdf in frames.items():
    print(ru.heading(m))
    for kind, sort_by in [("particles", "J"), ("chains", "chains")]:
        print(f"\n**{SWEEP[kind]}**\n")
        table = scaling.cost_table(mdf[mdf["scaling_type"] == kind], sort_by)
        print(ru.raw_html(table.to_html(classes="table table-striped table-hover", index=False)))

SPX

Varying J (fixed chains)

J Chains J × chains IF2 compile + M iters (s) IF2 per iter (s) IF2 ns / particle / iter IF2 marginal ns / particle pfilter warm (s) pfilter ns / particle Peak VRAM (MB)
1000 120 120,000 5.62 0.2050 1708.1 — 1.75 1455.79 77.0
2500 120 300,000 6.17 0.2539 846.4 272.0 1.89 628.42 103.0
5000 120 600,000 7.25 0.3455 575.8 305.1 2.47 412.49 146.0
10000 120 1,200,000 8.69 0.4902 408.5 241.2 3.11 259.37 277.0
20000 120 2,400,000 13.79 0.9730 405.4 402.3 4.75 197.87 448.0
40000 120 4,800,000 36.69 3.2520 677.5 949.6 10.57 220.13 969.0

Varying chains (fixed J)

J Chains J × chains IF2 compile + M iters (s) IF2 per iter (s) IF2 ns / particle / iter IF2 marginal ns / particle pfilter warm (s) pfilter ns / particle Peak VRAM (MB)
1000 125 125,000 5.11 0.1673 1338.8 — 1.45 1161.12 77.0
1000 250 250,000 5.67 0.2039 815.4 292.1 1.67 667.25 77.0
1000 500 500,000 6.45 0.2850 570.0 324.6 1.96 391.77 152.0
1000 1000 1,000,000 7.73 0.4250 425.0 280.0 2.68 268.02 249.0
1000 2000 2,000,000 11.76 0.7921 396.1 367.2 3.91 195.41 419.0
1000 4000 4,000,000 28.62 2.4910 622.7 849.4 7.33 183.21 779.0

Dhaka cholera

Varying J (fixed chains)

J Chains J × chains IF2 compile + M iters (s) IF2 per iter (s) IF2 ns / particle / iter IF2 marginal ns / particle pfilter warm (s) pfilter ns / particle Peak VRAM (MB)
1000 120 120,000 7.32 0.1898 1582.0 — 1.39 1155.10 150.0
2000 120 240,000 7.68 0.2425 1010.5 438.9 1.69 705.97 242.0
5000 120 600,000 11.83 0.6129 1021.5 1028.9 2.78 463.83 600.0
10000 120 1,200,000 19.38 1.2682 1056.8 1092.1 3.71 308.88 1297.0
15000 120 1,800,000 51.71 4.5590 2532.8 5484.8 4.99 277.34 1618.0
20000 120 2,400,000 68.60 6.2567 2607.0 2829.5 6.33 263.61 1827.0

Varying chains (fixed J)

J Chains J × chains IF2 compile + M iters (s) IF2 per iter (s) IF2 ns / particle / iter IF2 marginal ns / particle pfilter warm (s) pfilter ns / particle Peak VRAM (MB)
1000 125 125,000 7.04 0.1839 1471.4 — 1.35 1078.87 150.0
1000 250 250,000 7.90 0.2667 1066.7 661.9 1.66 663.13 278.0
1000 500 500,000 10.78 0.4867 973.4 880.1 2.49 498.30 450.0
1000 1000 1,000,000 14.97 0.9375 937.5 901.6 3.39 338.83 899.0
1000 1500 1,500,000 35.59 2.9782 1985.5 4081.5 4.35 289.88 1365.0
1000 2000 2,000,000 56.06 4.9613 2480.6 3966.1 5.54 276.78 1687.0

Measles

Varying J (fixed chains)

J Chains J × chains IF2 compile + M iters (s) IF2 per iter (s) IF2 ns / particle / iter IF2 marginal ns / particle pfilter warm (s) pfilter ns / particle Peak VRAM (MB)
1000 120 120,000 10.02 0.1768 1473.6 — 1.83 1525.28 80.0
2000 120 240,000 10.83 0.2530 1054.1 634.6 2.49 1037.94 128.0
5000 120 600,000 16.23 0.6356 1059.4 1062.8 6.98 1163.38 249.0
10000 120 1,200,000 24.48 1.4157 1179.8 1300.2 10.86 905.33 430.0
15000 120 1,800,000 27.83 1.7847 991.5 615.0 15.09 838.10 668.0
20000 120 2,400,000 37.09 2.6687 1112.0 1473.3 19.98 832.31 903.0
30000 120 3,600,000 52.70 4.2100 1169.4 1284.4 29.53 820.29 1181.0
40000 120 4,800,000 67.59 5.7082 1189.2 1248.5 56.63 1179.77 1569.0

Varying chains (fixed J)

J Chains J × chains IF2 compile + M iters (s) IF2 per iter (s) IF2 ns / particle / iter IF2 marginal ns / particle pfilter warm (s) pfilter ns / particle Peak VRAM (MB)
1000 125 125,000 10.20 0.1835 1467.9 — 1.89 1513.82 80.0
1000 250 250,000 11.53 0.2734 1093.8 719.6 2.71 1082.13 128.0
1000 500 500,000 15.90 0.6003 1200.5 1307.2 5.31 1061.17 257.0
1000 1000 1,000,000 22.08 1.1807 1180.7 1161.0 9.39 939.33 358.0
1000 1500 1,500,000 26.61 1.6876 1125.1 1013.8 13.17 878.24 551.0
1000 2000 2,000,000 32.79 2.2835 1141.7 1191.6 16.61 830.26 758.0
1000 3000 3,000,000 45.85 3.4923 1164.1 1208.8 25.67 855.61 1085.0
1000 4000 4,000,000 56.43 4.5819 1145.5 1089.6 34.61 865.36 1454.0

Panel measles

Varying J (fixed chains)

J Chains J × chains IF2 compile + M iters (s) IF2 per iter (s) IF2 ns / particle / iter IF2 marginal ns / particle pfilter warm (s) pfilter ns / particle Peak VRAM (MB)
1000 100 100,000 11.03 0.7033 7033.5 — 2.17 7218.84 80.0
2000 100 200,000 12.16 1.0340 5170.2 3306.9 3.13 5219.36 81.0
5000 100 500,000 18.09 2.5256 5051.3 4972.0 5.45 3631.50 204.0
10000 100 1,000,000 25.04 4.7261 4726.1 4400.9 10.03 3343.41 416.0
15000 100 1,500,000 28.96 6.1754 4116.9 2898.6 15.56 3457.56 576.0
20000 100 2,000,000 37.52 8.9944 4497.2 5638.1 30.30 5050.48 588.0
30000 100 3,000,000 53.90 14.0891 4696.4 5094.7 44.99 4998.41 961.0

Varying chains (fixed J)

J Chains J × chains IF2 compile + M iters (s) IF2 per iter (s) IF2 ns / particle / iter IF2 marginal ns / particle pfilter warm (s) pfilter ns / particle Peak VRAM (MB)
1000 125 125,000 10.88 0.7385 5908.0 — 2.23 5949.72 80.0
1000 250 250,000 12.51 1.1410 4563.8 3219.6 3.32 4431.48 102.0
1000 500 500,000 17.20 2.3814 4762.8 4961.8 6.20 4136.24 204.0
1000 1000 1,000,000 24.61 4.7423 4742.3 4721.9 11.93 3976.63 418.0
1000 1500 1,500,000 30.59 6.8492 4566.2 4213.8 18.15 4033.47 578.0
1000 2000 2,000,000 37.41 9.0165 4508.3 4334.6 26.44 4406.57 602.0
1000 3000 3,000,000 53.62 14.1141 4704.7 5097.6 39.41 4379.29 964.0

Appendix: settings and environment

Show Code 
print(ru.settings_sections(runs))

SPX

Setting / Parameter pypomp (GPU)
Algorithmic & Workload Settings
Run Level 4
IF2 Iterations ($N_{iter}$) 10
Evaluation Replicates ($N_{reps}$) 10
Random Seed 631409
Software & Environment
Pomp Framework pypomp 1.0.5
Backend / Engine JAX 0.11.2
Quant Git Commit e92c48a
Run Timestamp 2026-09-21 21:29:54
Hardware & Compute
Compute Device NVIDIA RTX PRO 6000 Blackwell Server Edition (1 GPU)
Slurm Partition gpu-rtx6000
Slurm Job ID 61671218

Dhaka cholera

Setting / Parameter pypomp (GPU)
Algorithmic & Workload Settings
Run Level 4
IF2 Iterations ($N_{iter}$) 10
Evaluation Replicates ($N_{reps}$) 10
Random Seed 631409
Software & Environment
Pomp Framework pypomp 1.0.5
Backend / Engine JAX 0.11.2
Quant Git Commit 5e9c248
Run Timestamp 2026-09-21 16:29:00
Hardware & Compute
Compute Device NVIDIA RTX PRO 6000 Blackwell Server Edition (1 GPU)
Slurm Partition gpu-rtx6000
Slurm Job ID 61640377

Measles

Setting / Parameter pypomp (GPU)
Algorithmic & Workload Settings
Run Level 4
IF2 Iterations ($N_{iter}$) 10
Evaluation Replicates ($N_{reps}$) 10
Unit(s) London
Random Seed 594709947
Software & Environment
Pomp Framework pypomp 1.0.8
Backend / Engine JAX 0.11.2
Quant Git Commit 4f0dcfd
Run Timestamp 2026-10-06 07:05:05
Hardware & Compute
Compute Device NVIDIA RTX PRO 6000 Blackwell Server Edition (1 GPU)
Slurm Partition gpu-rtx6000
Slurm Job ID 63288848

Panel measles

Setting / Parameter pypomp (GPU)
Algorithmic & Workload Settings
Run Level 4
MPIF Iterations ($N_{iter}$) 3
Pfilter Particles / Unit ($N_{p,eval}$) —
Evaluation Replicates ($N_{reps}$) 3
Unit(s) London, Halesworth, Hastings, Cardiff (4 units)
Random Seed 594709947
Software & Environment
Pomp Framework pypomp 1.0.8
Backend / Engine JAX 0.11.2
Quant Git Commit 4f0dcfd
Run Timestamp 2026-10-06 07:18:16
Hardware & Compute
Compute Device NVIDIA RTX PRO 6000 Blackwell Server Edition (1 GPU)
Slurm Partition gpu-rtx6000
Slurm Job ID 63288849

References

Sun, Weizhe. 2024. “Model Based Inference of Stochastic Volatility via Iterated Filtering.” Undergraduate Honors Thesis, University of Michigan.