Show Code
import os
import sys
sys.path.append("..")
import report_utils as ru
from IPython.display import display, HTML
display(HTML(f"<div style='margin-bottom: 20px;'>{ru.nav_bar('timing')}</div>"))Panel Measles Runtime & Performance Benchmark
Execution Speed, Acceleration, and Scalability: R panelPomp vs. pypomp
Introduction
The panel measles benchmark evaluates pypomp’s performance on a panel model with 4 UK cities (London, Halesworth, Hastings, and Cardiff) under a parameter structure including both shared and unit-specific parameters. Parameter estimation is performed via marginalized panel iterated filtering (MPIF).
Benchmark Settings & Environment
The table below summarizes algorithmic parameters, software environments, and compute hardware recorded in latest.json for each configuration.
Show Code
PLATFORMS = {
"R panelPomp (36 cores)": os.path.join("results", "R"),
"pypomp (GPU)": os.path.join("results", "gpu"),
}
runs = ru.load_timing_data(PLATFORMS)
display(HTML(ru.build_settings_comparison_html(runs, is_panel=True)))| Setting / Parameter | R panelPomp (36 cores) | pypomp (GPU) |
|---|---|---|
| Algorithmic & Workload Settings | ||
| Run Level | 4 | 4 |
| Starting Searches ($N_{starts}$) | 36 | 36 |
| MPIF Iterations ($N_{iter}$) | 100 | 100 |
| MPIF Particles / Unit ($N_{p,fit}$) | 5,000 | 5,000 |
| Pfilter Particles / Unit ($N_{p,eval}$) | 5,000 | 5,000 |
| Evaluation Replicates ($N_{reps}$) | 36 | 36 |
| Unit(s) | London, Halesworth, Hastings, Cardiff (4 units) | London, Halesworth, Hastings, Cardiff (4 units) |
| Shared Parameters | — | R0, sigma, gamma, sigmaSE, cohort, amplitude (6 params) |
| Random Seed | — | 594709947 |
| Software & Environment | ||
| Pomp Framework | panelPomp 1.7.0.0 (pomp 6.3) | pypomp 1.0.0rc1 |
| Backend / Engine | R 4.4.0 | JAX 0.11.1 |
| Quant Git Commit | b22249b |
630cc23 |
| Run Timestamp | 2026-08-12 13:20:17 | 2026-09-02 19:59:03 |
| Hardware & Compute | ||
| Compute Device | Intel(R) Xeon(R) Gold 6254 CPU @ 3.10GHz (36 cores) | NVIDIA RTX PRO 6000 Blackwell Server Edition (1 GPU) |
| Slurm Partition | standard | gpu-rtx6000 |
| Slurm Job ID | 57205144 | 59687806 |
Show Code
missing = [(label, r["dir"]) for label, r in runs.items() if not r["available"]]
for label, path in missing:
display(HTML(f"<div class='alert alert-warning'><strong>Missing results for {label}:</strong> Expected at <code>{path}</code>.</div>"))Runtime & Throughput Comparison
The table below compares execution speed across platforms. Speedup and throughput calculations privilege the cold-start particle filter (pfilter_cold) to account for initial execution and JIT compilation overhead incurred in practice. Throughput evaluates parallel acceleration relative to a single R CPU core (\(Speedup \times N_{cores}\)).
Show Code
timing_df = ru.build_timing_comparison_df(runs, is_panel=True)
display(HTML(timing_df.to_html(classes="table table-striped table-hover", index=False)))| Configuration | MPIF (s) | MPIF Speedup | Pfilter (s) | Pfilter Speedup | Total (s) | Total Speedup | Throughput (vs 1 R CPU core) |
|---|---|---|---|---|---|---|---|
| R panelPomp (36 cores) | 8187.0s (136.45m) | 1.00x | 67.4s (1.12m) | 1.00x | 8254.3s (137.57m) | 1.00x | 36.00x |
| pypomp (GPU) | 167.6s (2.79m) | 48.85x | 75.4s (1.26m) | 0.89x | 243.0s (4.05m) | 33.97x | 1223.05x |
JIT Compilation & Overhead Breakdown
The first pfilter call in JAX incurs one-time JIT compilation overhead. Subsequent warm evaluations run without recompilation.
Show Code
cold_warm_df = ru.build_cold_vs_warm_df(runs)
display(HTML(cold_warm_df.to_html(classes="table table-striped table-hover", index=False)))| Configuration | Pfilter Cold (s) | Pfilter Warm (s) | Compilation Overhead (s) |
|---|---|---|---|
| R panelPomp (36 cores) | 67.37s | 67.37s | 0.00s |
| pypomp (GPU) | 75.37s | 68.68s | 6.69s |
Performance Expectations & Takeaways
- GPU:
pypomp’s throughput on GPU should be thousands of times larger than Rpompwith 1 core.