Show Code
import os
import sys
sys.path.append("..")
import report_utils as ru
from IPython.display import display, HTML, Markdown
display(HTML(f"<div style='margin-bottom: 20px;'>{ru.nav_bar('timing')}</div>"))Dhaka Cholera: Runtime & Performance Benchmark
Execution Speed, Acceleration, and Scalability: R pomp vs. pypomp
Introduction
This benchmark measures the wall-clock execution time and throughput of IF2 parameter estimation and particle filtering (pfilter) for the Dhaka cholera model in pypomp (GPU and multi-threaded CPU) relative to parallel R pomp on CPU. Both implementations start from identical parameter vectors (../starting_parameters.csv) and execute identical workload iterations and particle numbers.
The Dhaka model’s state transition is lightweight but executed across 20 Euler interpolation steps per observation. As such, if this benchmark ever does particularly poorly relative to the others, it likely points to a problem with the Euler step overhead.
Benchmark Settings & Environment
The table below summarizes algorithmic parameters, software environments, and compute hardware recorded in latest.json for each configuration.
Show Code
PLATFORMS = {
"R pomp (36 cores)": os.path.join("results", "R"),
"pypomp (GPU)": os.path.join("results", "gpu"),
"pypomp (CPU, 36 cores)": os.path.join("results", "cpu"),
}
runs = ru.load_timing_data(PLATFORMS)
display(HTML(ru.build_settings_comparison_html(runs, is_panel=False)))| Setting / Parameter | R pomp (36 cores) | pypomp (GPU) | pypomp (CPU, 36 cores) |
|---|---|---|---|
| Algorithmic & Workload Settings | |||
| Run Level | 4 | 4 | 4 |
| Starting Searches ($N_{starts}$) | 36 | 36 | 36 |
| IF2 Iterations ($N_{iter}$) | 100 | 100 | 100 |
| IF2 Particles ($N_p$) | 5,000 | 5,000 | 5,000 |
| Pfilter Particles ($N_{p,eval}$) | 5,000 | 5,000 | 5,000 |
| Evaluation Replicates ($N_{reps}$) | 36 | 36 | 36 |
| Random Seed | — | 631409 | 631409 |
| Software & Environment | |||
| Pomp Framework | pomp 6.3 | pypomp 1.0.0rc1 | pypomp 1.0.0rc1 |
| Backend / Engine | R 4.4.0 | JAX 0.11.1 | JAX 0.11.1 |
| Quant Git Commit | 3646a0c |
630cc23 |
630cc23 |
| Run Timestamp | 2026-08-06 16:54:34 | 2026-09-02 18:35:29 | 2026-09-02 18:37:23 |
| Hardware & Compute | |||
| Compute Device | Intel(R) Xeon(R) Gold 6254 CPU @ 3.10GHz (36 cores) | NVIDIA RTX PRO 6000 Blackwell Server Edition (1 GPU) | Intel(R) Xeon(R) Gold 6154 CPU @ 3.00GHz (36 cores) |
| Slurm Partition | standard | gpu-rtx6000 | standard |
| Slurm Job ID | 56619140 | 59673749 | 59673750 |
Show Code
missing = [(label, r["dir"]) for label, r in runs.items() if not r["available"]]
for label, path in missing:
display(HTML(f"<div class='alert alert-warning'><strong>Missing results for {label}:</strong> Expected at <code>{path}</code>.</div>"))Runtime & Throughput Comparison
The table below compares execution speed across platforms. Speedup and throughput calculations privilege the cold-start particle filter (pfilter_cold) to account for initial execution and JIT compilation overhead incurred in practice. Throughput evaluates parallel acceleration relative to a single R CPU core (\(Speedup \times N_{cores}\)).
Show Code
timing_df = ru.build_timing_comparison_df(runs, is_panel=False)
display(HTML(timing_df.to_html(classes="table table-striped table-hover", index=False)))| Configuration | IF2 (s) | IF2 Speedup | Pfilter (s) | Pfilter Speedup | Total (s) | Total Speedup | Throughput (vs 1 R CPU core) |
|---|---|---|---|---|---|---|---|
| R pomp (36 cores) | 1565.7s (26.10m) | 1.00x | 358.9s (5.98m) | 1.00x | 1924.6s (32.08m) | 1.00x | 36.00x |
| pypomp (GPU) | 30.4s (0.51m) | 51.44x | 9.1s (0.15m) | 39.48x | 39.5s (0.66m) | 48.69x | 1752.94x |
| pypomp (CPU, 36 cores) | 398.3s (6.64m) | 3.93x | 68.7s (1.15m) | 5.22x | 467.0s (7.78m) | 4.12x | 148.35x |
JIT Compilation & Overhead Breakdown
The first pfilter call in JAX incurs one-time JIT compilation overhead. Subsequent warm evaluations run without recompilation.
Show Code
cold_warm_df = ru.build_cold_vs_warm_df(runs)
display(HTML(cold_warm_df.to_html(classes="table table-striped table-hover", index=False)))| Configuration | Pfilter Cold (s) | Pfilter Warm (s) | Compilation Overhead (s) |
|---|---|---|---|
| R pomp (36 cores) | 358.92s | 358.92s | 0.00s |
| pypomp (GPU) | 9.09s | 7.31s | 1.78s |
| pypomp (CPU, 36 cores) | 68.73s | 67.79s | 0.94s |
Performance Expectations & Takeaways
- GPU:
pypomp’s throughput on GPU should be thousands of times larger than Rpompwith 1 core. - CPU: The total speedup for CPU
pypomprelative to Rpompshould be about 4x.