Show Code
import os
import sys
sys.path.append("..")
import report_utils as ru
from IPython.display import display, HTML
display(HTML(f"<div style='margin-bottom: 20px;'>{ru.nav_bar('timing')}</div>"))SPX Runtime & Performance Benchmark
Execution Speed, Acceleration, and Scalability: R pomp vs. pypomp
Introduction
This benchmark evaluates execution speed and throughput of the S&P 500 (SPX) stochastic volatility model (Sun 2024) using pypomp on CPU and GPU relative to R pomp on CPU.
The SPX model features lightweight random number generation and a single rproc transition step per observation, making it sensitive to framework dispatch overhead during IF2 parameter optimization and particle filtering (pfilter).
Benchmark Settings & Environment
The table below summarizes algorithmic parameters, software environments, and compute hardware recorded in latest.json for each configuration.
Show Code
PLATFORMS = {
"R pomp (36 cores)": os.path.join("results", "R"),
"pypomp (GPU)": os.path.join("results", "gpu"),
"pypomp (CPU, 36 cores)": os.path.join("results", "cpu"),
}
runs = ru.load_timing_data(PLATFORMS)
display(HTML(ru.build_settings_comparison_html(runs, is_panel=False)))| Setting / Parameter | R pomp (36 cores) | pypomp (GPU) | pypomp (CPU, 36 cores) |
|---|---|---|---|
| Algorithmic & Workload Settings | |||
| Run Level | 4 | 4 | 4 |
| Starting Searches ($N_{starts}$) | 36 | 36 | 36 |
| IF2 Iterations ($N_{iter}$) | 200 | 200 | 200 |
| IF2 Particles ($N_p$) | 1,000 | 1,000 | 1,000 |
| Pfilter Particles ($N_{p,eval}$) | 1,000 | 1,000 | 1,000 |
| Evaluation Replicates ($N_{reps}$) | 24 | 24 | 24 |
| Random Seed | — | 631409 | 631409 |
| Software & Environment | |||
| Pomp Framework | pomp 6.3 | pypomp 1.0.0rc1 | pypomp 1.0.0rc1 |
| Backend / Engine | R 4.4.0 | JAX 0.11.1 | JAX 0.11.1 |
| Quant Git Commit | 3646a0c |
630cc23 |
630cc23 |
| Run Timestamp | 2026-08-06 16:10:58 | 2026-09-02 18:43:18 | 2026-09-02 18:36:48 |
| Hardware & Compute | |||
| Compute Device | Intel(R) Xeon(R) Gold 6154 CPU @ 3.00GHz (36 cores) | NVIDIA RTX PRO 6000 Blackwell Server Edition (1 GPU) | Intel(R) Xeon(R) Gold 6154 CPU @ 3.00GHz (36 cores) |
| Slurm Partition | standard | gpu-rtx6000 | standard |
| Slurm Job ID | 56616028 | 59673769 | 59673770 |
Show Code
missing = [(label, r["dir"]) for label, r in runs.items() if not r["available"]]
for label, path in missing:
display(HTML(f"<div class='alert alert-warning'><strong>Missing results for {label}:</strong> Expected at <code>{path}</code>.</div>"))Runtime & Throughput Comparison
The table below compares execution speed across platforms. Speedup and throughput calculations privilege the cold-start particle filter (pfilter_cold) to account for initial execution and JIT compilation overhead incurred in practice. Throughput evaluates parallel acceleration relative to a single R CPU core (\(Speedup \times N_{cores}\)).
Show Code
timing_df = ru.build_timing_comparison_df(runs, is_panel=False)
display(HTML(timing_df.to_html(classes="table table-striped table-hover", index=False)))| Configuration | IF2 (s) | IF2 Speedup | Pfilter (s) | Pfilter Speedup | Total (s) | Total Speedup | Throughput (vs 1 R CPU core) |
|---|---|---|---|---|---|---|---|
| R pomp (36 cores) | 702.5s (11.71m) | 1.00x | 36.4s (0.61m) | 1.00x | 738.9s (12.32m) | 1.00x | 36.00x |
| pypomp (GPU) | 60.2s (1.00m) | 11.68x | 8.4s (0.14m) | 4.33x | 68.6s (1.14m) | 10.78x | 388.03x |
| pypomp (CPU, 36 cores) | 282.5s (4.71m) | 2.49x | 7.3s (0.12m) | 4.96x | 289.9s (4.83m) | 2.55x | 91.77x |
JIT Compilation & Overhead Breakdown
The first pfilter call in JAX incurs one-time JIT compilation overhead. Subsequent warm evaluations run without recompilation.
Show Code
cold_warm_df = ru.build_cold_vs_warm_df(runs)
display(HTML(cold_warm_df.to_html(classes="table table-striped table-hover", index=False)))| Configuration | Pfilter Cold (s) | Pfilter Warm (s) | Compilation Overhead (s) |
|---|---|---|---|
| R pomp (36 cores) | 36.38s | 36.38s | 0.00s |
| pypomp (GPU) | 8.40s | 6.84s | 1.55s |
| pypomp (CPU, 36 cores) | 7.33s | 6.58s | 0.75s |
Performance Expectations & Takeaways
- GPU:
pypomp’s throughput on GPU should be hundreds of times larger than Rpompwith 1 core. - CPU:
pypompon CPU should run at least twice as fast as Rpompon CPU.
References
Sun, Weizhe. 2024. “Model Based Inference of Stochastic Volatility via Iterated Filtering.” Undergraduate Honors Thesis, University of Michigan.