SPX Fixed-Parameter Likelihood Validation (rep3600)

Particle Filter Distribution Validation: R pomp vs. pypomp

import os
import sys
sys.path.append("..")
import report_utils as ru
from IPython.display import display, HTML

display(HTML(f"<div style='margin-bottom: 20px;'>{ru.nav_bar('loglik')}</div>"))

Estimation  |  Timing & Throughput  |  Scaling  |  Likelihood Evaluation

Introduction

This section evaluates the particle filter’s likelihood calculation itself by running it multiple times on a single, fixed parameter vector (Sun 2024 estimates).

---

Benchmark Settings & Environment

The table below summarizes algorithmic parameters, software environments, and compute hardware recorded in latest.json for each configuration.

PLATFORMS = {
    "R pomp": os.path.join("results", "R"),
    "pypomp (GPU)": os.path.join("results", "gpu"),
}

runs = ru.load_timing_data(PLATFORMS)
display(HTML(ru.build_settings_comparison_html(runs, is_panel=False)))

Setting / Parameter	R pomp	pypomp (GPU)
Algorithmic & Workload Settings
Run Level	4	4
Pfilter Particles ($N_{p,eval}$)	1,000	1,000
Evaluation Replicates ($N_{reps}$)	3,600	3,600
Random Seed	—	631409
Software & Environment
Pomp Framework	pomp 6.3	pypomp 1.0.0rc1
Backend / Engine	R 4.4.0	JAX 0.11.1
Quant Git Commit	8853fc5	630cc23
Run Timestamp	2026-08-05 16:32:31	2026-09-02 18:59:37
Hardware & Compute
Compute Device	Intel(R) Xeon(R) Gold 6254 CPU @ 3.10GHz (36 cores)	Tesla V100-PCIE-16GB (1 GPU)
Slurm Partition	standard	gpu
Slurm Job ID	56537488	59673767

missing = [(label, r["dir"]) for label, r in runs.items() if not r["available"]]
for label, path in missing:
    display(HTML(f"<div class='alert alert-warning'><strong>Missing results for {label}:</strong> Expected at <code>{path}</code>.</div>"))

import os
import numpy as np
import pandas as pd
from scipy.stats import ks_2samp
from plotnine import ggplot, aes, geom_density, labs

os.environ["JAX_PLATFORMS"] = "cpu"

EVAL_CSV_PATH = os.path.join("results", "gpu", "pfilter_logliks.csv")
LL_rep3600 = pd.read_csv(EVAL_CSV_PATH)["logLik"].values.tolist() if os.path.exists(EVAL_CSV_PATH) else []

R_REF_PATH = os.path.join("results", "R", "pfilter_logliks.csv")
LL_rep3600_R = pd.read_csv(R_REF_PATH)["logLik"].values if os.path.exists(R_REF_PATH) else np.array([])

---

Likelihood Evaluation at Fixed Parameter (rep3600 Check)

Kolmogorov-Smirnov (KS) Test

ks_stat, ks_p = ks_2samp(LL_rep3600, LL_rep3600_R)

ks_table = pd.DataFrame({
    "Source": ["pypomp (GPU)", "R pomp"],
    "Replicates": [len(LL_rep3600), len(LL_rep3600_R)],
    "Mean": [np.mean(LL_rep3600), np.mean(LL_rep3600_R)],
    "SD": [np.std(LL_rep3600, ddof=1), np.std(LL_rep3600_R, ddof=1)],
    "Min": [np.min(LL_rep3600), np.min(LL_rep3600_R)],
    "Max": [np.max(LL_rep3600), np.max(LL_rep3600_R)],
})

display(HTML(ks_table.round(3).to_html(classes="table table-striped table-hover", index=False)))
print(
    f"Mean difference: {np.mean(LL_rep3600) - np.mean(LL_rep3600_R):+.4f} nats\n"
    f"Two-sample KS:   D = {ks_stat:.4f}, p = {ks_p:.4g}"
)

Source	Replicates	Mean	SD	Min	Max
pypomp (GPU)	3600	11848.131	2.091	11841.511	11855.983
R pomp	3600	11848.080	2.047	11841.868	11856.854

Mean difference: +0.0505 nats
Two-sample KS:   D = 0.0275, p = 0.1314

---

Density Comparison

rep3600_df = pd.concat([
    pd.DataFrame({"LL": LL_rep3600, "source": "python"}),
    pd.DataFrame({"LL": LL_rep3600_R, "source": "R"})
], ignore_index=True)

(
    ggplot(rep3600_df, aes(x="LL", fill="source", color="source"))
    + geom_density(alpha=0.3)
    + labs(
        title="Likelihood Evaluation at Fixed Parameter (rep3600)",
        subtitle="Log-Likelihood density distribution evaluated at a single benchmark parameter set",
        x="Log-Likelihood (LL)",
        y="Density",
        fill="Source",
        color="Source"
    )
    + ru.scale_fill_premium()
    + ru.scale_color_premium()
    + ru.theme_premium
)

Expectation

We expect the two log-likelihood density curves (pypomp vs pomp) to overlap heavily, confirming statistical consistency of particle filter evaluations across implementations.

---

References

Sun, Weizhe. 2024. “Model Based Inference of Stochastic Volatility via Iterated Filtering.” Undergraduate Honors Thesis, University of Michigan.