Does the CSP Opportunity Score actually predict anything?
Every Sunday we replay a reference trade — sell a 30-delta, 30-DTE cash-secured put — every month across ~2 years of daily bars for the top 100 tickers in our universe. Trades are bucketed by current composite score and we report three answers: a decile P&L chart (does the score sort trades?), a rolling Spearman IC dashboard (which sub-factors carry signal?), and a reliability diagram (are the predicted win-rates calibrated?).
1. Decile P&L — reference trade sorted by score
Each column is a decile of the current composite score. Y-axis is mean P&L per contract at expiry, walking forward monthly. If the score is informative, top deciles should be higher than bottom deciles. Overall Spearman IC and monotonicity of the bucket means are printed above the chart.
2. Rolling Spearman IC — score vs realized P&L
Cross-sectional Spearman rank correlation between the composite score and the same month's realized reference-trade P&L. Rolling 6-month window. Above zero and stable = signal.
3. Calibration — predicted vs realized win rate
Reliability diagram: predicted probability of finishing OTM (≈ 1 − |Δ|) bucketed into 10 bins vs the observed OTM frequency. Points on the diagonal = perfectly calibrated. Brier score and log-loss are the summary numbers.
- Implied vol is proxied from Yang-Zhang RV × (1 + 0.10 VRP tilt). Real historical option surfaces aren't cheaply available at this scale on our data plan — the backtest measures the strategy's structural premium capture, not tick-by-tick executable P&L.
- Deciles are tagged with the current composite score (regime-stationary assumption). When regime shifts sharply, historical decile ordering degrades. Live-tracked trades (grown via the calculator) fill the gap over time.
- Sample sizes per decile are small (~n_tickers × n_months ÷ 10). Confidence intervals are wide — don't over-interpret single-month IC dips.
- No spread/slippage in the backtest premium. Live executable ROC is meaningfully lower on illiquid names.