============================================================ UNCERTAINTY VALIDATION: Posterior Predictive Coverage ============================================================ Following Claassen (2019) validation framework Posterior predictive intervals account for both position uncertainty AND observation-level measurement noise. Using specified run directory: /projects/party4d/archive/party2d_replication/outputs/model_outputs/latest/run_2026-06-12_09-34-03 Loaded expert_dim.csv: 22994 observations Unique rr values: 4261 Item indices (var_exp_dim): [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12] Dimensions (dim_idx_exp): [1, 2] Loading 4 chain files (selective columns)... Need 8547 columns (4261 rr × 2 dims + 24 item params + 1 phi) Found 8547/8547 columns in chains Loading chain 1: chain_1.csv... 2000 samples, 39.3s Loading chain 2: chain_2.csv... 2000 samples, 22.4s Loading chain 3: chain_3.csv... 2000 samples, 24.5s Loading chain 4: chain_4.csv... 2000 samples, 16.7s Combined: 8000 total posterior draws V5 detected: using Beta(phi*K*mu, phi*K*(1-mu)) with per-observation K Computing posterior predictive coverage (95% level) Expert observations: 22994 Posterior draws: 8000 Progress: 5.0% (1149 / 22994) Progress: 10.0% (2298 / 22994) Progress: 15.0% (3447 / 22994) Progress: 20.0% (4596 / 22994) Progress: 25.0% (5745 / 22994) Progress: 30.0% (6894 / 22994) Progress: 35.0% (8043 / 22994) Progress: 40.0% (9192 / 22994) Progress: 45.0% (10341 / 22994) Progress: 50.0% (11490 / 22994) Progress: 55.0% (12639 / 22994) Progress: 60.0% (13788 / 22994) Progress: 65.0% (14937 / 22994) Progress: 70.0% (16086 / 22994) Progress: 75.0% (17235 / 22994) Progress: 80.0% (18384 / 22994) Progress: 84.9% (19533 / 22994) Progress: 89.9% (20682 / 22994) Progress: 94.9% (21831 / 22994) Progress: 99.9% (22980 / 22994) Progress: 100.0% (22994 / 22994) ============================================================ POSTERIOR PREDICTIVE COVERAGE (95%) ============================================================ economic left-right : 89.8% [89.1%, 90.5%] (6698/7455) By survey source: Project N PPC V-Party 5645 87.5% CHES 1177 97.3% POPPA 384 98.7% GPS 249 94.0% By decade: Decade N PPC 1970 658 88.1% 1980 712 88.9% 1990 1454 88.0% 2000 1758 89.4% 2010 2407 90.2% 2020 466 99.1% cultural cosmopolitan--traditionalist : 84.8% [84.2%, 85.3%] (13170/15539) By survey source: Project N PPC V-Party 14114 83.8% CHES 1176 94.3% GPS 249 91.2% By decade: Decade N PPC 1970 1633 86.2% 1980 1772 87.9% 1990 3485 85.5% 2000 3965 84.3% 2010 4426 82.1% 2020 258 97.7% Saved detailed posterior-predictive results to: revision/outputs/ppc ============================================================ RECOMPUTING WITH 80% LEVEL (Claassen comparison) ============================================================ V5 detected: using Beta(phi*K*mu, phi*K*(1-mu)) with per-observation K Computing posterior predictive coverage (80% level) Expert observations: 22994 Posterior draws: 8000 Progress: 5.0% (1149 / 22994) Progress: 10.0% (2298 / 22994) Progress: 15.0% (3447 / 22994) Progress: 20.0% (4596 / 22994) Progress: 25.0% (5745 / 22994) Progress: 30.0% (6894 / 22994) Progress: 35.0% (8043 / 22994) Progress: 40.0% (9192 / 22994) Progress: 45.0% (10341 / 22994) Progress: 50.0% (11490 / 22994) Progress: 55.0% (12639 / 22994) Progress: 60.0% (13788 / 22994) Progress: 65.0% (14937 / 22994) Progress: 70.0% (16086 / 22994) Progress: 75.0% (17235 / 22994) Progress: 80.0% (18384 / 22994) Progress: 84.9% (19533 / 22994) Progress: 89.9% (20682 / 22994) Progress: 94.9% (21831 / 22994) Progress: 99.9% (22980 / 22994) Progress: 100.0% (22994 / 22994) ============================================================ POSTERIOR PREDICTIVE COVERAGE (80%) ============================================================ economic left-right : 75.9% [74.9%, 76.8%] (5655/7455) By survey source: Project N PPC V-Party 5645 71.5% CHES 1177 90.1% POPPA 384 92.2% GPS 249 82.7% By decade: Decade N PPC 1970 658 69.3% 1980 712 73.5% 1990 1454 73.8% 2000 1758 74.1% 2010 2407 77.0% 2020 466 95.7% cultural cosmopolitan--traditionalist : 67.9% [67.1%, 68.6%] (10544/15539) By survey source: Project N PPC V-Party 14114 66.3% CHES 1176 83.9% GPS 249 77.5% By decade: Decade N PPC 1970 1633 68.4% 1980 1772 69.8% 1990 3485 67.5% 2000 3965 68.6% 2010 4426 65.0% 2020 258 94.2% Saved: revision/outputs/ppc/uncertainty_cic_summary_2026-08-12_16-50-24.csv Saved: revision/outputs/ppc/uncertainty_cic_80pct_summary_2026-08-12_16-50-24.csv Saved: revision/outputs/ppc/uncertainty_galtan_by_project_2026-08-12_16-50-24.csv Saved: revision/outputs/ppc/uncertainty_economic_lr_by_project_2026-08-12_16-50-24.csv ============================================================ COMPARISON WITH CLAASSEN (2019) BENCHMARKS ============================================================ Claassen's result: CIC (80% CI): 60.3% (Using credible intervals for θ, not posterior predictive) Our results (posterior predictive): ------------------------------------------------------------ Dimension PPC 95% PPC 80% Status ------------------------------------------------------------ economic_lr 89.8% 75.9% OK galtan 84.8% 67.9% OK ------------------------------------------------------------ Interpretation: 95% PPC ~95% = well-calibrated uncertainty 80% PPC > 60% = exceeds Claassen (2019) benchmark ============================================================ VALIDATION COMPLETE ============================================================