armin/party2d
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Keep diagnostics release to one PDF report

Browse Source
This commit is contained in:
aseimel
2026-06-15 16:48:41 +02:00
parent e2b0b1a025
commit c53c1e8ba2
10 changed files with 137 additions and 499 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
-1
View File
@@ -9,7 +9,6 @@ data/releases/
!data/releases/
data/releases/*
!data/releases/SHA256SUMS
!data/releases/party_2d_diagnostics_report_*.md
!data/releases/party_2d_diagnostics_report_*.pdf
data/*.csv
!data/text_data.csv
README.md
+1 -1
View File
@@ -87,7 +87,7 @@ After estimation and validation have been run, regenerate the diagnostics report
Rscript diagnostics/generate_diagnostics.R
```
The report is written to `diagnostics/generated/diagnostics_report.md` and copied to `data/releases/party_2d_diagnostics_report_v0.md`.
The report is written to `diagnostics/generated/diagnostics_report.pdf` and copied to `data/releases/party_2d_diagnostics_report_v0.pdf`.
## Data inputs
data/releases/SHA256SUMS
+1 -2
View File
@@ -1,4 +1,3 @@
b110155089daec726f781c5ab49d4ba5cb378f4402d1eddbbf34007b4b734dc8 party_2d_election_year_panel_v0.csv.gz
86773993d65f34d5813ca3ce1ebdde643d1c3a9993ccc342301fae3d1b106304 party_2d_annual_model_output_v0.csv.gz
bb69e71165dd3ea0276da3046fea6d1bf0422ddf48c71686ed498069fd323e26 party_2d_diagnostics_report_v0.md
11cbdfd3983bf2edc22bb18ca0b7c667f35a90b7081a74b9a16abc6824b41438 party_2d_diagnostics_report_v0.pdf
58bbe114494126387a9b4231721779f35ebc1788b7ab65cfe6f8e6dec2ce4e2d party_2d_diagnostics_report_v0.pdf
data/releases/party_2d_diagnostics_report_v0.md
-241
View File
@@ -1,241 +0,0 @@
# Diagnostics report
Generated: 2026-06-15 16:40:02 CEST
Release: v0
Model positions source: `party_positions_2026-06-12_16-35-01.csv`
## Purpose
The purpose of this report is to provide the appendix-style diagnostics that document how the released party-position estimates are constructed, checked, and validated. The main article reports the central validation evidence; this report keeps the larger technical tables with the release so readers can inspect item coverage, source coverage, coding orientation, harmonization, convergence, posterior uncertainty, and validation details in one reproducible place.
## Overview
This report follows the structure of the technical appendix material: data and item coverage, coding and scale orientation, party-union harmonization, construct checks, model convergence, and validation. It is generated from the model-ready inputs and completed model outputs; it is not part of the raw-data setup workflow.
## Data and item coverage
The model combines text-coded item counts, dimension-specific expert placements, and general left-right expert placements. Text items enter as positive/sample counts, expert items enter as aggregated ratings with scale and expert-count information, and general left-right ratings anchor the relationship between the two dimensions.
| source_file | dimension | items |
| --- | --- | --- |
| expert.csv | cultural cosmopolitan--traditionalist | 7 |
| expert.csv | economic left-right | 5 |
| lr_data.csv | general left-right | 3 |
| text_data.csv | cultural cosmopolitan--traditionalist | 11 |
| text_data.csv | economic left-right | 6 |
### Source coverage
| source_file | source | items | observations | party_years | parties | countries | min_year | max_year |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| expert.csv | CHES | 2 | 2639 | 1320 | 390 | 44 | 1999 | 2024 |
| expert.csv | GPS | 2 | 538 | 271 | 271 | 62 | 2019 | 2019 |
| expert.csv | POPPA | 1 | 413 | 413 | 225 | 31 | 2018 | 2023 |
| expert.csv | V-Party | 7 | 21488 | 3076 | 589 | 65 | 1970 | 2019 |
| lr_data.csv | CHES | 1 | 1320 | 1320 | 390 | 44 | 1999 | 2024 |
| lr_data.csv | Morgan | 1 | 471 | 471 | 72 | 11 | 1945 | 1973 |
| lr_data.csv | POPPA | 1 | 416 | 416 | 225 | 31 | 2018 | 2023 |
| text_data.csv | Manifesto Project | 8 | 36008 | 4501 | 713 | 65 | 1920 | 2025 |
| text_data.csv | PolDem | 9 | 2194 | 406 | 93 | 15 | 1972 | 2017 |
## Data coding and item orientation
All indicators are oriented toward the two reported dimensions: economic left-right and cultural cosmopolitan--traditionalist. For interpretability, generated diagnostics report whether higher observed values point toward the public high pole or are reversed for reporting. Original source variable names are preserved in the tables.
### Reversed items
| item | source | dimension | higher_values_indicate | observations | min_year | max_year |
| --- | --- | --- | --- | --- | --- | --- |
| internationalism_manifesto | Manifesto Project | cultural cosmopolitan--traditionalist | cosmopolitan | 4501 | 1920 | 2025 |
| multiculturalism_manifesto | Manifesto Project | cultural cosmopolitan--traditionalist | cosmopolitan | 4501 | 1920 | 2025 |
| economic_intervention_manifesto | Manifesto Project | economic left-right | pro_welfare | 4501 | 1920 | 2025 |
| social_services_manifesto | Manifesto Project | economic left-right | pro_welfare | 4501 | 1920 | 2025 |
| cultlib_poldem | PolDem | cultural cosmopolitan--traditionalist | cosmopolitan | 299 | 1972 | 2017 |
| euro_poldem | PolDem | cultural cosmopolitan--traditionalist | cosmopolitan | 93 | 1978 | 2017 |
| europe_poldem | PolDem | cultural cosmopolitan--traditionalist | cosmopolitan | 217 | 1972 | 2017 |
| immig_poldem | PolDem | cultural cosmopolitan--traditionalist | cosmopolitan | 236 | 1972 | 2017 |
| welfare_poldem | PolDem | economic left-right | pro_welfare | 349 | 1972 | 2017 |
## Party unions and electoral coalitions
Alliance and union labels are handled through constituent mappings so the released party identifiers represent individual parties. Shared text evidence can inform constituent parties through the union mapping while expert data continue to constrain individual parties directly.
| metric | category | value |
| --- | --- | --- |
| constituent_mappings | all | 106 |
| unique_union_or_alliance_ids | all | 76 |
| unique_constituent_party_ids | all | 100 |
| mappings_by_country | AR | 5 |
| mappings_by_country | AU | 2 |
| mappings_by_country | BA | 1 |
| mappings_by_country | BE | 4 |
| mappings_by_country | BG | 5 |
| mappings_by_country | BR | 6 |
| mappings_by_country | CH | 1 |
| mappings_by_country | CL | 8 |
| mappings_by_country | CZ | 4 |
| mappings_by_country | DE | 3 |
| mappings_by_country | EC | 1 |
| mappings_by_country | EE | 3 |
| mappings_by_country | ES | 3 |
| mappings_by_country | FR | 4 |
| mappings_by_country | GR | 3 |
| mappings_by_country | HR | 4 |
| mappings_by_country | IE | 1 |
| mappings_by_country | IL | 2 |
| mappings_by_country | IT | 8 |
| mappings_by_country | LK | 3 |
| mappings_by_country | LT | 4 |
| mappings_by_country | LU | 1 |
| mappings_by_country | LV | 6 |
| mappings_by_country | MD | 1 |
| mappings_by_country | ME | 1 |
| mappings_by_country | MX | 2 |
| mappings_by_country | NL | 1 |
## Party-family coverage
Party-family classifications are used for construct-validity diagnostics and coverage summaries. The table below reports coverage in the completed model output by family code.
| family | parties | party_years | countries | min_year | max_year |
| --- | --- | --- | --- | --- | --- |
| soc | 86 | 2892 | 37 | 1944 | 2025 |
| con | 82 | 2425 | 34 | 1944 | 2024 |
| lib | 81 | 2063 | 35 | 1944 | 2025 |
| chr | 41 | 1596 | 24 | 1945 | 2025 |
| com | 49 | 1241 | 27 | 1944 | 2025 |
| right | 50 | 975 | 26 | 1946 | 2025 |
| eco | 30 | 723 | 24 | 1960 | 2024 |
| agr | 12 | 592 | 10 | 1944 | 2024 |
| spec | 25 | 520 | 14 | 1949 | 2024 |
| other | 2 | 40 | 2 | 1992 | 2024 |
### Construct-validity family means
Substantive party-family means provide a construct-validity check: families should follow the expected ordering on the economic left-right and cultural cosmopolitan--traditionalist dimensions.
| family_name | n_parties | n_obs | mean_economic | sd_economic | mean_cultural | sd_cultural |
| --- | --- | --- | --- | --- | --- | --- |
| Communist/Far Left | 49 | 1241 | 0.12 | 0.09 | 0.383 | 0.18 |
| Green/Ecological | 30 | 723 | 0.267 | 0.115 | 0.251 | 0.092 |
| Social Democratic | 86 | 2892 | 0.331 | 0.124 | 0.378 | 0.138 |
| Christian Democratic | 41 | 1596 | 0.596 | 0.119 | 0.529 | 0.14 |
| Radical Right | 50 | 975 | 0.622 | 0.19 | 0.72 | 0.151 |
| Conservative | 82 | 2425 | 0.652 | 0.178 | 0.538 | 0.143 |
| Liberal | 81 | 2063 | 0.657 | 0.152 | 0.371 | 0.132 |
### Temporal-stability flags
The model permits movement through random walks, but unusually large one-year changes are flagged for inspection rather than treated as automatic errors.
| party_id | country | dimension | year_from | year_to | val_from | val_to | annual_change |
| --- | --- | --- | --- | --- | --- | --- | --- |
| 556 | LT | cultural cosmopolitan--traditionalist | 2019 | 2020 | 0.771 | 0.529 | 0.242 |
| 1663 | IL | cultural cosmopolitan--traditionalist | 2021 | 2022 | 0.142 | 0.356 | 0.214 |
| 455 | IL | cultural cosmopolitan--traditionalist | 1997 | 1998 | 0.457 | 0.643 | 0.186 |
| 455 | IL | cultural cosmopolitan--traditionalist | 1996 | 1997 | 0.277 | 0.457 | 0.18 |
| 8393 | LV | cultural cosmopolitan--traditionalist | 2018 | 2019 | 0.439 | 0.263 | 0.176 |
| 281 | BE | cultural cosmopolitan--traditionalist | 1977 | 1978 | 0.671 | 0.843 | 0.173 |
| 556 | LT | cultural cosmopolitan--traditionalist | 2001 | 2002 | 0.319 | 0.491 | 0.172 |
| 964 | IS | economic left-right | 2016 | 2017 | 0.691 | 0.521 | 0.17 |
| 298 | NL | cultural cosmopolitan--traditionalist | 2018 | 2019 | 0.761 | 0.593 | 0.168 |
| 901 | FI | economic left-right | 1992 | 1993 | 0.486 | 0.647 | 0.161 |
| 467 | SI | cultural cosmopolitan--traditionalist | 2018 | 2019 | 0.564 | 0.404 | 0.16 |
| 1221 | IT | economic left-right | 2007 | 2008 | 0.56 | 0.4 | 0.159 |
| 901 | FI | economic left-right | 1991 | 1992 | 0.327 | 0.486 | 0.159 |
| 455 | IL | cultural cosmopolitan--traditionalist | 1998 | 1999 | 0.643 | 0.797 | 0.154 |
| 1221 | IT | economic left-right | 2006 | 2007 | 0.709 | 0.56 | 0.149 |
| 2211 | UA | economic left-right | 2006 | 2007 | 0.417 | 0.562 | 0.145 |
| 631 | CH | economic left-right | 2018 | 2019 | 0.613 | 0.757 | 0.144 |
| 298 | NL | cultural cosmopolitan--traditionalist | 2019 | 2020 | 0.593 | 0.735 | 0.142 |
| 556 | LT | cultural cosmopolitan--traditionalist | 2000 | 2001 | 0.181 | 0.319 | 0.138 |
| 1417 | IL | cultural cosmopolitan--traditionalist | 1968 | 1969 | 0.498 | 0.635 | 0.137 |
## Model convergence diagnostics
Convergence is assessed using split R-hat and effective sample size over monitored parameters.
| metric | count | percentage |
| --- | --- | --- |
| R-hat < 1.01 | 35258 | 100 |
| R-hat 1.01-1.05 | 0 | 0 |
| R-hat > 1.05 | 0 | 0 |
| ESS > 1000 | 34853 | 98.85 |
| ESS 400-1000 | 405 | 1.15 |
| ESS < 400 | 0 | 0 |
### Convergence by parameter group
| dimension | parameters | mean_rhat | max_rhat | min_ess_bulk | mean_ess_bulk |
| --- | --- | --- | --- | --- | --- |
| cultural cosmopolitan--traditionalist | 17585 | 1.001 | 1.006 | 810.509 | 6889.246 |
| economic left-right | 17585 | 1 | 1.004 | 670.708 | 5573.456 |
| lr_country_offset | 65 | 1.001 | 1.004 | 1078.198 | 5434.868 |
| lr_decade_offset | 8 | 1 | 1.001 | 2069.676 | 3372.016 |
| lr_sigma | 3 | 1.001 | 1.002 | 1956.611 | 2582.401 |
| lr_source_offset | 3 | 1 | 1 | 3477.519 | 3814.374 |
| lr_weight | 3 | 1.002 | 1.002 | 1540.806 | 1560.217 |
| mean_sigma | 6 | 1.005 | 1.009 | 534.811 | 918.502 |
## Posterior uncertainty
The completed party-position output reports posterior standard errors and interval endpoints for both dimensions. These summaries describe the typical uncertainty in the release file used by the report.
| rows | parties | countries | min_year | max_year | mean_economic_se | median_economic_se | mean_cultural_se | median_cultural_se |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 17585 | 708 | 65 | 1944 | 2025 | 0.065 | 0.063 | 0.058 | 0.055 |
## Validation diagnostics
The validation diagnostics combine convergent and discriminant comparisons with expert surveys, posterior predictive coverage, construct checks, and out-of-sample validation when the corresponding outputs are available.
| diagnostic | dimension | n | pearson_r | spearman_r | mae | coverage |
| --- | --- | --- | --- | --- | --- | --- |
| convergent validity | economic left-right | 4637 | 0.904 | 0.901 | 0.076 | NA |
| convergent validity | cultural cosmopolitan--traditionalist | 1425 | 0.96 | 0.967 | 0.079 | NA |
| posterior predictive coverage | economic left-right | 7455 | NA | NA | NA | 0.899 |
| posterior predictive coverage | cultural cosmopolitan--traditionalist | 15539 | NA | NA | NA | 0.848 |
### Discriminant validity
| model_dim | expert_dim | r_pearson | r_spearman | n | type | diagnostic |
| --- | --- | --- | --- | --- | --- | --- |
| economic left-right | economic | 0.904 | 0.901 | 4637 | convergent | discriminant validity |
| cultural cosmopolitan--traditionalist | economic | 0.422 | 0.435 | 4637 | discriminant | discriminant validity |
| cultural cosmopolitan--traditionalist | cultural cosmopolitan--traditionalist | 0.96 | 0.967 | 1425 | convergent | discriminant validity |
| economic left-right | cultural cosmopolitan--traditionalist | 0.393 | 0.398 | 1425 | discriminant | discriminant validity |
### External validation correlations
| var | dimension | n | pearson_r | mean_absolute_error | coverage_95 |
| --- | --- | --- | --- | --- | --- |
| culsup_vparty | cultural cosmopolitan--traditionalist | 536 | 0.812 | 0.128 | 0.384 |
| galtan_ches | cultural cosmopolitan--traditionalist | 222 | 0.959 | 0.079 | 0.523 |
| gender_vparty | cultural cosmopolitan--traditionalist | 545 | 0.563 | 0.169 | 0.29 |
| immig_vparty | cultural cosmopolitan--traditionalist | 537 | 0.743 | 0.103 | 0.49 |
| lgbt_vparty | cultural cosmopolitan--traditionalist | 541 | 0.794 | 0.094 | 0.551 |
| relig_vparty | cultural cosmopolitan--traditionalist | 548 | 0.676 | 0.303 | 0.047 |
| lrecon_ches | economic left-right | 223 | 0.974 | 0.055 | 0.812 |
| lrecon_poppa | economic left-right | 74 | 0.98 | 0.066 | 0.662 |
| lrecon_vparty | economic left-right | 534 | 0.866 | 0.088 | 0.674 |
| welf_vparty | economic left-right | 534 | 0.682 | 0.176 | 0.363 |
## Generated tables
- `alliance_union_harmonization.csv`
- `construct_family_positions.csv`
- `construct_temporal_stability_flags.csv`
- `external_validation_correlations.csv`
- `item_coding_orientation.csv`
- `item_coverage.csv`
- `model_convergence_by_dimension.csv`
- `model_convergence_summary.csv`
- `party_family_coverage.csv`
- `party_year_source_coverage.csv`
- `posterior_uncertainty_summary.csv`
- `posterior_validation_convergent_summary.csv`
- `posterior_validation_discriminant_summary.csv`
- `posterior_validation_uncertainty_summary.csv`
- `reversed_items.csv`
- `source_coverage.csv`
data/releases/party_2d_diagnostics_report_v0.pdf
BIN
View File
Binary file not shown.
diagnostics/README.md
+1 -1
View File
@@ -14,4 +14,4 @@ If model outputs are stored outside the repository root, point the script to the
PARTY2D_OUTPUTS_DIR=/path/to/outputs Rscript diagnostics/generate_diagnostics.R
```
Generated files are written to `diagnostics/generated/`. The Markdown and PDF reports are also copied to `data/releases/` for the release bundle. PDF rendering uses R Markdown/Pandoc and requires a LaTeX engine such as `pdflatex`.
Generated files are written to `diagnostics/generated/`. The PDF report is also copied to `data/releases/` for the release bundle. PDF rendering uses R Markdown/Pandoc and requires a LaTeX engine such as `pdflatex`.
diagnostics/generate_diagnostics.R
+133 -11
View File
@@ -352,11 +352,6 @@ report_lines <- c(
""
)
report_path <- file.path(generated_dir, "diagnostics_report.md")
writeLines(report_lines, report_path)
release_report <- file.path(release_dir, paste0("party_2d_diagnostics_report_", release_version, ".md"))
invisible(file.copy(report_path, release_report, overwrite = TRUE))
pdf_source <- file.path(generated_dir, "diagnostics_report.Rmd")
pdf_file <- file.path(generated_dir, "diagnostics_report.pdf")
release_pdf <- file.path(release_dir, paste0("party_2d_diagnostics_report_", release_version, ".pdf"))
@@ -368,11 +363,139 @@ pdf_lines <- c(
" pdf_document:",
" toc: true",
" number_sections: true",
" latex_engine: pdflatex",
"geometry: margin=0.75in",
"fontsize: 9pt",
"fontsize: 10pt",
"header-includes:",
" - \\usepackage{booktabs}",
" - \\usepackage{longtable}",
" - \\usepackage{array}",
"---",
"",
report_lines[-1]
"```{r setup, include=FALSE}",
"knitr::opts_chunk$set(echo = FALSE, message = FALSE, warning = FALSE)",
"print_table <- function(x, n = Inf) {",
" if (nrow(x) == 0) { cat('No rows available.\\n'); return(invisible(NULL)) }",
" x <- head(x, n)",
" print(knitr::kable(x, format = 'latex', booktabs = TRUE, longtable = TRUE, digits = 3))",
"}",
"```",
"",
paste0("Generated: ", format(Sys.time(), "%Y-%m-%d %H:%M:%S %Z")),
"",
paste0("Release: ", release_version),
"",
paste0("Model positions source: `", display_path(model_positions_file), "`"),
"",
"# Purpose",
"",
"The purpose of this report is to provide the appendix-style diagnostics that document how the released party-position estimates are constructed, checked, and validated. The main article reports the central validation evidence; this report keeps the larger technical tables with the release so readers can inspect item coverage, source coverage, coding orientation, harmonization, convergence, posterior uncertainty, and validation details in one reproducible place.",
"",
"# Overview",
"",
"This report follows the structure of the technical appendix material: data and item coverage, coding and scale orientation, party-union harmonization, construct checks, model convergence, and validation. It is generated from the model-ready inputs and completed model outputs; it is not part of the raw-data setup workflow.",
"",
"# Data and item coverage",
"",
"The model combines text-coded item counts, dimension-specific expert placements, and general left-right expert placements. Text items enter as positive/sample counts, expert items enter as aggregated ratings with scale and expert-count information, and general left-right ratings anchor the relationship between the two dimensions.",
"",
"```{r item-counts, results='asis'}",
"print_table(item_counts)",
"```",
"",
"## Source coverage",
"",
"```{r source-coverage, results='asis'}",
"print_table(source_counts)",
"```",
"",
"# Data coding and item orientation",
"",
"All indicators are oriented toward the two reported dimensions: economic left-right and cultural cosmopolitan--traditionalist. For interpretability, generated diagnostics report whether higher observed values point toward the public high pole or are reversed for reporting. Original source variable names are preserved in the tables.",
"",
"## Reversed items",
"",
"```{r reversed-items, results='asis'}",
"print_table(reversed_items)",
"```",
"",
"# Party unions and electoral coalitions",
"",
"Alliance and union labels are handled through constituent mappings so the released party identifiers represent individual parties. Shared text evidence can inform constituent parties through the union mapping while expert data continue to constrain individual parties directly.",
"",
"```{r union-summary, results='asis'}",
"print_table(alliance_union_harmonization, n = 40)",
"```",
"",
"# Party-family and construct coverage",
"",
"Party-family classifications are used for construct-validity diagnostics and coverage summaries. The table below reports coverage in the completed model output by family code.",
"",
"```{r family-coverage, results='asis'}",
"print_table(party_family_coverage)",
"```",
"",
"## Construct-validity family means",
"",
"Substantive party-family means provide a construct-validity check: families should follow the expected ordering on the economic left-right and cultural cosmopolitan--traditionalist dimensions.",
"",
"```{r construct-family, results='asis'}",
"print_table(construct_family_positions %>% select(family_name, n_parties, n_obs, mean_economic, sd_economic, mean_cultural, sd_cultural) %>% mutate(across(where(is.numeric), ~ round(.x, 3))))",
"```",
"",
"## Temporal-stability flags",
"",
"The model permits movement through random walks, but unusually large one-year changes are flagged for inspection rather than treated as automatic errors.",
"",
"```{r temporal-stability, results='asis'}",
"print_table(construct_temporal_stability %>% select(party_id, country, dimension, year_from, year_to, val_from, val_to, annual_change) %>% mutate(across(where(is.numeric), ~ round(.x, 3))), n = 20)",
"```",
"",
"# Model convergence diagnostics",
"",
"Convergence is assessed using split R-hat and effective sample size over monitored parameters.",
"",
"```{r convergence-summary, results='asis'}",
"print_table(conv_display)",
"```",
"",
"## Convergence by parameter group",
"",
"```{r convergence-dim, results='asis'}",
"print_table(conv_dim_display)",
"```",
"",
"# Posterior uncertainty",
"",
"The completed party-position output reports posterior standard errors and interval endpoints for both dimensions. These summaries describe the typical uncertainty in the release file used by the report.",
"",
"```{r posterior-uncertainty, results='asis'}",
"print_table(posterior_uncertainty %>% mutate(across(where(is.numeric), ~ round(.x, 3))))",
"```",
"",
"# Validation diagnostics",
"",
"The validation diagnostics combine convergent and discriminant comparisons with expert surveys, posterior predictive coverage, construct checks, and out-of-sample validation when the corresponding outputs are available.",
"",
"```{r validation-summary, results='asis'}",
"print_table(val_display)",
"```",
"",
"## Discriminant validity",
"",
"```{r discriminant, results='asis'}",
"print_table(discriminant_summary %>% select(-any_of('source_file')) %>% mutate(across(where(is.numeric), ~ round(.x, 3))))",
"```",
"",
"## External validation correlations",
"",
"```{r external-validation, results='asis'}",
"print_table(external_validation_correlations %>% select(-any_of('source_file')) %>% mutate(across(where(is.numeric), ~ round(.x, 3))))",
"```",
"",
"# Generated tables",
"",
paste0("- `", list.files(generated_dir, pattern = "\\.csv$"), "`")
)
writeLines(pdf_lines, pdf_source)
if (!requireNamespace("rmarkdown", quietly = TRUE)) {
@@ -383,7 +506,8 @@ rmarkdown::render(
output_format = rmarkdown::pdf_document(toc = TRUE, number_sections = TRUE),
output_file = basename(pdf_file),
output_dir = generated_dir,
quiet = TRUE
quiet = TRUE,
envir = environment()
)
invisible(file.copy(pdf_file, release_pdf, overwrite = TRUE))
unlink(c(
@@ -406,7 +530,7 @@ generated_readme <- c(
"",
"The command requires completed model/post-estimation outputs. If those outputs are outside the repository root, set `PARTY2D_OUTPUTS_DIR` before running the script.",
"",
"The main reports are `diagnostics_report.md` and `diagnostics_report.pdf`; both are copied into `data/releases/` for the release files."
"The report file is `diagnostics_report.pdf`; the same PDF is copied into `data/releases/` for the release files."
)
writeLines(generated_readme, file.path(generated_dir, "README.md"))
@@ -414,7 +538,6 @@ sha_file <- file.path(release_dir, "SHA256SUMS")
release_files_for_sha <- c(
paste0("party_2d_election_year_panel_", release_version, ".csv.gz"),
paste0("party_2d_annual_model_output_", release_version, ".csv.gz"),
basename(release_report),
basename(release_pdf)
)
existing_release_files <- release_files_for_sha[file.exists(file.path(release_dir, release_files_for_sha))]
@@ -427,5 +550,4 @@ sha_lines <- vapply(existing_release_files, function(f) {
writeLines(sha_lines, sha_file)
message("Diagnostics written to diagnostics/generated")
message("Release diagnostics report written to ", release_report)
message("Release diagnostics PDF written to ", release_pdf)
diagnostics/generated/README.md
+1 -1
View File
@@ -8,4 +8,4 @@ Rscript diagnostics/generate_diagnostics.R
The command requires completed model/post-estimation outputs. If those outputs are outside the repository root, set `PARTY2D_OUTPUTS_DIR` before running the script.
The main reports are `diagnostics_report.md` and `diagnostics_report.pdf`; both are copied into `data/releases/` for the release files.
The report file is `diagnostics_report.pdf`; the same PDF is copied into `data/releases/` for the release files.
diagnostics/generated/diagnostics_report.md
-241
View File
@@ -1,241 +0,0 @@
# Diagnostics report
Generated: 2026-06-15 16:40:02 CEST
Release: v0
Model positions source: `party_positions_2026-06-12_16-35-01.csv`
## Purpose
The purpose of this report is to provide the appendix-style diagnostics that document how the released party-position estimates are constructed, checked, and validated. The main article reports the central validation evidence; this report keeps the larger technical tables with the release so readers can inspect item coverage, source coverage, coding orientation, harmonization, convergence, posterior uncertainty, and validation details in one reproducible place.
## Overview
This report follows the structure of the technical appendix material: data and item coverage, coding and scale orientation, party-union harmonization, construct checks, model convergence, and validation. It is generated from the model-ready inputs and completed model outputs; it is not part of the raw-data setup workflow.
## Data and item coverage
The model combines text-coded item counts, dimension-specific expert placements, and general left-right expert placements. Text items enter as positive/sample counts, expert items enter as aggregated ratings with scale and expert-count information, and general left-right ratings anchor the relationship between the two dimensions.
| source_file | dimension | items |
| --- | --- | --- |
| expert.csv | cultural cosmopolitan--traditionalist | 7 |
| expert.csv | economic left-right | 5 |
| lr_data.csv | general left-right | 3 |
| text_data.csv | cultural cosmopolitan--traditionalist | 11 |
| text_data.csv | economic left-right | 6 |
### Source coverage
| source_file | source | items | observations | party_years | parties | countries | min_year | max_year |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| expert.csv | CHES | 2 | 2639 | 1320 | 390 | 44 | 1999 | 2024 |
| expert.csv | GPS | 2 | 538 | 271 | 271 | 62 | 2019 | 2019 |
| expert.csv | POPPA | 1 | 413 | 413 | 225 | 31 | 2018 | 2023 |
| expert.csv | V-Party | 7 | 21488 | 3076 | 589 | 65 | 1970 | 2019 |
| lr_data.csv | CHES | 1 | 1320 | 1320 | 390 | 44 | 1999 | 2024 |
| lr_data.csv | Morgan | 1 | 471 | 471 | 72 | 11 | 1945 | 1973 |
| lr_data.csv | POPPA | 1 | 416 | 416 | 225 | 31 | 2018 | 2023 |
| text_data.csv | Manifesto Project | 8 | 36008 | 4501 | 713 | 65 | 1920 | 2025 |
| text_data.csv | PolDem | 9 | 2194 | 406 | 93 | 15 | 1972 | 2017 |
## Data coding and item orientation
All indicators are oriented toward the two reported dimensions: economic left-right and cultural cosmopolitan--traditionalist. For interpretability, generated diagnostics report whether higher observed values point toward the public high pole or are reversed for reporting. Original source variable names are preserved in the tables.
### Reversed items
| item | source | dimension | higher_values_indicate | observations | min_year | max_year |
| --- | --- | --- | --- | --- | --- | --- |
| internationalism_manifesto | Manifesto Project | cultural cosmopolitan--traditionalist | cosmopolitan | 4501 | 1920 | 2025 |
| multiculturalism_manifesto | Manifesto Project | cultural cosmopolitan--traditionalist | cosmopolitan | 4501 | 1920 | 2025 |
| economic_intervention_manifesto | Manifesto Project | economic left-right | pro_welfare | 4501 | 1920 | 2025 |
| social_services_manifesto | Manifesto Project | economic left-right | pro_welfare | 4501 | 1920 | 2025 |
| cultlib_poldem | PolDem | cultural cosmopolitan--traditionalist | cosmopolitan | 299 | 1972 | 2017 |
| euro_poldem | PolDem | cultural cosmopolitan--traditionalist | cosmopolitan | 93 | 1978 | 2017 |
| europe_poldem | PolDem | cultural cosmopolitan--traditionalist | cosmopolitan | 217 | 1972 | 2017 |
| immig_poldem | PolDem | cultural cosmopolitan--traditionalist | cosmopolitan | 236 | 1972 | 2017 |
| welfare_poldem | PolDem | economic left-right | pro_welfare | 349 | 1972 | 2017 |
## Party unions and electoral coalitions
Alliance and union labels are handled through constituent mappings so the released party identifiers represent individual parties. Shared text evidence can inform constituent parties through the union mapping while expert data continue to constrain individual parties directly.
| metric | category | value |
| --- | --- | --- |
| constituent_mappings | all | 106 |
| unique_union_or_alliance_ids | all | 76 |
| unique_constituent_party_ids | all | 100 |
| mappings_by_country | AR | 5 |
| mappings_by_country | AU | 2 |
| mappings_by_country | BA | 1 |
| mappings_by_country | BE | 4 |
| mappings_by_country | BG | 5 |
| mappings_by_country | BR | 6 |
| mappings_by_country | CH | 1 |
| mappings_by_country | CL | 8 |
| mappings_by_country | CZ | 4 |
| mappings_by_country | DE | 3 |
| mappings_by_country | EC | 1 |
| mappings_by_country | EE | 3 |
| mappings_by_country | ES | 3 |
| mappings_by_country | FR | 4 |
| mappings_by_country | GR | 3 |
| mappings_by_country | HR | 4 |
| mappings_by_country | IE | 1 |
| mappings_by_country | IL | 2 |
| mappings_by_country | IT | 8 |
| mappings_by_country | LK | 3 |
| mappings_by_country | LT | 4 |
| mappings_by_country | LU | 1 |
| mappings_by_country | LV | 6 |
| mappings_by_country | MD | 1 |
| mappings_by_country | ME | 1 |
| mappings_by_country | MX | 2 |
| mappings_by_country | NL | 1 |
## Party-family coverage
Party-family classifications are used for construct-validity diagnostics and coverage summaries. The table below reports coverage in the completed model output by family code.
| family | parties | party_years | countries | min_year | max_year |
| --- | --- | --- | --- | --- | --- |
| soc | 86 | 2892 | 37 | 1944 | 2025 |
| con | 82 | 2425 | 34 | 1944 | 2024 |
| lib | 81 | 2063 | 35 | 1944 | 2025 |
| chr | 41 | 1596 | 24 | 1945 | 2025 |
| com | 49 | 1241 | 27 | 1944 | 2025 |
| right | 50 | 975 | 26 | 1946 | 2025 |
| eco | 30 | 723 | 24 | 1960 | 2024 |
| agr | 12 | 592 | 10 | 1944 | 2024 |
| spec | 25 | 520 | 14 | 1949 | 2024 |
| other | 2 | 40 | 2 | 1992 | 2024 |
### Construct-validity family means
Substantive party-family means provide a construct-validity check: families should follow the expected ordering on the economic left-right and cultural cosmopolitan--traditionalist dimensions.
| family_name | n_parties | n_obs | mean_economic | sd_economic | mean_cultural | sd_cultural |
| --- | --- | --- | --- | --- | --- | --- |
| Communist/Far Left | 49 | 1241 | 0.12 | 0.09 | 0.383 | 0.18 |
| Green/Ecological | 30 | 723 | 0.267 | 0.115 | 0.251 | 0.092 |
| Social Democratic | 86 | 2892 | 0.331 | 0.124 | 0.378 | 0.138 |
| Christian Democratic | 41 | 1596 | 0.596 | 0.119 | 0.529 | 0.14 |
| Radical Right | 50 | 975 | 0.622 | 0.19 | 0.72 | 0.151 |
| Conservative | 82 | 2425 | 0.652 | 0.178 | 0.538 | 0.143 |
| Liberal | 81 | 2063 | 0.657 | 0.152 | 0.371 | 0.132 |
### Temporal-stability flags
The model permits movement through random walks, but unusually large one-year changes are flagged for inspection rather than treated as automatic errors.
| party_id | country | dimension | year_from | year_to | val_from | val_to | annual_change |
| --- | --- | --- | --- | --- | --- | --- | --- |
| 556 | LT | cultural cosmopolitan--traditionalist | 2019 | 2020 | 0.771 | 0.529 | 0.242 |
| 1663 | IL | cultural cosmopolitan--traditionalist | 2021 | 2022 | 0.142 | 0.356 | 0.214 |
| 455 | IL | cultural cosmopolitan--traditionalist | 1997 | 1998 | 0.457 | 0.643 | 0.186 |
| 455 | IL | cultural cosmopolitan--traditionalist | 1996 | 1997 | 0.277 | 0.457 | 0.18 |
| 8393 | LV | cultural cosmopolitan--traditionalist | 2018 | 2019 | 0.439 | 0.263 | 0.176 |
| 281 | BE | cultural cosmopolitan--traditionalist | 1977 | 1978 | 0.671 | 0.843 | 0.173 |
| 556 | LT | cultural cosmopolitan--traditionalist | 2001 | 2002 | 0.319 | 0.491 | 0.172 |
| 964 | IS | economic left-right | 2016 | 2017 | 0.691 | 0.521 | 0.17 |
| 298 | NL | cultural cosmopolitan--traditionalist | 2018 | 2019 | 0.761 | 0.593 | 0.168 |
| 901 | FI | economic left-right | 1992 | 1993 | 0.486 | 0.647 | 0.161 |
| 467 | SI | cultural cosmopolitan--traditionalist | 2018 | 2019 | 0.564 | 0.404 | 0.16 |
| 1221 | IT | economic left-right | 2007 | 2008 | 0.56 | 0.4 | 0.159 |
| 901 | FI | economic left-right | 1991 | 1992 | 0.327 | 0.486 | 0.159 |
| 455 | IL | cultural cosmopolitan--traditionalist | 1998 | 1999 | 0.643 | 0.797 | 0.154 |
| 1221 | IT | economic left-right | 2006 | 2007 | 0.709 | 0.56 | 0.149 |
| 2211 | UA | economic left-right | 2006 | 2007 | 0.417 | 0.562 | 0.145 |
| 631 | CH | economic left-right | 2018 | 2019 | 0.613 | 0.757 | 0.144 |
| 298 | NL | cultural cosmopolitan--traditionalist | 2019 | 2020 | 0.593 | 0.735 | 0.142 |
| 556 | LT | cultural cosmopolitan--traditionalist | 2000 | 2001 | 0.181 | 0.319 | 0.138 |
| 1417 | IL | cultural cosmopolitan--traditionalist | 1968 | 1969 | 0.498 | 0.635 | 0.137 |
## Model convergence diagnostics
Convergence is assessed using split R-hat and effective sample size over monitored parameters.
| metric | count | percentage |
| --- | --- | --- |
| R-hat < 1.01 | 35258 | 100 |
| R-hat 1.01-1.05 | 0 | 0 |
| R-hat > 1.05 | 0 | 0 |
| ESS > 1000 | 34853 | 98.85 |
| ESS 400-1000 | 405 | 1.15 |
| ESS < 400 | 0 | 0 |
### Convergence by parameter group
| dimension | parameters | mean_rhat | max_rhat | min_ess_bulk | mean_ess_bulk |
| --- | --- | --- | --- | --- | --- |
| cultural cosmopolitan--traditionalist | 17585 | 1.001 | 1.006 | 810.509 | 6889.246 |
| economic left-right | 17585 | 1 | 1.004 | 670.708 | 5573.456 |
| lr_country_offset | 65 | 1.001 | 1.004 | 1078.198 | 5434.868 |
| lr_decade_offset | 8 | 1 | 1.001 | 2069.676 | 3372.016 |
| lr_sigma | 3 | 1.001 | 1.002 | 1956.611 | 2582.401 |
| lr_source_offset | 3 | 1 | 1 | 3477.519 | 3814.374 |
| lr_weight | 3 | 1.002 | 1.002 | 1540.806 | 1560.217 |
| mean_sigma | 6 | 1.005 | 1.009 | 534.811 | 918.502 |
## Posterior uncertainty
The completed party-position output reports posterior standard errors and interval endpoints for both dimensions. These summaries describe the typical uncertainty in the release file used by the report.
| rows | parties | countries | min_year | max_year | mean_economic_se | median_economic_se | mean_cultural_se | median_cultural_se |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 17585 | 708 | 65 | 1944 | 2025 | 0.065 | 0.063 | 0.058 | 0.055 |
## Validation diagnostics
The validation diagnostics combine convergent and discriminant comparisons with expert surveys, posterior predictive coverage, construct checks, and out-of-sample validation when the corresponding outputs are available.
| diagnostic | dimension | n | pearson_r | spearman_r | mae | coverage |
| --- | --- | --- | --- | --- | --- | --- |
| convergent validity | economic left-right | 4637 | 0.904 | 0.901 | 0.076 | NA |
| convergent validity | cultural cosmopolitan--traditionalist | 1425 | 0.96 | 0.967 | 0.079 | NA |
| posterior predictive coverage | economic left-right | 7455 | NA | NA | NA | 0.899 |
| posterior predictive coverage | cultural cosmopolitan--traditionalist | 15539 | NA | NA | NA | 0.848 |
### Discriminant validity
| model_dim | expert_dim | r_pearson | r_spearman | n | type | diagnostic |
| --- | --- | --- | --- | --- | --- | --- |
| economic left-right | economic | 0.904 | 0.901 | 4637 | convergent | discriminant validity |
| cultural cosmopolitan--traditionalist | economic | 0.422 | 0.435 | 4637 | discriminant | discriminant validity |
| cultural cosmopolitan--traditionalist | cultural cosmopolitan--traditionalist | 0.96 | 0.967 | 1425 | convergent | discriminant validity |
| economic left-right | cultural cosmopolitan--traditionalist | 0.393 | 0.398 | 1425 | discriminant | discriminant validity |
### External validation correlations
| var | dimension | n | pearson_r | mean_absolute_error | coverage_95 |
| --- | --- | --- | --- | --- | --- |
| culsup_vparty | cultural cosmopolitan--traditionalist | 536 | 0.812 | 0.128 | 0.384 |
| galtan_ches | cultural cosmopolitan--traditionalist | 222 | 0.959 | 0.079 | 0.523 |
| gender_vparty | cultural cosmopolitan--traditionalist | 545 | 0.563 | 0.169 | 0.29 |
| immig_vparty | cultural cosmopolitan--traditionalist | 537 | 0.743 | 0.103 | 0.49 |
| lgbt_vparty | cultural cosmopolitan--traditionalist | 541 | 0.794 | 0.094 | 0.551 |
| relig_vparty | cultural cosmopolitan--traditionalist | 548 | 0.676 | 0.303 | 0.047 |
| lrecon_ches | economic left-right | 223 | 0.974 | 0.055 | 0.812 |
| lrecon_poppa | economic left-right | 74 | 0.98 | 0.066 | 0.662 |
| lrecon_vparty | economic left-right | 534 | 0.866 | 0.088 | 0.674 |
| welf_vparty | economic left-right | 534 | 0.682 | 0.176 | 0.363 |
## Generated tables
- `alliance_union_harmonization.csv`
- `construct_family_positions.csv`
- `construct_temporal_stability_flags.csv`
- `external_validation_correlations.csv`
- `item_coding_orientation.csv`
- `item_coverage.csv`
- `model_convergence_by_dimension.csv`
- `model_convergence_summary.csv`
- `party_family_coverage.csv`
- `party_year_source_coverage.csv`
- `posterior_uncertainty_summary.csv`
- `posterior_validation_convergent_summary.csv`
- `posterior_validation_discriminant_summary.csv`
- `posterior_validation_uncertainty_summary.csv`
- `reversed_items.csv`
- `source_coverage.csv`
diagnostics/generated/diagnostics_report.pdf
BIN
View File
Binary file not shown.