Keep diagnostics release to one PDF report
This commit is contained in:
aseimel
@@ -352,11 +352,6 @@ report_lines <- c(
|
||||
""
|
||||
)
|
||||
|
||||
report_path <- file.path(generated_dir, "diagnostics_report.md")
|
||||
writeLines(report_lines, report_path)
|
||||
release_report <- file.path(release_dir, paste0("party_2d_diagnostics_report_", release_version, ".md"))
|
||||
invisible(file.copy(report_path, release_report, overwrite = TRUE))
|
||||
|
||||
pdf_source <- file.path(generated_dir, "diagnostics_report.Rmd")
|
||||
pdf_file <- file.path(generated_dir, "diagnostics_report.pdf")
|
||||
release_pdf <- file.path(release_dir, paste0("party_2d_diagnostics_report_", release_version, ".pdf"))
|
||||
@@ -368,11 +363,139 @@ pdf_lines <- c(
|
||||
" pdf_document:",
|
||||
" toc: true",
|
||||
" number_sections: true",
|
||||
" latex_engine: pdflatex",
|
||||
"geometry: margin=0.75in",
|
||||
"fontsize: 9pt",
|
||||
"fontsize: 10pt",
|
||||
"header-includes:",
|
||||
" - \\usepackage{booktabs}",
|
||||
" - \\usepackage{longtable}",
|
||||
" - \\usepackage{array}",
|
||||
"---",
|
||||
"",
|
||||
report_lines[-1]
|
||||
"```{r setup, include=FALSE}",
|
||||
"knitr::opts_chunk$set(echo = FALSE, message = FALSE, warning = FALSE)",
|
||||
"print_table <- function(x, n = Inf) {",
|
||||
" if (nrow(x) == 0) { cat('No rows available.\\n'); return(invisible(NULL)) }",
|
||||
" x <- head(x, n)",
|
||||
" print(knitr::kable(x, format = 'latex', booktabs = TRUE, longtable = TRUE, digits = 3))",
|
||||
"}",
|
||||
"```",
|
||||
"",
|
||||
paste0("Generated: ", format(Sys.time(), "%Y-%m-%d %H:%M:%S %Z")),
|
||||
"",
|
||||
paste0("Release: ", release_version),
|
||||
"",
|
||||
paste0("Model positions source: `", display_path(model_positions_file), "`"),
|
||||
"",
|
||||
"# Purpose",
|
||||
"",
|
||||
"The purpose of this report is to provide the appendix-style diagnostics that document how the released party-position estimates are constructed, checked, and validated. The main article reports the central validation evidence; this report keeps the larger technical tables with the release so readers can inspect item coverage, source coverage, coding orientation, harmonization, convergence, posterior uncertainty, and validation details in one reproducible place.",
|
||||
"",
|
||||
"# Overview",
|
||||
"",
|
||||
"This report follows the structure of the technical appendix material: data and item coverage, coding and scale orientation, party-union harmonization, construct checks, model convergence, and validation. It is generated from the model-ready inputs and completed model outputs; it is not part of the raw-data setup workflow.",
|
||||
"",
|
||||
"# Data and item coverage",
|
||||
"",
|
||||
"The model combines text-coded item counts, dimension-specific expert placements, and general left-right expert placements. Text items enter as positive/sample counts, expert items enter as aggregated ratings with scale and expert-count information, and general left-right ratings anchor the relationship between the two dimensions.",
|
||||
"",
|
||||
"```{r item-counts, results='asis'}",
|
||||
"print_table(item_counts)",
|
||||
"```",
|
||||
"",
|
||||
"## Source coverage",
|
||||
"",
|
||||
"```{r source-coverage, results='asis'}",
|
||||
"print_table(source_counts)",
|
||||
"```",
|
||||
"",
|
||||
"# Data coding and item orientation",
|
||||
"",
|
||||
"All indicators are oriented toward the two reported dimensions: economic left-right and cultural cosmopolitan--traditionalist. For interpretability, generated diagnostics report whether higher observed values point toward the public high pole or are reversed for reporting. Original source variable names are preserved in the tables.",
|
||||
"",
|
||||
"## Reversed items",
|
||||
"",
|
||||
"```{r reversed-items, results='asis'}",
|
||||
"print_table(reversed_items)",
|
||||
"```",
|
||||
"",
|
||||
"# Party unions and electoral coalitions",
|
||||
"",
|
||||
"Alliance and union labels are handled through constituent mappings so the released party identifiers represent individual parties. Shared text evidence can inform constituent parties through the union mapping while expert data continue to constrain individual parties directly.",
|
||||
"",
|
||||
"```{r union-summary, results='asis'}",
|
||||
"print_table(alliance_union_harmonization, n = 40)",
|
||||
"```",
|
||||
"",
|
||||
"# Party-family and construct coverage",
|
||||
"",
|
||||
"Party-family classifications are used for construct-validity diagnostics and coverage summaries. The table below reports coverage in the completed model output by family code.",
|
||||
"",
|
||||
"```{r family-coverage, results='asis'}",
|
||||
"print_table(party_family_coverage)",
|
||||
"```",
|
||||
"",
|
||||
"## Construct-validity family means",
|
||||
"",
|
||||
"Substantive party-family means provide a construct-validity check: families should follow the expected ordering on the economic left-right and cultural cosmopolitan--traditionalist dimensions.",
|
||||
"",
|
||||
"```{r construct-family, results='asis'}",
|
||||
"print_table(construct_family_positions %>% select(family_name, n_parties, n_obs, mean_economic, sd_economic, mean_cultural, sd_cultural) %>% mutate(across(where(is.numeric), ~ round(.x, 3))))",
|
||||
"```",
|
||||
"",
|
||||
"## Temporal-stability flags",
|
||||
"",
|
||||
"The model permits movement through random walks, but unusually large one-year changes are flagged for inspection rather than treated as automatic errors.",
|
||||
"",
|
||||
"```{r temporal-stability, results='asis'}",
|
||||
"print_table(construct_temporal_stability %>% select(party_id, country, dimension, year_from, year_to, val_from, val_to, annual_change) %>% mutate(across(where(is.numeric), ~ round(.x, 3))), n = 20)",
|
||||
"```",
|
||||
"",
|
||||
"# Model convergence diagnostics",
|
||||
"",
|
||||
"Convergence is assessed using split R-hat and effective sample size over monitored parameters.",
|
||||
"",
|
||||
"```{r convergence-summary, results='asis'}",
|
||||
"print_table(conv_display)",
|
||||
"```",
|
||||
"",
|
||||
"## Convergence by parameter group",
|
||||
"",
|
||||
"```{r convergence-dim, results='asis'}",
|
||||
"print_table(conv_dim_display)",
|
||||
"```",
|
||||
"",
|
||||
"# Posterior uncertainty",
|
||||
"",
|
||||
"The completed party-position output reports posterior standard errors and interval endpoints for both dimensions. These summaries describe the typical uncertainty in the release file used by the report.",
|
||||
"",
|
||||
"```{r posterior-uncertainty, results='asis'}",
|
||||
"print_table(posterior_uncertainty %>% mutate(across(where(is.numeric), ~ round(.x, 3))))",
|
||||
"```",
|
||||
"",
|
||||
"# Validation diagnostics",
|
||||
"",
|
||||
"The validation diagnostics combine convergent and discriminant comparisons with expert surveys, posterior predictive coverage, construct checks, and out-of-sample validation when the corresponding outputs are available.",
|
||||
"",
|
||||
"```{r validation-summary, results='asis'}",
|
||||
"print_table(val_display)",
|
||||
"```",
|
||||
"",
|
||||
"## Discriminant validity",
|
||||
"",
|
||||
"```{r discriminant, results='asis'}",
|
||||
"print_table(discriminant_summary %>% select(-any_of('source_file')) %>% mutate(across(where(is.numeric), ~ round(.x, 3))))",
|
||||
"```",
|
||||
"",
|
||||
"## External validation correlations",
|
||||
"",
|
||||
"```{r external-validation, results='asis'}",
|
||||
"print_table(external_validation_correlations %>% select(-any_of('source_file')) %>% mutate(across(where(is.numeric), ~ round(.x, 3))))",
|
||||
"```",
|
||||
"",
|
||||
"# Generated tables",
|
||||
"",
|
||||
paste0("- `", list.files(generated_dir, pattern = "\\.csv$"), "`")
|
||||
)
|
||||
writeLines(pdf_lines, pdf_source)
|
||||
if (!requireNamespace("rmarkdown", quietly = TRUE)) {
|
||||
@@ -383,7 +506,8 @@ rmarkdown::render(
|
||||
output_format = rmarkdown::pdf_document(toc = TRUE, number_sections = TRUE),
|
||||
output_file = basename(pdf_file),
|
||||
output_dir = generated_dir,
|
||||
quiet = TRUE
|
||||
quiet = TRUE,
|
||||
envir = environment()
|
||||
)
|
||||
invisible(file.copy(pdf_file, release_pdf, overwrite = TRUE))
|
||||
unlink(c(
|
||||
@@ -406,7 +530,7 @@ generated_readme <- c(
|
||||
"",
|
||||
"The command requires completed model/post-estimation outputs. If those outputs are outside the repository root, set `PARTY2D_OUTPUTS_DIR` before running the script.",
|
||||
"",
|
||||
"The main reports are `diagnostics_report.md` and `diagnostics_report.pdf`; both are copied into `data/releases/` for the release files."
|
||||
"The report file is `diagnostics_report.pdf`; the same PDF is copied into `data/releases/` for the release files."
|
||||
)
|
||||
writeLines(generated_readme, file.path(generated_dir, "README.md"))
|
||||
|
||||
@@ -414,7 +538,6 @@ sha_file <- file.path(release_dir, "SHA256SUMS")
|
||||
release_files_for_sha <- c(
|
||||
paste0("party_2d_election_year_panel_", release_version, ".csv.gz"),
|
||||
paste0("party_2d_annual_model_output_", release_version, ".csv.gz"),
|
||||
basename(release_report),
|
||||
basename(release_pdf)
|
||||
)
|
||||
existing_release_files <- release_files_for_sha[file.exists(file.path(release_dir, release_files_for_sha))]
|
||||
@@ -427,5 +550,4 @@ sha_lines <- vapply(existing_release_files, function(f) {
|
||||
writeLines(sha_lines, sha_file)
|
||||
|
||||
message("Diagnostics written to diagnostics/generated")
|
||||
message("Release diagnostics report written to ", release_report)
|
||||
message("Release diagnostics PDF written to ", release_pdf)
|
||||
|
||||
Reference in New Issue
Block a user