Code and processed model inputs for generating two-dimensional party-position estimates from text and expert data. The model combines manifesto and media text indicators with expert survey placements in a Bayesian dynamic item-response framework.

Repository contents

data-setup/ — local-only source download and rebuild workflow. It is included because the original source files cannot be redistributed in this public repository.
src/julia/ — Stan data preparation, model fitting, post-estimation, enrichment, and validation.
models/ — Stan model specification.
data/ — five processed party-level inputs used by the Julia/Stan model.
metadata/ — data dictionary and source-support documentation.
docs/ — raw data source documentation, coding decisions, and operational notes.

Processed inputs needed by the model are included in data/ so the estimation step can be reproduced from the model-ready data.

Source data and redistribution

The public repository intentionally does not contain the original raw/source files. Several inputs are third-party datasets with their own terms of use, and the Morgan historical file is a local OCR/transcription source. Instead, the repository provides:

committed model-ready inputs in data/, sufficient for fitting the Julia/Stan model; and
data-setup/, a local-only workflow that downloads or checks raw sources and rebuilds comparable model-ready inputs under ignored _local/ directories.

data-setup/ automatically downloads all script-accessible sources:

PolDem
PartyFacts crosswalk
CHES family files
POPPA from Harvard Dataverse
Global Party Survey 2019 from Harvard Dataverse
V-Party through the provider's download form

Two inputs require user-provided access/material:

Manifesto Project: users must provide their own Manifesto Project API key via MANIFESTO_API_KEY or PARTY2D_MANIFESTO_API_KEY.
Morgan historical expert data: morgan_positions_raw.csv is not publicly downloadable; it can be provided on request and should be placed locally under _local/raw/morgan/.

Raw downloads, intermediate build files, regenerated inputs, R package installs, and comparison reports are written only under _local/, which is ignored by git. The setup workflow never overwrites committed files in data/. See data-setup/README.md and docs/RAW_DATA_SOURCES.md for exact commands and source details.

Running the pipeline

Run the full workflow with:

bash run_estimation.sh full

This checks that model-ready inputs are present, then executes the Julia/Stan workflow scripts:

bash scripts/01_prepare_data.sh   # checks model-ready inputs; does not rebuild raw data
bash scripts/02_fit_model.sh
bash scripts/03_extract_estimates.sh
bash scripts/04_enrich_estimates.sh
bash scripts/05_validate_estimates.sh

The numbered scripts can also be run manually in that order.

The Bayesian model is computationally expensive. The production run used 4 cores on an AMD Ryzen 9 7945HX and took 60,372 seconds, approximately 16 hours 46 minutes.

If model output is already available, rebuild estimates without refitting Stan:

bash run_estimation.sh reuse

reuse verifies the model-ready inputs, then reruns post-estimation, enrichment, and validation while skipping the Stan fitting step.

To check the local setup without fitting the model, run:

bash run_estimation.sh dry-run

Data inputs

The model-ready inputs are included under data/:

text_data.csv
expert.csv
lr_data.csv
union_mapping.csv
party_families.csv

Original raw source files are not redistributed. Rebuilding inputs from raw files is separate from the normal estimation workflow and writes only to ignored _local/ directories; it never replaces committed data/ inputs automatically.

Output variables

The two position variables are scaled from 0 to 1:

economic_lr: economic left to economic right.
galtan: cosmopolitan/socially liberal to traditionalist/nationalist.

Column definitions are in metadata/data_dictionary.csv.