armin/party2d
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Clarify source data setup requirements

Browse Source
This commit is contained in:
aseimel
2026-06-15 14:48:55 +02:00
parent d9c648bd54
commit 8353d02761
3 changed files with 67 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+25 -2
View File
@@ -4,7 +4,7 @@ Code and processed model inputs for generating two-dimensional party-position es
## Repository contents
- `data-setup/` — optional scripts and source manifest for rebuilding model-ready inputs from local raw files.
- `data-setup/` — local-only source download and rebuild workflow. It is included because the original source files cannot be redistributed in this public repository.
- `src/julia/` — Stan data preparation, model fitting, post-estimation, enrichment, and validation.
- `models/` — Stan model specification.
- `data/` — five processed party-level inputs used by the Julia/Stan model.
@@ -13,6 +13,29 @@ Code and processed model inputs for generating two-dimensional party-position es
Processed inputs needed by the model are included in `data/` so the estimation step can be reproduced from the model-ready data.
## Source data and redistribution
The public repository intentionally does **not** contain the original raw/source files. Several inputs are third-party datasets with their own terms of use, and the Morgan historical file is a local OCR/transcription source. Instead, the repository provides:
1. committed model-ready inputs in `data/`, sufficient for fitting the Julia/Stan model; and
2. `data-setup/`, a local-only workflow that downloads or checks raw sources and rebuilds comparable model-ready inputs under ignored `_local/` directories.
`data-setup/` automatically downloads all script-accessible sources:
- PolDem
- PartyFacts crosswalk
- CHES family files
- POPPA from Harvard Dataverse
- Global Party Survey 2019 from Harvard Dataverse
- V-Party through the provider's download form
Two inputs require user-provided access/material:
- **Manifesto Project**: users must provide their own Manifesto Project API key via `MANIFESTO_API_KEY` or `PARTY2D_MANIFESTO_API_KEY`.
- **Morgan historical expert data**: `morgan_positions_raw.csv` is not publicly downloadable; it can be provided on request and should be placed locally under `_local/raw/morgan/`.
Raw downloads, intermediate build files, regenerated inputs, R package installs, and comparison reports are written only under `_local/`, which is ignored by git. The setup workflow never overwrites committed files in `data/`. See `data-setup/README.md` and `docs/RAW_DATA_SOURCES.md` for exact commands and source details.
## Running the pipeline
Run the full workflow with:
@@ -59,7 +82,7 @@ The model-ready inputs are included under `data/`:
- `union_mapping.csv`
- `party_families.csv`
Original raw source files are not redistributed. See `docs/RAW_DATA_SOURCES.md` and `data-setup/` for the list of original data sources, access information, and expected local filenames for regenerating the processed inputs. Rebuilding inputs from raw files is separate from the normal estimation workflow and writes only to ignored `_local/` test directories; it never replaces committed `data/` inputs automatically.
Original raw source files are not redistributed. Rebuilding inputs from raw files is separate from the normal estimation workflow and writes only to ignored `_local/` directories; it never replaces committed `data/` inputs automatically.
## Output variables