party2d/docs/RAW_DATA_SOURCES.md

Raw data sources and local-only setup

The public replication repository includes processed inputs under data/, but it does not redistribute licensed or third-party raw source files. Raw files needed to regenerate processed inputs should be kept in a local-only directory outside the replication git repository.

Recommended local layout:

_local/raw/
  poldem/poldem-election_all.csv
  manifesto/MPDataset_MPDS2025a.csv

The scripts read PARTY2D_RAW_DATA_DIR when it is set. For another location, use:

export PARTY2D_RAW_DATA_DIR=/path/to/local/raw

Required raw inputs

Source	Raw file	Local path below $PARTY2D_RAW_DATA_DIR	Used by	Redistribution
PolDem Election Campaigns, all issues	poldem-election_all.csv	poldem/poldem-election_all.csv	src/r/00c_process_poldem.R	Not redistributed in this repo
Manifesto Project Dataset	MPDataset_MPDS2025a.csv	manifesto/MPDataset_MPDS2025a.csv	src/r/00a_process_manifesto.R if data/manifesto_data.csv is regenerated	Not redistributed in this repo

PolDem download

Dataset: poldem-election_all from the PolDem Election Campaigns collection.

• Overview: https://poldem.eui.eu/data-overview/
• Download page: https://poldem.eui.eu/download/election-campaigns/
• CSV URL used locally on 2026-06-11: https://poldem.eui.eu/downloads/cosa/poldem-election_all.csv

Observed checksum after download on 2026-06-11:

sha256  2cd8c9108b1b0b9c1b6594bb21acee709c70259cd02f450bc69fc09b505fc9fb

Preflight check

From the repository root, run:

bash src/sh/check_raw_data.sh

This checks required local raw inputs and prints byte sizes and SHA-256 checksums.

Processed files kept in this repository

The processed files under data/ are intended to be part of this repository, including:

• data/poldem_data.csv
• data/manifesto_data.csv
• data/text_data.csv
• data/expert.csv
• data/lr_data.csv
• data/model_data.csv

These files document the analysis-ready inputs while avoiding redistribution of the underlying raw source data.