Tighten source setup documentation

README.md

@@ -4,7 +4,7 @@ Code and processed model inputs for generating two-dimensional party-position es
 
 ## Repository contents
 
-- `data-setup/` — local-only source download and rebuild workflow. It is included because the original source files cannot be redistributed in this public repository.
+- `data-setup/` — source download and rebuild workflow for raw files that cannot be redistributed here.
 - `src/julia/` — Stan data preparation, model fitting, post-estimation, enrichment, and validation.
 - `models/` — Stan model specification.
 - `data/` — five processed party-level inputs used by the Julia/Stan model.
@@ -15,10 +15,10 @@ Processed inputs needed by the model are included in `data/` so the estimation s
 
 ## Source data and redistribution
 
-The public repository intentionally does **not** contain the original raw/source files. Several inputs are third-party datasets with their own terms of use, and the Morgan historical file is a local OCR/transcription source. Instead, the repository provides:
+The public repository does **not** contain the original raw/source files. Several inputs are third-party datasets with their own terms of use, and the Morgan historical file is a local OCR/transcription source. Instead, this repository provides:
 
 1. committed model-ready inputs in `data/`, sufficient for fitting the Julia/Stan model; and
-2. `data-setup/`, a local-only workflow that downloads or checks raw sources and rebuilds comparable model-ready inputs under ignored `_local/` directories.
+2. `data-setup/`, a workflow that downloads or checks raw sources and rebuilds comparable model-ready inputs locally.
 
 `data-setup/` automatically downloads all script-accessible sources:
 
@@ -34,7 +34,7 @@ Two inputs require user-provided access/material:
 - **Manifesto Project**: users must provide their own Manifesto Project API key via `MANIFESTO_API_KEY` or `PARTY2D_MANIFESTO_API_KEY`.
 - **Morgan historical expert data**: `morgan_positions_raw.csv` is not publicly downloadable; it can be provided on request and should be placed locally under `_local/raw/morgan/`.
 
-Raw downloads, intermediate build files, regenerated inputs, R package installs, and comparison reports are written only under `_local/`, which is ignored by git. The setup workflow never overwrites committed files in `data/`. See `data-setup/README.md` and `docs/RAW_DATA_SOURCES.md` for exact commands and source details.
+The setup workflow never overwrites committed files in `data/`. See `data-setup/README.md` and `docs/RAW_DATA_SOURCES.md` for exact commands and source details.
 
 ## Running the pipeline
 
@@ -82,7 +82,7 @@ The model-ready inputs are included under `data/`:
 - `union_mapping.csv`
 - `party_families.csv`
 
-Original raw source files are not redistributed. Rebuilding inputs from raw files is separate from the normal estimation workflow and writes only to ignored `_local/` directories; it never replaces committed `data/` inputs automatically.
+Original raw source files are not redistributed. Rebuilding inputs from raw files is separate from the normal estimation workflow and never replaces committed `data/` inputs automatically.
 
 ## Output variables
 

data-setup/README.md

@@ -1,12 +1,12 @@
 # Data setup
 
-This directory exists because the public repository cannot redistribute the original raw/source files. It provides a reproducible, local-only workflow to download the source files that can be downloaded automatically, check user-provided restricted files, rebuild model-ready inputs, and compare the rebuilt inputs with the committed files in `../data/`.
+This directory exists because the public repository cannot redistribute the original raw/source files. It downloads the sources that can be fetched automatically, checks user-provided restricted files, rebuilds model-ready inputs, and compares them with the committed files in `../data/`.
 
 The main estimation workflow does not run these scripts. Once the five model-ready CSVs exist in `data/`, fitting and post-estimation are Julia/Stan-only.
 
-The setup workflow is intentionally local-only. It never overwrites committed files in `data/`. Raw downloads, intermediate files, regenerated inputs, and comparison reports go under `_local/`, which is ignored by git.
+The setup workflow never overwrites committed files in `data/`.
 
-Raw source files are not redistributed in this repository. Put them in `_local/raw/` or set `PARTY2D_RAW_DATA_DIR` to another local directory. See `source_manifest.csv` and `../docs/RAW_DATA_SOURCES.md` for source-specific access notes.
+Put raw source files in `_local/raw/` or set `PARTY2D_RAW_DATA_DIR` to another local directory. See `source_manifest.csv` and `../docs/RAW_DATA_SOURCES.md` for source-specific access notes.
 
 ## What downloads automatically?
 
@@ -23,7 +23,7 @@ Raw source files are not redistributed in this repository. Put them in `_local/r
 | Manifesto Project | Yes, with credentials | set your own `MANIFESTO_API_KEY` or `PARTY2D_MANIFESTO_API_KEY` |
 | Morgan historical expert data | No | place `morgan_positions_raw.csv` locally; available on request |
 
-Do not commit any downloaded or user-provided source files. All default paths are under ignored `_local/` directories.
+Do not commit downloaded or user-provided source files.
 
 Recommended local layout:
 
@@ -55,20 +55,20 @@ Check setup without downloading or rebuilding:
 bash data-setup/run_data_setup.sh --dry-run
 ```
 
-Download all automatically downloadable sources. This also downloads Manifesto if you set a Manifesto API key and V-Party if you set a V-Dem form email:
+Download all script-accessible sources. Manifesto requires an API key; V-Party requires an email for the provider form:
 
 ```bash
 bash data-setup/run_data_setup.sh --download-only
 ```
 
-For V-Party, set an email address accepted by the provider's download form:
+V-Party:
 
 ```bash
 export PARTY2D_VDEM_EMAIL='you@example.org'
 export PARTY2D_VDEM_GENDER=''   # blank means prefer not to say
 ```
 
-For Manifesto Project, users must use their own API key from their Manifesto Project account:
+Manifesto Project:
 
 ```bash
 export MANIFESTO_API_KEY='...'
@@ -76,13 +76,13 @@ export MANIFESTO_API_KEY='...'
 export PARTY2D_MANIFESTO_API_KEY='...'
 ```
 
-`morgan/morgan_positions_raw.csv` is a local OCR/transcription source derived from Morgan (1976), not a public provider download. It can be provided on request. Place it under `_local/raw/morgan/` before a full rebuild test:
+Morgan is not a public provider download. The local OCR/transcription file can be provided on request and should be placed at:
 
 ```text
 _local/raw/morgan/morgan_positions_raw.csv
 ```
 
-Rebuild model-ready inputs after placing all required local files. This writes to `_local/generated-inputs/`, not `data/`:
+Rebuild model-ready inputs after placing all required local files:
 
 ```bash
 bash data-setup/run_data_setup.sh --build-test

docs/RAW_DATA_SOURCES.md

@@ -3,11 +3,10 @@
 This repository includes model-ready inputs under `data/`, but it does not
 redistribute licensed, restricted, or third-party raw source files. Raw files
 needed to regenerate processed inputs should be kept in `_local/raw/` or another
-local-only directory selected with `PARTY2D_RAW_DATA_DIR`.
+directory selected with `PARTY2D_RAW_DATA_DIR`.
 
-The source setup scripts are provided specifically to avoid redistributing those
+The source setup scripts document how to obtain and test those files locally.
-source files while still documenting how to obtain and test them locally. Users
+Users must use their own Manifesto Project API key and obtain the Morgan OCR/
-must use their own Manifesto Project API key and must obtain the Morgan OCR/
 transcription file separately.
 
 The data setup workflow writes intermediates to `_local/build/`, regenerated