Require explicit data setup option
This commit is contained in:
+19
-5
@@ -49,13 +49,27 @@ _local/reports/ # comparison reports
|
||||
|
||||
## Commands
|
||||
|
||||
Check setup without downloading or rebuilding:
|
||||
Run the setup script with one explicit option:
|
||||
|
||||
```bash
|
||||
bash data-setup/run_data_setup.sh <option>
|
||||
```
|
||||
|
||||
Available options:
|
||||
|
||||
- `--dry-run` — check setup scripts only.
|
||||
- `--download-only` — download script-accessible sources.
|
||||
- `--build-test` — rebuild model-ready inputs from existing local sources.
|
||||
- `--compare` — compare regenerated inputs with committed `data/` inputs.
|
||||
- `--full-test` — download, rebuild, and compare locally.
|
||||
|
||||
Minimal check:
|
||||
|
||||
```bash
|
||||
bash data-setup/run_data_setup.sh --dry-run
|
||||
```
|
||||
|
||||
Download all script-accessible sources. Manifesto requires an API key; V-Party requires an email for the provider form:
|
||||
Download sources:
|
||||
|
||||
```bash
|
||||
bash data-setup/run_data_setup.sh --download-only
|
||||
@@ -82,19 +96,19 @@ Morgan is not a public provider download. The local OCR/transcription file can b
|
||||
_local/raw/morgan/morgan_positions_raw.csv
|
||||
```
|
||||
|
||||
Rebuild model-ready inputs after placing all required local files:
|
||||
Rebuild model-ready inputs:
|
||||
|
||||
```bash
|
||||
bash data-setup/run_data_setup.sh --build-test
|
||||
```
|
||||
|
||||
Compare regenerated inputs with the committed inputs:
|
||||
Compare regenerated inputs:
|
||||
|
||||
```bash
|
||||
bash data-setup/run_data_setup.sh --compare
|
||||
```
|
||||
|
||||
Run the full local test sequence — source downloads, raw-file preflight, local rebuild, and comparison — after setting required environment variables and placing Morgan locally:
|
||||
Full local source test:
|
||||
|
||||
```bash
|
||||
bash data-setup/run_data_setup.sh --full-test
|
||||
|
||||
@@ -1,14 +1,32 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
MODE="${1:---dry-run}"
|
||||
usage() {
|
||||
cat >&2 <<'EOF'
|
||||
Usage: bash data-setup/run_data_setup.sh <option>
|
||||
|
||||
Options:
|
||||
--dry-run check setup scripts only
|
||||
--download-only download source files where scriptable
|
||||
--build-test rebuild inputs from existing local sources
|
||||
--compare compare regenerated inputs with committed data/
|
||||
--full-test download, rebuild, and compare locally
|
||||
EOF
|
||||
}
|
||||
|
||||
if [ "$#" -ne 1 ]; then
|
||||
usage
|
||||
exit 1
|
||||
fi
|
||||
|
||||
MODE="$1"
|
||||
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd -P)"
|
||||
cd "$repo_root"
|
||||
|
||||
case "$MODE" in
|
||||
--dry-run|--download-only|--build-test|--compare|--full-test) ;;
|
||||
*)
|
||||
echo "Usage: $0 [--dry-run|--download-only|--build-test|--compare|--full-test]" >&2
|
||||
usage
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
@@ -69,8 +69,7 @@ From the repository root, run:
|
||||
bash data-setup/run_data_setup.sh --dry-run
|
||||
```
|
||||
|
||||
This parses the setup scripts without downloading or rebuilding. To check local
|
||||
raw file placement and print byte sizes/checksums, run:
|
||||
To check local raw file placement and print byte sizes/checksums, run:
|
||||
|
||||
```bash
|
||||
bash data-setup/check_raw_data.sh
|
||||
|
||||
Reference in New Issue
Block a user