Require explicit data setup option
This commit is contained in:
+19
-5
@@ -49,13 +49,27 @@ _local/reports/ # comparison reports
|
|||||||
|
|
||||||
## Commands
|
## Commands
|
||||||
|
|
||||||
Check setup without downloading or rebuilding:
|
Run the setup script with one explicit option:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash data-setup/run_data_setup.sh <option>
|
||||||
|
```
|
||||||
|
|
||||||
|
Available options:
|
||||||
|
|
||||||
|
- `--dry-run` — check setup scripts only.
|
||||||
|
- `--download-only` — download script-accessible sources.
|
||||||
|
- `--build-test` — rebuild model-ready inputs from existing local sources.
|
||||||
|
- `--compare` — compare regenerated inputs with committed `data/` inputs.
|
||||||
|
- `--full-test` — download, rebuild, and compare locally.
|
||||||
|
|
||||||
|
Minimal check:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
bash data-setup/run_data_setup.sh --dry-run
|
bash data-setup/run_data_setup.sh --dry-run
|
||||||
```
|
```
|
||||||
|
|
||||||
Download all script-accessible sources. Manifesto requires an API key; V-Party requires an email for the provider form:
|
Download sources:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
bash data-setup/run_data_setup.sh --download-only
|
bash data-setup/run_data_setup.sh --download-only
|
||||||
@@ -82,19 +96,19 @@ Morgan is not a public provider download. The local OCR/transcription file can b
|
|||||||
_local/raw/morgan/morgan_positions_raw.csv
|
_local/raw/morgan/morgan_positions_raw.csv
|
||||||
```
|
```
|
||||||
|
|
||||||
Rebuild model-ready inputs after placing all required local files:
|
Rebuild model-ready inputs:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
bash data-setup/run_data_setup.sh --build-test
|
bash data-setup/run_data_setup.sh --build-test
|
||||||
```
|
```
|
||||||
|
|
||||||
Compare regenerated inputs with the committed inputs:
|
Compare regenerated inputs:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
bash data-setup/run_data_setup.sh --compare
|
bash data-setup/run_data_setup.sh --compare
|
||||||
```
|
```
|
||||||
|
|
||||||
Run the full local test sequence — source downloads, raw-file preflight, local rebuild, and comparison — after setting required environment variables and placing Morgan locally:
|
Full local source test:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
bash data-setup/run_data_setup.sh --full-test
|
bash data-setup/run_data_setup.sh --full-test
|
||||||
|
|||||||
@@ -1,14 +1,32 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
MODE="${1:---dry-run}"
|
usage() {
|
||||||
|
cat >&2 <<'EOF'
|
||||||
|
Usage: bash data-setup/run_data_setup.sh <option>
|
||||||
|
|
||||||
|
Options:
|
||||||
|
--dry-run check setup scripts only
|
||||||
|
--download-only download source files where scriptable
|
||||||
|
--build-test rebuild inputs from existing local sources
|
||||||
|
--compare compare regenerated inputs with committed data/
|
||||||
|
--full-test download, rebuild, and compare locally
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
if [ "$#" -ne 1 ]; then
|
||||||
|
usage
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
MODE="$1"
|
||||||
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd -P)"
|
repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd -P)"
|
||||||
cd "$repo_root"
|
cd "$repo_root"
|
||||||
|
|
||||||
case "$MODE" in
|
case "$MODE" in
|
||||||
--dry-run|--download-only|--build-test|--compare|--full-test) ;;
|
--dry-run|--download-only|--build-test|--compare|--full-test) ;;
|
||||||
*)
|
*)
|
||||||
echo "Usage: $0 [--dry-run|--download-only|--build-test|--compare|--full-test]" >&2
|
usage
|
||||||
exit 1
|
exit 1
|
||||||
;;
|
;;
|
||||||
esac
|
esac
|
||||||
|
|||||||
@@ -69,8 +69,7 @@ From the repository root, run:
|
|||||||
bash data-setup/run_data_setup.sh --dry-run
|
bash data-setup/run_data_setup.sh --dry-run
|
||||||
```
|
```
|
||||||
|
|
||||||
This parses the setup scripts without downloading or rebuilding. To check local
|
To check local raw file placement and print byte sizes/checksums, run:
|
||||||
raw file placement and print byte sizes/checksums, run:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
bash data-setup/check_raw_data.sh
|
bash data-setup/check_raw_data.sh
|
||||||
|
|||||||
Reference in New Issue
Block a user