Make data setup a no-option entry point

This commit is contained in:
Armin
2026-06-15 17:53:31 +02:00
parent 9b925f340f
commit 55b4895872
4 changed files with 15 additions and 89 deletions
+4 -37
View File
@@ -55,25 +55,10 @@ Run the full source-data setup workflow with:
bash data-setup/run_data_setup.sh
```
Maintenance options:
- `--dry-run` — check setup scripts only.
- `--download-only` — download script-accessible sources.
- `--build-test` — rebuild model-ready inputs from existing local sources.
- `--compare` — compare regenerated inputs with committed `data/` inputs.
- no option — download, rebuild, and compare locally.
Minimal check:
```bash
bash data-setup/run_data_setup.sh --dry-run
```
Download sources:
```bash
bash data-setup/run_data_setup.sh --download-only
```
The command downloads script-accessible sources, checks required local files,
rebuilds model-ready inputs under `_local/generated-inputs/`, and writes a
comparison report under `_local/reports/`. It never replaces committed files in
`data/`.
V-Party:
@@ -96,24 +81,6 @@ Morgan is not a public provider download. The local OCR/transcription file can b
_local/raw/morgan/morgan_positions_raw.csv
```
Rebuild model-ready inputs:
```bash
bash data-setup/run_data_setup.sh --build-test
```
Compare regenerated inputs:
```bash
bash data-setup/run_data_setup.sh --compare
```
Full local source test:
```bash
bash data-setup/run_data_setup.sh
```
The comparison writes `_local/reports/input_comparison.md`. Replacing committed inputs, if ever needed, is a separate manual decision and is not done by these scripts.
Known behavior: the current public/rebuilt sources run through the workflow successfully, but regenerated `text_data.csv`, `expert.csv`, and `lr_data.csv` are not byte-identical to the committed model-ready inputs because of source-version and linkage differences. The comparison report records those differences explicitly.