Reference data¶
Yeliztli annotates your variants against public scientific datasets. They fall into two groups: prebuilt bundles that Yeliztli publishes, and pipeline sources fetched from their original providers. They also have two setup roles:
- Required databases must be downloaded or built and pass integrity checks before the first-time setup wizard can complete.
- Optional sources add context or optional features. Most can be installed later from Settings → Database Management; manual or bring-your-own sources use the local ingest path documented for that source.
The setup-blocking required databases are ClinVar, gnomAD, dbNSFP, CPIC,
GWAS Catalog, dbSNP, and MONDO/HPO. If any required database is Failed or not
integrity-ready, setup remains active and the dashboard is not reachable until you resume,
retry, or clean that database. Reference data is stored under your data directory (default
~/.yeliztli/).
All downloads are resumable and integrity-checked. You can inspect, resume, verify, or clean any of them under Settings → System Health → Database Health.
First-run duration¶
Full reference-data setup is a long one-time operation. It uses more than 60 GB at peak, with ~80 GB recommended for headroom, and commonly takes on the order of an hour or more. Slow networks, slower disks, or optional databases can make it considerably longer.
The dbNSFP step dominates the runtime. Its source archive is a large network download,
and then Yeliztli parses, builds, and indexes a multi-GB SQLite database. Download progress is
shown as bytes transferred; the later build/index phase is CPU- and disk-bound and may not
show the same moving progress bar. If Settings → System Health → Database Health shows
the database as Downloading or Building, setup is still active. If it shows Failed, use
the repair/resume controls from that page.
Prebuilt bundles¶
These are published as GitHub release assets, pinned by version and SHA-256 checksum in
bundles/manifest.json.
| Bundle | Setup role | Approx. size | What it provides |
|---|---|---|---|
| gnomAD allele frequencies | Required | ~1.30 GB download / ~2.85 GB installed | Population allele frequencies, observed allele counts, and homozygous counts — CC0 / public domain. |
| VEP consequence bundle | Optional | ~360 MB | Pre-computed variant consequences, HGVS, and transcript context for the genotyped sites. |
| PGS scores | Optional | ~104 MB | Polygenic-score weight sets used by the risk modules. |
| Ancestry PCA bundle | Optional (ships with app) | ~0.4 MB | Ancestry-informative markers and PCA loadings — ships inside the app, no download. |
| Ancestry LAI bundle | Optional | ~1.7 GB | Local-ancestry-inference models + phasing reference for Tier-2 chromosome painting. Requires Java 8+; only download it if you want chromosome-level ancestry. |
Pipeline sources¶
These are downloaded from the original providers. Each retains its own license — the full
attribution list lives in the repository
NOTICE file.
| Source | Setup role | Purpose | Approx. setup footprint | License |
|---|---|---|---|---|
| ClinVar (NCBI) | Required | Clinical variant classifications | ~250 MB | Public domain |
| dbNSFP | Required | In-silico pathogenicity predictions (REVEL, CADD, ...) | ~50 GB transient ZIP + ~10+ GB built DB | Academic / non-commercial |
| CPIC | Required | Pharmacogenomics allele & guideline data | ~5 MB | CC0-1.0 |
| ClinGen | Optional | Gene-disease validity & dosage | ~1 MB | CC0-1.0 |
| PharmVar | Supporting source for CPIC | Pharmacogene star-allele definitions | Small metadata source | Open |
| AlphaMissense | Optional | Missense pathogenicity predictions | ~3.5 GB when installed | CC-BY-4.0 |
| GWAS Catalog (EBI) | Required | Trait/disease associations for risk modules | ~100 MB | Open |
| dbSNP (NCBI) | Required | rsID merge/identity resolution | ~20 MB | Public domain |
| MONDO/HPO (Monarch) | Required | Disease & phenotype associations | ~15 MB | Open |
| PharmGKB | Optional context | Clinical drug annotations | Small metadata source | CC-BY-SA-4.0 |
| FDA drug labels (via PharmGKB) | Optional context | Pharmacogenomic labeling | Small metadata source | CC-BY-SA-4.0 |
| GTEx eQTL | Optional | Tissue eQTLs for functional context | ~3 GB when installed | Open-access summary stats |
| SpliceAI | Optional / manual | User-supplied splice-effect prediction database | Depends on local ingest | Illumina non-commercial terms |
| ENCODE cCREs | Optional | Candidate cis-regulatory elements for Genome Browser tracks | ~30 MB when installed | ENCODE data terms |
UCSC hg19 FASTA + RefSeq (refGene) |
Optional local browser reference | Fully local Genome Browser reference and gene track | ~4 GB when installed | UCSC Genome Browser data terms |
dbNSFP license
dbNSFP is distributed under an academic / non-commercial license. Make sure your use complies with its terms. Its setup footprint is also large: the source archive is removed after a successful build, but an interrupted build may keep the completed archive so setup can resume without starting another large download. The build and index step can take a long time after the archive has already downloaded.
HIBAG HLA model files¶
The HLA (imputed) feature uses HIBAG pre-fit model files, but those files are not Yeliztli release bundles or setup-wizard downloads. They are bring-your-own external inputs because HIBAG itself is an operator-installed R/Bioconductor runtime and the model files carry their own distribution terms.
If you enable HLA imputation, fetch the model files yourself, place them in the directory
configured by YELIZTLI_HIBAG_MODEL_DIR, and keep their expected names:
European-HLA4.RData, Asian-HLA4.RData, Hispanic-HLA4.RData, or African-HLA4.RData.
Then run scripts/predict_hla.py for each sample to populate hla_calls.
Genome Browser local reference files¶
The Genome Browser can run without contacting IGV.js reference hosts when these local files are installed:
grch37.fagrch37.fa.faigrch37_refseq.bedgenome_browser_reference_manifest.json
By default Yeliztli looks for them in the data directory. You can point to local runtime files
with YELIZTLI_GRCH37_FASTA_PATH and
YELIZTLI_GENOME_BROWSER_REFSEQ_TRACK_PATH, but the files are accepted only when the manifest
describes the expected UCSC hg19 FASTA / refGene build and the FASTA index matches GRCh37/hg19
sentinel chromosome lengths. If any file is missing or validation fails, the Genome Browser keeps
the disclosure-gated hosted hg19 fallback.
Maintainers can build the local reference files from UCSC sources with
scripts/build_genome_browser_reference.py.
The SLURM procedure and provenance checklist are in the
Genome Browser reference bundle runbook.
Updating reference data¶
The per-database auto-update table in Settings → Database Management covers the sources registered in the update manager: ClinVar, dbNSFP, CPIC, GWAS Catalog, dbSNP, MONDO/HPO, ENCODE cCREs, the published gnomAD, VEP, LAI, and PGS score bundles, and the app-shipped ancestry PCA bundle. For those sources you can configure auto-update toggles and an optional bandwidth window for large downloads. See updating.
Static / manual-refresh sources¶
Some registered sources are install-time or bring-your-own context layers, not auto-updated reference data:
- AlphaMissense is built when installed and is not version-checked by the automatic update system.
- GTEx eQTL is built when installed and is not version-checked by the automatic update system.
- ClinGen gene-disease validity data is built when installed and is not version-checked by the automatic update system.
- SpliceAI is a manual, user-supplied database and is never downloaded or updated by Yeliztli.
These sources do not appear in the per-database auto-update table and do not have automatic upstream release checks. To refresh them, use the relevant manual rebuild, local ingest, or setup flow; then re-annotate samples whose results should reflect the refreshed snapshot.