Skip to content

Genome Browser reference bundle

This runbook builds the optional local Genome Browser reference bundle. When the files are installed in the Yeliztli data directory, IGV.js uses local GRCh37/hg19 sequence and RefSeq genes instead of contacting hosted IGV reference endpoints.

Artifact contents

The runtime bundle is a directory, usually archived as yeliztli_genome_browser_reference_<date>.tar.zst, containing:

  • grch37.fa — UCSC hg19 FASTA, decompressed and renamed for the app.
  • grch37.fa.fai — FASTA index generated by the build script.
  • grch37_refseq.bed — BED12 track generated from UCSC refGene.
  • genome_browser_reference_manifest.json — source URLs, accessed date, license note, source SHA-256 values, output SHA-256 values, file sizes, and build statistics.

The source inputs are:

  • https://hgdownload.soe.ucsc.edu/goldenPath/hg19/bigZips/hg19.fa.gz
  • https://hgdownload.soe.ucsc.edu/goldenPath/hg19/database/refGene.txt.gz

UCSC's license page says Genome Browser raw data files and database table dumps are freely available for public and commercial use, while noting that source databases can impose separate terms. The artifact is treated as a BYO/provider-fetched local build: do not attach it to Yeliztli releases or redistribute it as an official Yeliztli asset without a separate license/release review. The verification checklist below keeps the manifest and attribution text with the local artifact so operators can inspect that posture directly.

Local smoke build

Use this only for script changes and tiny local fixtures. The real UCSC FASTA expands to multiple GB and should be built on SLURM.

python scripts/build_genome_browser_reference.py \
  --output-dir data/genome-browser-reference \
  --accessed-date 2026-07-01

After a successful build, copy or unpack the three runtime files into the configured Yeliztli data directory. The app discovers them by name, unless YELIZTLI_GRCH37_FASTA_PATH or YELIZTLI_GENOME_BROWSER_REFSEQ_TRACK_PATH points elsewhere.

SLURM build

Run the real build on the gpu partition and stage under node-local scratch. This follows the repo's heavy-job rule: the FASTA download and decompressed output are large enough that they should not be built on a laptop or on the login shell.

export YELIZTLI_SLURM_LOGIN=<slurm-login-host>
ssh "$YELIZTLI_SLURM_LOGIN" 'bash -ls' <<'EOF'
mkdir -p "$HOME/jobs" "$HOME/artifacts/genome-browser-reference"
job_id=$(sbatch --parsable -p gpu --job-name=gb-ref --time=02:00:00 \
  --cpus-per-task=2 --mem=12G --output="$HOME/jobs/%x-%j.log" <<'SBATCH'
#!/usr/bin/env bash
set -euo pipefail

work=/tmp/$USER-$SLURM_JOB_ID
mkdir -p "$work"
trap 'rm -rf "$work"' EXIT

repo=$work/Yeliztli
git clone --depth 1 https://github.com/bioedca/Yeliztli.git "$repo"
cd "$repo"

out=$work/genome-browser-reference
python3.12 scripts/build_genome_browser_reference.py \
  --output-dir "$out" \
  --source-dir "$work/sources" \
  --accessed-date "$(date -u +%F)"

artifact="yeliztli_genome_browser_reference_$(date -u +%Y%m%d).tar.zst"
target_dir="$HOME/artifacts/genome-browser-reference"
mkdir -p "$target_dir"
tar --use-compress-program "zstd -19 -T0" -cf "$target_dir/$artifact" -C "$out" .
sha256sum "$target_dir/$artifact" > "$target_dir/$artifact.sha256"
stat -c "%n %s" "$target_dir/$artifact" > "$target_dir/$artifact.size"
SBATCH
)
echo "Submitted $job_id"
EOF

Monitor it:

ssh "$YELIZTLI_SLURM_LOGIN" 'squeue -u "$USER"'
ssh "$YELIZTLI_SLURM_LOGIN" 'tail -n 80 "$HOME/jobs/gb-ref-<jobid>.log"'

Retrieve the artifact into a gitignored local download directory:

mkdir -p data/downloads/genome-browser-reference
rsync -av "$YELIZTLI_SLURM_LOGIN":artifacts/genome-browser-reference/ \
  data/downloads/genome-browser-reference/

Verification checklist

Before installing the artifact locally or handing it to another operator under an approved site process:

  1. Unpack the archive and inspect genome_browser_reference_manifest.json.
  2. Confirm sources.fasta.url, sources.refgene.url, accessed_date, source SHA-256, output SHA-256, and output sizes are populated.
  3. Confirm the runtime files are named exactly grch37.fa, grch37.fa.fai, and grch37_refseq.bed.
  4. Confirm the FASTA index contains GRCh37/hg19 sentinel lengths, including chr1=249250621, chr2=243199373, chr10=135534747, chrX=155270560, chrY=59373566, and chrM=16571.
  5. Install the three runtime files plus genome_browser_reference_manifest.json into a Yeliztli data directory and verify /api/igv-tracks/reference/status reports "mode": "local".
  6. Open the Genome Browser and confirm no outbound IGV reference or RefSeq requests occur in browser network logs.

The hosted IGV hg19 fallback remains the correct behavior when any one of the three runtime files is missing.