← VCF Desk / API
Tokens

Drive VCF Desk from your own code

VCF Desk reads a variant call file and returns one of three structured reviews over that same file: a triage pass that says whether the callset can be trusted and what to filter, an interpret pass that groups the variants by the genes the file itself annotates and writes the database queries for everything it refuses to invent, or a cohort pass that says whether the callset can be loaded into a population-scale variant store. Everything the web page does is available over HTTP, so the natural uses are a pipeline step that triages every callset as the caller finishes, a nightly sweep that re-checks cohort readiness across a delivery directory, and a gate that refuses to hand a callset to an analyst while a blocker is open.

The reply is always one JSON object — no prose, no Markdown, no code fence — so it drops straight into a script. Everything below is the exact contract the app itself speaks; there is no second, friendlier API behind it.

Base URL and the envelope

Every endpoint lives under https://api.skillsafe.ai/v1/app-api and every response uses the same envelope, so one helper covers the whole API: 

{ "ok": true,  "data":  { ... } }
{ "ok": false, "error": { "code": "...", "message": "...", "details": { ... } } }

On success read data; on failure read error.code and branch on that — message is for humans and details carries whatever the endpoint could say about the specific field or job that failed. A client that checks ok once, in one place, never has to check it again. Authentication is one header on every call: Authorization: Bearer <token>.

There is no X-App-Slug header

The slug vcf-desk travels in exactly one place: the JSON body of POST /guest. After that the token itself carries the app binding, and the only headers any request needs are Content-Type: application/json and Authorization: Bearer <token>, plus Idempotency-Key on run calls. If you have copied a snippet from another SkillSafe app that sets X-App-Slug, delete that line; it is not read here, and its presence hides the real cause when something 401s.

Endpoints

endpointmethodmeteredwhat it does
/guestPOSTnoMints a guest token from {"slug":"vcf-desk"}. A guest can read /me and price a run; it cannot start one.
/meGETnoWho the token belongs to, and the credit balance.
/estimatePOSTno — but authenticatedPrices a run. Creates no job, charges nothing, and 401s without a token.
/runPOSTyesStarts a review, returns {"job_id"}.
/jobs/{job_id}GETnoJob status and, once terminal, the output and the real charge.
/run-streamPOSTyesThe same review as Server-Sent Events. The default for anything long.

Error codes

codestatuswhat it means here, and what to do
UNAUTHORIZED401No token, a malformed token, or an expired one. It is also what /estimate returns before you have minted anything. Mint a fresh token, or copy the one this browser holds from the token page.
FORBIDDEN403The token is real but not allowed to do this: a token minted for a different app, or a guest token calling /run. Runs are for signed-in accounts; re-mint against {"slug":"vcf-desk"} for the free endpoints, or sign in for a personal token.
NOT_FOUND404An unknown job_id in GET /jobs/{job_id}, or a path that does not exist. Job ids are opaque: echo back what /run gave you, never construct one.
VALIDATION_ERROR400The body is not valid JSON, vcf is missing, or a field has the wrong type — a string where prescan_facts should be an object, a list where context belongs. details names the field. A body wrapped in an "input" key does not land here; see the warning in step 4.
INSUFFICIENT_CREDITS402The balance is below min_credits. Call /estimate for the lane you are about to run, compare against credits from /me, and top up — a bare retry cannot succeed.
RATE_LIMITED429Too many requests in too little time. Back off exponentially; do not tight-loop. Two seconds between GET /jobs/{job_id} polls is plenty.
INTERNAL500A server-side failure. Retry with the same Idempotency-Key so the retry cannot be billed as a second review.

The input contract — read this before anything else

This app has three lanes over one file, and the run input is a single JSON object whose first field decides which one you get. Getting task right is the difference between a filter plan and an ingest plan; nothing later in this page matters as much.

task — the router field, and it comes first

task is one of exactly three lane ids. It selects the prompt that runs, the enum verdict is drawn from, which extra keys come back in the reply, and which of the other input fields are read at all.

taskthe question it answersalso readsadds to the reply
triageCan this callset be trusted, and what do I filter?intentmetrics_read, checks, findings, filter_plan
interpretWhich variants and genes come next, and what do I look up?focusannotation_state, gene_groups, shortlist, lookup_plan, evidence_gaps
cohortCan this go into the population variant store?cohort_contextreadiness, blockers, ingest_plan, cohort_risks

"triage" — QC review of the callset

The lane to run first, and the one to run in CI. It reads the header against the records, quotes the metrics the prescan computed, walks fifteen named checks in a fixed order, ranks findings by consequence, and ends in an ordered filter_plan of real bcftools commands where every threshold is justified by a number in the file or by the caller's own documented convention. verdict is release | filter | recall — release means structurally sound and consistent with intent, filter means usable once corrected, and recall means filtering cannot fix it and the variants have to be produced again upstream.

"interpret" — gene-level follow-up

Groups the variants by gene, ranks a shortlist, and writes the lookups. verdict is ready-to-prioritise | annotate-first | not-interpretable — ready-to-prioritise when the file carries functional annotation (CSQ, ANN or BCSQ) so gene grouping is possible from the file itself, annotate-first when the file names no genes, and not-interpretable for a gVCF, a sites-only reference panel, or a file too broken to read.

"cohort" — population-store ingestion readiness

Eleven fixed readiness requirements, the blockers that stand in the way, and the attribute schema the header can actually support, ending in ordered TileDB-VCF ingest commands. verdict is ingest | fix-first | do-not-ingest — do-not-ingest is reserved for the cases where adding the file would corrupt the store: a build mismatch, a contig-naming mismatch, a gVCF into a callset store, samples that already exist.

If task is absent or is not one of those three ids, the run is not refused: the model picks the closest lane, does that lane's contract in full, and names the choice it made in the first sentence of headline. That is convenient interactively and a bad idea in a pipeline — send the lane explicitly, and read lane back off the reply before you branch on the lane-specific keys. Two lanes are never blended into one reply.

Input fields

fieldtypelanesnotes
taskstringall Required in practice. One of triage, interpret, cohort. If omitted the model picks the closest lane and names its choice in headline.
vcfstringall Required. The VCF file text including its ## header lines. Without the header there are no declarations to validate the records against, and half the review collapses into unknown: no ##contig lengths means no reference build, no ##INFO and ##FORMAT means no arity checking and no attribute schema. Must be tab-delimited text — a file whose tabs were turned into spaces by a chat client reads as a single malformed column. A gzipped .vcf.gz or a BCF must be decompressed first (bcftools view writes plain VCF to stdout); this endpoint takes text, not bytes.
prescan_factsobjectall The facts the browser computed before the run. Optional over the API and strongly recommended: the model is instructed to quote its numbers from here rather than recompute them, and to reconcile every prescan_facts.flags[].id exactly once in coverage_check. Shape below.
intentstringtriage What kind of callset this is: germline-wgs, germline-exome, somatic, joint-cohort, panel, long-read, unsure. It changes which QC expectations apply — a somatic callset is not judged against germline Ti/Tv or het/hom ratios, and a panel is not expected to look like a genome. Send unsure rather than a wrong value; a wrong intent produces confidently wrong metrics_read rows.
focusstringinterpret Optional: a gene list, a phenotype, or a free-text question. It reorders the shortlist and the lookup plan; it cannot conjure annotation the file does not carry.
cohort_contextstringcohort Optional: what is already in the store — its build, its contig naming, its sample count, its callers, its partitioning. When it is absent, store-compatibility checks (naming, build, sample collision) are evaluated against the file alone and come back unknown rather than ready. That is the honest answer, and it is not the answer you want from a gate.
contextstringall Optional free text: how the callset was produced, what is already known to be wrong, what happens to it next. A known-bad sample stated here comes back as an assumption honoured rather than as a finding rediscovered.
retry_notestringall Optional; send it only when a previous reply failed to parse. The instruction is obeyed literally, so say what went wrong: "the previous reply was wrapped in a code fence" or "coverage_check was missing VD-GVCF".

prescan_facts, field by field

In the browser this object is produced for free, before any run, by a local VCF reader that parses every meta line into its structured form, validates each record against the header's declared Number and Type, pins the reference build from declared contig lengths and computes the QC metrics. Over the API you may send it, send a subset, or omit it. This is the whole shape: 

{
  "is_vcf":           true,          // did it parse as VCF at all
  "fileformat":       "VCFv4.2",     // from ##fileformat, or null
  "reference_line":   "file:///ref/GRCh38.fa",   // from ##reference, or null
  "caller":           "GATK HaplotypeCaller",    // detected from the header, or null
  "assembly":         { "name": "GRCh38", "confidence": "high",
                        "evidence": "chr1 length 248956422 from ##contig",
                        "species": "Homo sapiens" },   // or null
  "contig_style":     "chr-prefixed",            // or "plain", or "mixed"
  "contigs_declared": 25,            // how many ##contig lines
  "contigs_seen":     ["chr1", "chr2", "chrX"],  // first 30, in file order
  "samples":          ["NA12878"],
  "sample_count":     1,
  "data_lines":       184203,        // non-header lines in the text
  "records_parsed":   50000,         // how many were actually parsed
  "parse_ceiling_hit": true,         // records_parsed < data_lines: the window is a prefix
  "is_gvcf":          false,         // END / NON_REF seen: a reference-block file
  "counts":  { "snv": 41022, "ins": 3110, "del": 3401, "mnv": 92, "symbolic": 0,
               "breakend": 0, "star": 118, "missingAlt": 0, "invalid": 3,
               "multiallelic": 1204 },
  "titv":    { "ti": 27400, "tv": 13600, "ratio": 2.01 },
  "qual":    { "missing": 0, "scored": 50000, "min": 10.4,
               "median": 412.9, "mean": 508.2, "max": 24011.0 },
  "filters": { "PASS": 46120, "LowGQ": 3880 },   // FILTER tally over parsed records
  "filter_undeclared": ["LowGQ"],    // used in records, never declared in ##FILTER
  "info_declared":    ["AC", "AN", "AF", "DP", "ExcessHet", "CSQ"],
  "info_used":        ["AC", "AN", "AF", "DP", "CSQ"],
  "info_undeclared":  ["MQRankSum"], // used but not declared: the ones that break tools
  "info_unused":      ["ExcessHet"], // declared but never used
  "format_declared":  ["GT", "AD", "DP", "GQ", "PL"],
  "format_used":      ["GT", "AD", "DP", "GQ", "PL"],
  "format_undeclared": [],
  "per_sample": [ { "name": "NA12878", "called": 49120, "nocall": 880,
                    "het": 28800, "homref": 120, "homvar": 20200,
                    "missingRate": 0.0176, "hetHomRatio": 1.426,
                    "meanDp": 41.2, "medianDp": 38, "meanGq": 71.4,
                    "ploidy": "2" } ],
  "sort":       { "pos_inversions": 2, "contig_re_entries": 1, "duplicate_sites": 4 },
  "annotation": { "kind": "CSQ", "tool": "VEP",
                  "fields": ["Allele", "Consequence", "IMPACT", "SYMBOL",
                             "Gene", "Feature_type", "Feature", "BIOTYPE",
                             "HGVSc", "HGVSp"] },   // or null when unannotated
  "genes_seen": ["BRCA2", "TP53", "CFTR"],          // first 25 symbols found in the text
  "flags": [ { "id": "VD-FILTER-UNDECLARED", "severity": "high",
               "title": "FILTER token used but not declared",
               "evidence": "LowGQ on 3,880 records; no ##FILTER=<ID=LowGQ>" } ]
}

Every key is optional. An empty {} is legitimate and the review still runs — the model reads vcf either way. What sending facts buys you is twofold. First, the model is told to quote counts, ratios, depths, missingness and QUAL statistics from the facts rather than recompute them from the text it was handed, which matters because vcf may be an excerpt while the facts were computed over the whole parsed window: set records_parsed and parse_ceiling_hit honestly and the review knows what it is not seeing.

Second, the reconciliation contract: every id in flags comes back exactly once in coverage_check — no duplicates, none missing, and no ids you did not send. Each entry is either "addressed", with where naming the finding, check or readiness row that carries it, or "set-aside", with note saying why it does not matter for this file. set-aside is a correct answer and a different thing from silence: a mixed-ploidy warning on a chrX-only callset, or a high-missingness flag on a deliberately sparse joint call, is better set aside with a reason than agreed with. A flag that never appears at all is a failed review, and it is worth asserting that in your client — see the invariants at the end of this page.

Flag ids are stable strings of the form VD-*: VD-NO-FILEFORMAT, VD-FILEFORMAT-LATE, VD-NO-CONTIGS, VD-CONTIG-UNDECLARED, VD-CONTIG-MIXED, VD-CONTIG-INTERLEAVED, VD-ASSEMBLY-UNKNOWN, VD-ASSEMBLY-AMBIGUOUS, VD-INFO-UNDECLARED, VD-INFO-ARITY, VD-FORMAT-UNDECLARED, VD-FORMAT-ARITY, VD-FILTER-UNDECLARED, VD-COLUMN-COUNT, VD-UNSORTED-POS, VD-DUPLICATE-SITES, VD-GT-RANGE, VD-AC-GT-AN, VD-QUAL-ABSENT, VD-TITV-LOW, VD-TITV-HIGH, VD-GVCF, VD-SITES-ONLY, VD-NO-ANNOTATION, VD-PLOIDY-MIXED, VD-PARSE-CEILING and the per-sample forms VD-MISSING-<sample>, VD-LOWDP-<sample>, VD-HETHOM-HIGH-<sample>, VD-HETHOM-LOW-<sample>. You are free to raise your own ids from your own tooling; the reconciliation contract binds whatever you send.

1. A tiny client, and where the token comes from

One helper that adds the headers, unwraps data and raises on error. Because the envelope never changes, this is the only place in your program that needs to know what a SkillSafe response looks like.

Step 1 needs a token — the token page shows the one this browser already holds, masked, with copy buttons: Copy token for the raw value and Copy shell export for a ready-made export SKILLSAFE_TOKEN=… line. Nothing there needs DevTools. A guest token, minted from POST /guest with {"slug":"vcf-desk"}, is enough for /me and /estimate; a review is metered, so /run and /run-stream need a personal token from signing in. 

# Every call is the same three things: the base URL, your bearer token and a JSON
# body. No X-App-Slug header anywhere -- the slug only appears in POST /guest.
BASE="https://api.skillsafe.ai/v1/app-api"
TOKEN="$SKILLSAFE_TOKEN"   # from https://vcf-desk.skillsafe.ai/tokens.html

call() {                   # call <path> [json-body]
  if [ -n "$2" ]; then
    curl -sS -X POST "$BASE/$1" \
      -H "Authorization: Bearer $TOKEN" \
      -H "Content-Type: application/json" \
      -d "$2"
  else
    curl -sS "$BASE/$1" -H "Authorization: Bearer $TOKEN"
  fi
}

# A guest token, if you do not want to sign in yet. Enough for /me and /estimate.
curl -sS -X POST "$BASE/guest" -H "Content-Type: application/json" \
  -d '{"slug":"vcf-desk"}'
# {"ok":true,"data":{"token":"sk_guest_...","subject_type":"guest","expires_at":"..."}}

# These docs unwrap with python3 -c for portability; jq works just as well.
data() { python3 -c 'import sys,json;print(json.dumps(json.load(sys.stdin)["data"]))'; }

2. Session state with /me

Free, and the cheapest way to find out what your token actually is before you build on it. Three fields matter: subject_type is user or guest — a guest can price a review but a run comes back FORBIDDEN — username is the signed-in account, and credits is the wallet balance in credits. Compare credits against min_credits from the next step before you run, so a shortfall surfaces as your own clear message rather than as a 402 halfway through a batch. 

call me
# {"ok":true,"data":{"subject_type":"user","username":"you","credits":51234}}

# A guest token answers too, with subject_type "guest" and no username:
# {"ok":true,"data":{"subject_type":"guest","credits":0}}

3. Price it with POST /estimate — free

/estimate takes the same input object a run takes and returns the pricing. It creates no job and charges nothing. It does need a token, so it 401s if you have not minted one.

fieldmeaning
modelThe concrete model this run binds to.
model_aliasThe stable alias the app is written against — gpt-terra.
markup_bpsPlatform markup in basis points. 1000 is 10%.
hold_creditsWhat gets reserved when the run starts.
min_creditsThe balance you must clear for the run to be accepted at all.
sponsor_enabledWhether the app is covering this run rather than your wallet.

Said plainly, because it is the number people misread: hold_credits is a reservation priced against the full output cap, not the price. It is what the run would cost if the model wrote to its very last permitted token. The settled charged_credits on the finished job is usually far lower — often a small fraction of the hold, because a review that finds a clean callset is short. Budget against hold_credits; report and reconcile against charged_credits.

hold_credits differs per lane

The three lanes carry different prompts and different output caps — a triage reply has fifteen checks, a findings array and a filter plan; a cohort reply has eleven readiness rows and an ingest plan; an interpret reply can be a long lookup plan or almost nothing at all when the file is unannotated. So estimate the lane you are about to run, with the same task you will send to /run. An estimate taken for interpret and spent on triage is not a quote for anything. In a batch that walks all three lanes over one file, price all three. 

# Build the body from the real file rather than escaping a VCF by hand. Note the
# header lines go in with it: bcftools view keeps them, `grep -v '^#'` does not.
INPUT=$(python3 - <<'PY'
import json, pathlib
print(json.dumps({
    "task": "triage",
    "vcf": pathlib.Path("callset.vcf").read_text(encoding="utf-8"),
    "intent": "germline-exome",
    "context": "GATK HaplotypeCaller then SelectVariants; about to hand it to an analyst.",
    "prescan_facts": {},
}))
PY
)

call estimate "$INPUT"
# {"ok":true,"data":{"model":"gpt-5.6-terra","model_alias":"gpt-terra",
#   "markup_bps":1000,"hold_credits":3180,"min_credits":420,"sponsor_enabled":false}}
#
# estimate is FREE: no job, no charge. hold_credits is a RESERVATION against the
# full output cap; charged_credits after settlement is normally much lower. Price
# each lane separately -- the caps differ.

4. Run it with POST /run, then poll GET /jobs/{job_id}

This is the metered call. POST /run returns a job_id immediately; poll GET /jobs/{job_id} every couple of seconds until status is succeeded or failed. The review is the JSON string at data.output.output — parse it once. The terminal job also carries charged_credits, the real price, and truncated, which says the reply was cut against the output cap and is a prefix rather than a review.

The request body is the input object itself

Send {"task": "...", "vcf": "...", ...} as the whole body. Do not wrap it in an "input" key. A wrapped body does not fail validation — it returns 200 and starts a run in which the model never sees task at all, so you pay for a lane you did not choose. If a reply comes back naming a lane in headline that you thought you had specified, check for that wrapper first.

Always send an Idempotency-Key. Derive it from the lane and a hash of the input, as in vcf-desk:<task>:<hash>:a1; a retried request carrying the same key returns the same job instead of billing a second review, which is what makes a pipeline retry safe after a network blip. There is a whole section on this below, and one rule that cannot wait: the key must include the lane. 

# The key carries the lane, because triage and cohort over the same VCF are two
# distinct runs that must not collide on one key.
HASH=$(printf '%s' "$INPUT" | shasum -a 256 | cut -c1-16)
KEY="vcf-desk:triage:$HASH:a1"

JOB=$(curl -sS -X POST "$BASE/run" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $KEY" \
  -d "$INPUT" | python3 -c 'import sys,json;print(json.load(sys.stdin)["data"]["job_id"])')

while :; do
  OUT=$(call "jobs/$JOB")
  STATUS=$(printf '%s' "$OUT" | python3 -c 'import sys,json;print(json.load(sys.stdin)["data"]["status"])')
  [ "$STATUS" = "succeeded" ] && break
  [ "$STATUS" = "failed" ] && echo "$OUT" && exit 1
  sleep 2
done

# {"ok":true,"data":{"job_id":"job_...","status":"succeeded",
#   "output":{"output":"{\"lane\":\"triage\",\"verdict\":\"filter\", ...}"},
#   "charged_credits":602,"truncated":false}}
printf '%s' "$OUT" | python3 -c '
import sys, json
job = json.load(sys.stdin)["data"]
review = json.loads(job["output"]["output"])
print(review["lane"], review["verdict"], "-", review["headline"])
print("charged", job["charged_credits"], "truncated", job["truncated"])'

5. Or stream it with POST /run-stream

Same body, same Idempotency-Key, same metering — the difference is that the reply arrives as Server-Sent Events while it is being written. This is the default for anything long, and a whole-genome triage with fifteen checks and a filter plan is long: a stream gives you a first token in a second or two and no idle connection waiting on a poll interval. The web app streams; a batch job that only wants the final object is the case where polling is simpler.

Four event types, each with a JSON data: line:

eventdatawhat to do with it
job{"job_id": "job_..."}Arrives first. Record it — if the connection drops you can still read the outcome from GET /jobs/{job_id}.
delta{"text": "..."}Append text to a buffer. The concatenation of every delta is the review JSON string; do not try to parse a partial buffer.
done{"job_id", "status", "charged_credits", "output": {"output": "..."}, "truncated"}Terminal. output.output is the same string the polling path returns, so you can ignore your own buffer and parse this instead.
error{"code", "message", "job_id"}Terminal failure. Branch on code exactly as with the envelope's error.code.

A pending event in place of done means the request was an idempotent replay of a run that is still in flight; read the outcome from GET /jobs/{job_id}. And if the response comes back with a content-type that is not text/event-stream, it is a plain envelope — a replay of a finished job — so handle that case rather than assuming a stream. 

# -N disables buffering so the deltas arrive as they are written.
curl -sSN -X POST "$BASE/run-stream" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -H "Accept: text/event-stream" \
  -H "Idempotency-Key: vcf-desk:triage:$HASH:a1" \
  -d "$INPUT"

# event: job
# data: {"job_id":"job_01J..."}
#
# event: delta
# data: {"text":"{\"lane\":\"triage\",\"title\":\"NA12878 exome"}
#
# event: delta
# data: {"text":"\",\"headline\":\"Usable after two fixes"}
#
# event: done
# data: {"job_id":"job_01J...","status":"succeeded","charged_credits":602,
#        "truncated":false,"output":{"output":"{\"lane\":\"triage\", ...}"}}

# Or let a reader assemble it. This prints just the finished review JSON:
curl -sSN -X POST "$BASE/run-stream" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -H "Accept: text/event-stream" -H "Idempotency-Key: vcf-desk:triage:$HASH:a1" \
  -d "$INPUT" | python3 -c '
import sys, json
buf = []
for line in sys.stdin:
    if not line.startswith("data:"):
        continue
    payload = json.loads(line[5:].strip())
    if "text" in payload:
        buf.append(payload["text"])
    elif "output" in payload:
        print(payload["output"]["output"])
        break
'

The output contract

The reply is exactly one JSON object and nothing else — no prose before it, no code fence around it, no commentary after it. Arrays are always present even when empty; an array key is never null. Every id the reply mints is stable within that reply and prefixed per lane: T-001 for triage findings, I-001 for interpret, C-001 for cohort blockers. Strings are plain text rather than Markdown, except command fields, which are shell one-liners meant to be read and run by a person.

The common envelope — present on every lane

keytypemeaning
laneenumtriage | interpret | cohort. Echoes the task you sent. Assert that it matches before you touch the lane-specific keys — if task was absent this is where the model's choice is recorded.
titlestringA short name for this callset, drawn from the sample names or the contigs.
headlinestringOne sentence: the thing you most need to know. When the lane was inferred rather than given, the first sentence says which lane was chosen.
verdictenumThe lane's own enum. triage: release | filter | recall. interpret: ready-to-prioritise | annotate-first | not-interpretable. cohort: ingest | fix-first | do-not-ingest. This is the field a gate branches on.
verdict_reasonstringThe one fact that decides the verdict, quoted from the file or from prescan_facts. If it does not quote something, treat the verdict as soft.
assemblystringThe reference build as established from the header — usually from declared ##contig lengths, which is evidence, unlike a file name. "unidentified" when the header cannot say.
callerstringThe caller named in the header, or "not named in the header". It matters because filter conventions are caller-specific.
record_summarystringOne line: how many records, how many samples, and what kind of file this is (single-sample callset, joint call, sites-only, gVCF).
assumptionsstring[]What had to be assumed because the file is silent, including anything you asserted in context that was taken at face value. [] when none.
open_questionsstring[]Questions whose answers would change the review. In a pipeline these are the prompts for the next input, not decoration.
coverage_checkobject[]{flag_id, handling, where, note}, where handling is "addressed" or "set-aside". Exactly one entry per prescan_facts.flags[].id you sent, no duplicates, and no ids you did not send. where names the finding, check or readiness row that carries it; note carries the reason when it was set aside.
summarystringTwo to four sentences a colleague could paste into a ticket.

Three grounding rules run through all of it, and they are worth knowing before you read a reply as data. The pasted file is the only evidence, so every claim is traceable to a line, a header declaration or a facts value. Numbers are quoted from prescan_facts rather than recomputed from an excerpt. And "unknown" is a legitimate status: a sites-only VCF's per-sample missingness is unknown, not pass.

triage adds

keyshapemeaning
metrics_read [{metric, value, expected, reading, note}] The QC numbers, read against intent. value is quoted from prescan_facts; expected says what this intent should show and why; reading is consistent | borderline | inconsistent | not-applicable. not-applicable is the honest reading for a germline Ti/Tv expectation on a somatic callset, and you will see it used that way.
checks [{check, status, evidence, requirement}] Fifteen entries, always, in a fixed order — render by index and two reviews of the same callset diff row by row. status is pass | fail | partial | unknown. requirement is what would have to be true for a pass.
findings [{id, severity, area, title, evidence, why, fix, commands[]}] Ids are T-001, T-002, … severity is blocker | high | medium | low | info and is about consequence, not tidiness: blocker means a downstream tool will fail or silently produce a wrong answer, and a header cosmetic is never high. area is header | records | genotypes | quality | sorting | annotation. why names what breaks downstream, concretely; commands is read-only advice you run yourself.
filter_plan {strategy, steps: [{order, goal, tool, command, expected_effect, risk}]} Executable in order. risk is low | medium | high. No cutoff is invented from nowhere: every threshold is justified either by a number in prescan_facts or by the caller's own documented convention, and the step says which. No command overwrites your input in place — they write to a new path.

The fifteen checks arrive in this order, every time:

 1.  fileformat and header order      9.  coordinate sort order
 2.  reference build identified      10.  duplicate sites
 3.  contig declarations             11.  allele and genotype validity
 4.  contig naming consistency       12.  QUAL usability
 5.  INFO declared vs used           13.  per-sample missingness
 6.  FORMAT declared vs used         14.  Ti/Tv sanity
 7.  FILTER declared vs used         15.  annotation present
 8.  column integrity

A worked reply, abbreviated but structurally complete:

{
  "lane": "triage",
  "title": "NA12878 single-sample exome, GRCh38",
  "headline": "Usable after two fixes: an undeclared FILTER token and four duplicate sites.",
  "verdict": "filter",
  "verdict_reason": "LowGQ appears on 3,880 records with no ##FILTER=<ID=LowGQ> declaration, so bcftools view -f PASS silently keeps them.",
  "assembly": "GRCh38 (chr1 length 248956422 from ##contig, high confidence)",
  "caller": "GATK HaplotypeCaller 4.5.0.0 (##GATKCommandLine)",
  "record_summary": "50,000 of 184,203 data lines parsed, 1 sample, single-sample called VCF, not a gVCF.",
  "assumptions": [
    "The exome kit is not named in the header; Ti/Tv is read against a generic exome expectation.",
    "context says this is pre-analyst QC, so the plan optimises for correctness over yield."
  ],
  "open_questions": [
    "Which target BED was used? Off-target calls cannot be separated without it.",
    "Was SelectVariants run with --exclude-filtered? The FILTER tally suggests not."
  ],
  "metrics_read": [
    { "metric": "Ti/Tv (biallelic SNVs)", "value": "2.01 (27,400 Ti / 13,600 Tv)",
      "expected": "2.0-2.1 for a germline exome; lower suggests false-positive load",
      "reading": "consistent", "note": "Computed over the parsed window, not the whole file." },
    { "metric": "no-call rate (NA12878)", "value": "0.0176",
      "expected": "< 0.05 for a single-sample exome at this depth",
      "reading": "consistent", "note": "" },
    { "metric": "het/hom-alt ratio (NA12878)", "value": "1.426",
      "expected": "1.4-2.0 for a germline exome", "reading": "consistent",
      "note": "Would be not-applicable had intent been somatic." },
    { "metric": "median DP (NA12878)", "value": "38",
      "expected": "> 30 for confident exome genotypes", "reading": "borderline",
      "note": "meanDp 41.2 with median 38 means a right-tailed distribution." }
  ],
  "checks": [
    { "check": "fileformat and header order", "status": "pass",
      "evidence": "##fileformat=VCFv4.2 on line 1", "requirement": "##fileformat first" },
    { "check": "reference build identified", "status": "pass",
      "evidence": "chr1 length 248956422", "requirement": "a build derivable from ##contig" },
    { "check": "contig declarations", "status": "pass",
      "evidence": "25 ##contig lines, 24 contigs seen",
      "requirement": "every CHROM declared" },
    { "check": "contig naming consistency", "status": "pass",
      "evidence": "contig_style chr-prefixed throughout", "requirement": "one naming scheme" },
    { "check": "INFO declared vs used", "status": "fail",
      "evidence": "MQRankSum used, never declared",
      "requirement": "every INFO key in a ##INFO line" },
    { "check": "FORMAT declared vs used", "status": "pass",
      "evidence": "GT:AD:DP:GQ:PL all declared", "requirement": "every FORMAT key declared" },
    { "check": "FILTER declared vs used", "status": "fail",
      "evidence": "LowGQ on 3,880 records, no ##FILTER line",
      "requirement": "every FILTER token declared" },
    { "check": "column integrity", "status": "pass",
      "evidence": "10 tab-separated columns on every parsed record",
      "requirement": "8 fixed columns plus FORMAT and one per sample" },
    { "check": "coordinate sort order", "status": "fail",
      "evidence": "sort.pos_inversions 2, contig_re_entries 1",
      "requirement": "POS ascending within each contig, contigs in blocks" },
    { "check": "duplicate sites", "status": "fail",
      "evidence": "sort.duplicate_sites 4",
      "requirement": "no repeated CHROM:POS:REF:ALT" },
    { "check": "allele and genotype validity", "status": "partial",
      "evidence": "3 records with counts.invalid; AC exceeds AN on 41 records",
      "requirement": "GT indices within ALT, AC <= AN" },
    { "check": "QUAL usability", "status": "pass",
      "evidence": "qual.missing 0, median 412.9", "requirement": "QUAL scored and spread" },
    { "check": "per-sample missingness", "status": "pass",
      "evidence": "missingRate 0.0176", "requirement": "no sample dominated by ./." },
    { "check": "Ti/Tv sanity", "status": "pass",
      "evidence": "2.01", "requirement": "within the expectation for intent" },
    { "check": "annotation present", "status": "unknown",
      "evidence": "annotation null; no CSQ, ANN or BCSQ",
      "requirement": "a functional annotation field, if interpretation follows" }
  ],
  "findings": [
    { "id": "T-001", "severity": "high", "area": "header",
      "title": "FILTER token LowGQ is used but never declared",
      "evidence": "3,880 records carry FILTER=LowGQ; no ##FILTER=<ID=LowGQ> in the header",
      "why": "bcftools view -f PASS and most downstream readers treat an undeclared token as an unknown filter; some keep the record, some warn and drop it. The same command gives different answers on two tools.",
      "fix": "Add the missing ##FILTER declaration, or re-run the filtering step so the token it writes is the token it declares.",
      "commands": [
        "bcftools view -h callset.vcf > hdr.txt",
        "printf '##FILTER=<ID=LowGQ,Description=\"GQ below 25 in any called sample\">\\n' >> hdr.txt",
        "bcftools reheader -h hdr.txt -o callset.reheadered.vcf callset.vcf"
      ] },
    { "id": "T-002", "severity": "high", "area": "sorting",
      "title": "Two position inversions and one contig re-entry",
      "evidence": "sort.pos_inversions 2, sort.contig_re_entries 1",
      "why": "tabix refuses to index an unsorted file, and any tool that assumes sorted input will silently miss records in a region query.",
      "fix": "Sort, then index. Do not hand-edit the offending lines.",
      "commands": [
        "bcftools sort -Oz -o callset.sorted.vcf.gz callset.vcf",
        "bcftools index -t callset.sorted.vcf.gz"
      ] },
    { "id": "T-003", "severity": "medium", "area": "records",
      "title": "Four duplicate sites",
      "evidence": "sort.duplicate_sites 4",
      "why": "A duplicate site double-counts in any per-sample or per-region aggregate, and a merge downstream will keep both copies.",
      "fix": "De-duplicate after sorting, keeping the first record at each CHROM:POS:REF:ALT.",
      "commands": ["bcftools norm -d exact -Oz -o callset.dedup.vcf.gz callset.sorted.vcf.gz"] },
    { "id": "T-004", "severity": "medium", "area": "genotypes",
      "title": "AC exceeds AN on 41 records",
      "evidence": "41 records where the INFO AC sum is greater than AN",
      "why": "These counts describe the parent callset this sample was subset out of. Any allele-frequency filter reading INFO/AF is reading the wrong cohort.",
      "fix": "Recompute the counts for this subset rather than trusting the inherited ones.",
      "commands": ["bcftools +fill-tags callset.dedup.vcf.gz -Oz -o callset.tagged.vcf.gz -- -t AC,AN,AF"] },
    { "id": "T-005", "severity": "low", "area": "header",
      "title": "MQRankSum used in INFO but not declared",
      "evidence": "prescan info_undeclared: MQRankSum",
      "why": "A strict parser drops the field; a lenient one guesses its Type. Either way a filter on it is not portable.",
      "fix": "Declare it with the Number and Type GATK writes, or stop emitting it.",
      "commands": [] }
  ],
  "filter_plan": {
    "strategy": "Fix the structural problems before any quality filtering, because sorting and de-duplication change what the counts mean. Then recompute the allele counts for this single sample, and only then apply a genotype-quality threshold. The GQ cutoff is the caller's own LowGQ boundary of 25, not a number chosen here; the depth floor is the parsed median of 38 halved, which keeps the left tail visible rather than pretending it is absent.",
    "steps": [
      { "order": 1, "goal": "Declare the FILTER token so -f PASS behaves the same everywhere",
        "tool": "bcftools reheader", "command": "bcftools reheader -h hdr.txt -o step1.vcf callset.vcf",
        "expected_effect": "No records change; the header gains one ##FILTER line.", "risk": "low" },
      { "order": 2, "goal": "Sort and index", "tool": "bcftools sort",
        "command": "bcftools sort -Oz -o step2.vcf.gz step1.vcf && bcftools index -t step2.vcf.gz",
        "expected_effect": "Record count unchanged; the file becomes indexable.", "risk": "low" },
      { "order": 3, "goal": "Remove the four duplicate sites", "tool": "bcftools norm",
        "command": "bcftools norm -d exact -Oz -o step3.vcf.gz step2.vcf.gz",
        "expected_effect": "About 4 records removed.", "risk": "low" },
      { "order": 4, "goal": "Recompute AC, AN and AF for this sample", "tool": "bcftools +fill-tags",
        "command": "bcftools +fill-tags step3.vcf.gz -Oz -o step4.vcf.gz -- -t AC,AN,AF",
        "expected_effect": "INFO counts stop describing the parent cohort.", "risk": "low" },
      { "order": 5, "goal": "Drop genotypes below the caller's own GQ boundary", "tool": "bcftools filter",
        "command": "bcftools filter -S . -e 'FMT/GQ<25' -Oz -o step5.vcf.gz step4.vcf.gz",
        "expected_effect": "Roughly the 3,880 LowGQ records become no-calls rather than being deleted.",
        "risk": "medium" }
    ]
  },
  "coverage_check": [
    { "flag_id": "VD-FILTER-UNDECLARED", "handling": "addressed", "where": "T-001, filter_plan step 1",
      "note": "" },
    { "flag_id": "VD-PARSE-CEILING", "handling": "set-aside", "where": "record_summary",
      "note": "50,000 records is a large enough window for these metrics; the ceiling changes no verdict here, but the counts are a window, not a total." }
  ],
  "summary": "Structurally this is a normal GATK exome with three fixable problems: an undeclared FILTER token, two sort inversions with one contig re-entry, and four duplicate sites. The QC picture is consistent with a germline exome - Ti/Tv 2.01, no-call rate 1.8%, het/hom 1.43 - so nothing here suggests recalling. Run the five-step plan in order and the callset is fit to hand to an analyst. Recompute the allele counts before any frequency-based filter: the inherited AC/AN describe the cohort this sample came out of."
}

interpret adds

keyshapemeaning
annotation_state {present, kind, tool, fields_available[], note} present is a boolean and it governs the rest of the lane. kind is the annotation field found (CSQ, ANN, BCSQ), tool the annotator named in the header, fields_available the sub-fields parsed out of the Description. Read this first: it tells you how much of the reply can exist.
gene_groups [{gene, variant_count, consequences[], loci[], from_file, unknown_without_lookup}] The variants gathered by gene. from_file is what the file itself says about them; unknown_without_lookup is what cannot be said yet, and why. [] when the file is unannotated — see the rule below.
shortlist [{rank, locus, ref_alt, gene, consequence, reason, confidence}] Ranked, rank 1 first. gene is "not annotated" and consequence is "unknown" when the file does not say. reason may cite the reference and alternate alleles, the consequence string the file carries, the FILTER value, the depth and the genotype — and nothing else. confidence is high | medium | low.
lookup_plan [{order, question, database, command, what_would_change}] The queries that would settle what the file cannot. command is a real gget invocation; database names where the answer lives (Ensembl, UniProt, ClinVar, gnomAD, COSMIC, …); what_would_change says what a given answer would do to the shortlist, which is what makes the row worth running.
evidence_gaps string[] What the file cannot answer at all — no transcript set, no population frequencies, no phasing, no parental genotypes.

The grounding rule — the one that matters most on this lane

gene_groups may only contain gene symbols that appear as text in the pasted file. If the VCF is unannotated, gene_groups is [] and annotation_state.present is false. It is not populated from what is known to sit at those coordinates.

The model will not assert gene function, pathogenicity, population frequency or which transcript is canonical. Those become lookup_plan rows — real gget commands you can run — rather than sentences. This is deliberate, and the reason is worth stating plainly: the app has no database access, and a confident uncited gene-disease claim is the worst possible output here because it looks exactly like a real finding. It would be pasted into a report and believed. A lookup_plan of ten precise queries is a better answer than three fluent sentences about gene function, and it is the answer you get.

The practical consequence for your client: on an unannotated file, do not treat an empty gene_groups as a failed run. It is the contract working. Read annotation_state.present, expect verdict: "annotate-first", and run the annotator the reply names before re-running this lane.

A worked reply, on an unannotated file — the case people misread:

{
  "lane": "interpret",
  "title": "NA12878 exome, 50,000 records parsed, no functional annotation",
  "headline": "This file names no genes: annotate it first, then re-run. The shortlist below is positional only.",
  "verdict": "annotate-first",
  "verdict_reason": "prescan annotation is null and no INFO key among CSQ, ANN or BCSQ is declared or used, so there is no gene symbol anywhere in the file.",
  "assembly": "GRCh38 (chr1 length 248956422 from ##contig, high confidence)",
  "caller": "GATK HaplotypeCaller 4.5.0.0",
  "record_summary": "50,000 of 184,203 data lines parsed, 1 sample, called VCF with GT:AD:DP:GQ:PL.",
  "annotation_state": {
    "present": false, "kind": "", "tool": "", "fields_available": [],
    "note": "No CSQ, ANN or BCSQ. VEP, SnpEff and bcftools csq all leave a declared INFO field behind; none is here."
  },
  "gene_groups": [],
  "shortlist": [
    { "rank": 1, "locus": "chr7:117559590", "ref_alt": "ATCT>A", "gene": "not annotated",
      "consequence": "unknown",
      "reason": "A three-base deletion, FILTER=PASS, GQ 99, DP 52, genotype 0/1. Length not a multiple of three, so if it falls in coding sequence it is frameshifting - the file cannot say whether it does.",
      "confidence": "medium" },
    { "rank": 2, "locus": "chr17:7676154", "ref_alt": "C>T", "gene": "not annotated",
      "consequence": "unknown",
      "reason": "Highest QUAL in the parsed window at 24011.0, FILTER=PASS, DP 71, genotype 1/1. Homozygous alternate at high depth is a confident call, whatever it turns out to mean.",
      "confidence": "medium" },
    { "rank": 3, "locus": "chr2:47806090", "ref_alt": "G>GA", "gene": "not annotated",
      "consequence": "unknown",
      "reason": "Single-base insertion, FILTER=LowGQ, GQ 21. Listed because it is the kind of call the filter plan would drop; confirm it before spending a lookup on it.",
      "confidence": "low" }
  ],
  "lookup_plan": [
    { "order": 1, "question": "Which gene and transcript overlap chr7:117559590 on GRCh38?",
      "database": "Ensembl", "command": "gget search -s homo_sapiens --limit 5 7:117559590",
      "what_would_change": "Turns an unnamed deletion into a gene-level finding, or rules it out as intergenic." },
    { "order": 2, "question": "What is the consequence of the chr7 deletion on the canonical transcript?",
      "database": "Ensembl VEP", "command": "gget vep -s homo_sapiens 7:117559590:ATCT:A",
      "what_would_change": "Confirms or removes the frameshift hypothesis in rank 1's reason." },
    { "order": 3, "question": "Is chr17:7676154 C>T a known clinical variant?",
      "database": "ClinVar", "command": "gget cosmic -s clinvar 17:7676154",
      "what_would_change": "A known interpretation would move rank 2 to the top or dismiss it." },
    { "order": 4, "question": "What is the population frequency of each shortlisted allele?",
      "database": "gnomAD", "command": "gget vep -s homo_sapiens --af 17:7676154:C:T",
      "what_would_change": "A common allele stops being a candidate for anything rare." },
    { "order": 5, "question": "Which of the three loci fall inside coding exons at all?",
      "database": "Ensembl", "command": "gget seq --translate -o loci.fa ENSG00000141510",
      "what_would_change": "Removes intergenic and deep-intronic loci from the shortlist entirely." }
  ],
  "assumptions": [
    "The shortlist ranks on file-internal evidence only: QUAL, DP, GQ, FILTER, genotype and the shape of REF/ALT.",
    "GRCh38 coordinates, from the header; every lookup command above assumes that build."
  ],
  "open_questions": [
    "Which annotator is available in your pipeline? The lookup plan is written for gget, but a local VEP cache answers rows 1, 2 and 4 in one pass.",
    "Is there a phenotype or gene list to focus on? focus was empty, so the ranking is generic."
  ],
  "evidence_gaps": [
    "No gene symbols, consequences or impact classes anywhere in the file.",
    "No population frequencies: the INFO AF present describes the parent cohort, not any population.",
    "No phasing and no parental genotypes, so compound heterozygosity cannot be assessed.",
    "Only the first 50,000 of 184,203 data lines were parsed; a variant of interest may sit outside the window."
  ],
  "coverage_check": [
    { "flag_id": "VD-NO-ANNOTATION", "handling": "addressed",
      "where": "annotation_state, verdict annotate-first", "note": "" },
    { "flag_id": "VD-PARSE-CEILING", "handling": "addressed", "where": "evidence_gaps",
      "note": "" }
  ],
  "summary": "There is nothing to interpret at gene level yet: the file carries no CSQ, ANN or BCSQ field, so no gene symbol can be named without inventing one. Annotate with VEP, SnpEff or bcftools csq and re-run this lane - the same three loci will come back with real consequences attached. In the meantime the shortlist is positional: a three-base deletion at chr7:117559590 that is frameshifting if it is coding, a very high-confidence homozygous SNV at chr17:7676154, and a low-GQ insertion worth confirming before spending time on it. The five gget queries above are what the file cannot answer."
}

cohort adds

keyshapemeaning
readiness [{requirement, status, evidence, remediation}] Eleven entries, always, in a fixed order. status is ready | blocked | unknown. Without cohort_context the store-comparison rows are unknown rather than ready — that is the honest answer, and it is why a gate should require cohort_context.
blockers [{id, title, why, fix, command}] Ids are C-001, C-002, … One command per blocker, not a list: the single next thing to run. A do-not-ingest verdict always has at least one blocker, and an ingest verdict has none.
ingest_plan {store, partitioning, attributes: [{name, source, type, note}], steps: [{order, action, command, note}]} store is the store shape being assumed, partitioning the choice and the reason given this file's contigs. attributes is derived from the header's own ##INFO and ##FORMAT declarations — an undeclared key cannot become an attribute, so the plan says which declared fields become attributes, which are dropped, and what that costs.
cohort_risks string[] What could still go wrong after a clean ingest — a schema that cannot absorb the next caller, counts that stop being comparable after a subset, a naming convention one delivery away from breaking.

The eleven readiness requirements arrive in this order, every time:

 1.  single reference build              7.  INFO and FORMAT fields declared
 2.  contig naming matches the store     8.  multi-allelic representation decided
 3.  contig set is declared              9.  gVCF versus called VCF
 4.  coordinate sorted and indexable    10.  FILTER semantics agreed
 5.  sample names are unique and stable 11.  allele counts recomputed after any subsetting
 6.  no duplicate sites

Multi-allelic sites are treated as a decision, not a defect: the plan says whether to split them with bcftools norm -m-any or keep them, and what each choice costs at query time. Splitting is not presented as unconditionally correct, because it is not.

A worked reply:

{
  "lane": "cohort",
  "title": "NA12878 exome into a 1,400-sample GRCh38 TileDB-VCF store",
  "headline": "Two blockers: the file is not sorted or indexable, and its INFO counts describe a different cohort.",
  "verdict": "fix-first",
  "verdict_reason": "sort.pos_inversions is 2 with one contig re-entry, so tabix cannot index the file and TileDB-VCF ingestion requires an indexable input.",
  "assembly": "GRCh38 (chr1 length 248956422 from ##contig, high confidence)",
  "caller": "GATK HaplotypeCaller 4.5.0.0",
  "record_summary": "50,000 of 184,203 data lines parsed, 1 sample, called VCF, not a gVCF.",
  "readiness": [
    { "requirement": "single reference build", "status": "ready",
      "evidence": "GRCh38 from contig lengths; cohort_context says the store is GRCh38",
      "remediation": "" },
    { "requirement": "contig naming matches the store", "status": "ready",
      "evidence": "contig_style chr-prefixed; the store is chr-prefixed", "remediation": "" },
    { "requirement": "contig set is declared", "status": "ready",
      "evidence": "25 ##contig lines cover the 24 contigs seen", "remediation": "" },
    { "requirement": "coordinate sorted and indexable", "status": "blocked",
      "evidence": "sort.pos_inversions 2, contig_re_entries 1",
      "remediation": "bcftools sort, then bcftools index -t. See C-001." },
    { "requirement": "sample names are unique and stable", "status": "unknown",
      "evidence": "sample NA12878; cohort_context gives a sample count but not the roster",
      "remediation": "Check the store for an existing NA12878 before ingesting; a collision overwrites or duplicates depending on the mode." },
    { "requirement": "no duplicate sites", "status": "blocked",
      "evidence": "sort.duplicate_sites 4",
      "remediation": "bcftools norm -d exact after sorting. See C-002." },
    { "requirement": "INFO and FORMAT fields declared", "status": "blocked",
      "evidence": "info_undeclared: MQRankSum; filter_undeclared: LowGQ",
      "remediation": "Declare both, or accept that neither can become an attribute. See C-003." },
    { "requirement": "multi-allelic representation decided", "status": "unknown",
      "evidence": "counts.multiallelic 1,204; cohort_context does not say what the store did",
      "remediation": "Match the store. If the existing 1,400 samples were split with bcftools norm -m-any, split this one too; if they were kept, keep it." },
    { "requirement": "gVCF versus called VCF", "status": "ready",
      "evidence": "is_gvcf false: no END or NON_REF", "remediation": "" },
    { "requirement": "FILTER semantics agreed", "status": "blocked",
      "evidence": "LowGQ used on 3,880 records and never declared",
      "remediation": "A store that filters on FILTER at query time needs the token to mean the same thing in every sample. Declare it. See C-003." },
    { "requirement": "allele counts recomputed after any subsetting", "status": "blocked",
      "evidence": "AC exceeds AN on 41 records: the counts came from the parent callset",
      "remediation": "bcftools +fill-tags before ingest. See C-004." }
  ],
  "blockers": [
    { "id": "C-001", "title": "Not sorted, therefore not indexable",
      "why": "TileDB-VCF ingests from an indexed file; an unsorted input either fails outright or ingests a region map that does not match the data.",
      "fix": "Sort, then build a tabix index, and ingest from the sorted copy.",
      "command": "bcftools sort -Oz -o NA12878.sorted.vcf.gz callset.vcf && bcftools index -t NA12878.sorted.vcf.gz" },
    { "id": "C-002", "title": "Four duplicate sites",
      "why": "A duplicate site becomes two cells at one coordinate for one sample. Every count over that region is then wrong for this sample only, which is the hardest kind of error to notice in a cohort.",
      "fix": "De-duplicate exactly, after sorting.",
      "command": "bcftools norm -d exact -Oz -o NA12878.dedup.vcf.gz NA12878.sorted.vcf.gz" },
    { "id": "C-003", "title": "Undeclared INFO key and FILTER token",
      "why": "The store's attribute schema is built from ##INFO and ##FORMAT. MQRankSum cannot become an attribute while it is undeclared, and an undeclared LowGQ makes FILTER mean something different in this sample than in the other 1,400.",
      "fix": "Add both declarations by reheadering; do not touch the records.",
      "command": "bcftools reheader -h hdr.fixed.txt -o NA12878.reheadered.vcf.gz NA12878.dedup.vcf.gz" },
    { "id": "C-004", "title": "Inherited allele counts",
      "why": "AC, AN and AF describe the cohort this sample was subset out of. Ingested as attributes they would sit alongside 1,400 samples' correct counts and be indistinguishable from them.",
      "fix": "Recompute the tags for this sample before ingest.",
      "command": "bcftools +fill-tags NA12878.reheadered.vcf.gz -Oz -o NA12878.ready.vcf.gz -- -t AC,AN,AF" }
  ],
  "ingest_plan": {
    "store": "An existing TileDB-VCF store of 1,400 GRCh38 samples, chr-prefixed, partitioned by contig, per cohort_context.",
    "partitioning": "Keep the store's existing per-contig partitioning. This file's 24 contigs map onto it directly, and re-partitioning to accommodate one sample would rewrite fragments for the other 1,400.",
    "attributes": [
      { "name": "GT", "source": "##FORMAT=<ID=GT>", "type": "string",
        "note": "The only attribute every store needs. Declared." },
      { "name": "DP", "source": "##FORMAT=<ID=DP,Number=1,Type=Integer>", "type": "int32",
        "note": "Declared and used; per-sample depth is worth the storage." },
      { "name": "GQ", "source": "##FORMAT=<ID=GQ,Number=1,Type=Integer>", "type": "int32",
        "note": "Declared. Needed if LowGQ is going to be re-derivable at query time." },
      { "name": "AD", "source": "##FORMAT=<ID=AD,Number=R,Type=Integer>", "type": "int32[]",
        "note": "Number=R, so its arity depends on the multi-allelic decision above." },
      { "name": "AC/AN/AF", "source": "##INFO", "type": "int32 / float",
        "note": "Only after C-004. Ingesting them as they stand imports the parent cohort's counts." },
      { "name": "MQRankSum", "source": "used but not declared", "type": "-",
        "note": "Dropped. An undeclared key cannot become an attribute; the cost is that mapping-quality-based filtering will not be available in the store for this sample." },
      { "name": "ExcessHet", "source": "##INFO=<ID=ExcessHet>", "type": "float",
        "note": "Declared but never used in the records, so nothing to store. No cost." }
    ],
    "steps": [
      { "order": 1, "action": "Produce the clean input",
        "command": "bcftools sort -Oz -o s.vcf.gz callset.vcf && bcftools norm -d exact -Oz -o d.vcf.gz s.vcf.gz && bcftools reheader -h hdr.fixed.txt -o r.vcf.gz d.vcf.gz && bcftools +fill-tags r.vcf.gz -Oz -o NA12878.ready.vcf.gz -- -t AC,AN,AF",
        "note": "Blockers C-001 through C-004, in the only order that works: sorting before de-duplication, counts last." },
      { "order": 2, "action": "Index the prepared file",
        "command": "bcftools index -t NA12878.ready.vcf.gz",
        "note": "Ingestion reads the index, not the file order." },
      { "order": 3, "action": "Confirm the sample name is not already in the store",
        "command": "tiledbvcf list --uri s3://cohort/store | grep -x NA12878",
        "note": "No output is the answer you want. A match means resolve the collision first." },
      { "order": 4, "action": "Dry-run the ingest against a copy",
        "command": "tiledbvcf store --uri s3://cohort/store-staging NA12878.ready.vcf.gz",
        "note": "Staging first keeps a schema surprise away from the real store." },
      { "order": 5, "action": "Ingest",
        "command": "tiledbvcf store --uri s3://cohort/store NA12878.ready.vcf.gz",
        "note": "One sample, existing partitioning, no schema change." },
      { "order": 6, "action": "Verify the round trip",
        "command": "tiledbvcf export --uri s3://cohort/store --sample-names NA12878 --regions chr17:7676154-7676154",
        "note": "Read back one known record and compare it to the source line." }
    ]
  },
  "cohort_risks": [
    "MQRankSum is dropped for this sample. If a later delivery declares it, the store will hold it for some samples and not others, and a filter on it will silently compare unequal populations.",
    "The multi-allelic decision is unknown. Getting it wrong in either direction makes this sample's AD arity inconsistent with the rest of the store.",
    "cohort_context gives a sample count but no roster, so the NA12878 collision check in step 3 is the only thing standing between an overwrite and a duplicate.",
    "Only 50,000 of 184,203 data lines were parsed, so the sort and duplicate findings are lower bounds: the rest of the file has not been examined."
  ],
  "assumptions": [
    "cohort_context is accurate about the store: GRCh38, chr-prefixed, per-contig partitioning, all DeepVariant.",
    "This callset came from GATK while the store is DeepVariant, so FILTER semantics differ by construction; that is the reason FILTER semantics agreed is blocked rather than merely unknown."
  ],
  "open_questions": [
    "Were the existing 1,400 samples split at multi-allelic sites, or kept?",
    "Does the store already hold an NA12878, from any earlier delivery?",
    "Is mapping-quality filtering used at query time? If it is, declaring MQRankSum is worth doing before ingest rather than after."
  ],
  "coverage_check": [
    { "flag_id": "VD-UNSORTED-POS", "handling": "addressed", "where": "C-001, readiness row 4", "note": "" },
    { "flag_id": "VD-DUPLICATE-SITES", "handling": "addressed", "where": "C-002, readiness row 6", "note": "" },
    { "flag_id": "VD-FILTER-UNDECLARED", "handling": "addressed", "where": "C-003, readiness rows 7 and 10", "note": "" },
    { "flag_id": "VD-INFO-UNDECLARED", "handling": "addressed", "where": "C-003, ingest_plan attributes", "note": "" },
    { "flag_id": "VD-AC-GT-AN", "handling": "addressed", "where": "C-004, readiness row 11", "note": "" },
    { "flag_id": "VD-PARSE-CEILING", "handling": "set-aside", "where": "cohort_risks",
      "note": "The ceiling does not change the verdict - the blockers found inside the window are enough to require fixing - but it does mean the counts are lower bounds." }
  ],
  "summary": "This callset can go into the store, but not as it stands. Four blockers, in order: sort and index it, de-duplicate the four repeated sites, declare MQRankSum and LowGQ so the attribute schema and FILTER semantics line up with the other 1,400 samples, and recompute AC/AN/AF so the counts describe this sample rather than the cohort it was subset from. Two things remain unknown because cohort_context does not say: whether the store split multi-allelic sites, and whether an NA12878 already exists in it. Resolve both before step 5, not after."
}

Invariants worth asserting

Six assertions catch almost every way a reply can be wrong while still parsing. Put them in the client, not in a comment:

  1. review.lane === input.task. A mismatch means the wrapper bug, a missing task, or a lane inferred where you meant to specify one.
  2. verdict is in the lane's enum. Anything else is a parse you should reject rather than normalise.
  3. Every prescan_facts.flags[].id you sent appears in coverage_check exactly once, and no id you did not send appears at all.
  4. On triage, checks.length === 15; on cohort, readiness.length === 11. Both arrive in a fixed order, so index the table rather than searching it by name.
  5. On interpret, if annotation_state.present === false then gene_groups is empty. A gene symbol appearing with present: false is the one failure mode that matters on that lane.
  6. job.truncated === false. A truncated reply is a prefix: the checks may be complete while filter_plan or summary is cut mid-string. Retry with a retry_note and a bumped attempt suffix; do not repair truncated JSON by appending braces, which produces something that parses and is not what the model meant.

Idempotency — and why the key must carry the lane

Send an Idempotency-Key header on every /run and /run-stream call. It is not formally required by the endpoint and it is required in practice: a retried request carrying the same key returns the same job instead of billing a second review, which is what makes a pipeline retry safe after a timeout or a dropped connection. Replaying a key with a different body is a conflict, so bump an attempt suffix whenever the input actually changed.

The shape that works here is vcf-desk:<task>:<input-hash>:a<attempt>. The lane must be in the key. Two lanes over the same VCF are two distinct runs with two different prompts, two different output caps and two different prices — and if the hash is taken over a body that contains task, a key built from the hash alone looks distinct until the day something normalises the body, drops an empty field, or reorders keys. Then a cohort run replays a triage job, returns a reply whose lane is triage, and your ingest gate reads a filter plan. Naming the lane in the key makes that collision impossible rather than unlikely. 

vcf-desk:triage:9f2a17c4d8b0e651:a1      # first attempt
vcf-desk:triage:9f2a17c4d8b0e651:a2      # same VCF, retried after a truncated reply
vcf-desk:cohort:9f2a17c4d8b0e651:a1      # same VCF, different lane, different run
vcf-desk:interpret:9f2a17c4d8b0e651:a1   # and again

Retry a 500 or a network failure with the same key so you cannot be billed twice for one review. Change the key only when the input changed.

Metering, in one place

Runs are metered against the gpt-terra model alias with a platform markup of 1000 basis points — 10% — and they need a signed-in account: a guest token calling /run or /run-stream gets FORBIDDEN. /me and /estimate are free and a guest token can call both, which makes them the right place to build and test your input-shaping code before you spend anything. The in-browser VCF reader that produces prescan_facts is free too, and it is where most of the structural findings come from — it is worth sending its output rather than paying the model to rediscover it.

hold_credits is reserved at the start and released at settlement; charged_credits on the terminal job is the real price and is usually much smaller. When the balance sits between min_credits and hold_credits the run is not refused — it executes with a reduced output cap and comes back with truncated: true, which is the case invariant 6 exists to catch.

Not a clinical or diagnostic tool

VCF Desk reviews a file. It does not diagnose anyone, it does not report clinical significance, and nothing it returns is a medical device, a clinical decision, or a substitute for one. The interpret lane deliberately refuses to assert pathogenicity, gene function or population frequency and hands you the queries instead — that is the whole design, and it is the reason the output is safe to read as engineering evidence rather than as a finding. Commands are read-only advice for you to run and check yourself. Every downstream use of a callset reviewed here remains the responsibility of the people who produced it.