Skip to content

0039 — Multi-curator split-file label merge-back (append-only owner-pull, adopt-or-surface)

  • Status: accepted
  • Date: 2026-07-07
  • Deciders: bioedca
  • PRD anchor: §5.1 (/labels provenance + "the molecule_key + labeler identity enable multi-curator reconciliation"; "the owner-curator's retrain recomputes every row's weight from the merged label set … no weight reconciliation is needed"), §7.5 ("other members curate into their own split/subset files whose labeled rows … merge back as an append-only owner-pull … joined on molecule_key. The owner's retrain then recomputes every row's weight … and human-vs-human disagreement on the same molecule surfaces through a §7.4-style per-trace reconcile prompt"), §7.10 (FR-CONCURRENCY split-file curation), §7.4 (per-trace reconcile prompt)
  • Milestone: M5

Context and problem statement

Concurrent curation on OneDrive/network shares is served by the split-file path ([ADR-0023]'s /labels log, §7.10): a non-owner lab member curates into their own subset .tether rather than writing the single-writer-locked central project. The third and final concern of the M5 "seeding + drift flag + multi-curator split-merge" PR (after the drift flag [ADR-0037] and the provisional-prior training fold [ADR-0038]) is the owner-pull that folds those split-file human labels back into the condition-owner's canonical project so its ranker retrain sees them. The PRD specifies what must happen (append-only, joined on molecule_key, central weight recompute, disagreement surfaces) but not the mechanics, which force four decisions:

  1. Which rows merge. A split file's /labels can carry both human curation and provisional seed priors. Do the seeds merge too?
  2. How adoption reaches the ranker. The training fold ([ADR-0038]) reads a molecule's human signal from /molecules.curation_label, not from /labels human rows (only provisional priors come from /labels). So appending a contributor's human /labels row alone has no training effect — the merge must also update curation_label for the contributor's labels to matter.
  3. What to do on disagreement. Updating curation_label from a contributor is exactly the "silent overwrite" §7.4/§7.5 forbid when the owner already holds a different human decision.
  4. Idempotency. "Append-only owner-pull … at the video boundary" invites re-pulling the same split file more than once; a naïve append duplicates the whole contributor history each time.

Getting (3) wrong is the subtle trap: silently majority-voting or averaging away a human-vs-human disagreement discards the signal that annotator disagreement carries and erases one curator's judgement without review.

Decision drivers

  • PRD §5.1/§7.5/§7.10 fidelity — append-only, join on molecule_key, weights recomputed centrally (per-split-file weights advisory), disagreement surfaces via a §7.4 reconcile prompt.
  • Never silently overwrite a human decision; never silently drop (§7.5) — a foreign decision that conflicts with the owner's is surfaced, not applied; an unattributable row is surfaced, not dropped.
  • The merge must actually train — non-conflicting contributor labels have to reach the signal the ranker reads (/molecules.curation_label), or the feature is inert.
  • Schema-guard green — additive /labels rows + a value-only curation_label rewrite; no group/dataset/dtype/field change ([ADR-0005], [ADR-0023]).
  • Established practice — annotator disagreement carries signal that single-ground-truth aggregation destroys [Davani2021]; disagreement should be resolved by a deliberate human procedure, not silent voting [Oortwijn2021].

Considered options

  • Which rows merge: human-only vs. human + provisional. The two provisional sources (deeplasi-provisional / cross-condition-seed) are the owner's own machine priors, regenerated by the seeding path ([ADR-0038]); pulling a contributor's copy would duplicate machine priors the owner already holds and conflate "another human curated this" with "a classifier guessed this." Human-only, with the skipped non-human count reported (never silently ignored).
  • How adoption reaches the ranker: append-to-/labels-only (inert — training ignores human /labels rows) vs. also set curation_label for non-conflicting decisions. Only the latter has any training effect, so it is required; the report lists every adopted key so it is transparent, not silent.
  • Disagreement handling: silent last-writer-wins / majority-vote vs. adopt-or-surface. A contributor decision on an uncurated owner molecule is adopted; one that agrees is a no-op; one that disagrees is returned as a MergeConflict with the owner's curation_label untouched — the §7.4 reconcile case. The conflicting row is still appended to the append-only /labels audit (with labeler provenance), so the record is complete, but it does not move the training signal until a human resolves it.
  • Resolution primitive: a bespoke resolve_conflict API vs. reuse the ordinary owner action. A reconcile decision is just the owner setting curation_label, i.e. tether.project.labels.accept/reject — no new primitive; the (deferred) GUI reconcile dialog calls those.
  • Idempotency: accept duplication vs. dedup by event provenance. Two /labels rows equal on (molecule_key, labeler, timestamp, source_file, source, label_value) are the same event; re-pulling skips them (weight and condition_id are excluded — weight is advisory/recomputed and condition_id is rewritten to the owner's authoritative value on append, so neither identifies the event). Append-only still holds: only genuinely new events are appended.

Decision outcome

New tether.project.merge.merge_labels(owner, source, *, recompute_weights=True) -> MergeReport:

  1. Reads the contributor's /labels (read-only), partitions human rows (non-human counted into skipped_non_human), and joins on molecule_key against the owner's /molecules; a key absent from the owner is collected into unmatched, and a key mapping to more than one owner condition_id (a same-sha cross-condition coordinate collision, §7.10) is collected into ambiguous and refused — mirroring set_curation_label's multi-condition guard rather than silently tagging one condition and adopting across both — both surfaced, neither appended nor adopted.
  2. Appends the genuinely-new matched human rows to the owner's /labels/table (the set_curation_label resize + slice-assign pattern), rewriting each appended row's condition_id to the owner's authoritative value for that molecule so the central recompute groups it correctly; already-present events (same provenance) count into skipped_duplicate.
  3. Classifies each matched molecule's latest contributor human decision vs. the owner's curation_label: owner-UNCURATED → adopt (write curation_label on all matched rows); equal → agreement; different → a MergeConflict (owner curation_label left untouched).
  4. Unless recompute_weights=False, calls recompute_label_weights(owner) ([ADR-0036]) over the merged set — per-split-file weights are superseded centrally.

MergeReport fully itemizes the merge (appended / adopted / agreements / conflicts / unmatched / skipped_non_human / skipped_duplicate / weights_recomputed), so every source row and every classified molecule is accounted for — nothing is silently dropped.

Consequences

  • Good: schema-guard stays green — additive /labels rows + a value-only curation_label rewrite over the frozen store; no new group/dataset/dtype/field. No conda-lock change (h5py + numpy, the set_curation_label/recompute_label_weights stack) and no new §11.2 tunable (the central recompute reuses the existing w₀).
  • Good: a re-pull of the same split file is idempotent (dedup + idempotent adoption), so the append-only pull can run every video boundary without bloating the log or double-counting.
  • Good: human-vs-human disagreement is preserved and surfaced for a deliberate reconcile [Davani2021, Oortwijn2021] and a conflicting contributor decision cannot silently overwrite the owner's — while its audit row is still recorded (the ranker's training signal, curation_label, is unaffected until the owner resolves it).
  • Trade-off: adoption uses the contributor's latest human event as their current decision (a contributor un-reject, label_value = UNCURATED, withdraws the decision — no adopt/conflict); a mid-history flip is not separately surfaced, consistent with "current decision" semantics elsewhere.
  • Follow-up: the GUI reconcile dialog that walks MergeReport.conflicts per-trace (accept/reject each) is deferred to the M5 GUI PR (computer-use gate); the headless core surfaces the conflicts it will consume. With this, all three concerns of the split M5 "seeding + drift + multi-curator" PR ([ADR-0037], [ADR-0038], this) have landed; the active-learning non-reordering badge is the last M5 PR before the v0.5.0 exit.

More information

  • PRD §5.1 (/labels provenance + multi-curator reconciliation), §7.5 (curation & per-condition ML), §7.10 (FR-CONCURRENCY split-file curation), §7.4 (per-trace reconcile prompt); PLAN §9 M5.
  • Builds on [ADR-0023] (the /labels codec + append-only log + human-only curation_label), [ADR-0036] (recompute_label_weights), and [ADR-0038] (the training fold that reads human labels from curation_label, provisional from /labels).
  • [Davani2021] Davani, Díaz & Prabhakaran, "Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations," TACL (2021) — annotator disagreement captures signal that aggregation to a single ground truth ignores.
  • [Oortwijn2021] Oortwijn, Ossenkoppele & Betti, "Interrater Disagreement Resolution: A Systematic Procedure to Reach Consensus in Annotation Tasks" (2021) — disagreement in ground-truth construction should be resolved by a systematic human procedure, not silent aggregation.