reTraining/docs/requirements.md

# Requirements

Status: **agreed** (planning session, 2026-07-31). Changes only with the user's approval.

## Goal

A system for **enriching a dataset and improving an existing detection model**, iteratively,
from an archive of recorded video. One full round:

> pick a project → browse the video archive → pick a batch → trim a time range →
> extract frames → auto-annotate with SAM3 → review and correct every frame → approve →
> merge into the master dataset → fine-tune from the base model → compare against the base.

The system is **generic**: the sack case is only the first project. Other cases are
created as new projects with their own base model, classes, and video archive — no code
changes.

## Non-goals (for this version)

- Login, multi-user, tenants, quotas. The architecture leaves room for them; the features
  are not built.
- Tracking or annotation propagation between frames.
- Collaborative annotation by several people at once.
- Public internet deployment.

---

## A. Project

- **REQ-001** — The user can create, list, and delete projects. A project has: name, label
  type, base model, video archive root, and a class list.
- **REQ-002** — Each project picks a **label type**: `bbox` (YOLO detect) or `polygon`
  (YOLO segment). This determines the export format, the editor's behaviour, and which
  model variant is trained. It cannot be changed once a batch has been merged.
- **REQ-003** — The user uploads a **base model** `.pt`. The class list is read from the
  model (`model.names`). It cannot drift on its own: nothing adds or removes a class as a
  side effect of another action. Deliberate deletion is REQ-007.
- **REQ-004** — A project may be created **without** a base model. In that case the user
  types the class list, and the first training starts from pretrained weights
  (`yolo11n.pt` / `yolo11n-seg.pt`).
- **REQ-005** — Each class carries its own **SAM3 text prompt**, which may differ from the
  class name (e.g. class `sack` with prompt `"woven plastic sack"`). Prompts can be
  edited at any time without affecting existing data.
- **REQ-006** — All of a project's data (base model, master dataset, batch frames, trained
  weights) lives under one project folder, so it can be copied or backed up whole.
- **REQ-007** — The user can **delete a class** at any point in a project's life, including
  after batches have been merged. Deleting one:
  - removes every annotation of that class, in batches under review and in the master
    dataset alike;
  - **renumbers the classes above it**, in the database *and* in every label file already
    written to disk, because a YOLO label is an integer index and leaving a gap would make
    old labels silently name the wrong class;
  - regenerates `data.yaml`;
  - is refused for a project's last remaining class.

  Before confirming, the user is told how many shapes will be destroyed. The action cannot
  be undone. If the project's base model was trained on the old class list, it stops being
  comparable — which REQ-063 already reports rather than hides.
- **REQ-008** — The user can **add a new class** (name & prompt) to an existing project at any time.
  The new class receives the next sequential `class_id`, and `data.yaml` is regenerated if a
  master dataset exists.

## B. Video archive

- **REQ-010** — The video archive lives at a local path (disk or mount); videos are **not
  uploaded** through the browser.
- **REQ-011** — Archive structure: `<video_root>/<date>/<batch>.<ext>`. The system lists the
  dates, and within each date the videos with their batch labels parsed from the filename.
- **REQ-012** — Each video shows its duration, resolution, and whether it has already been
  used as a batch in this project.
- **REQ-013** — Videos play in the browser with seeking (HTTP Range), without copying the
  file first.

## C. Trim & frame extraction

- **REQ-020** — The user sets the in/out range with a timeline slider on the player, and can
  also type precise timestamps.
- **REQ-021** — The user sets the extraction **frames per second** (default 1 fps). The
  resulting frame count is shown before extraction runs.
- **REQ-022** — Extraction runs as a background job with progress, producing sequentially
  numbered JPEG files inside the batch folder.
- **REQ-023** — One video may be used more than once with different time ranges; each
  extraction produces its own batch.

## D. Auto-annotation

- **REQ-030** — Once frames are extracted, the system runs SAM3 over all of them using each
  class's prompt, as a background job with progress and cancellation.
- **REQ-031** — Detections that overlap across prompts are deduplicated (greedy IoU NMS), so
  one object is not labelled as two classes at once.
- **REQ-032** — The confidence threshold is configurable per job.
- **REQ-033** — A frame with no detections is valid and still enters the dataset as a
  negative sample — it is not a failure.
- **REQ-034** — Auto-annotation can be re-run on the same batch; previous automatic results
  are replaced, but **the user's manual corrections must never be lost**.
- **REQ-035** — Auto-annotation can be started in **resume** mode, which skips frames that
  already carry automatic annotations. Resume is always an explicit choice and never the
  default, because a full re-run is also how the confidence threshold (REQ-032) is changed —
  the system cannot tell the two intentions apart, so it asks. A frame SAM3 legitimately
  found nothing on (REQ-033) writes no annotations, so a resume re-does it; that is accepted
  rather than tracked.

## E. Review & correction

- **REQ-040** — The user reviews frames one at a time, with fast navigation (left/right
  arrows, thumbnail filmstrip, jump to the next unreviewed frame).
- **REQ-041** — Each frame has a status: `pending`, `approved`, or `rejected`. Rejected
  frames never enter the dataset.
- **REQ-042** — The user can draw a new shape, move it, resize it, delete it, and change its
  class.
- **REQ-043** — The user can ask SAM3 for help inside the editor: click or drag a box around
  one object and the model produces its shape.
- **REQ-044** — All annotations and review statuses are **persistent** — they survive a
  server restart, unlike today's in-memory sessions.
- **REQ-045** — Review progress is visible (e.g. "120/300 reviewed"), and a batch can only
  be approved once no frame is still `pending`.
- **REQ-046** — The user can delete/clear all annotations of a specific class across all frames in
  the current batch from the Review editor.

## F. Master dataset

- **REQ-050** — Approving a batch **merges** its approved frames and their labels into the
  project's master dataset (accumulating across batches).
- **REQ-051** — The master dataset is train-ready YOLO format: `images/{train,val}`,
  `labels/{train,val}`, and a `data.yaml` regenerated from the project's class list.
- **REQ-052** — **Stable val split**: once a frame is placed in `val`, it stays in `val`
  across every later merge. New frames are split with an every-Nth pattern.
- **REQ-053** — The system records which batches have entered the master dataset, when, and
  how many images/labels each added.
- **REQ-054** — The master dataset can be downloaded as a `.zip` (e.g. to import into
  Roboflow or train on another machine).

## F2. Data Prep as the merge gate

- **REQ-130** — The Batches page supports multi-select. "Prepare & Merge Selected" opens
  Data Prep scoped to exactly those batches (`#/projects/{id}/data-prep?batches=1,2,3`).
  No dataset exists at this point.
- **REQ-131** — Data Prep is the merge gate. Filters and augmentation are tuned against the
  selected batches' shapes; "Confirm merge" names or picks the target dataset and queues
  **one** merge job for the whole selection. There is no path from Batches or Review
  straight to a dataset.
- **REQ-132** — The rules in force when a merge is confirmed are **snapshotted onto the
  dataset** (`datasets.rules_json`). The merge runs under the snapshot, and later edits to
  the project's rules never rewrite an existing dataset. Only an explicit Resync adopts
  today's rules — and it re-stamps the snapshot with them.

## F3. Counting correctness

- **REQ-140** — Ghost rejection (`entry_travel_min`) and spatial dedup
  (`dedup_radius`) are separate parameters. They pull in opposite directions, so one
  number cannot serve both.
- **REQ-141** — A track that vanishes parks its history; a new track id born within
  `handoff_radius` of its velocity-projected position inherits it. This is what keeps an
  ID switch at the counting line from either losing a count (the sack's "was above"
  evidence dies with the old id) or duplicating one (the new id has no "already counted"
  verdict).
- **REQ-142** — A counted direction is the track's *last* verdict, not a permanent one. A
  sack genuinely taken back out and reloaded counts again; unloading requires
  `unload_confirm_frames` sustained frames above the band, so repositioning by hand cannot
  cancel a real count.
- **REQ-143** — Per-track state is evicted once a track has been gone for `track_ttl`, so a
  long shift does not grow state without bound.
- **REQ-144** — Every finished track is written to a per-session JSONL with its trajectory
  and the reason it did or did not count, so a miss can be attributed to the model, the
  tracker, or the counter.

## F4. Counting accuracy bench

- **REQ-150** — A page lists every archive video as a row: date, batch, length, and the
  counter's `counted in` / `counted out` / `net` for it. Videos never counted are still
  rows — the table is the work list.
- **REQ-151** — Each row has an editable **ground truth** (what a human counted). The
  scored figure is the signed delta `counted_in - ground_truth`, so over- and under-counting
  stay distinguishable. Accuracy is `1 - |delta| / ground_truth`.
- **REQ-152** — Accuracy totals are computed **only** over rows where a ground truth is
  filled in. An uncounted or unscored video never enters the denominator.
- **REQ-153** — Counting runs as a queued background job over a selection of videos, or all
  of them, holding the GPU lock. It renders nothing — no annotated frame, no JPEG encode —
  which is what makes counting a 30-minute video practical. A run records the parameters and
  model it used.

## F5. Real recording times and working days

- **REQ-160** — Each recording's start time is read from the timestamp the camera burns into
  the top-right of every frame. Folder names and file mtimes are both unreliable: mtimes are
  file *copy* times, not recording times.
- **REQ-161** — A **working day runs 06:00 to 06:00**. A recording that started before 06:00
  belongs to the previous working day. A recording spanning the boundary is assigned by its
  start.
- **REQ-162** — Recordings are numbered 1..N within a working day, ordered by real start
  time. The original file path stays the file's identity and is shown beside the number, so
  results already recorded against it survive.
- **REQ-164** — The counting table is grouped into collapsible **cycles**, newest first, with
  the recordings inside each one in the order they were made. A cycle's header carries its
  video count, its AI/ground-truth totals, signed delta and accuracy, and how many of its rows
  have an unverified start time. Only the newest cycle is expanded by default.
- **REQ-165** — The Video Archive page browses the archive **by cycle**, not by folder. The
  left-hand list holds cycles newest first; the table shows the recordings of the selected
  cycle in the order they were made, with their cycle batch number and the time read from the
  overlay. A recording pulled in from another folder is marked with the folder it sits in.
- **REQ-166** — Each recording is checked for a truck with the project's newest model,
  sampling a handful of frames rather than the whole file. The recording trigger is truck
  arrival and departure, so one file is one batch — this check is what proves that assumption
  per file, and flags any recording where it does not hold.
- **REQ-167** — The production counter's counting day turns over at the same hour as the
  archive's cycles, 06:00, so `batch_number` on the Jetson and the batch order in Video
  Archive mean the same thing. It stays overridable per deployment via `DAILY_CUTOFF_TIME`.
- **REQ-168** — The recorder writes each archive file at the frame rate the stream actually
  delivers, and paces writes against the wall clock, so a file's duration equals the real
  duration of the recording however unevenly the capture loop runs.
- **REQ-170** — Recording happens once, on the streaming server, and is stored as a rolling
  buffer. The truck detector does not encode video: when a session ends it downloads that time
  range as a **copy**, so the archive keeps the camera's own codec, resolution and frame rate.
  Each clip carries a sidecar with the server's start time, which the app trusts over reading
  the burned-in overlay. Archive folder names remain calendar dates; the app derives cycles
  from the real start time, so the folder name is never read as a date.
- **REQ-163** — Nothing in the archive is moved, renamed or written to; it is mounted
  read-only. The grouping lives in an index beside it. Timestamps that could not be read, or
  were read with low confidence, are flagged and can be hand-entered; a hand-entered time
  outranks any reading and is never overwritten by a rescan.

## G. Training & evaluation

- **REQ-060** — The user starts training from the project page. Training **fine-tunes from
  the project's base model** on the merged master dataset (old + new).
- **REQ-061** — A fallback option "train on the latest batch only" (lower LR, fewer epochs)
  exists for cases where the old dataset is unavailable. It is not the default, and the UI
  warns about catastrophic forgetting.
- **REQ-062** — Default `batch`, `imgsz`, and `device` are derived from the hardware detected
  at runtime (VRAM), and all of them can be overridden — so moving to a bigger machine needs
  no code change.
- **REQ-063** — After training, the system validates **the base model and the new model on
  the exact same val set**, then shows mAP50 and mAP50-95 for both side by side with the
  delta.
- **REQ-064** — Each training run produces a stored model version (weights + metrics). The
  user can download the weights and **promote that version to be the project's new base
  model** for the next round.
- **REQ-065** — SAM3 and training must never hold VRAM at the same time; the system releases
  the SAM3 model before training starts.

## H. System

- **REQ-070** — Heavy work (extraction, auto-annotation, training) runs as queued jobs, one
  at a time, because there is a single GPU. Jobs show progress and logs, and can be cancelled.
- **REQ-071** — Jobs and their progress are persistent; after a server restart the job list
  is still there with its final statuses.
- **REQ-072** — The application runs via `docker compose up` with GPU access, and every path
  (video archive, data folder) is configured through environment/volumes — never hardcoded.
- **REQ-073** — The health endpoint reports: detected device/GPU, ffmpeg availability,
  whether the HuggingFace token was picked up, and database reachability.
- **REQ-074** — The system never writes anything into the user's video archive folder.