# Design Serves `./requirements.md`. Each section names the REQs it fulfils. ## Overview ``` Browser (React + Vite, served by nginx) │ /api → proxy ▼ FastAPI ──► jobs (1 worker thread, 1 GPU) │ ├─ extract : ffmpeg (CPU) │ ├─ autolabel: SAM3 (GPU) │ ├─ merge : copy + write labels (CPU) │ └─ train : Ultralytics + eval (GPU) ├──► SQLite (metadata & status) └──► data/ (frames, master dataset, weights) ``` Storage split rule: **SQLite holds metadata and status; the disk holds pixels, final labels, and weights.** The master dataset must stay useful even if the database is lost (REQ-006, REQ-054). ## Disk layout (REQ-006) ``` data/ # Docker volume app.db # SQLite (WAL) projects// base/model.pt # the project's active base model (REQ-003) dataset/ # MASTER, accumulative (REQ-050…052) images/{train,val}/…jpg labels/{train,val}/…txt data.yaml batches// frames/000001.jpg … # extraction output (REQ-022) models// best.pt metrics.json # base vs new metrics (REQ-063) runs/ # Ultralytics run directory ``` Master dataset filenames: `__.jpg` — unique across batches and self-documenting about where each image came from. The user's video archive is read-only (REQ-074). ## SQLite schema Created by an idempotent migration in `backend/db.py` at startup. ```sql projects( id, slug UNIQUE, name, label_type CHECK(bbox|polygon), base_model_path, base_model_kind CHECK(uploaded|pretrained|trained), video_root, val_every DEFAULT 5, created_at) project_classes( id, project_id → projects, class_id INT, name, prompt, UNIQUE(project_id, class_id)) -- class_id = the YOLO class index (REQ-003/005) batches( id, project_id → projects, video_path, date_label, batch_label, start_sec REAL, end_sec REAL, fps REAL, status CHECK(extracting|extracted|labeling|reviewing|approved|merged|failed), frame_count INT, created_at, merged_at) frames( id, batch_id → batches, idx INT, filename, width INT, height INT, review_status CHECK(pending|approved|rejected) DEFAULT 'pending', UNIQUE(batch_id, idx)) annotations( id, frame_id → frames, class_id INT, geometry TEXT, -- JSON; see "Geometry format" score REAL, source CHECK(auto|manual), created_at) dataset_items( -- master dataset membership (REQ-052) id, project_id → projects, frame_id → frames UNIQUE, split CHECK(train|val), image_rel, label_rel, added_at) model_versions( id, project_id → projects, version INT, weights_path, parent_model_path, metrics TEXT, base_metrics TEXT, created_at, UNIQUE(project_id, version)) jobs( -- persistent (REQ-071) id, project_id, batch_id, type CHECK(extract|autolabel|merge|train), status CHECK(queued|running|done|failed|cancelled), progress INT, total INT, message, error, log TEXT, created_at, started_at, finished_at) ``` **The stable val split (REQ-052)** is enforced by `dataset_items`: an existing row never changes its `split`. On merge, only frames without a row are assigned, using a per-project round-robin counter (`val_every`) that continues from the previous count. **Geometry format.** One JSON column covers both label types (REQ-002): - `bbox` → `{"type":"bbox","points":[x0,y0,x1,y1]}` - `polygon` → `{"type":"polygon","points":[[x,y], …]}` Coordinates are stored **normalized 0–1** against the frame size, so neither the editor nor the exporter needs to know the display size. SAM3 mask → polygon conversion is `review.mask_to_polygons()`; for `bbox` projects the mask is only used to take its bounding box. ## Backend modules `app/` moves to `backend/`. Reuse existing code wherever possible: | Module | Role | Status | |---|---|---| | `sam3_engine.py` | SAM3 singleton, `open_state`/`apply_prompts`/`segment_at` | reused, plus a `release()` for REQ-065 | | `labeling.py` | per-frame detection + cross-prompt NMS (REQ-031) | reused; the folder-walking half went with the old flow | | `exporters.py` | ~~YOLO label writing~~ | **deleted** — `dataset.py` writes labels, `mask_to_polygons` moved to `review.py` | | `sessions.py` | ~~exemplar/tap interaction~~ | **deleted** — see below | | `jobs.py` | single-worker queue | extended: job types + persistence | | `training.py` | Ultralytics fine-tune | changed: starts from the base model, args from `hardware.py` | | `db.py` | connection + migration | **new** | | `projects.py` | project CRUD, reads classes from a `.pt` | **new** | | `library.py` | scans `//` | **new** | | `video.py` | `ffprobe`, Range streaming, `ffmpeg` extraction | **new** | | `batches.py` | batch lifecycle | **new** | | `review.py` | annotation CRUD, frame status, click-assist | **new** | | `autolabel.py` | the SAM3 job over a whole batch | **new** | | `dataset.py` | merge into the master dataset, stable split | **new** | | `evaluate.py` | validate base vs new model | **new** | | `hardware.py` | VRAM detection → training defaults | **new** | | `api/` | the FastAPI routes, one module per domain | **new** | Removed: `uploads.py`, `static/index.html`, and the old flow's endpoints. `sessions.py` was meant to be reused for click-assist, but it existed to hold GPU-resident state for an interactive session — a whole eviction policy, an undo stack, and a per-session annotation store, all of which the database and the stateless `review.assist` now cover. Adapting 357 lines to do what 60 lines do was not worth it, so the module is gone. The one thing it knew that mattered — SAM3 wants exemplar boxes as normalized centre-x, centre-y, width, height — moved with it. The 400-line file limit (see `../AGENTS.md`) applies to all of the above. It is why the routes live in `backend/api/{projects,batches,review,models,jobs}.py` rather than in `main.py`, which now only builds the app and owns startup. Route modules import their domain module under an alias (`from backend import projects as project_store`) so the two namespaces stay distinguishable. ## API contract ``` GET /api/health REQ-073 GET /api/projects REQ-001 POST /api/projects REQ-001,002,004,005 GET /api/projects/{id} # includes label_type_locked: bool (REQ-002) DELETE /api/projects/{id} PATCH /api/projects/{id} # class prompts, val_every (REQ-005) POST /api/projects/{id}/classes # add new class {name, prompt} (REQ-008) DELETE /api/projects/{id}/classes/{class_id} # delete class, delete shapes, reindex classes (REQ-007) POST /api/projects/{id}/base-model # upload .pt, read classes (REQ-003) GET /api/projects/{id}/dataset # master dataset summary (REQ-053) GET /api/projects/{id}/dataset/download # zip (REQ-054) GET /api/projects/{id}/library # list of dates (REQ-011) GET /api/projects/{id}/library/{date} # videos + duration/resolution (REQ-012) GET /api/projects/{id}/video?rel=… # Range streaming (REQ-013) POST /api/projects/{id}/batches # {rel, start_sec, end_sec, fps} → extract job GET /api/batches/{id} # status + review progress (REQ-045) GET /api/batches/{id}/frames # frames + statuses POST /api/batches/{id}/autolabel # {threshold} → job (REQ-030,032,034) DELETE /api/batches/{id}/classes/{class_id}/annotations # clear all shapes of class in batch (REQ-046) POST /api/batches/{ids}/approve # one or many, comma-separated → one merge job (REQ-131) GET /api/batches/{ids}/triage/summary # one or many, comma-separated (REQ-130) GET /api/batches/{ids}/triage/shapes GET /api/batches/{ids}/triage/suggest POST /api/batches/{ids}/triage/simulate GET /api/frames/{id}/image?w=… # frame image / thumbnail GET /api/frames/{id}/annotations POST /api/frames/{id}/annotations # add a manual shape (REQ-042) PATCH /api/annotations/{id} # move/resize/reclass DELETE /api/annotations/{id} POST /api/frames/{id}/assist # click/box → SAM3 shape (REQ-043) POST /api/frames/{id}/status # approved | rejected | pending (REQ-041) POST /api/projects/{id}/train # → train job (REQ-060,061,062) GET /api/projects/{id}/models # versions + metrics (REQ-063,064) GET /api/models/{id}/weights # download best.pt POST /api/models/{id}/promote # make it the project's base model (REQ-064) GET /api/jobs?project_id=… REQ-070,071 GET /api/jobs/{id} POST /api/jobs/{id}/cancel ``` ## Job flows **extract (REQ-020…023).** `ffmpeg -ss -to -i