chore: sync annotation UI, shortcuts, and misc updates

AGENTS.md

@@ -5,8 +5,6 @@ guidelines](https://github.com/multica-ai/andrej-karpathy-skills) and the
 [Chain of Truth](https://faridsurya-dev.github.io/Vibe-Coding-Research/en/welcome)
 method: **validated artifacts are the source of truth, AI is a generator and accelerator.**
 
-Please Response in extremely concise and precise.
-
 ## 1. Think Before Coding
 
 **Don't assume. Don't hide confusion. Surface tradeoffs.**
@@ -56,14 +54,12 @@ endpoint, run the job, look at the files it produced.
 
 `docs/` is the source of truth, not the chat prompt.
 
-
 | Document               | Contents                                                                            |
 | ---------------------- | ----------------------------------------------------------------------------------- |
 | `docs/requirements.md` | Numbered `REQ-xxx` requirements. Changes only with the user's approval.             |
 | `docs/design.md`       | Data schema, API contract, disk layout. Each section names the `REQ-xxx` it serves. |
 | `docs/tasks.md`        | Implementation steps + verification criteria, status `[TODO]`/`[DONE]`.             |
 
-
 The rules:
 
 - Before writing feature code, make sure a `REQ-xxx` covers it. If none does, propose
@@ -82,14 +78,45 @@ scripts with `uv run`. The Docker image installs the same file, so the two envir
 cannot drift.
 - **File size limit: 400 lines.** Any new or refactored file that exceeds it must be split
 into smaller, logical modules.
-- `**sam3/` is a vendor copy** of Meta's library. It's a dependency, not app code — don't
-add scripts there or edit anything inside it.
+- **`sam3/` is a vendored git submodule** (Meta's `facebookresearch/sam3`). It's a
+dependency, not app code — don't add scripts there or edit anything inside it. It's
+installed with `uv pip install -e ./sam3`.
 - **Never write into the user's video archive.** All output goes under `data/`.
 - Secrets (`HF_TOKEN`) come from `.env` only; they never belong in code or docs.
 
-## 7. UI/UX
+## 7. Architecture
 
-Frontend work follows [ui-ux-pro-max](https://github.com/nextlevelbuilder/ui-ux-pro-max-skill):
+**Two backends can coexist** — bare-metal (port 8000) and Docker (port 9010). The
+`start.sh` script auto-detects GPU, generates `docker-compose.override.yml` for CDI
+GPU passthrough, and falls back to bare-metal if Docker is unavailable.
+
+```
+backend/          FastAPI app (main.py), API routes in backend/api/
+frontend/         React 19 + Vite 7 SPA; dev on 5173, Docker on 9000
+algoritma-batch/  Counting engine (tracker, line counter). In Docker image it
+                  is copied to /app/src/ and /app/cfg/ — imports resolve as
+                  `src.*`, not `algoritma-batch.*`.
+data/             SQLite DB, projects, extracted frames, datasets, model weights
+docs/             Chain of Truth: requirements, design, tasks, UI spec
+```
+
+**Port mapping:**
+
+| Service         | Bare-metal | Docker |
+|-----------------|:----------:|:------:|
+| Backend API     | 8000       | 9010   |
+| Frontend Web UI | 5173       | 9000   |
+
+**Key entrypoints:**
+
+- `backend/main.py` — FastAPI app, lifespan, health endpoint, router wiring
+- `backend/jobs.py` — single-threaded GPU-locked job worker (extract, autolabel, merge, train, count, clock-scan, truck-scan)
+- `backend/sam3_engine.py` — SAM3 vision backbone, loaded once per process
+- `backend/hardware.py` — GPU VRAM detection and training batch-size defaults
+
+## 8. UI/UX
+
+Frontend work follows [ui-ux-pro-max](https://github.com/nextlevelbuilder/ui--pro-max-skill):
 generate the design system first (style, palette, typography), then build against it, then
 validate before delivering. Consistency across pages beats per-page cleverness.
 
@@ -109,18 +136,19 @@ Pre-delivery checklist — a UI task is not `[DONE]` until all of it passes:
 
 The review editor also has to survive keyboard-only use — see `docs/design.md`, "Frontend".
 
-## 8. Domain invariants
+## 9. Domain invariants
 
 Two things are easy to break without noticing, and breaking either makes the whole system
 lie:
 
 1. **Stable val split.** Once a frame lands in `val`, it stays in `val` forever. Otherwise
-he base-vs-new mAP comparison is meaningless.
+the base-vs-new mAP comparison is meaningless.
 2. **One `set_image` per image.** `Sam3Processor.set_image()` runs the vision backbone;
-set_text_prompt()`only re-runs the grounding head against the cached`backbone_out`. n N-prompt job calls` set_image` **once per image** and loops prompts over that same
-tate. Don't restructure this into set_image-per-prompt.
+`set_text_prompt()` only re-runs the grounding head against the cached `backbone_out`. An
+N-prompt job calls `set_image` **once per image** and loops prompts over that same state.
+Don't restructure this into set_image-per-prompt.
 
-## 9. Scalability & Portability
+## 10. Scalability & Portability
 
 **Never hardcode something that will change across environments.**
 

backend/api/review.py

@@ -41,9 +41,11 @@ class AssistRequest(BaseModel):
 
 
 class PoolExemplar(BaseModel):
-    # Normalized xyxy against the frame, as drawn on the review canvas.
-    box: List[float]
+    # Normalized xyxy box or [x, y] point against the frame.
+    box: Optional[List[float]] = None
+    point: Optional[List[float]] = None
     positive: bool = True
+    kind: Optional[str] = None
 
 
 class ExemplarLabelRequest(BaseModel):

backend/exemplar.py

@@ -143,8 +143,17 @@ def label(frame_id: int, class_id: int, exemplars: List[dict],
 
     positives, negatives = [], []
     for item in exemplars:
-        box = review.validate({"type": "bbox", "points": item["box"]}, "bbox")["points"]
-        (positives if item.get("positive", True) else negatives).append(box)
+        is_pos = bool(item.get("positive", True))
+        target_list = positives if is_pos else negatives
+        if item.get("point") is not None:
+            px, py = float(item["point"][0]), float(item["point"][1])
+            target_list.append({"kind": "point", "cxcywh": [px, py, 0.03, 0.03], "point": [px, py]})
+        elif item.get("box") is not None:
+            box = review.validate({"type": "bbox", "points": item["box"]}, "bbox")["points"]
+            target_list.append({"kind": "box", "cxcywh": _cxcywh(box), "box": box})
+        elif "points" in item and len(item["points"]) == 2:
+            px, py = float(item["points"][0]), float(item["points"][1])
+            target_list.append({"kind": "point", "cxcywh": [px, py, 0.03, 0.03], "point": [px, py]})
     if not positives and not negatives:
         raise review.ReviewError("No exemplars to run")
 
@@ -154,15 +163,11 @@ def label(frame_id: int, class_id: int, exemplars: List[dict],
     if not jobs.gpu_lock.acquire(timeout=5):
         busy = jobs.running_types()
         kind = busy[0] if busy else "background"
-        drawn = [{"geometry": _rect(box, label_type), "score": 1.0, "source": "manual"}
-                 for box in positives]
+        drawn = [{"geometry": _rect(p["box"], label_type), "score": 1.0, "source": "manual"}
+                 for p in positives if p["kind"] == "box"]
         if apply:
-            # Never the replace path here: with no detections to put back, it
-            # would wipe the class and leave only the drawings. Applying a
-            # detection-less run just files the drawings and honours the
-            # negatives.
             _append_drawn(frame_id, class_id, drawn)
-            _drop_negative_overlaps(frame_id, class_id, negatives)
+            _drop_negative_overlaps(frame_id, class_id, [n["box"] for n in negatives if n["kind"] == "box"])
         return _result(frame_id, drawn, apply, redetected=False,
                        message=f"The GPU is busy with a {kind} job — this is your drawing "
                                "only, nothing was detected")
@@ -178,8 +183,8 @@ def label(frame_id: int, class_id: int, exemplars: List[dict],
             state = engine.open_state(image)
             found = engine.apply_prompts(
                 state, threshold=threshold, text=prompt,
-                exemplars=[{"box": _cxcywh(box), "positive": True} for box in positives]
-                          + [{"box": _cxcywh(box), "positive": False} for box in negatives],
+                exemplars=[{"box": p["cxcywh"], "positive": True} for p in positives]
+                          + [{"box": n["cxcywh"], "positive": False} for n in negatives],
             )
     finally:
         jobs.gpu_lock.release()
@@ -198,9 +203,11 @@ def label(frame_id: int, class_id: int, exemplars: List[dict],
     detections = [(_norm_box(d.box, width, height), d) for d in found]
     items: List[dict] = []
 
-    # The user's own boxes first, so the duplicate check below measures against
-    # what they drew rather than the other way round.
-    for box in positives:
+    # Manual drawn boxes first:
+    for item in positives:
+        if item["kind"] != "box":
+            continue
+        box = item["box"]
         geometry = _rect(box, label_type)
         if label_type != "bbox":
             snapped = _snap(box, detections, width, height)
@@ -208,10 +215,16 @@ def label(frame_id: int, class_id: int, exemplars: List[dict],
                 geometry = snapped
         items.append({"geometry": geometry, "score": 1.0, "source": "manual"})
 
+    manual_boxes = [p["box"] for p in positives if p["kind"] == "box"]
+    negative_boxes = [n["box"] for n in negatives if n["kind"] == "box"]
+    negative_points = [n["point"] for n in negatives if n["kind"] == "point"]
+
     for norm, detection in detections:
-        if any(_iou(norm, box) >= NEGATIVE_IOU for box in negatives):
+        if any(_iou(norm, box) >= NEGATIVE_IOU for box in negative_boxes):
             continue
-        if any(_iou(norm, box) >= DUPLICATE_IOU for box in positives):
+        if any(norm[0] <= pt[0] <= norm[2] and norm[1] <= pt[1] <= norm[3] for pt in negative_points):
+            continue
+        if any(_iou(norm, box) >= DUPLICATE_IOU for box in manual_boxes):
             continue
         for geometry in _detection_shapes(detection, norm, width, height, label_type):
             items.append({"geometry": geometry, "score": detection.score, "source": "auto"})

frontend/src/app.css

@@ -710,7 +710,7 @@ main.page {
 
 .shortcuts { margin: 0; display: grid; gap: 5px; font-size: 12px; }
 .shortcuts > div { display: flex; gap: 10px; align-items: baseline; }
-.shortcuts dt { flex: none; width: 62px; }
+.shortcuts dt { flex: none; width: 72px; }
 .shortcuts dd { margin: 0; color: var(--text-muted); }
 
 kbd {

frontend/src/components/AnnotationCanvas.jsx

@@ -75,6 +75,17 @@ export default function AnnotationCanvas({
 
   const additive = useRef(false)
 
+  function handleDoubleClick(event) {
+    if (selecting) return
+    const [x, y] = pointAt(event)
+    if (assistMode) {
+      const seed = [Math.max(0, x - 0.015), Math.max(0, y - 0.015), Math.min(1, x + 0.015), Math.min(1, y + 0.015)]
+      onAssist(seed)
+    } else if (onExemplar) {
+      onExemplar([x, y], !event.shiftKey, { kind: 'point' })
+    }
+  }
+
   function startDraw(event) {
     if (event.button !== 0) return
     const [x, y] = pointAt(event)
@@ -218,6 +229,7 @@ export default function AnnotationCanvas({
         onPointerDown={startDraw}
         onPointerMove={onPointerMove}
         onPointerUp={onPointerUp}
+        onDoubleClick={handleDoubleClick}
       >
         {/* A preview replaces this class on the frame wholesale, so its stored
             shapes step aside for the proposals — left on screen they read as
@@ -270,7 +282,7 @@ export default function AnnotationCanvas({
           )
         })}
 
-        {!selecting && exemplars.map((item, i) => {
+        {!selecting && exemplars.filter((item) => item.kind !== 'point' && item.box).map((item, i) => {
           const [x0, y0, x1, y1] = item.box
           return (
             <rect

frontend/src/components/ShortcutsPanel.jsx

@@ -7,6 +7,10 @@ export default function ShortcutsPanel() {
           <dt><kbd>Drag</kbd></dt>
           <dd>Example of this class → preview a re-detect</dd>
         </div>
+        <div>
+          <dt><kbd>Dbl Click</kbd></dt>
+          <dd>Quick exemplar point → preview a re-detect</dd>
+        </div>
         <div>
           <dt><kbd>Enter</kbd> <kbd>Esc</kbd></dt>
           <dd>Apply / discard the preview</dd>
@@ -15,6 +19,10 @@ export default function ShortcutsPanel() {
           <dt><kbd>Shift</kbd>+<kbd>Drag</kbd></dt>
           <dd>Not this → drop it and re-detect</dd>
         </div>
+        <div>
+          <dt><kbd>Shift</kbd>+<kbd>Dbl</kbd></dt>
+          <dd>Quick negative exemplar point</dd>
+        </div>
         <div>
           <dt><kbd>V</kbd></dt>
           <dd>Draw ⇄ Select mode</dd>

frontend/src/hooks/useExemplarPool.js

@@ -100,9 +100,12 @@ export default function useExemplarPool({ frameId, classId, onApplied, onError,
     timer.current = setTimeout(() => runRef.current?.(), delay)
   }, [])
 
-  const add = useCallback((box, positive) => {
+  const add = useCallback((target, positive, { kind = 'box' } = {}) => {
     if (!frameId) return
-    poolRef.current = [...poolRef.current, { box, positive }]
+    const entry = kind === 'point'
+      ? { point: target, positive, kind: 'point' }
+      : { box: target, positive, kind: 'box' }
+    poolRef.current = [...poolRef.current, entry]
     setExemplars(poolRef.current)
     setActive(true)
     // A drag during a pass cannot be in that pass's result, so it always earns

frontend/src/pages/ReviewPage.jsx

@@ -500,7 +500,7 @@ export default function ReviewPage({ batchId: rawBatchId, projectId, onProject }
                   ? `${pool.exemplars.filter((e) => e.positive).length} example(s), `
                     + `${pool.exemplars.filter((e) => !e.positive).length} negative — `
                     + 'tune the filters, then Apply'
-                  : 'Drag an example of this class · Shift-drag = not this'}
+                  : 'Drag or double-click an example · Shift-drag/click = not this'}
               </span>
             )}
           </div>