feat: configurable track forward, track to end, skip-and-mark (REQ-189)

Track 5 Frames becomes Track Forward with a 1-100 count input, remembered in
localStorage (default 5). A separate -> End action runs to the last frame
behind a confirm and can be cancelled mid-flight.

Frames that already hold a shape of the tracked class are skipped instead of
re-annotated: hidden and dimmed shapes count as present. Skips are reported in
the result banner and marked with a dot plus a class name on the filmstrip
thumbnail. frames() now returns the class ids each frame holds, the same query
row, no migration.

Chaining and seed growth stay out: every frame is still seeded from the drawn
box only.
This commit is contained in:
asus committed 2026-10-05 13:46:24 +07:00
1 parent f6b0335c93
commit 4f15335092
9 files changed
+217 -43

No files matched your search

+7 -2
View File
@@ -137,11 +137,16 @@ def frames(batch_id: int) -> List[dict]:
with db.cursor() as cur:
cur.execute(
"""SELECT f.*, (SELECT COUNT(*) FROM annotations a WHERE a.frame_id = f.id)
AS annotation_count
AS annotation_count,
(SELECT GROUP_CONCAT(DISTINCT a.class_id) FROM annotations a
WHERE a.frame_id = f.id) AS class_ids
FROM frames f WHERE f.batch_id = ? ORDER BY f.idx""",
(batch_id,),
)
return [dict(row) for row in cur.fetchall()]
rows = [dict(row) for row in cur.fetchall()]
for row in rows:
row["class_ids"] = [int(v) for v in row["class_ids"].split(",")] if row["class_ids"] else []
return rows
def frame_path(frame_id: int) -> Optional[str]:
+15 -6
View File
@@ -169,12 +169,21 @@ changes.
- **REQ-043** — The user can ask SAM3 for help inside the editor: click or drag a box around
one object and the model produces its shape.
- **REQ-189** — In the review editor, the user can push the **selected shape's** box forward
with *Track 5 Frames* (`[T]`): the next five frames each get their own REQ-043 box-assist
run, seeded with that box, and the shape is written as `source=manual`. Both geometry types
work — a bbox shape supplies its own box, a polygon is reduced to its bounding box. Each
frame is an **independent** run: a frame SAM3 refuses (nothing inside the box) is reported
and the rest still run; nothing selected, an unusable shape, or the last frame says so
instead of doing nothing. This is one-shot hand-off across the next frames, **not**
with *Track Forward* (`[T]`): the next N frames each get their own REQ-043 box-assist run,
seeded with that box, and the shape is written as `source=manual`. N is **configurable**,
1–100, remembered between sessions, default 5; a separate **Track to End** action runs to
the last frame of the batch and confirms before starting. Both geometry types work — a bbox
shape supplies its own box, a polygon is reduced to its bounding box. Each frame is an
**independent** run: a frame SAM3 refuses (nothing inside the box) is reported and the rest
still run; nothing selected, an unusable shape, or the last frame says so instead of doing
nothing. A frame that **already holds a shape of the tracked class is skipped** — hidden or
dimmed shapes count as present — and the skip is reported in the result banner and marked
with a dot on that frame's filmstrip thumbnail, never silently passed over. A long run can be
**cancelled** mid-flight: cancelling stops the remaining frames, but a request already in
flight is not aborted server-side and may still save its shape. The box is never grown and never reseeded from a result, so there is
no size or position readjustment between frames: a fast-moving object can still leave the
seed window (denser extraction or a video-propagation requirement, not yet written, is the
fix). This is one-shot hand-off across the next frames, **not**
propagation of drawn exemplars (which stays a non-goal): nothing is carried frame to frame,
and the runs do not depend on each other.
- **REQ-044** — All annotations and review statuses are **persistent** — they survive a
+41
View File
@@ -1405,6 +1405,47 @@ read it.
(accent border + glow) — no CSS added; **browser eyeball still owed by the user**
(375 px 2-col wrap, long class names).
## Task — Configurable Track Forward + track-to-end (REQ-189) `[DONE]`
> Status caveat: code, build and docs are verified below; the **end-to-end GPU run in the
> user's browser is still owed**, so the feature is not proven on real shapes yet.
1. **Backend** `backend/batches.py:136` `frames()` — the skip check needs *which* classes a
frame holds, and the frontend only ever received `annotation_count`. Added
`GROUP_CONCAT(DISTINCT a.class_id)` to the existing subquery (same row, no extra round trip,
no migration) and parsed it to `[int]`, `[]` when NULL.
→ verify: **[DONE]** `uv run` against the real DB: batch 19 returns 507 frames, the five
frames of the earlier manual track run (`22829`–`22833`) come back `class_ids: [2]`, frames
with none return `[]`. `EXPLAIN QUERY PLAN` shows both subqueries seek
`idx_annotations_frame` (no per-frame scan). Other consumers (`autolabel.py:95`,
`dataset.py:480`, `preview.py:35`) only read dict keys — unaffected.
2. **Cancel plumbing** `frontend/src/api.js` — `request` takes `signal`, `assist(frameId, body,
{ signal })` forwards it. No other caller touched.
3. **UI** `frontend/src/pages/ReviewPage.jsx` — `trackForward(toEnd)` now takes an explicit
count: N input (1–100, clamped and **persisted on blur** — an effect would save the
half-typed value, so typing "15" survives), *Track Forward [T]* with a dynamic tooltip,
*→ End* with a confirm showing the real frame count, `AbortController` per run so the
"asking SAM3…" indicator becomes a **Cancel** button, skip-on-existing-class, and a banner
reading `Tracked N of M — skipped K already had <class> — <failures>`. The banner shows when
something was skipped *or* failed — a fully clean run stays silent, as before. A successful
assist also unions its `class_id` into the local frame's `class_ids`, so a second run in the
same session skips its own output instead of writing duplicates.
4. **Filmstrip** `frontend/src/components/Filmstrip.jsx` — optional `markedIds` + `markLabel`
props; `frontend/src/app.css` `.thumb .skip-dot` (`--warn` + 2 px dark ring for contrast on
light thumbs, top-left so it never collides with the count badge), `.track-count` for the
input, and `flex-wrap` on `.frame-bar` so the bar still fits 375 px. Skipped frames also name
the class in the thumb tooltip.
→ verify: **[DONE]** `npm run build` green; clamp covers `"abc"`, `0`, `999`, `-3`, `""`,
`Infinity` and non-numeric `localStorage` garbage (`Number.isFinite` guard → 5), and the
slice bound is computed from `Number(trackFrames)` so a string can never concatenate into the
slice end; `slice(index+1, index+1+N)` and `slice(index+1)` both bounded; cancel lands as
`cancelled — frame n may still be saved by the server` and keeps the shapes already written;
skip check reads the `class_ids` array, not `annotation_count`; marks cleared at the start of
every run; `Shift+T` deliberately **not** bound (a fat-fingered 500-frame GPU run is not worth
a keystroke). Reviewer pass done — 4 findings fixed. **Still owed, user browser + GPU**: set
N=3 → 3 shapes, reload → N remembered; to-end over the already-reviewed part of batch 19 →
banner reports the skips and dots appear on those thumbs; Cancel mid-run stops it.
## Task — Track 5 Frames works at all (REQ-189) `[DONE]`
1. `ReviewPage.jsx:trackForward` read `geometry.coordinates` — a key the backend never emits
+21 -11
View File
@@ -205,7 +205,7 @@ SAM3: Ready | Off ← green when ready
```
Icons are SVG (Heroicons/Lucide-style), never emoji. *(Some emoji survive inside page bodies —
`🔄 Reset Auto`, `🏷️ Next Shape`, `📋 Copy Prev`, `🚀 Track 5`, `🤖`, `📦`, `⚡`, `🖼️`. These are
`🔄 Reset Auto`, `🏷️ Next Shape`, `📋 Copy Prev`, `🚀 Track Forward`, `🤖`, `📦`, `⚡`, `🖼️`. These are
legacy and **should** become SVG icons in a rebuild; that is the one place the current UI is
off-spec.)*
@@ -897,7 +897,7 @@ worked". The jump happens once per load, and again after an auto-label job finis
### 5.2 Data model in play
```
frame { id, idx, filename, width, height, review_status, annotation_count }
frame { id, idx, filename, width, height, review_status, annotation_count, class_ids }
annotation { id, class_id, geometry, score, source: "auto"|"manual" }
batch { …, review: { pending, approved, rejected }, annotation_count, status }
```
@@ -982,27 +982,37 @@ previous geometry** and shows the error. Same optimistic-with-rollback pattern f
`POST /annotations/bulk-reclass` and `POST /annotations/bulk-delete`.
**Frame bar.** `←` · `n / N` + a status pill (`pending`/`approved`/`rejected`) · `→` ·
**Next Shape [N]** · **Copy Prev [C]** · **Track 5 Frames [T]** · an "asking SAM3…" indicator ·
**Next Shape [N]** · **Copy Prev [C]** · **Track Forward [T]** (+ N input, *→ End*) ·
an "asking SAM3…" indicator ·
**Reject [X]** · **Approve [A]**.
- *Next Shape* jumps to the next frame with annotations, wrapping to the first.
- *Copy Prev* copies every annotation from frame `n-1` onto this frame (one POST per shape);
disabled at index 0 or when the previous frame is empty.
- *Track 5 Frames* takes the selected shape's bounding box and runs SAM3 assist on the next 5
frames with it. Works for both geometry types: a bbox shape supplies its own box, a polygon
is reduced to its bounding box. Nothing selected, an unusable shape, or the last frame
- *Track Forward* takes the selected shape's bounding box and runs SAM3 assist on the next N
frames with it, N from the number input beside the button (1–100, remembered, default 5).
*→ End* runs to the last frame of the batch behind a confirm. Works for both geometry types:
a bbox shape supplies its own box, a polygon is reduced to its bounding box. Nothing
selected, an unusable shape, or the last frame
says so in the error slot instead of silently doing nothing; a frame SAM3 refuses (nothing
inside the box) does not stop the rest — the banner then **starts** with `Tracked N of M`
(M = frames actually ahead, ≤ 5) followed by the per-frame failures. A second press while
a run is in flight is ignored. Shapes are written `source=manual`, exactly as a hand-drawn
one (REQ-189).
(M = frames actually ahead) followed by `skipped K already had <class>` and the per-frame
failures. Skipped frames are those that already hold a shape of the tracked class (hidden or
dimmed counts); they get a dot on their filmstrip thumb and the thumb's tooltip says why. The
"asking SAM3…" indicator becomes a **Cancel** button while a run is in flight (Cancel stops the
remaining frames; the frame already in flight can still be saved by the server, and the banner
says so). A second
press while a run is in flight is ignored. Shapes are written `source=manual`, exactly as a
hand-drawn one (REQ-189).
**Quick reclass bar** appears whenever a shape is selected: one button per class
(`[n] name`, class-coloured) plus *Delete [Del]*.
**Filmstrip.** Every frame as a thumbnail (`GET /frames/{id}/image?w=120`, lazy-loaded), border
coloured by review status, an annotation-count badge, `aria-current` on the active one, which
is scrolled into view (`block: nearest, inline: center`) on every index change.
is scrolled into view (`block: nearest, inline: center`) on every index change. A frame that
Track Forward skipped carries a `--warn` dot top-left and a tooltip naming the class it already
had; the marking clears on the next run (REQ-189).
### 5.6 The exemplar flow — the most intricate part of the app
@@ -1114,7 +1124,7 @@ Handlers are attached in the **capture** phase on `document`, and:
| `U` | jump to next pending frame (`GET /batches/{id}/next-pending?after_idx=`) | same |
| `N` | jump to next annotated frame | same |
| `C` | copy annotations from previous frame | same |
| `T` | track selected shape forward 5 frames | same |
| `T` | track selected shape forward N frames (N = the count input) | same |
| `1`–`9` | set active class; reclass the selected shape if any | reclass **all marked** shapes |
| `Del` / `Backspace` | delete the selected shape | delete **all marked** shapes (confirmed) |
| `H` | hide/show the selected shape(s); no-op with nothing selected | same |
+3 -3
View File
@@ -3,8 +3,8 @@
* FastAPI reports failures as {"detail": "..."} — unwrapped here so callers can
* show the message the backend actually wrote instead of "500". */
async function request(path, { method = 'GET', body, form } = {}) {
const options = { method, headers: {} }
async function request(path, { method = 'GET', body, form, signal } = {}) {
const options = { method, headers: {}, signal }
if (form) {
options.body = form
} else if (body !== undefined) {
@@ -135,7 +135,7 @@ export const api = {
method: 'POST',
body: { annotation_ids: annotationIds, class_id: classId },
}),
assist: (frameId, body) => request(`/frames/${frameId}/assist`, { method: 'POST', body }),
assist: (frameId, body, { signal } = {}) => request(`/frames/${frameId}/assist`, { method: 'POST', body, signal }),
// The whole frame-local exemplar pool, re-sent on every drag (REQ-173/174).
exemplarLabel: (frameId, body) =>
request(`/frames/${frameId}/exemplar-label`, { method: 'POST', body }),
+29
View File
@@ -558,6 +558,7 @@ main.page {
.frame-bar {
display: flex;
align-items: center;
flex-wrap: wrap;
gap: 10px;
padding: 8px 12px;
background: rgba(17, 24, 39, 0.45);
@@ -628,6 +629,34 @@ main.page {
line-height: 16px;
}
/* Track-forward skipped this frame: it already held the class (REQ-189). */
.thumb .skip-dot {
position: absolute;
left: 3px;
top: 3px;
width: 8px;
height: 8px;
border-radius: 999px;
background: var(--warn);
box-shadow: 0 0 0 2px rgba(0, 0, 0, 0.75);
}
/* Frame count for Track Forward, in the frame bar next to the button. */
.track-count {
display: inline-flex;
align-items: center;
gap: 4px;
font-size: 0.78rem;
color: var(--text-muted);
white-space: nowrap;
}
.track-count input {
width: 64px;
padding: 2px 6px;
font-size: 0.78rem;
}
.review-side {
display: flex;
flex-direction: column;
+8 -4
View File
@@ -1,21 +1,25 @@
import React from 'react'
import { api } from '../api'
export default function Filmstrip({ frames, index, onSelectIndex, stripRef }) {
export default function Filmstrip({ frames, index, onSelectIndex, stripRef, markedIds, markLabel = 'already has it' }) {
return (
<div className="filmstrip" ref={stripRef}>
{frames.map((item, position) => (
{frames.map((item, position) => {
const marked = markedIds?.has(item.id)
return (
<button
key={item.id}
className={`thumb ${item.review_status}`}
aria-current={position === index}
onClick={() => onSelectIndex(position)}
title={`${item.filename} — ${item.review_status}`}
title={`${item.filename} — ${item.review_status}${marked ? ` — ${markLabel}` : ''}`}
>
<img src={api.frameUrl(item.id, 120)} alt="" loading="lazy" />
{item.annotation_count > 0 && <span className="badge">{item.annotation_count}</span>}
{marked && <span className="skip-dot" aria-hidden="true" />}
</button>
))}
)
})}
</div>
)
}
+1 -1
View File
@@ -61,7 +61,7 @@ export default function ShortcutsPanel() {
</div>
<div>
<dt><kbd>C</kbd> <kbd>T</kbd></dt>
<dd>Copy prev / Track 5 frames</dd>
<dd>Copy prev / Track forward</dd>
</div>
</dl>
</div>
+92 -16
View File
@@ -26,6 +26,13 @@ export default function ReviewPage({ batchId: rawBatchId, projectId, onProject }
const [hiddenClasses, setHiddenClasses] = useState(() => new Set())
const [hiddenShapeIds, setHiddenShapeIds] = useState(() => new Set())
const [overriddenShapeIds, setOverriddenShapeIds] = useState(() => new Set())
const [trackFrames, setTrackFrames] = useState(() => {
const stored = Number(localStorage.getItem('trackFrames'))
return Number.isFinite(stored) && stored >= 1 ? Math.min(100, Math.round(stored)) : 5
})
const [trackAbort, setTrackAbort] = useState(null)
const [skippedIds, setSkippedIds] = useState(() => new Set())
const [skippedLabel, setSkippedLabel] = useState('')
const batchId = batch?.id || rawBatchId
const stripRef = useRef(null)
@@ -354,11 +361,11 @@ export default function ReviewPage({ batchId: rawBatchId, projectId, onProject }
}
}, [frames, index, frame, hiddenClasses])
const trackForward = useCallback(async () => {
const trackForward = useCallback(async (toEnd = false) => {
if (busy) return
if (!frames?.length) return
if (selectedId == null) {
setError('Select a shape first — Track 5 Frames follows one shape forward.')
setError('Select a shape first — Track Forward follows one shape forward.')
return
}
const current = annotations.find((a) => a.id === selectedId)
@@ -384,37 +391,71 @@ export default function ReviewPage({ batchId: rawBatchId, projectId, onProject }
box = [minX, minY, maxX, maxY]
}
if (!box) {
setError('That shape has no usable box — Track 5 Frames needs one.')
setError('That shape has no usable box — Track Forward needs one.')
return
}
const ahead = frames.slice(index + 1, index + 6)
const count = Number(trackFrames) || 5
const ahead = toEnd ? frames.slice(index + 1) : frames.slice(index + 1, index + 1 + count)
if (!ahead.length) {
setError('Last frame — Track 5 Frames has no frames ahead to follow into.')
setError('Last frame — nothing ahead to follow into.')
return
}
if (toEnd && !window.confirm(`Track ${ahead.length} frames to the end of the batch? That is one GPU call per frame.`)) {
return
}
const className = (project?.classes ?? []).find((item) => item.class_id === current.class_id)?.name || 'that class'
const controller = new AbortController()
setTrackAbort(controller)
setBusy(true)
setError('')
setSkippedIds(new Set())
setSkippedLabel('')
try {
let tracked = 0
let skipped = 0
const failures = []
const marked = new Set()
for (let i = 0; i < ahead.length; i++) {
const targetFrame = ahead[i]
// A frame that already holds this class is left alone — hidden or
// dimmed shapes count. It is reported, never silently passed over.
if ((targetFrame.class_ids ?? []).includes(current.class_id)) {
skipped++
marked.add(targetFrame.id)
continue
}
try {
await api.assist(targetFrame.id, { box, class_id: current.class_id })
await api.assist(targetFrame.id, { box, class_id: current.class_id }, { signal: controller.signal })
tracked++
patchFrameLocally(targetFrame.id, { annotation_count: (targetFrame.annotation_count ?? 0) + 1 })
patchFrameLocally(targetFrame.id, {
annotation_count: (targetFrame.annotation_count ?? 0) + 1,
// Keeps the skip check honest for a second run in this session.
class_ids: (targetFrame.class_ids ?? []).includes(current.class_id)
? targetFrame.class_ids
: [...(targetFrame.class_ids ?? []), current.class_id],
})
} catch (exc) {
if (controller.signal.aborted) {
failures.push(`cancelled — frame ${index + i + 2} may still be saved by the server`)
break
}
failures.push(`frame ${index + i + 2}: ${exc.message}`)
}
}
if (failures.length) {
setError(`Tracked ${tracked} of ${ahead.length} — ${failures.join(' · ')}`)
if (marked.size) {
setSkippedIds(marked)
setSkippedLabel(className)
}
const parts = [`Tracked ${tracked} of ${ahead.length}`]
if (skipped) parts.push(`skipped ${skipped} already had ${className}`)
if (failures.length) parts.push(failures.join(' · '))
if (tracked === 0 || skipped || failures.length) setError(parts.join(' — '))
} finally {
setTrackAbort(null)
setBusy(false)
}
}, [annotations, selectedId, index, frames, busy])
}, [annotations, selectedId, index, frames, busy, trackFrames, project])
const stateRef = useRef({})
stateRef.current = { frames, index, project, selectedId, setStatus, removeSelected, reclass, jumpToPending, jumpToNextAnnotated, setAssistMode, copyPrevious, trackForward, mode, setMode, markedIds, removeMarked, reclassMarked, setMarkedIds, toggleHideSelected }
@@ -451,7 +492,7 @@ export default function ReviewPage({ batchId: rawBatchId, projectId, onProject }
else if (key === 'u' || key === 'U') jumpToPending?.()
else if (key === 'n' || key === 'N') jumpToNextAnnotated?.()
else if (key === 'c' || key === 'C') copyPrevious?.()
else if (key === 't' || key === 'T') trackForward?.()
else if (key === 't' || key === 'T') trackForward?.(false)
else if (key === 'h' || key === 'H') toggleHideSelected?.()
else if (key === 'Delete' || key === 'Backspace') removeSelected?.()
else if (/^[1-9]$/.test(key)) {
@@ -685,18 +726,49 @@ export default function ReviewPage({ batchId: rawBatchId, projectId, onProject }
>
📋 Copy Prev [C]
</button>
<label className="track-count" htmlFor="track-count-input">
Track
<input
id="track-count-input" type="number" min="1" max="100" step="1"
value={trackFrames}
onChange={(e) => setTrackFrames(e.target.value)}
onBlur={() => {
// Clamp and persist here, not in an effect: an effect would
// save the half-typed value on every keystroke.
const n = Number(trackFrames)
const next = Number.isFinite(n) && n >= 1 ? Math.min(100, Math.round(n)) : 5
setTrackFrames(next)
localStorage.setItem('trackFrames', String(next))
}}
/>
frames
</label>
<button
type="button"
className="btn"
style={{ padding: '2px 8px', fontSize: '0.78rem', borderColor: 'rgba(56, 189, 248, 0.5)', color: '#38bdf8', background: 'rgba(56, 189, 248, 0.1)' }}
onClick={trackForward}
onClick={() => trackForward(false)}
disabled={selectedId == null || index >= frames.length - 1}
title="SAM3 auto-track selected shape for next 5 frames [T]"
title={`SAM3 auto-track selected shape forward ${trackFrames} frames [T]`}
>
🚀 Track 5 Frames [T]
🚀 Track Forward [T]
</button>
<button
type="button"
className="btn"
style={{ padding: '2px 8px', fontSize: '0.78rem', borderColor: 'rgba(56, 189, 248, 0.5)', color: '#38bdf8', background: 'rgba(56, 189, 248, 0.1)' }}
onClick={() => trackForward(true)}
disabled={selectedId == null || index >= frames.length - 1}
title="SAM3 auto-track selected shape forward to the end of the batch"
>
→ End
</button>
<span className="spacer" />
{busy && <span className="muted">asking SAM3…</span>}
{busy && (
trackAbort
? <button type="button" className="btn" onClick={() => trackAbort.abort()}>Cancel</button>
: <span className="muted">asking SAM3…</span>
)}
<button className="btn btn-danger" onClick={() => setStatus('rejected')}><XIcon size={14} /> Reject [X]</button>
<button className="btn btn-primary" onClick={() => setStatus('approved')}><CheckIcon size={14} /> Approve [A]</button>
</div>
@@ -705,7 +777,11 @@ export default function ReviewPage({ batchId: rawBatchId, projectId, onProject }
<QuickReclassBar classesList={classesList} reclass={reclass} removeSelected={removeSelected} />
)}
<Filmstrip frames={frames} index={index} onSelectIndex={setIndex} stripRef={stripRef} />
<Filmstrip
frames={frames} index={index} onSelectIndex={setIndex} stripRef={stripRef}
markedIds={skippedIds}
markLabel={`already has ${skippedLabel}`}
/>
</div>
<ReviewSidebar