feat: setup dataset enrichment app codebase and scripts

This commit is contained in:
asus committed 2026-08-05 11:52:27 +07:00
1 parent b5c28cc98a
commit d07578462e
72 files changed
+11370

No files matched your search

+85
View File
@@ -0,0 +1,85 @@
# Python
__pycache__/
*.pyc
*.pyo
*.pyd
.venv/
venv/
*.egg-info/
build/
dist/
# Application Data & Environment
data/
.env
.env.local
# Frontend
frontend/node_modules/
frontend/dist/
frontend/*.local
# OS / IDE
.DS_Store
.vscode/
.idea/
# Video formats
*.mp4
*.avi
*.mov
*.mkv
*.webm
*.flv
*.wmv
*.m4v
*.mpg
*.mpeg
# Image formats & Datasets
*.jpg
*.jpeg
*.png
*.webp
*.bmp
*.tiff
*.gif
datasets/
dataset/
# ML Models & Checkpoints
*.pt
*.pth
*.bin
*.safetensors
*.ckpt
*.onnx
*.engine
*.plan
*.trt
*.h5
runs/
checkpoints/
weights/
# Binary & Data Arrays
*.npy
*.npz
*.parquet
*.feather
*.pkl
*.pickle
*.joblib
*.db
*.sqlite
*.sqlite3
# Training Logs & Caches
*.log
*.tfevents*
wandb/
mlruns/
.cache/
.pytest_cache/
.mypy_cache/
.ruff_cache/
+3
View File
@@ -0,0 +1,3 @@
[submodule "sam3"]
path = sam3
url = https://github.com/facebookresearch/sam3.git
+129
View File
@@ -0,0 +1,129 @@
# Agent Instructions
Working rules for agents in this repo. A merge of the [Karpathy coding
guidelines](https://github.com/multica-ai/andrej-karpathy-skills) and the
[Chain of Truth](https://faridsurya-dev.github.io/Vibe-Coding-Research/en/welcome)
method: **validated artifacts are the source of truth, AI is a generator and accelerator.**
Please Response in extremely concise and precise.
## 1. Think Before Coding
**Don't assume. Don't hide confusion. Surface tradeoffs.**
- State your assumptions explicitly. If uncertain, ask.
- If multiple interpretations exist, present them — don't pick silently.
- If a simpler approach exists, say so. Push back when warranted.
- If something is unclear, stop. Name what's confusing. Ask.
## 2. Simplicity First
**Minimum code that solves the problem. Nothing speculative.**
- No features beyond what was asked.
- No abstractions for single-use code.
- No "flexibility" or "configurability" that wasn't requested.
- No error handling for impossible scenarios.
- If you write 200 lines and it could be 50, rewrite it.
## 3. Surgical Changes
**Touch only what you must. Clean up only your own mess.**
- Don't "improve" adjacent code, comments, or formatting.
- Don't refactor things that aren't broken.
- Match existing style, even if you'd do it differently.
- If you notice unrelated dead code, mention it — don't delete it.
- Remove imports/variables/functions that *your* changes made unused.
The test: every changed line should trace directly to the user's request.
## 4. Goal-Driven Execution
**Define success criteria. Loop until verified.**
Turn tasks into verifiable goals, and for multi-step work state a brief plan:
```
1. [Step] → verify: [check]
2. [Step] → verify: [check]
```
This repo has no automated tests, so verification means **running something**: hit the
endpoint, run the job, look at the files it produced.
## 5. Chain of Truth — documents first, then code
`docs/` is the source of truth, not the chat prompt.
| Document | Contents |
| ---------------------- | ----------------------------------------------------------------------------------- |
| `docs/requirements.md` | Numbered `REQ-xxx` requirements. Changes only with the user's approval. |
| `docs/design.md` | Data schema, API contract, disk layout. Each section names the `REQ-xxx` it serves. |
| `docs/tasks.md` | Implementation steps + verification criteria, status `[TODO]`/`[DONE]`. |
The rules:
- Before writing feature code, make sure a `REQ-xxx` covers it. If none does, propose
adding one to the user first.
- Once a task is finished **and verified**, flip its status in `docs/tasks.md` to `[DONE]`
in the same commit.
- If the implementation diverges from `docs/design.md`, update the design — never let a
document lie.
- Use relative paths in markdown (`./`, `../`), not absolute ones.
## 6. Repo rules
- **Package manager: `uv`.** No `pip`, `poetry`, or bare `python`/`python3`. Dependencies
live in `requirements.txt`; install them with `uv pip install -r requirements.txt` and run
scripts with `uv run`. The Docker image installs the same file, so the two environments
cannot drift.
- **File size limit: 400 lines.** Any new or refactored file that exceeds it must be split
into smaller, logical modules.
- `**sam3/` is a vendor copy** of Meta's library. It's a dependency, not app code — don't
add scripts there or edit anything inside it.
- **Never write into the user's video archive.** All output goes under `data/`.
- Secrets (`HF_TOKEN`) come from `.env` only; they never belong in code or docs.
## 7. UI/UX
Frontend work follows [ui-ux-pro-max](https://github.com/nextlevelbuilder/ui-ux-pro-max-skill):
generate the design system first (style, palette, typography), then build against it, then
validate before delivering. Consistency across pages beats per-page cleverness.
This app is a **dense internal tool**, not a landing page. Its screens are for long review
sessions in front of a screen: the video frame and the annotation canvas are the content,
everything else is chrome and stays quiet. No decorative gradients, no marketing motion.
Pre-delivery checklist — a UI task is not `[DONE]` until all of it passes:
- [ ] No emoji as icons (SVG only: Heroicons/Lucide)
- [ ] `cursor: pointer` on every clickable element
- [ ] Hover states with smooth transitions (150–300 ms)
- [ ] Text contrast at least 4.5:1
- [ ] Focus states visible for keyboard navigation
- [ ] `prefers-reduced-motion` respected
- [ ] Responsive at 375 / 768 / 1024 / 1440 px
The review editor also has to survive keyboard-only use — see `docs/design.md`, "Frontend".
## 8. Domain invariants
Two things are easy to break without noticing, and breaking either makes the whole system
lie:
1. **Stable val split.** Once a frame lands in `val`, it stays in `val` forever. Otherwise
he base-vs-new mAP comparison is meaningless.
2. **One `set_image` per image.** `Sam3Processor.set_image()` runs the vision backbone;
set_text_prompt()`only re-runs the grounding head against the cached`backbone_out`. n N-prompt job calls` set_image` **once per image** and loops prompts over that same
tate. Don't restructure this into set_image-per-prompt.
## 9. Scalability & Portability
**Never hardcode something that will change across environments.**
- **Hardware Agnosticism:** Do not hardcode hardware requirements (e.g., GPU configurations in `docker-compose.yml`) directly into base configuration files. Instead, use dynamic startup scripts (like `start.sh`) or environment overrides to detect the host's capabilities and inject the appropriate settings automatically.
- **Portability:** The app must be fully deployable and scalable on any device (from a CPU-only laptop to a massive multi-GPU rig) without requiring manual code edits to run.
- **Dynamic Configuration:** Do not hardcode absolute IP addresses, local network paths, or machine-specific environment variables in code. Rely on relative paths and configuration files to ensure maximum scalability.
Symlink
+1
View File
@@ -0,0 +1 @@
AGENTS.md
+35
View File
@@ -0,0 +1,35 @@
FROM python:3.12-slim
ENV PYTHONUNBUFFERED=1 \
UV_SYSTEM_PYTHON=1 \
APP_DATA_DIR=/data \
VIDEO_ARCHIVE=/videos
# ffmpeg does the trimming and frame extraction; libgl1/libglib2.0-0 are what
# OpenCV needs once ultralytics pulls in the non-headless build.
RUN apt-get update && apt-get install -y --no-install-recommends \
ffmpeg libgl1 libglib2.0-0 \
&& rm -rf /var/lib/apt/lists/*
# uv comes from PyPI rather than `COPY --from=ghcr.io/astral-sh/uv`: pulling it
# from a second registry made every build fail whenever ghcr.io was unreachable,
# even with all the dependency layers already cached. This is the only pip call
# in the project — bootstrapping the tool that installs everything else.
RUN pip install --no-cache-dir uv
WORKDIR /app
# Dependencies first so code edits don't re-download ~3 GB of CUDA wheels.
COPY requirements.txt ./
RUN uv pip install -r requirements.txt
# The vendored SAM3 declares timm/ftfy/regex/tqdm itself, so this install keeps
# its dependencies (unlike einops + pycocotools, which it needs but never
# declares — those are pinned in requirements.txt).
COPY sam3/ ./sam3/
RUN uv pip install -e ./sam3
COPY backend/ ./backend/
EXPOSE 8000
CMD ["uvicorn", "backend.main:app", "--host", "0.0.0.0", "--port", "8000"]
View File
Whitespace-only changes.
View File
Whitespace-only changes.
+152
View File
@@ -0,0 +1,152 @@
"""Batch, frame and auto-annotation routes (REQ-020…034)."""
import os
from typing import Optional
from fastapi import APIRouter, HTTPException, Response
from fastapi.responses import FileResponse
from pydantic import BaseModel
from backend import autolabel, dataset, library
from backend import batches as batch_store
from backend import review as review_store
from backend.api.common import project_or_404, thumbnail
router = APIRouter(tags=["batches"])
class BatchRequest(BaseModel):
rel: str
start_sec: float = 0.0
end_sec: float
fps: float = 1.0
class AutolabelRequest(BaseModel):
engine: str = "sam3"
engines: Optional[list[str]] = None
class_ids: Optional[list[int]] = None
engine_classes: Optional[dict[str, list[str]]] = None
threshold: float = autolabel.DEFAULT_THRESHOLD
iou_threshold: float = autolabel.DEFAULT_IOU
min_box_frac: float = 0.0
resume: bool = False
@router.post("/api/projects/{project_id}/batches")
def create_batch(project_id: int, request: BatchRequest) -> dict:
try:
return batch_store.create(project_id, request.rel, request.start_sec,
request.end_sec, request.fps)
except (batch_store.BatchError, library.LibraryError) as exc:
raise HTTPException(400, str(exc))
@router.get("/api/projects/{project_id}/batches")
def list_batches(project_id: int) -> dict:
project_or_404(project_id)
return {"batches": batch_store.listing(project_id)}
class BatchPatch(BaseModel):
batch_label: Optional[str] = None
date_label: Optional[str] = None
status: Optional[str] = None
@router.get("/api/batches/{batch_id}")
def read_batch(batch_id: int) -> dict:
batch = batch_store.get(batch_id)
if batch is None:
raise HTTPException(404, "No such batch")
return batch
@router.patch("/api/batches/{batch_id}")
def update_batch(batch_id: int, request: BatchPatch) -> dict:
try:
return batch_store.update(batch_id, request.model_dump(exclude_unset=True))
except batch_store.BatchError as exc:
raise HTTPException(400, str(exc))
@router.delete("/api/batches/{batch_id}")
def delete_batch(batch_id: int) -> dict:
if not batch_store.delete(batch_id):
raise HTTPException(404, "No such batch")
return {"deleted": True}
@router.get("/api/batches/{batch_id}/frames")
def list_frames(batch_id: int) -> dict:
if batch_store.get(batch_id) is None:
raise HTTPException(404, "No such batch")
return {"frames": batch_store.frames(batch_id)}
@router.post("/api/batches/{batch_id}/autolabel")
def start_autolabel(batch_id: int, request: AutolabelRequest) -> dict:
try:
engine_list = request.engines if (request.engines and len(request.engines) > 0) else [request.engine]
return autolabel.start(batch_id, request.threshold, request.iou_threshold,
request.min_box_frac, resume=request.resume,
engines=engine_list, class_ids=request.class_ids,
engine_classes=request.engine_classes)
except batch_store.BatchError as exc:
raise HTTPException(400, str(exc))
@router.post("/api/batches/{batch_id}/approve-all")
def approve_all_batch_frames(batch_id: int) -> dict:
if batch_store.get(batch_id) is None:
raise HTTPException(404, "No such batch")
updated = batch_store.approve_all_frames(batch_id)
return {"approved_count": updated}
@router.post("/api/batches/{batch_id}/approve")
def approve_batch(batch_id: int) -> dict:
try:
return dataset.approve(batch_id)
except dataset.DatasetError as exc:
raise HTTPException(400, str(exc))
@router.get("/api/projects/{project_id}/dataset")
def dataset_summary(project_id: int) -> dict:
project_or_404(project_id)
return dataset.summary(project_id)
@router.get("/api/projects/{project_id}/dataset/download")
def dataset_download(project_id: int):
project = project_or_404(project_id)
try:
path = dataset.zip_path(project)
except dataset.DatasetError as exc:
raise HTTPException(400, str(exc))
return FileResponse(path, media_type="application/zip",
filename=f"{project['slug']}-dataset.zip")
@router.get("/api/frames/{frame_id}/image")
def frame_image(frame_id: int, w: int = 0):
path = batch_store.frame_path(frame_id)
if path is None or not os.path.isfile(path):
raise HTTPException(404, "No such frame")
if w and 16 <= w <= 2048:
return Response(content=thumbnail(path, w), media_type="image/jpeg",
headers={"Cache-Control": "public, max-age=3600"})
return FileResponse(path, media_type="image/jpeg",
headers={"Cache-Control": "public, max-age=3600"})
@router.delete("/api/batches/{batch_id}/classes/{class_id}/annotations")
def clear_batch_class_annotations(batch_id: int, class_id: int) -> dict:
if batch_store.get(batch_id) is None:
raise HTTPException(404, "No such batch")
deleted = review_store.clear_batch_class_annotations(batch_id, class_id)
return {"deleted": deleted}
+35
View File
@@ -0,0 +1,35 @@
"""Helpers shared by the route modules."""
import io
import os
from functools import lru_cache
from fastapi import HTTPException
from backend import projects as project_store
def project_or_404(project_id: int) -> dict:
project = project_store.get(project_id)
if project is None:
raise HTTPException(404, "No such project")
return project
@lru_cache(maxsize=512)
def _thumbnail_cached(path: str, width: int, mtime: float) -> bytes:
from PIL import Image
with Image.open(path) as image:
image = image.convert("RGB")
if image.width > width:
height = max(1, round(image.height * width / image.width))
image = image.resize((width, height), Image.BILINEAR)
buffer = io.BytesIO()
image.save(buffer, "JPEG", quality=72)
return buffer.getvalue()
def thumbnail(path: str, width: int) -> bytes:
# mtime is part of the key so a re-extracted frame never serves a stale one.
return _thumbnail_cached(path, width, os.path.getmtime(path))
+29
View File
@@ -0,0 +1,29 @@
"""Job routes (REQ-070, REQ-071)."""
from typing import Optional
from fastapi import APIRouter, HTTPException
from backend import jobs as job_store
router = APIRouter(prefix="/api/jobs", tags=["jobs"])
@router.get("")
def list_jobs(project_id: Optional[int] = None, limit: int = 50) -> dict:
return {"jobs": [job.to_dict() for job in job_store.listing(project_id, limit)]}
@router.get("/{job_id}")
def get_job(job_id: int) -> dict:
job = job_store.get(job_id)
if job is None:
raise HTTPException(404, "No such job")
return job.to_dict()
@router.post("/{job_id}/cancel")
def cancel_job(job_id: int) -> dict:
if not job_store.cancel(job_id):
raise HTTPException(400, "Job is not cancellable")
return {"ok": True}
+63
View File
@@ -0,0 +1,63 @@
"""Training and model-version routes (REQ-060…065)."""
import os
from typing import Optional, Union
from fastapi import APIRouter, HTTPException
from fastapi.responses import FileResponse
from pydantic import BaseModel
from backend import hardware, training
from backend.api.common import project_or_404
router = APIRouter(tags=["models"])
class TrainRequest(BaseModel):
epochs: int = 50
batch: Optional[int] = None
imgsz: Optional[int] = None
device: Optional[Union[int, str]] = None
batch_ids: Optional[list] = None
@router.get("/api/hardware")
def read_hardware() -> dict:
return hardware.defaults()
@router.post("/api/projects/{project_id}/train")
def start_training(project_id: int, request: TrainRequest) -> dict:
project_or_404(project_id)
try:
return training.start(
project_id, request.epochs,
{"batch": request.batch, "imgsz": request.imgsz, "device": request.device},
batch_ids=request.batch_ids,
)
except training.TrainingError as exc:
raise HTTPException(400, str(exc))
@router.get("/api/projects/{project_id}/models")
def list_models(project_id: int) -> dict:
project = project_or_404(project_id)
return {"models": training.listing(project_id),
"base_model_kind": project["base_model_kind"]}
@router.get("/api/models/{model_id}/weights")
def download_weights(model_id: int):
version = training.get_version(model_id)
if version is None or not os.path.isfile(version["weights_path"]):
raise HTTPException(404, "No weights for that version")
return FileResponse(version["weights_path"], media_type="application/octet-stream",
filename=f"v{version['version']}-best.pt")
@router.post("/api/models/{model_id}/promote")
def promote_model(model_id: int) -> dict:
try:
return training.promote(model_id)
except training.TrainingError as exc:
raise HTTPException(400, str(exc))
+207
View File
@@ -0,0 +1,207 @@
"""Project, library and video-streaming routes (REQ-001…013)."""
import os
import shutil
import tempfile
from typing import Dict, List, Optional
from fastapi import APIRouter, File, HTTPException, Request, UploadFile
from fastapi.responses import FileResponse, StreamingResponse
from pydantic import BaseModel, Field
from backend import config, library, video
from backend import projects as project_store
from backend.api.common import project_or_404
router = APIRouter(prefix="/api/projects", tags=["projects"])
VIDEO_MEDIA = {".mp4": "video/mp4", ".m4v": "video/mp4", ".mkv": "video/x-matroska",
".mov": "video/quicktime", ".webm": "video/webm", ".avi": "video/x-msvideo"}
STREAM_CHUNK = 1024 * 512
class ClassSpec(BaseModel):
name: str
prompt: Optional[str] = None
class ProjectRequest(BaseModel):
name: str
label_type: str = "bbox"
video_root: str
classes: List[ClassSpec] = Field(default_factory=list)
val_every: int = 5
class ProjectPatch(BaseModel):
prompts: Optional[Dict[int, str]] = None
val_every: Optional[int] = None
video_root: Optional[str] = None
@router.get("")
def list_projects() -> dict:
return {"projects": project_store.listing(), "video_root_default": config.VIDEO_ROOT}
@router.post("")
def create_project(request: ProjectRequest) -> dict:
try:
return project_store.create(
name=request.name,
label_type=request.label_type,
video_root=request.video_root,
classes=[item.model_dump() for item in request.classes],
val_every=request.val_every,
)
except project_store.ProjectError as exc:
raise HTTPException(400, str(exc))
@router.get("/{project_id}")
def read_project(project_id: int) -> dict:
return project_or_404(project_id)
@router.patch("/{project_id}")
def patch_project(project_id: int, request: ProjectPatch) -> dict:
try:
return project_store.update(project_id, prompts=request.prompts,
val_every=request.val_every,
video_root=request.video_root)
except project_store.ProjectError as exc:
raise HTTPException(400, str(exc))
@router.post("/{project_id}/base-model")
async def upload_base_model(project_id: int, file: UploadFile = File(...)) -> dict:
if not (file.filename or "").endswith(".pt"):
raise HTTPException(400, "Base model must be a .pt checkpoint")
# Staged to a temp file first: reading the classes can fail, and a rejected
# upload must not leave a broken model.pt in the project.
with tempfile.NamedTemporaryFile(suffix=".pt", delete=False) as staged:
shutil.copyfileobj(file.file, staged)
staged_path = staged.name
await file.close()
try:
return project_store.set_base_model(project_id, staged_path)
except project_store.ProjectError as exc:
raise HTTPException(400, str(exc))
finally:
os.unlink(staged_path)
@router.post("/{project_id}/secondary-model")
async def upload_secondary_model(project_id: int, file: UploadFile = File(...)) -> dict:
project_or_404(project_id)
if not file.filename.endswith(".pt"):
raise HTTPException(400, "The secondary model must be a .pt file")
with tempfile.NamedTemporaryFile(suffix=".pt", delete=False) as staged:
shutil.copyfileobj(file.file, staged)
staged_path = staged.name
await file.close()
try:
return project_store.set_secondary_model(project_id, staged_path, name=file.filename)
except project_store.ProjectError as exc:
raise HTTPException(400, str(exc))
finally:
os.unlink(staged_path)
@router.post("/{project_id}/classes")
def add_class(project_id: int, request: ClassSpec) -> dict:
try:
return project_store.add_class(project_id, request.name, request.prompt)
except project_store.ProjectError as exc:
raise HTTPException(400, str(exc))
@router.delete("/{project_id}/classes/{class_id}")
def delete_class(project_id: int, class_id: int) -> dict:
try:
return project_store.delete_class(project_id, class_id)
except project_store.ProjectError as exc:
raise HTTPException(400, str(exc))
@router.delete("/{project_id}")
def delete_project(project_id: int) -> dict:
return {"deleted": project_store.delete(project_id)}
@router.get("/{project_id}/library")
def list_library(project_id: int) -> dict:
project = project_or_404(project_id)
try:
return {"video_root": project["video_root"],
"dates": library.list_dates(project["video_root"])}
except library.LibraryError as exc:
raise HTTPException(400, str(exc))
@router.get("/{project_id}/library/{date}")
def list_library_date(project_id: int, date: str) -> dict:
project = project_or_404(project_id)
try:
return {"date": date,
"videos": library.list_videos(project["video_root"], date, project_id)}
except library.LibraryError as exc:
raise HTTPException(400, str(exc))
@router.get("/{project_id}/video/info")
def video_info(project_id: int, rel: str) -> dict:
project = project_or_404(project_id)
try:
path = library.resolve(project["video_root"], rel)
info = dict(video.probe(path))
except (library.LibraryError, video.VideoError) as exc:
raise HTTPException(400, str(exc))
info.update({"rel": rel, "batch_label": library.batch_label(os.path.basename(path)),
"date_label": rel.split("/", 1)[0]})
return info
@router.get("/{project_id}/video")
def stream_video(project_id: int, rel: str, request: Request):
"""Serve a video with Range support so the player can seek (REQ-013)."""
project = project_or_404(project_id)
try:
path = library.resolve(project["video_root"], rel)
except library.LibraryError as exc:
raise HTTPException(404, str(exc))
media = VIDEO_MEDIA.get(os.path.splitext(path)[1].lower(), "application/octet-stream")
size = os.path.getsize(path)
header = request.headers.get("range")
if not header or not header.startswith("bytes="):
return FileResponse(path, media_type=media, headers={"Accept-Ranges": "bytes"})
start_text, _, end_text = header[6:].partition("-")
start = int(start_text) if start_text else 0
end = int(end_text) if end_text else size - 1
start, end = max(0, start), min(end, size - 1)
if start > end:
raise HTTPException(416, "Requested range is not satisfiable")
def chunks():
with open(path, "rb") as handle:
handle.seek(start)
remaining = end - start + 1
while remaining > 0:
block = handle.read(min(STREAM_CHUNK, remaining))
if not block:
break
remaining -= len(block)
yield block
return StreamingResponse(
chunks(), status_code=206, media_type=media,
headers={
"Content-Range": f"bytes {start}-{end}/{size}",
"Accept-Ranges": "bytes",
"Content-Length": str(end - start + 1),
},
)
+91
View File
@@ -0,0 +1,91 @@
"""Annotation and review-state routes (REQ-040…045)."""
from typing import Any, Dict, List, Optional
from fastapi import APIRouter, HTTPException
from pydantic import BaseModel
from backend import batches as batch_store
from backend import review as review_store
router = APIRouter(tags=["review"])
class AnnotationRequest(BaseModel):
class_id: int = 0
geometry: Dict[str, Any]
class AnnotationPatch(BaseModel):
class_id: Optional[int] = None
geometry: Optional[Dict[str, Any]] = None
class StatusRequest(BaseModel):
status: str
class AssistRequest(BaseModel):
box: List[float]
class_id: int = 0
threshold: float = 0.5
@router.get("/api/frames/{frame_id}/annotations")
def list_annotations(frame_id: int) -> dict:
target = review_store.frame(frame_id)
if target is None:
raise HTTPException(404, "No such frame")
return {
"frame": {
"id": target["id"], "batch_id": target["batch_id"], "idx": target["idx"],
"width": target["width"], "height": target["height"],
"review_status": target["review_status"], "label_type": target["label_type"],
},
"annotations": review_store.listing(frame_id),
}
@router.post("/api/frames/{frame_id}/annotations")
def add_annotation(frame_id: int, request: AnnotationRequest) -> dict:
try:
return review_store.add(frame_id, request.class_id, request.geometry)
except review_store.ReviewError as exc:
raise HTTPException(400, str(exc))
@router.patch("/api/annotations/{annotation_id}")
def patch_annotation(annotation_id: int, request: AnnotationPatch) -> dict:
try:
return review_store.update(annotation_id, request.class_id, request.geometry)
except review_store.ReviewError as exc:
raise HTTPException(400, str(exc))
@router.delete("/api/annotations/{annotation_id}")
def delete_annotation(annotation_id: int) -> dict:
return {"deleted": review_store.delete(annotation_id)}
@router.post("/api/frames/{frame_id}/assist")
def assist(frame_id: int, request: AssistRequest) -> dict:
try:
return review_store.assist(frame_id, request.box, request.class_id,
request.threshold)
except review_store.ReviewError as exc:
raise HTTPException(400, str(exc))
@router.post("/api/frames/{frame_id}/status")
def set_status(frame_id: int, request: StatusRequest) -> dict:
try:
return review_store.set_status(frame_id, request.status)
except review_store.ReviewError as exc:
raise HTTPException(400, str(exc))
@router.get("/api/batches/{batch_id}/next-pending")
def next_pending(batch_id: int, after_idx: int = -1) -> dict:
if batch_store.get(batch_id) is None:
raise HTTPException(404, "No such batch")
return {"frame_id": review_store.next_pending(batch_id, after_idx)}
+241
View File
@@ -0,0 +1,241 @@
"""The auto-annotation job: SAM3 over every frame of a batch (REQ-030…034).
Detection itself is `labeling.label_image`, unchanged — one `set_image` per
frame with the class prompts looped over that cached state, then greedy IoU NMS
across prompts. This module's job is only to turn its output into rows and to
keep the user's own corrections out of the way.
"""
import os
from typing import List, Optional
from backend import batches, db, jobs, labeling, projects, review
from backend.batches import BatchError
DEFAULT_THRESHOLD = 0.35
DEFAULT_IOU = 0.8
def start(batch_id: int, threshold: float = DEFAULT_THRESHOLD,
iou_threshold: float = DEFAULT_IOU, min_box_frac: float = 0.0,
resume: bool = False, engine: str = "sam3",
engines: Optional[List[str]] = None,
class_ids: Optional[List[int]] = None,
engine_classes: Optional[dict[str, List[str]]] = None) -> dict:
batch = batches.get(batch_id)
if batch is None:
raise batches.BatchError("No such batch")
if batch["frame_count"] == 0:
raise batches.BatchError("This batch has no frames yet")
active_engines = engines if (engines and len(engines) > 0) else [engine]
job = jobs.create(
"autolabel",
params={"batch_id": batch_id, "threshold": threshold,
"iou_threshold": iou_threshold, "min_box_frac": min_box_frac,
"resume": resume, "engine": active_engines[0], "engines": active_engines,
"class_ids": class_ids, "engine_classes": engine_classes},
project_id=batch["project_id"],
batch_id=batch_id,
message=f"{batch['date_label']}/{batch['batch_label']} ({'+'.join(e.upper() for e in active_engines)})",
)
return job.to_dict()
def _geometries(detection, width: int, height: int, label_type: str) -> List[dict]:
if label_type == "bbox":
x0, y0, x1, y1 = detection.box
return [review.bbox(x0 / width, y0 / height, x1 / width, y1 / height)]
shapes = []
for points in review.mask_to_polygons(detection.mask):
if len(points) >= 3:
shapes.append(review.polygon([(x / width, y / height) for x, y in points]))
return shapes
@jobs.handler("autolabel")
def _run_autolabel(job) -> None:
batch = batches.get(job.params["batch_id"])
if batch is None:
raise batches.BatchError("The batch disappeared before labeling started")
project = projects.get(batch["project_id"])
raw_active = job.params.get("engines") or [job.params.get("engine", "sam3")]
expanded_engines = []
for eng in raw_active:
if eng == "both":
expanded_engines.extend(["base_model", "secondary_model"])
elif eng == "sam3+model1":
expanded_engines.extend(["sam3", "base_model"])
else:
expanded_engines.append(eng)
expanded_engines = list(dict.fromkeys(expanded_engines))
frames = batches.frames(batch["id"])
batches.set_status(batch["id"], "labeling")
job.progress(0, len(frames))
skip = review.frames_with_auto(batch["id"]) if job.params.get("resume") else set()
if skip:
job.log(f"Resuming: skipping {len(skip)} frame(s) that already have automatic shapes")
directory = batches.frames_dir(batch["project_slug"], batch["id"])
written = 0
attempted = 0
failures = []
from ultralytics import YOLO
m1_path = projects.training_start_point(project)
with db.cursor() as cur:
cur.execute("SELECT weights_path FROM model_versions WHERE project_id = ? ORDER BY version DESC LIMIT 1", (project["id"],))
row = cur.fetchone()
if row and os.path.isfile(row[0]):
m1_path = row[0]
yolo_models = {}
if "base_model" in expanded_engines or "yolo" in expanded_engines:
job.log(f"Loading Base/Trained Model: {os.path.basename(m1_path)}...")
yolo_models["base_model"] = YOLO(m1_path)
if "secondary_model" in expanded_engines:
m2_path = project["secondary_model_path"] if (project.get("secondary_model_path") and os.path.isfile(project["secondary_model_path"])) else m1_path
label_name = project.get("secondary_model_name") or os.path.basename(m2_path)
job.log(f"Loading Secondary Model: {label_name}...")
yolo_models["secondary_model"] = YOLO(m2_path)
allowed_class_ids = set(job.params["class_ids"]) if job.params.get("class_ids") is not None else None
engine_classes = job.params.get("engine_classes") or {}
sam3_target_classes = []
if "sam3" in expanded_engines:
sam3_classes = engine_classes.get("sam3")
if sam3_classes is not None:
allowed_set = {c.strip().lower() for c in sam3_classes}
sam3_target_classes = [
c for c in project["classes"]
if c["name"].strip().lower() in allowed_set or c["prompt"].strip().lower() in allowed_set
]
else:
sam3_target_classes = [
c for c in project["classes"]
if (allowed_class_ids is None or c["class_id"] in allowed_class_ids)
]
prompts = [c["prompt"] for c in sam3_target_classes]
if prompts:
from backend.sam3_engine import engine_is_loaded, get_engine
if not engine_is_loaded():
job.log("Loading SAM3 (the first run downloads ~3.4 GB from HuggingFace)…")
engine = get_engine()
job.log(f"SAM3 ready on {engine.device}; prompts: {', '.join(prompts)}")
else:
job.log("SAM3 selected but 0 prompts match class filter.")
name_to_class_id = {item["name"].strip().lower(): item["class_id"] for item in project["classes"]}
conf = job.params.get("threshold", DEFAULT_THRESHOLD)
iou_thresh = job.params.get("iou_threshold", DEFAULT_IOU)
job.log(f"Starting multi-engine auto-labeling ({', '.join(expanded_engines)})...")
for index, frame in enumerate(frames):
if job.cancelled:
job.log(f"Cancelled after {index} frame(s)")
break
if frame["id"] in skip:
job.progress(index + 1, len(frames))
continue
attempted += 1
try:
frame_file = os.path.join(directory, frame["filename"])
all_raw_detections = []
for eng_key, y_model in yolo_models.items():
allowed_for_eng = engine_classes.get(eng_key)
if allowed_for_eng is not None and len(allowed_for_eng) == 0:
continue
results = y_model.predict(frame_file, conf=conf, verbose=False)
if results and len(results) > 0:
model_names = results[0].names
for box in results[0].boxes:
cls_idx = int(box.cls[0].item())
cls_name = str(model_names.get(cls_idx, cls_idx)).strip().lower()
if allowed_for_eng is not None and cls_name not in [c.strip().lower() for c in allowed_for_eng]:
continue
if cls_name not in name_to_class_id:
try:
updated_proj = projects.add_class(project["id"], {"name": cls_name, "prompt": cls_name})
project["classes"] = updated_proj["classes"]
name_to_class_id = {item["name"].strip().lower(): item["class_id"] for item in project["classes"]}
except Exception:
pass
if cls_name in name_to_class_id:
target_class_id = name_to_class_id[cls_name]
else:
continue
score = float(box.conf[0].item())
xyxyn = box.xyxyn[0].tolist()
all_raw_detections.append(labeling.Detection(
class_id=target_class_id,
class_name=cls_name,
box=[xyxyn[0]*frame["width"], xyxyn[1]*frame["height"], xyxyn[2]*frame["width"], xyxyn[3]*frame["height"]],
score=score,
mask=None
))
if "sam3" in expanded_engines and sam3_target_classes:
prompts = [c["prompt"] for c in sam3_target_classes]
res = labeling.label_image(
frame_file, frame["filename"], prompts, conf,
iou_threshold=iou_thresh, min_box_frac=job.params.get("min_box_frac", 0.0)
)
if not res.error and res.detections:
for det in res.detections:
if 0 <= det.class_id < len(sam3_target_classes):
real_cls = sam3_target_classes[det.class_id]
det.class_id = real_cls["class_id"]
det.class_name = real_cls["name"]
all_raw_detections.append(det)
kept = labeling.deduplicate(all_raw_detections, iou_threshold=iou_thresh)
items = []
for det in kept:
if project["label_type"] == "bbox" or det.mask is None:
geom = review.bbox(det.box[0]/frame["width"], det.box[1]/frame["height"], det.box[2]/frame["width"], det.box[3]/frame["height"])
items.append({"class_id": det.class_id, "geometry": geom, "score": det.score})
else:
for geometry in _geometries(det, frame["width"], frame["height"], project["label_type"]):
items.append({"class_id": det.class_id, "geometry": geometry, "score": det.score})
review.replace_auto(frame["id"], items)
written += len(items)
job.progress(index + 1, len(frames), f"{frame['filename']}: {len(items)} shape(s)")
except Exception as exc:
failures.append(str(exc))
job.log(f"[ERROR] {frame['filename']}: {exc}")
job.progress(index + 1, len(frames))
# "Every frame failed" is not a finished job with no findings — it is a
# broken run, and reporting `done` for it would be the system lying about
# its own state. An empty frame is fine (REQ-033); an errored one is not.
if attempted and len(failures) == attempted:
batches.set_status(batch["id"], "failed")
raise BatchError(f"All {attempted} frame(s) failed. First error: {failures[0]}")
batches.set_status(batch["id"], "reviewing")
_reset_reviewed(batch["id"])
if failures:
job.log(f"{len(failures)} of {attempted} frame(s) failed — see the errors above")
job.log(f"Wrote {written} shape(s) across {attempted - len(failures)} frame(s)")
def _reset_reviewed(batch_id: int) -> None:
"""Approvals were given against the previous labels, so a re-run puts those
frames back in the queue. Manual shapes stay; the sign-off does not."""
with db.cursor() as cur:
cur.execute(
"UPDATE frames SET review_status = 'pending' WHERE batch_id = ? "
"AND review_status = 'approved'",
(batch_id,),
)
+253
View File
@@ -0,0 +1,253 @@
"""Batch lifecycle: one trimmed range of one video, turned into frames.
A batch is the unit of work everything downstream hangs off — auto-annotation,
review, and the merge into the master dataset all address a batch. The same
video can produce many batches with different ranges (REQ-023).
extracting -> extracted -> labeling -> reviewing -> approved -> merged
\\-> failed
"""
import os
import time
from typing import List, Optional
from PIL import Image
from backend import config, db, jobs, library, projects, video
class BatchError(Exception):
pass
def batch_dir(project_slug: str, batch_id: int) -> str:
return os.path.join(config.project_dir(project_slug), "batches", str(batch_id))
def frames_dir(project_slug: str, batch_id: int) -> str:
return os.path.join(batch_dir(project_slug, batch_id), "frames")
def create(project_id: int, rel: str, start_sec: float, end_sec: float, fps: float) -> dict:
"""Register a batch and queue its extraction job (REQ-020…022)."""
project = projects.get(project_id)
if project is None:
raise BatchError("No such project")
try:
video_path = library.resolve(project["video_root"], rel)
except library.LibraryError as exc:
raise BatchError(str(exc))
try:
info = video.probe(video_path)
except video.VideoError as exc:
raise BatchError(str(exc))
end_sec = min(end_sec, info["duration"]) if info["duration"] else end_sec
if end_sec <= start_sec:
raise BatchError("The end of the range must be after its start")
if fps <= 0:
raise BatchError("fps must be greater than 0")
date_label, filename = rel.split("/", 1)
with db.cursor() as cur:
cur.execute(
"""INSERT INTO batches (project_id, video_path, date_label, batch_label,
start_sec, end_sec, fps, status, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?, 'extracting', ?)""",
(project_id, video_path, date_label, library.batch_label(filename),
float(start_sec), float(end_sec), float(fps), time.time()),
)
batch_id = cur.lastrowid
jobs.create(
"extract",
params={"batch_id": batch_id},
project_id=project_id,
batch_id=batch_id,
message=f"{date_label}/{library.batch_label(filename)}",
)
return get(batch_id)
def get(batch_id: int) -> Optional[dict]:
with db.cursor() as cur:
cur.execute(
"""SELECT b.*, p.slug AS project_slug, p.name AS project_name
FROM batches b JOIN projects p ON p.id = b.project_id WHERE b.id = ?""",
(batch_id,),
)
row = cur.fetchone()
if row is None:
return None
return _row_to_dict(cur, row)
def listing(project_id: int) -> List[dict]:
with db.cursor() as cur:
cur.execute(
"""SELECT b.*, p.slug AS project_slug, p.name AS project_name
FROM batches b JOIN projects p ON p.id = b.project_id
WHERE b.project_id = ? ORDER BY b.created_at DESC""",
(project_id,),
)
return [_row_to_dict(cur, row) for row in cur.fetchall()]
def _row_to_dict(cur, row) -> dict:
cur.execute(
"""SELECT review_status, COUNT(*) FROM frames WHERE batch_id = ?
GROUP BY review_status""",
(row["id"],),
)
review = {"pending": 0, "approved": 0, "rejected": 0}
for status, count in cur.fetchall():
review[status] = count
cur.execute(
"""SELECT COUNT(*) FROM annotations a JOIN frames f ON f.id = a.frame_id
WHERE f.batch_id = ?""",
(row["id"],),
)
annotation_count = cur.fetchone()[0]
return {
"id": row["id"],
"project_id": row["project_id"],
"project_slug": row["project_slug"],
"project_name": row["project_name"],
"video_path": row["video_path"],
"date_label": row["date_label"],
"batch_label": row["batch_label"],
"start_sec": row["start_sec"],
"end_sec": row["end_sec"],
"fps": row["fps"],
"status": row["status"],
"frame_count": row["frame_count"],
"created_at": row["created_at"],
"merged_at": row["merged_at"],
"review": review,
"reviewed": review["approved"] + review["rejected"],
"annotation_count": annotation_count,
}
def frames(batch_id: int) -> List[dict]:
with db.cursor() as cur:
cur.execute(
"""SELECT f.*, (SELECT COUNT(*) FROM annotations a WHERE a.frame_id = f.id)
AS annotation_count
FROM frames f WHERE f.batch_id = ? ORDER BY f.idx""",
(batch_id,),
)
return [dict(row) for row in cur.fetchall()]
def frame_path(frame_id: int) -> Optional[str]:
with db.cursor() as cur:
cur.execute(
"""SELECT f.filename, b.id AS batch_id, p.slug
FROM frames f
JOIN batches b ON b.id = f.batch_id
JOIN projects p ON p.id = b.project_id
WHERE f.id = ?""",
(frame_id,),
)
row = cur.fetchone()
if row is None:
return None
return os.path.join(frames_dir(row["slug"], row["batch_id"]), row["filename"])
def set_status(batch_id: int, status: str) -> None:
with db.cursor() as cur:
cur.execute("UPDATE batches SET status = ? WHERE id = ?", (status, batch_id))
@jobs.handler("extract")
def _run_extract(job) -> None:
batch = get(job.params["batch_id"])
if batch is None:
raise BatchError("The batch disappeared before extraction started")
out_dir = frames_dir(batch["project_slug"], batch["id"])
expected = video.frame_count(batch["start_sec"], batch["end_sec"], batch["fps"])
job.log(f"Extracting {expected} frame(s) at {batch['fps']} fps from "
f"{batch['date_label']}/{batch['batch_label']} "
f"[{batch['start_sec']:.1f}s – {batch['end_sec']:.1f}s]")
job.progress(0, expected)
try:
names = video.extract_frames(
batch["video_path"], out_dir,
batch["start_sec"], batch["end_sec"], batch["fps"],
on_progress=lambda written: job.progress(written, expected),
should_stop=lambda: job.cancelled,
)
except video.VideoError as exc:
set_status(batch["id"], "failed")
raise BatchError(str(exc))
if not names:
set_status(batch["id"], "failed")
raise BatchError("ffmpeg produced no frames for that range")
with Image.open(os.path.join(out_dir, names[0])) as first:
width, height = first.size
with db.cursor() as cur:
cur.executemany(
"INSERT OR IGNORE INTO frames (batch_id, idx, filename, width, height) "
"VALUES (?, ?, ?, ?, ?)",
[(batch["id"], index, name, width, height) for index, name in enumerate(names)],
)
cur.execute("UPDATE batches SET frame_count = ?, status = 'extracted' WHERE id = ?",
(len(names), batch["id"]))
job.progress(len(names), len(names))
job.log(f"Extracted {len(names)} frame(s) at {width}×{height}")
def update(batch_id: int, patch: dict) -> dict:
batch = get(batch_id)
if batch is None:
raise BatchError("No such batch")
fields = []
args = []
if "batch_label" in patch and patch["batch_label"] is not None:
fields.append("batch_label = ?")
args.append(str(patch["batch_label"]).strip())
if "date_label" in patch and patch["date_label"] is not None:
fields.append("date_label = ?")
args.append(str(patch["date_label"]).strip())
if "status" in patch and patch["status"] is not None:
fields.append("status = ?")
args.append(str(patch["status"]).strip())
if fields:
args.append(batch_id)
with db.cursor() as cur:
cur.execute(f"UPDATE batches SET {', '.join(fields)} WHERE id = ?", args)
return get(batch_id)
def delete(batch_id: int) -> bool:
import shutil
batch = get(batch_id)
if batch is None:
return False
with db.cursor() as cur:
cur.execute("DELETE FROM batches WHERE id = ?", (batch_id,))
shutil.rmtree(batch_dir(batch["project_slug"], batch_id), ignore_errors=True)
return True
def approve_all_frames(batch_id: int) -> int:
with db.cursor() as cur:
cur.execute("UPDATE frames SET review_status = 'approved' WHERE batch_id = ?", (batch_id,))
return cur.rowcount
+39
View File
@@ -0,0 +1,39 @@
"""Paths and settings. Everything is environment-driven so the same image runs
locally and in Docker without code changes (REQ-072)."""
import os
from dotenv import load_dotenv
load_dotenv()
# huggingface_hub reads HF_TOKEN / HUGGING_FACE_HUB_TOKEN; accept either name in
# .env so pasting a token under the obvious name just works.
if os.environ.get("HF_TOKEN") and not os.environ.get("HUGGING_FACE_HUB_TOKEN"):
os.environ["HUGGING_FACE_HUB_TOKEN"] = os.environ["HF_TOKEN"]
REPO_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
DATA_DIR = os.path.abspath(os.environ.get("APP_DATA_DIR", os.path.join(REPO_ROOT, "data")))
PROJECTS_DIR = os.path.join(DATA_DIR, "projects")
DB_PATH = os.path.join(DATA_DIR, "app.db")
# Where the video archive is mounted. Projects store a path relative to nothing —
# they store an absolute one — but this is the default the UI starts browsing from.
VIDEO_ROOT = os.path.abspath(os.environ.get("VIDEO_ARCHIVE", os.path.join(DATA_DIR, "archive")))
# Vite dev server needs cross-origin access; in Docker nginx proxies /api and
# this is irrelevant.
CORS_ORIGINS = [
origin.strip()
for origin in os.environ.get("CORS_ORIGINS", "http://localhost:5173").split(",")
if origin.strip()
]
def ensure_dirs() -> None:
os.makedirs(PROJECTS_DIR, exist_ok=True)
def project_dir(slug: str) -> str:
return os.path.join(PROJECTS_DIR, slug)
+269
View File
@@ -0,0 +1,269 @@
"""The master dataset: approved frames merged in, batch after batch (REQ-050…054).
The one rule that matters here is the stable val split. A frame's membership is
recorded once in `dataset_items` and never revised, so an image that was in
`val` for the last comparison is still in `val` for the next one. Without that,
a rising mAP could just mean an easier val set.
Label files are plain YOLO:
detect class_id cx cy w h (normalized)
segment class_id x1 y1 x2 y2 … (normalized polygon)
"""
import os
import shutil
import time
from backend import batches, config, db, jobs, projects, review
class DatasetError(Exception):
pass
def dataset_dir(project_slug: str) -> str:
return os.path.join(config.project_dir(project_slug), "dataset")
def approve(batch_id: int) -> dict:
"""Sign a batch off and queue its merge (REQ-045, REQ-050)."""
batch = batches.get(batch_id)
if batch is None:
raise DatasetError("No such batch")
if batch["status"] == "merged":
raise DatasetError("This batch is already in the master dataset")
if batch["review"]["pending"] > 0:
raise DatasetError(
f"{batch['review']['pending']} frame(s) still need a decision before this "
"batch can be approved"
)
if batch["review"]["approved"] == 0:
raise DatasetError("Every frame was rejected — there is nothing to merge")
batches.set_status(batch_id, "approved")
job = jobs.create(
"merge",
params={"batch_id": batch_id},
project_id=batch["project_id"],
batch_id=batch_id,
message=f"{batch['date_label']}/{batch['batch_label']}",
)
return job.to_dict()
def _label_line(class_id: int, geometry: dict, label_type: str) -> str:
if label_type == "bbox":
x0, y0, x1, y1 = review.to_box(geometry)
return (f"{class_id} {(x0 + x1) / 2:.6f} {(y0 + y1) / 2:.6f} "
f"{x1 - x0:.6f} {y1 - y0:.6f}")
points = geometry["points"]
if geometry["type"] == "bbox":
x0, y0, x1, y1 = geometry["points"]
points = [[x0, y0], [x1, y0], [x1, y1], [x0, y1]]
coords = " ".join(f"{value:.6f}" for point in points for value in point)
return f"{class_id} {coords}"
def _next_split(cur, project_id: int, val_every: int) -> str:
"""Continue the every-Nth pattern from wherever the last merge left off."""
if val_every <= 0:
return "train"
cur.execute("SELECT COUNT(*) FROM dataset_items WHERE project_id = ?", (project_id,))
position = cur.fetchone()[0]
return "val" if position % val_every == val_every - 1 else "train"
def write_data_yaml(project: dict, batch_ids: list = None) -> str:
"""Rebuild data.yaml from the project's classes (REQ-051)."""
root = dataset_dir(project["slug"])
os.makedirs(root, exist_ok=True)
counts = summary(project["id"])["splits"]
names = ", ".join(f"'{item['name']}'" for item in project["classes"])
if batch_ids:
with db.cursor() as cur:
placeholders = ",".join("?" for _ in batch_ids)
cur.execute(
f"""SELECT d.image_rel, d.split FROM dataset_items d
JOIN frames f ON f.id = d.frame_id
WHERE d.project_id = ? AND f.batch_id IN ({placeholders})""",
[project["id"]] + list(batch_ids),
)
rows = cur.fetchall()
train_files = [row[0] for row in rows if row[1] == "train"]
val_files = [row[0] for row in rows if row[1] == "val"] or train_files
train_txt = os.path.join(root, "selected_train.txt")
val_txt = os.path.join(root, "selected_val.txt")
with open(train_txt, "w", encoding="utf-8") as handle:
handle.write("\n".join(os.path.join(root, rel) for rel in train_files) + "\n")
with open(val_txt, "w", encoding="utf-8") as handle:
handle.write("\n".join(os.path.join(root, rel) for rel in val_files) + "\n")
path = os.path.join(root, "selected_data.yaml")
with open(path, "w", encoding="utf-8") as handle:
handle.write(f"path: {root}\n")
handle.write(f"train: {train_txt}\n")
handle.write(f"val: {val_txt}\n\n")
handle.write(f"nc: {len(project['classes'])}\n")
handle.write(f"names: [{names}]\n")
return path
path = os.path.join(root, "data.yaml")
with open(path, "w", encoding="utf-8") as handle:
handle.write(f"path: {root}\n")
handle.write("train: images/train\n")
handle.write(f"val: images/{'val' if counts['val'] > 0 else 'train'}\n\n")
handle.write(f"nc: {len(project['classes'])}\n")
handle.write(f"names: [{names}]\n")
return path
def summary(project_id: int) -> dict:
with db.cursor() as cur:
cur.execute(
"SELECT split, COUNT(*) FROM dataset_items WHERE project_id = ? GROUP BY split",
(project_id,),
)
splits = {"train": 0, "val": 0}
for split, count in cur.fetchall():
splits[split] = count
cur.execute(
"""SELECT b.id, b.date_label, b.batch_label, b.merged_at,
COUNT(d.id) AS images
FROM batches b
LEFT JOIN frames f ON f.batch_id = b.id
LEFT JOIN dataset_items d ON d.frame_id = f.id
WHERE b.project_id = ? AND b.status = 'merged'
GROUP BY b.id ORDER BY b.merged_at""",
(project_id,),
)
merged = [dict(row) for row in cur.fetchall()]
return {"splits": splits, "total": splits["train"] + splits["val"], "batches": merged}
def drop_class_from_labels(project: dict, class_id: int) -> dict:
"""Rewrite every label file on disk after a class is deleted (REQ-007).
Two edits per file: lines of the deleted class go, and every id above it
comes down by one. Skipping this would leave `2` in old files meaning a
class that is now `1` — labels that quietly name the wrong thing are worse
than labels that are missing.
"""
root = dataset_dir(project["slug"])
with db.cursor() as cur:
cur.execute("SELECT label_rel FROM dataset_items WHERE project_id = ?",
(project["id"],))
label_files = [row[0] for row in cur.fetchall()]
rewritten = 0
dropped = 0
for rel in label_files:
path = os.path.join(root, rel)
if not os.path.isfile(path):
continue
with open(path, encoding="utf-8") as handle:
lines = handle.read().splitlines()
kept, touched = [], False
for line in lines:
if not line.strip():
continue
head, _, rest = line.partition(" ")
try:
current = int(head)
except ValueError:
kept.append(line)
continue
if current == class_id:
dropped += 1
touched = True
continue
if current > class_id:
current -= 1
touched = True
kept.append(f"{current} {rest}")
if touched:
# An emptied file stays as an empty file: the image is still a valid
# negative sample (REQ-033), it just has nothing on it any more.
with open(path, "w", encoding="utf-8") as handle:
handle.write("\n".join(kept) + ("\n" if kept else ""))
rewritten += 1
return {"label_files_rewritten": rewritten, "dataset_lines_removed": dropped}
def zip_path(project: dict) -> str:
"""Zip the master dataset for download (REQ-054)."""
root = dataset_dir(project["slug"])
if not os.path.isdir(os.path.join(root, "images")):
raise DatasetError("This project's dataset is still empty")
archive = os.path.join(config.project_dir(project["slug"]), "dataset")
return shutil.make_archive(archive, "zip", root)
@jobs.handler("merge")
def _run_merge(job) -> None:
batch = batches.get(job.params["batch_id"])
if batch is None:
raise DatasetError("The batch disappeared before the merge started")
project = projects.get(batch["project_id"])
root = dataset_dir(project["slug"])
for split in ("train", "val"):
os.makedirs(os.path.join(root, "images", split), exist_ok=True)
os.makedirs(os.path.join(root, "labels", split), exist_ok=True)
frames = [f for f in batches.frames(batch["id"]) if f["review_status"] == "approved"]
source_dir = batches.frames_dir(project["slug"], batch["id"])
job.progress(0, len(frames))
job.log(f"Merging {len(frames)} approved frame(s) into the master dataset")
added = {"train": 0, "val": 0}
skipped = 0
for index, frame in enumerate(frames):
if job.cancelled:
job.log(f"Cancelled after {index} frame(s)")
break
with db.cursor() as cur:
cur.execute("SELECT 1 FROM dataset_items WHERE frame_id = ?", (frame["id"],))
if cur.fetchone() is not None:
skipped += 1
job.progress(index + 1, len(frames))
continue
split = _next_split(cur, project["id"], project["val_every"])
stem = f"{batch['id']}__{os.path.splitext(frame['filename'])[0]}"
image_rel = f"images/{split}/{stem}.jpg"
label_rel = f"labels/{split}/{stem}.txt"
shutil.copyfile(os.path.join(source_dir, frame["filename"]),
os.path.join(root, image_rel))
lines = [_label_line(item["class_id"], item["geometry"], project["label_type"])
for item in review.listing(frame["id"])]
# An approved frame with nothing on it is a negative sample, and an
# empty .txt is how YOLO spells that (REQ-033).
with open(os.path.join(root, label_rel), "w", encoding="utf-8") as handle:
handle.write("\n".join(lines) + ("\n" if lines else ""))
cur.execute(
"""INSERT INTO dataset_items (project_id, frame_id, split, image_rel,
label_rel, added_at)
VALUES (?, ?, ?, ?, ?, ?)""",
(project["id"], frame["id"], split, image_rel, label_rel, time.time()),
)
added[split] += 1
job.progress(index + 1, len(frames))
with db.cursor() as cur:
cur.execute("UPDATE batches SET status = 'merged', merged_at = ? WHERE id = ?",
(time.time(), batch["id"]))
path = write_data_yaml(projects.get(project["id"]))
totals = summary(project["id"])["splits"]
job.log(f"Added {added['train']} train / {added['val']} val"
+ (f", skipped {skipped} already merged" if skipped else ""))
job.log(f"Master dataset now {totals['train']} train / {totals['val']} val — {path}")
+178
View File
@@ -0,0 +1,178 @@
"""SQLite storage for metadata and status.
The split is deliberate: this database holds *what* and *where*, the disk holds
the pixels, the final YOLO labels, and the weights. A master dataset stays
trainable even if this file is deleted (REQ-006, REQ-054).
Connections are per-call rather than shared, because the job worker runs on its
own thread and SQLite connections are not safely shared across threads. WAL mode
lets that worker write while requests read.
"""
import os
import sqlite3
from contextlib import contextmanager
from typing import Iterator
from backend import config
SCHEMA = [
"""
CREATE TABLE IF NOT EXISTS projects (
id INTEGER PRIMARY KEY AUTOINCREMENT,
slug TEXT NOT NULL UNIQUE,
name TEXT NOT NULL,
label_type TEXT NOT NULL CHECK (label_type IN ('bbox', 'polygon')),
base_model_path TEXT,
base_model_kind TEXT CHECK (base_model_kind IN ('uploaded', 'pretrained', 'trained')),
video_root TEXT NOT NULL,
val_every INTEGER NOT NULL DEFAULT 5,
created_at REAL NOT NULL
)
""",
"""
CREATE TABLE IF NOT EXISTS project_classes (
id INTEGER PRIMARY KEY AUTOINCREMENT,
project_id INTEGER NOT NULL REFERENCES projects(id) ON DELETE CASCADE,
class_id INTEGER NOT NULL,
name TEXT NOT NULL,
prompt TEXT NOT NULL,
UNIQUE (project_id, class_id)
)
""",
"""
CREATE TABLE IF NOT EXISTS batches (
id INTEGER PRIMARY KEY AUTOINCREMENT,
project_id INTEGER NOT NULL REFERENCES projects(id) ON DELETE CASCADE,
video_path TEXT NOT NULL,
date_label TEXT NOT NULL,
batch_label TEXT NOT NULL,
start_sec REAL NOT NULL,
end_sec REAL NOT NULL,
fps REAL NOT NULL,
status TEXT NOT NULL CHECK (status IN (
'extracting', 'extracted', 'labeling', 'reviewing',
'approved', 'merged', 'failed')),
frame_count INTEGER NOT NULL DEFAULT 0,
created_at REAL NOT NULL,
merged_at REAL
)
""",
"""
CREATE TABLE IF NOT EXISTS frames (
id INTEGER PRIMARY KEY AUTOINCREMENT,
batch_id INTEGER NOT NULL REFERENCES batches(id) ON DELETE CASCADE,
idx INTEGER NOT NULL,
filename TEXT NOT NULL,
width INTEGER NOT NULL,
height INTEGER NOT NULL,
review_status TEXT NOT NULL DEFAULT 'pending'
CHECK (review_status IN ('pending', 'approved', 'rejected')),
UNIQUE (batch_id, idx)
)
""",
"""
CREATE TABLE IF NOT EXISTS annotations (
id INTEGER PRIMARY KEY AUTOINCREMENT,
frame_id INTEGER NOT NULL REFERENCES frames(id) ON DELETE CASCADE,
class_id INTEGER NOT NULL,
geometry TEXT NOT NULL,
score REAL NOT NULL DEFAULT 1.0,
source TEXT NOT NULL CHECK (source IN ('auto', 'manual')),
created_at REAL NOT NULL
)
""",
"""
CREATE TABLE IF NOT EXISTS dataset_items (
id INTEGER PRIMARY KEY AUTOINCREMENT,
project_id INTEGER NOT NULL REFERENCES projects(id) ON DELETE CASCADE,
frame_id INTEGER NOT NULL REFERENCES frames(id) ON DELETE CASCADE UNIQUE,
split TEXT NOT NULL CHECK (split IN ('train', 'val')),
image_rel TEXT NOT NULL,
label_rel TEXT NOT NULL,
added_at REAL NOT NULL
)
""",
"""
CREATE TABLE IF NOT EXISTS model_versions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
project_id INTEGER NOT NULL REFERENCES projects(id) ON DELETE CASCADE,
version INTEGER NOT NULL,
weights_path TEXT NOT NULL,
parent_model_path TEXT,
metrics TEXT,
base_metrics TEXT,
created_at REAL NOT NULL,
UNIQUE (project_id, version)
)
""",
"""
CREATE TABLE IF NOT EXISTS jobs (
id INTEGER PRIMARY KEY AUTOINCREMENT,
project_id INTEGER REFERENCES projects(id) ON DELETE CASCADE,
batch_id INTEGER REFERENCES batches(id) ON DELETE CASCADE,
type TEXT NOT NULL CHECK (type IN ('extract', 'autolabel', 'merge', 'train')),
status TEXT NOT NULL CHECK (status IN (
'queued', 'running', 'done', 'failed', 'cancelled')),
params TEXT NOT NULL DEFAULT '{}',
progress INTEGER NOT NULL DEFAULT 0,
total INTEGER NOT NULL DEFAULT 0,
message TEXT NOT NULL DEFAULT '',
error TEXT NOT NULL DEFAULT '',
log TEXT NOT NULL DEFAULT '',
created_at REAL NOT NULL,
started_at REAL,
finished_at REAL
)
""",
"CREATE INDEX IF NOT EXISTS idx_frames_batch ON frames(batch_id, idx)",
"CREATE INDEX IF NOT EXISTS idx_annotations_frame ON annotations(frame_id)",
"CREATE INDEX IF NOT EXISTS idx_batches_project ON batches(project_id)",
"CREATE INDEX IF NOT EXISTS idx_jobs_project ON jobs(project_id, created_at)",
"CREATE INDEX IF NOT EXISTS idx_dataset_items_project ON dataset_items(project_id)",
]
def connect() -> sqlite3.Connection:
os.makedirs(os.path.dirname(config.DB_PATH), exist_ok=True)
connection = sqlite3.connect(config.DB_PATH, timeout=30.0)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA journal_mode = WAL")
connection.execute("PRAGMA foreign_keys = ON")
connection.execute("PRAGMA busy_timeout = 30000")
return connection
@contextmanager
def cursor() -> Iterator[sqlite3.Cursor]:
"""Transactional cursor: commits on success, rolls back on exception."""
connection = connect()
try:
with connection:
yield connection.cursor()
finally:
connection.close()
def migrate() -> None:
"""Create every table and index. Idempotent — safe on every startup."""
with cursor() as cur:
for statement in SCHEMA:
cur.execute(statement)
cur.execute("PRAGMA table_info(projects)")
cols = [column[1] for column in cur.fetchall()]
if "secondary_model_path" not in cols:
cur.execute("ALTER TABLE projects ADD COLUMN secondary_model_path TEXT")
if "secondary_model_name" not in cols:
cur.execute("ALTER TABLE projects ADD COLUMN secondary_model_name TEXT")
if "secondary_model_classes" not in cols:
cur.execute("ALTER TABLE projects ADD COLUMN secondary_model_classes TEXT")
def healthy() -> bool:
try:
with cursor() as cur:
cur.execute("SELECT 1")
return True
except sqlite3.Error:
return False
+64
View File
@@ -0,0 +1,64 @@
"""Base model versus new model, on the same val set (REQ-063).
Both models are validated against one `data.yaml`, so the numbers differ only
because the weights differ. Combined with the stable val split in `dataset.py`,
that is what makes "the model improved" a claim rather than a hope.
"""
from typing import Optional
class EvaluateError(Exception):
pass
def class_names(weights: str) -> list:
from ultralytics import YOLO
names = YOLO(weights).names
return [names[key] for key in sorted(names)] if isinstance(names, dict) else list(names)
def validate(weights: str, data_yaml: str, imgsz: int = 640, device=0,
batch: int = 8) -> dict:
from ultralytics import YOLO
metrics = YOLO(weights).val(
data=data_yaml, imgsz=imgsz, device=device, batch=batch,
split="val", plots=False, verbose=False,
)
box = metrics.box
return {
"map50": round(float(box.map50), 4),
"map50_95": round(float(box.map), 4),
"precision": round(float(box.mp), 4),
"recall": round(float(box.mr), 4),
}
def compare(base_weights: Optional[str], new_weights: str, data_yaml: str,
expected_classes: list, imgsz: int = 640, device=0,
batch: int = 8) -> dict:
"""Validate both models where that is meaningful, and say so when it is not.
A base model whose class list differs from the project's cannot be scored on
this dataset — its class ids mean something else. Reporting nothing beats
reporting a number that looks like a regression but is a mismatch.
"""
new_metrics = validate(new_weights, data_yaml, imgsz, device, batch)
base_metrics = None
skipped = None
if not base_weights:
skipped = "This project has no base model yet — nothing to compare against."
else:
try:
base_metrics = validate(base_weights, data_yaml, imgsz, device, batch)
except Exception as exc:
base_metrics, skipped = None, f"Could not evaluate base model: {exc}"
delta = None
if base_metrics:
delta = {key: round(new_metrics[key] - base_metrics[key], 4) for key in new_metrics}
return {"base": base_metrics, "new": new_metrics, "delta": delta, "skipped": skipped}
+66
View File
@@ -0,0 +1,66 @@
"""Training defaults derived from the machine this happens to be running on.
The point is REQ-062: moving to a bigger GPU should change the numbers in the
form, not the code. Everything here is a *default* — the user can override all
of it per run.
"""
from typing import Optional
SAM3_RESIDENT_GB = 3.9
SAM3_HEADROOM_GB = 0.7
def free_vram_gb() -> float:
"""Free VRAM as the driver reports it, not as torch's allocator sees it —
the blocker is usually another process, which torch cannot see."""
import torch
if not torch.cuda.is_available():
return 0.0
free, _total = torch.cuda.mem_get_info()
return round(free / (1024 ** 3), 2)
def detect() -> dict:
import torch
if not torch.cuda.is_available():
return {"device": "cpu", "gpu": None, "vram_gb": 0.0}
properties = torch.cuda.get_device_properties(0)
return {
"device": "cuda",
"gpu": properties.name,
"vram_gb": round(properties.total_memory / (1024 ** 3), 1),
}
def defaults(epochs: int = 50) -> dict:
"""Batch size and image size that should fit, given the VRAM we can see."""
info = detect()
vram = info["vram_gb"]
if info["device"] == "cpu":
settings = {"batch": 4, "imgsz": 512, "device": "cpu", "workers": 2}
note = "No GPU visible — training on CPU will be very slow."
elif vram < 8:
settings = {"batch": 8, "imgsz": 640, "device": 0, "workers": 2}
note = f"{vram} GB of VRAM: small batches, 640 px."
elif vram <= 16:
settings = {"batch": 32, "imgsz": 640, "device": 0, "workers": 8}
note = f"{vram} GB of VRAM: optimized batch 32, 640 px."
else:
settings = {"batch": 32, "imgsz": 768, "device": 0, "workers": 8}
note = f"{vram} GB of VRAM: room for larger batches and 768 px."
return {**info, **settings, "epochs": epochs, "note": note}
def resolve(overrides: Optional[dict] = None, epochs: int = 50) -> dict:
"""Defaults with the user's overrides applied on top."""
settings = defaults(epochs)
for key, value in (overrides or {}).items():
if value is not None and key in ("batch", "imgsz", "device", "epochs", "workers"):
settings[key] = value
return settings
+255
View File
@@ -0,0 +1,255 @@
"""Job queue: one worker thread, because there is one GPU (REQ-070).
Jobs are rows in SQLite, so the list survives a restart (REQ-071). A job that was
still running when the process died is marked failed at the next startup — it
cannot be resumed, and pretending otherwise would be worse than saying so.
Handlers register themselves by job type:
@jobs.handler("extract")
def _extract(job: Job) -> None:
...
A handler reports progress with `job.progress(i, n, "...")`, writes user-facing
lines with `job.log("...")`, and checks `job.cancelled` between units of work.
Raising anything marks the job failed with that exception on it.
"""
import json
import queue
import threading
import time
import traceback
from typing import Callable, Dict, List, Optional
from backend import db
MAX_LOG_LINES = 500
PROGRESS_FLUSH_SECONDS = 0.5
JOB_TYPES = ("extract", "autolabel", "merge", "train")
GPU_JOB_TYPES = ("autolabel", "train")
"""`extract` is ffmpeg and `merge` is file copying — neither touches the card,
so neither should be able to block an interactive assist."""
gpu_lock = threading.Lock()
"""Held for the duration of any GPU work. The job worker takes it around a
handler; the interactive assist route takes it around one SAM3 call. One card,
one holder (REQ-065)."""
class Job:
"""One queued unit of work. The database row is the source of truth; this
object is the handle a handler writes through."""
def __init__(self, row):
self.id: int = row["id"]
self.type: str = row["type"]
self.project_id: Optional[int] = row["project_id"]
self.batch_id: Optional[int] = row["batch_id"]
self.params: dict = json.loads(row["params"])
self.status: str = row["status"]
self.current: int = row["progress"]
self.total: int = row["total"]
self.message: str = row["message"]
self.error: str = row["error"]
self.lines: List[str] = row["log"].splitlines() if row["log"] else []
self.created_at: float = row["created_at"]
self.started_at: Optional[float] = row["started_at"]
self.finished_at: Optional[float] = row["finished_at"]
self._flushed_at = 0.0
# ---- what handlers call ------------------------------------------------
@property
def cancelled(self) -> bool:
return self.id in _cancelled
def progress(self, current: int, total: Optional[int] = None,
message: Optional[str] = None) -> None:
self.current = current
if total is not None:
self.total = total
if message is not None:
self.message = message
# Throttled: a 3000-frame job would otherwise write 3000 times.
if time.time() - self._flushed_at >= PROGRESS_FLUSH_SECONDS:
self.flush()
def log(self, message: str) -> None:
self.lines.append(f"[{time.strftime('%H:%M:%S')}] {message}")
if len(self.lines) > MAX_LOG_LINES:
del self.lines[: len(self.lines) - MAX_LOG_LINES]
self.flush()
def flush(self) -> None:
self._flushed_at = time.time()
with db.cursor() as cur:
cur.execute(
"""UPDATE jobs SET status = ?, progress = ?, total = ?, message = ?,
error = ?, log = ?, started_at = ?, finished_at = ?
WHERE id = ?""",
(self.status, self.current, self.total, self.message, self.error,
"\n".join(self.lines), self.started_at, self.finished_at, self.id),
)
# ---- serialisation -----------------------------------------------------
def to_dict(self) -> dict:
end = self.finished_at or time.time()
return {
"id": self.id,
"type": self.type,
"project_id": self.project_id,
"batch_id": self.batch_id,
"params": self.params,
"status": self.status,
"progress": self.current,
"total": self.total,
"message": self.message,
"error": self.error,
"log": self.lines,
"created_at": self.created_at,
"started_at": self.started_at,
"finished_at": self.finished_at,
"elapsed": end - (self.started_at or self.created_at),
}
_handlers: Dict[str, Callable[[Job], None]] = {}
_queue: "queue.Queue[int]" = queue.Queue()
_cancelled: set = set()
_worker: Optional[threading.Thread] = None
_worker_lock = threading.Lock()
def handler(job_type: str):
"""Register the function that runs jobs of this type."""
if job_type not in JOB_TYPES:
raise ValueError(f"Unknown job type: {job_type}")
def decorate(function: Callable[[Job], None]) -> Callable[[Job], None]:
_handlers[job_type] = function
return function
return decorate
def create(job_type: str, params: Optional[dict] = None, project_id: Optional[int] = None,
batch_id: Optional[int] = None, message: str = "") -> Job:
if job_type not in _handlers:
raise ValueError(f"No handler registered for job type: {job_type}")
with db.cursor() as cur:
cur.execute(
"""INSERT INTO jobs (project_id, batch_id, type, status, params, message, created_at)
VALUES (?, ?, ?, 'queued', ?, ?, ?)""",
(project_id, batch_id, job_type, json.dumps(params or {}), message, time.time()),
)
job_id = cur.lastrowid
job = get(job_id)
assert job is not None
_queue.put(job_id)
_ensure_worker()
return job
def get(job_id: int) -> Optional[Job]:
with db.cursor() as cur:
cur.execute("SELECT * FROM jobs WHERE id = ?", (job_id,))
row = cur.fetchone()
return Job(row) if row else None
def listing(project_id: Optional[int] = None, limit: int = 50) -> List[Job]:
with db.cursor() as cur:
if project_id is None:
cur.execute("SELECT * FROM jobs ORDER BY created_at DESC LIMIT ?", (limit,))
else:
cur.execute(
"SELECT * FROM jobs WHERE project_id = ? ORDER BY created_at DESC LIMIT ?",
(project_id, limit),
)
return [Job(row) for row in cur.fetchall()]
def running_types() -> List[str]:
"""Job types occupying the worker right now — the GPU is shared with the
interactive SAM3 calls the review editor makes."""
with db.cursor() as cur:
cur.execute("SELECT DISTINCT type FROM jobs WHERE status = 'running'")
return [row[0] for row in cur.fetchall()]
def cancel(job_id: int) -> bool:
job = get(job_id)
if job is None or job.status in ("done", "failed", "cancelled"):
return False
_cancelled.add(job_id)
if job.status == "queued":
# Never started, so no handler will notice the flag — close it out here.
job.status = "cancelled"
job.finished_at = time.time()
job.log("Cancelled before it started")
else:
job.log("Cancellation requested")
return True
def recover() -> int:
"""Close out jobs left behind by a previous process (REQ-071)."""
with db.cursor() as cur:
cur.execute(
"""UPDATE jobs SET status = 'failed', error = 'interrupted by a server restart',
finished_at = ?
WHERE status IN ('queued', 'running')""",
(time.time(),),
)
return cur.rowcount
def _ensure_worker() -> None:
global _worker
with _worker_lock:
if _worker is None or not _worker.is_alive():
_worker = threading.Thread(target=_worker_loop, name="job-worker", daemon=True)
_worker.start()
def _worker_loop() -> None:
while True:
job_id = _queue.get()
job = get(job_id)
if job is None:
continue
if job.id in _cancelled:
_finish(job, "cancelled")
continue
_run(job)
def _run(job: Job) -> None:
job.status = "running"
job.started_at = time.time()
job.flush()
try:
if job.type in GPU_JOB_TYPES:
with gpu_lock:
_handlers[job.type](job)
else:
_handlers[job.type](job)
except Exception as exc:
job.error = f"{type(exc).__name__}: {exc}"
job.log(f"FAILED: {job.error}")
job.log(traceback.format_exc().strip().splitlines()[-1])
_finish(job, "failed")
return
_finish(job, "cancelled" if job.cancelled else "done")
def _finish(job: Job, status: str) -> None:
_cancelled.discard(job.id)
job.status = status
job.finished_at = time.time()
job.message = ""
job.flush()
+81
View File
@@ -0,0 +1,81 @@
"""Run every class prompt against one frame and return the surviving instances.
Each class is its own prompt, so prompt index is class id. Prompts overlap in
practice ("sack" and "woven plastic sack" both fire on the same object), so
detections are deduplicated across prompts by IoU, keeping the higher-scoring
one (REQ-031).
The set_image-once-per-image rule lives in `sam3_engine.detect`, which this
calls — see the domain invariants in `../AGENTS.md`.
"""
from dataclasses import dataclass, field
from typing import List, Optional
from PIL import Image
from backend.sam3_engine import Detection, get_engine
@dataclass
class ImageResult:
image_path: str
rel_path: str
width: int
height: int
detections: List[Detection] = field(default_factory=list)
error: Optional[str] = None
def _iou(box_a: List[float], box_b: List[float]) -> float:
ax0, ay0, ax1, ay1 = box_a
bx0, by0, bx1, by1 = box_b
inter_w = max(0.0, min(ax1, bx1) - max(ax0, bx0))
inter_h = max(0.0, min(ay1, by1) - max(ay0, by0))
inter = inter_w * inter_h
if inter <= 0:
return 0.0
area_a = max(0.0, ax1 - ax0) * max(0.0, ay1 - ay0)
area_b = max(0.0, bx1 - bx0) * max(0.0, by1 - by0)
union = area_a + area_b - inter
return inter / union if union > 0 else 0.0
def deduplicate(detections: List[Detection], iou_threshold: float = 0.8) -> List[Detection]:
"""Greedy NMS across all prompts: highest score wins an overlapping region."""
kept: List[Detection] = []
for det in sorted(detections, key=lambda d: d.score, reverse=True):
if all(_iou(det.box, k.box) < iou_threshold for k in kept):
kept.append(det)
return kept
def label_image(
image_path: str,
rel_path: str,
prompts: List[str],
threshold: float,
iou_threshold: float = 0.8,
min_box_frac: float = 0.0,
) -> ImageResult:
"""Detect every prompt in one image and return the surviving instances."""
try:
image = Image.open(image_path).convert("RGB")
except Exception as exc: # unreadable/corrupt frame: report, don't abort the job
return ImageResult(image_path, rel_path, 0, 0, error=str(exc))
width, height = image.size
try:
detections = get_engine().detect(image, prompts, threshold)
except Exception as exc:
return ImageResult(image_path, rel_path, width, height, error=str(exc))
if min_box_frac > 0:
floor = width * height * min_box_frac
detections = [
d for d in detections
if (d.box[2] - d.box[0]) * (d.box[3] - d.box[1]) >= floor
]
return ImageResult(image_path, rel_path, width, height,
deduplicate(detections, iou_threshold))
+117
View File
@@ -0,0 +1,117 @@
"""The video archive, read as `<video_root>/<date>/<batch>.<ext>` (REQ-010…012).
Read-only by construction: this module only ever lists and stats files, never
writes into the user's recordings (REQ-074).
"""
import os
import re
from typing import List, Optional
from backend import config, db, video
class LibraryError(Exception):
pass
def _safe_join(video_root: str, *parts: str) -> str:
"""Join under the archive root, refusing anything that escapes it."""
root = os.path.realpath(video_root)
target = os.path.realpath(os.path.join(root, *parts))
if target != root and not target.startswith(root + os.sep):
raise LibraryError("Path is outside the video archive")
return target
def batch_label(filename: str) -> str:
"""`batch-4.mp4` -> `batch-4`. The filename is the batch's identity."""
return os.path.splitext(filename)[0]
def _batch_sort_key(filename: str):
"""Sort batch-2 before batch-10, and keep unnumbered names after them."""
numbers = re.findall(r"\d+", batch_label(filename))
return (0, int(numbers[0])) if numbers else (1, 0), filename.lower()
def _effective_root(video_root: str) -> str:
"""Return video_root if it exists, otherwise fall back to config.VIDEO_ROOT."""
if os.path.isdir(video_root):
return video_root
if os.path.isdir(config.VIDEO_ROOT):
return config.VIDEO_ROOT
return video_root
def list_dates(video_root: str) -> List[dict]:
"""Date folders, newest name first, with how many videos each holds."""
video_root = _effective_root(video_root)
if not os.path.isdir(video_root):
raise LibraryError(f"Video archive folder not found: {video_root}")
dates = []
for name in sorted(os.listdir(video_root), reverse=True):
path = os.path.join(video_root, name)
if not os.path.isdir(path) or name.startswith("."):
continue
try:
count = sum(1 for f in os.listdir(path) if f.lower().endswith(video.VIDEO_EXTS))
except OSError:
continue
dates.append({"date": name, "video_count": count})
return dates
def list_videos(video_root: str, date: str, project_id: Optional[int] = None) -> List[dict]:
"""Videos in one date folder, with metadata and how often each was used."""
video_root = _effective_root(video_root)
folder = _safe_join(video_root, date)
if not os.path.isdir(folder):
raise LibraryError(f"No such date in the archive: {date}")
used = _usage(project_id)
videos = []
for filename in sorted(
(f for f in os.listdir(folder) if f.lower().endswith(video.VIDEO_EXTS)),
key=_batch_sort_key,
):
path = os.path.join(folder, filename)
entry = {
"rel": f"{date}/{filename}",
"filename": filename,
"batch_label": batch_label(filename),
"used_count": used.get(os.path.realpath(path), 0),
}
try:
entry.update(video.probe(path))
except video.VideoError as exc:
# A file ffprobe cannot read still belongs in the list, flagged —
# hiding it would look like the archive is missing recordings.
entry.update({"duration": 0.0, "width": 0, "height": 0, "fps": 0.0,
"error": str(exc)})
videos.append(entry)
return videos
def _usage(project_id: Optional[int]) -> dict:
"""How many batches already came out of each video path (REQ-012)."""
if project_id is None:
return {}
with db.cursor() as cur:
cur.execute(
"SELECT video_path, COUNT(*) FROM batches WHERE project_id = ? GROUP BY video_path",
(project_id,),
)
return {os.path.realpath(row[0]): row[1] for row in cur.fetchall()}
def resolve(video_root: str, rel: str) -> str:
"""Turn a `<date>/<file>` reference into an absolute path inside the archive."""
video_root = _effective_root(video_root)
path = _safe_join(video_root, rel)
if not os.path.isfile(path):
raise LibraryError(f"No such video: {rel}")
if not path.lower().endswith(video.VIDEO_EXTS):
raise LibraryError("That file is not a video")
return path
+75
View File
@@ -0,0 +1,75 @@
"""FastAPI server for the dataset enrichment platform.
Run it with: .venv/bin/uvicorn backend.main:app --port 8000
or: docker compose up
Routes live in `backend/api/`, one module per domain; this file only wires them
together and owns startup.
"""
import os
import shutil
from contextlib import asynccontextmanager
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from backend import config, db, jobs
from backend.api import batches, jobs as job_routes, models, projects, review
@asynccontextmanager
async def lifespan(_app: FastAPI):
config.ensure_dirs()
db.migrate()
from backend import projects as project_store
project_store.ensure_seed_project()
interrupted = jobs.recover()
if interrupted:
print(f"[startup] closed {interrupted} job(s) interrupted by the last restart")
yield
app = FastAPI(title="Dataset Enrichment", lifespan=lifespan)
app.add_middleware(
CORSMiddleware,
allow_origins=config.CORS_ORIGINS,
allow_methods=["*"],
allow_headers=["*"],
)
app.include_router(projects.router)
app.include_router(batches.router)
app.include_router(review.router)
app.include_router(models.router)
app.include_router(job_routes.router)
@app.get("/api/health")
def health() -> dict:
import torch
from backend import hardware
free_vram = hardware.free_vram_gb()
needed = hardware.SAM3_RESIDENT_GB + hardware.SAM3_HEADROOM_GB
return {
"device": "cuda" if torch.cuda.is_available() else "cpu",
"gpu": torch.cuda.get_device_name(0) if torch.cuda.is_available() else None,
"vram_free_gb": free_vram,
"sam3_ready": free_vram >= needed or _engine_loaded(),
"ffmpeg": shutil.which("ffmpeg") is not None,
"ffprobe": shutil.which("ffprobe") is not None,
"hf_token": bool(os.environ.get("HUGGING_FACE_HUB_TOKEN")),
"db": db.healthy(),
"data_dir": config.DATA_DIR,
"video_root": config.VIDEO_ROOT,
"model_loaded": _engine_loaded(),
}
def _engine_loaded() -> bool:
from backend.sam3_engine import engine_is_loaded
return engine_is_loaded()
+438
View File
@@ -0,0 +1,438 @@
"""Projects: the unit that makes this system reusable (REQ-001…006).
A project owns a base model, a locked class list, a video archive root, and its
own accumulating master dataset. Everything it produces lives under one folder,
so a project can be copied or backed up whole.
Classes come from the base model whenever there is one — `model.names` is the
only thing that keeps the master dataset, the auto-annotation prompts, and the
fine-tune consistent with each other (REQ-003).
"""
import os
import re
import shutil
import time
from typing import List, Optional
from backend import config, db
LABEL_TYPES = ("bbox", "polygon")
# Starting points when a project has no base model of its own (REQ-004).
PRETRAINED = {"bbox": "yolo11n.pt", "polygon": "yolo11n-seg.pt"}
class ProjectError(Exception):
"""Something the user can fix: a bad name, a missing folder, a locked field."""
def slugify(name: str) -> str:
slug = re.sub(r"[^a-z0-9]+", "-", name.strip().lower()).strip("-")
return slug or "project"
def _unique_slug(cur, name: str) -> str:
base = slugify(name)
slug, suffix = base, 2
while True:
cur.execute("SELECT 1 FROM projects WHERE slug = ?", (slug,))
if cur.fetchone() is None:
return slug
slug, suffix = f"{base}-{suffix}", suffix + 1
def read_model_classes(weights_path: str) -> List[str]:
"""Class names in a YOLO checkpoint, in class-id order."""
from ultralytics import YOLO
try:
names = YOLO(weights_path).names
except Exception as exc:
raise ProjectError(f"Could not read classes from that model: {exc}")
if isinstance(names, dict):
return [names[key] for key in sorted(names)]
return list(names)
def _project_paths(slug: str) -> dict:
root = config.project_dir(slug)
return {
"root": root,
"base": os.path.join(root, "base"),
"dataset": os.path.join(root, "dataset"),
"batches": os.path.join(root, "batches"),
"models": os.path.join(root, "models"),
}
def create(name: str, label_type: str, video_root: str, classes: Optional[List[dict]] = None,
base_model_path: Optional[str] = None, val_every: int = 5) -> dict:
"""Create a project. `classes` is [{"name": ..., "prompt": ...}, …] and is
ignored when a base model is given — that model's names win."""
if not name.strip():
raise ProjectError("A project name is required")
if label_type not in LABEL_TYPES:
raise ProjectError(f"label_type must be one of {LABEL_TYPES}")
video_root = os.path.abspath(os.path.expanduser(video_root))
if not os.path.isdir(video_root):
raise ProjectError(f"Video archive folder not found: {video_root}")
if base_model_path:
names = read_model_classes(base_model_path)
classes = [{"name": n, "prompt": n} for n in names]
if not classes:
raise ProjectError("Give a base model to read classes from, or list the classes")
cleaned = []
for index, item in enumerate(classes):
class_name = str(item.get("name", "")).strip()
if not class_name:
raise ProjectError(f"Class {index} has no name")
cleaned.append({"name": class_name,
"prompt": str(item.get("prompt") or class_name).strip()})
if len({c["name"] for c in cleaned}) != len(cleaned):
raise ProjectError("Class names must be unique")
with db.cursor() as cur:
slug = _unique_slug(cur, name)
paths = _project_paths(slug)
for path in paths.values():
os.makedirs(path, exist_ok=True)
stored_model = ""
kind = "pretrained"
if base_model_path:
stored_model = os.path.join(paths["base"], "model.pt")
shutil.copyfile(base_model_path, stored_model)
kind = "uploaded"
cur.execute(
"""INSERT INTO projects (slug, name, label_type, base_model_path,
base_model_kind, video_root, val_every, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)""",
(slug, name.strip(), label_type, stored_model, kind, video_root,
max(0, val_every), time.time()),
)
project_id = cur.lastrowid
_write_classes(cur, project_id, cleaned)
return get(project_id)
def _write_classes(cur, project_id: int, classes: List[dict]) -> None:
cur.execute("DELETE FROM project_classes WHERE project_id = ?", (project_id,))
cur.executemany(
"INSERT INTO project_classes (project_id, class_id, name, prompt) VALUES (?, ?, ?, ?)",
[(project_id, index, item["name"], item["prompt"])
for index, item in enumerate(classes)],
)
def _row_to_dict(cur, row) -> dict:
cur.execute(
"SELECT class_id, name, prompt FROM project_classes WHERE project_id = ? ORDER BY class_id",
(row["id"],),
)
classes = [dict(item) for item in cur.fetchall()]
# How many shapes hang off each class — the number the user needs before
# agreeing to delete one (REQ-007).
cur.execute(
"""SELECT a.class_id, COUNT(*) FROM annotations a
JOIN frames f ON f.id = a.frame_id
JOIN batches b ON b.id = f.batch_id
WHERE b.project_id = ? GROUP BY a.class_id""",
(row["id"],),
)
usage = dict(cur.fetchall())
for item in classes:
item["annotation_count"] = usage.get(item["class_id"], 0)
cur.execute("SELECT COUNT(*) FROM batches WHERE project_id = ?", (row["id"],))
batch_count = cur.fetchone()[0]
cur.execute(
"SELECT split, COUNT(*) FROM dataset_items WHERE project_id = ? GROUP BY split",
(row["id"],),
)
dataset = {"train": 0, "val": 0}
for split, count in cur.fetchall():
dataset[split] = count
sec_classes = []
if "secondary_model_classes" in row.keys() and row["secondary_model_classes"]:
try:
sec_classes = json.loads(row["secondary_model_classes"])
except Exception:
sec_classes = []
paths = _project_paths(row["slug"])
return {
"id": row["id"],
"slug": row["slug"],
"name": row["name"],
"label_type": row["label_type"],
"base_model_path": row["base_model_path"],
"base_model_kind": row["base_model_kind"],
"secondary_model_path": row["secondary_model_path"] if "secondary_model_path" in row.keys() else None,
"secondary_model_name": row["secondary_model_name"] if "secondary_model_name" in row.keys() else None,
"secondary_model_classes": sec_classes,
"base_model_fallback": PRETRAINED[row["label_type"]],
"video_root": row["video_root"],
"val_every": row["val_every"],
"created_at": row["created_at"],
"classes": classes,
"batch_count": batch_count,
"dataset": dataset,
# Once anything has been merged the label type is settled (REQ-002).
"label_type_locked": (dataset["train"] + dataset["val"]) > 0,
"paths": paths,
}
def get(project_id: int) -> Optional[dict]:
with db.cursor() as cur:
cur.execute("SELECT * FROM projects WHERE id = ?", (project_id,))
row = cur.fetchone()
return _row_to_dict(cur, row) if row else None
def listing() -> List[dict]:
with db.cursor() as cur:
cur.execute("SELECT * FROM projects ORDER BY created_at DESC")
return [_row_to_dict(cur, row) for row in cur.fetchall()]
def update(project_id: int, prompts: Optional[dict] = None, val_every: Optional[int] = None,
video_root: Optional[str] = None) -> dict:
"""Edit the things that are safe to change: prompts, split ratio, archive root.
Class names and label type are not among them."""
if get(project_id) is None:
raise ProjectError("No such project")
with db.cursor() as cur:
if val_every is not None:
cur.execute("UPDATE projects SET val_every = ? WHERE id = ?",
(max(0, val_every), project_id))
if video_root is not None:
resolved = os.path.abspath(os.path.expanduser(video_root))
if not os.path.isdir(resolved):
raise ProjectError(f"Video archive folder not found: {resolved}")
cur.execute("UPDATE projects SET video_root = ? WHERE id = ?",
(resolved, project_id))
for class_id, prompt in (prompts or {}).items():
cur.execute(
"UPDATE project_classes SET prompt = ? WHERE project_id = ? AND class_id = ?",
(str(prompt).strip(), project_id, int(class_id)),
)
return get(project_id)
def add_class(project_id: int, name: str, prompt: Optional[str] = None) -> dict:
"""Append a class to an existing project (REQ-008).
Appending is the easy direction: the new class takes the next id, so no
existing annotation or label file means anything different afterwards. Only
`data.yaml` has to be rewritten, because it carries `nc`.
"""
project = get(project_id)
if project is None:
raise ProjectError("No such project")
clean = name.strip()
if not clean:
raise ProjectError("A class needs a name")
if any(item["name"] == clean for item in project["classes"]):
raise ProjectError(f'This project already has a class called "{clean}"')
next_id = max((item["class_id"] for item in project["classes"]), default=-1) + 1
with db.cursor() as cur:
cur.execute(
"INSERT INTO project_classes (project_id, class_id, name, prompt) "
"VALUES (?, ?, ?, ?)",
(project_id, next_id, clean, (prompt or clean).strip()),
)
updated = get(project_id)
if updated["dataset"]["train"] + updated["dataset"]["val"] > 0:
from backend import dataset
dataset.write_data_yaml(updated)
return updated
def delete_class(project_id: int, class_id: int) -> dict:
"""Remove a class and renumber the ones above it, everywhere (REQ-007).
"Everywhere" is the whole point: the database rows, and the label files
already written into the master dataset. A YOLO label is an integer index,
so a class list and a set of label files that disagree do not fail loudly —
they train a model on the wrong names.
"""
project = get(project_id)
if project is None:
raise ProjectError("No such project")
target = next((c for c in project["classes"] if c["class_id"] == class_id), None)
if target is None:
raise ProjectError(f"This project has no class {class_id}")
if len(project["classes"]) == 1:
raise ProjectError("A project needs at least one class")
from backend import dataset
with db.cursor() as cur:
frames_of_project = """
SELECT f.id FROM frames f
JOIN batches b ON b.id = f.batch_id
WHERE b.project_id = ?
"""
cur.execute(
f"DELETE FROM annotations WHERE class_id = ? AND frame_id IN ({frames_of_project})",
(class_id, project_id),
)
removed = cur.rowcount
cur.execute(
f"""UPDATE annotations SET class_id = class_id - 1
WHERE class_id > ? AND frame_id IN ({frames_of_project})""",
(class_id, project_id),
)
cur.execute("DELETE FROM project_classes WHERE project_id = ? AND class_id = ?",
(project_id, class_id))
cur.execute(
"UPDATE project_classes SET class_id = class_id - 1 "
"WHERE project_id = ? AND class_id > ?",
(project_id, class_id),
)
report = dataset.drop_class_from_labels(project, class_id)
updated = get(project_id)
dataset.write_data_yaml(updated)
return {
"project": updated,
"removed": {
"class": target["name"],
"annotations": removed,
**report,
},
}
def set_base_model(project_id: int, weights_path: str) -> dict:
"""Point the project at a new base model and re-read its classes (REQ-003).
Refused once the master dataset exists and the new model's classes differ —
a dataset labelled against one class list cannot be trained against another.
"""
project = get(project_id)
if project is None:
raise ProjectError("No such project")
names = read_model_classes(weights_path)
existing = [item["name"] for item in project["classes"]]
if project["dataset"]["train"] + project["dataset"]["val"] > 0 and set(names) != set(existing):
raise ProjectError(
"That model's classes differ from the ones this project's dataset was "
f"labelled with ({existing} vs {names}). Create a new project for it."
)
paths = _project_paths(project["slug"])
os.makedirs(paths["base"], exist_ok=True)
stored = os.path.join(paths["base"], "model.pt")
if os.path.abspath(weights_path) != os.path.abspath(stored):
shutil.copyfile(weights_path, stored)
with db.cursor() as cur:
cur.execute(
"UPDATE projects SET base_model_path = ?, base_model_kind = 'uploaded' WHERE id = ?",
(stored, project_id),
)
_write_classes(cur, project_id,
[{"name": n, "prompt": p}
for n, p in zip(names, _kept_prompts(project, names))])
return get(project_id)
def set_secondary_model(project_id: int, weights_path: str, name: str = "") -> dict:
"""Point the project at a secondary model for auto-annotation."""
project = get(project_id)
if project is None:
raise ProjectError("No such project")
paths = _project_paths(project["slug"])
os.makedirs(paths["base"], exist_ok=True)
stored = os.path.join(paths["base"], "secondary_model.pt")
if os.path.abspath(weights_path) != os.path.abspath(stored):
shutil.copyfile(weights_path, stored)
names = read_model_classes(weights_path)
model_label = name.strip() or os.path.basename(weights_path)
with db.cursor() as cur:
cur.execute(
"UPDATE projects SET secondary_model_path = ?, secondary_model_name = ?, secondary_model_classes = ? WHERE id = ?",
(stored, model_label, json.dumps(names), project_id),
)
return get(project_id)
def _kept_prompts(project: dict, names: List[str]) -> List[str]:
"""Keep the prompt the user already wrote for a class that survives a
base-model swap; fall back to the class name for new ones."""
known = {item["name"]: item["prompt"] for item in project["classes"]}
return [known.get(name, name) for name in names]
def training_start_point(project: dict) -> str:
"""The weights a training run should start from (REQ-060, REQ-004)."""
path = project["base_model_path"]
if path and os.path.isfile(path):
return path
return PRETRAINED[project["label_type"]]
def delete(project_id: int) -> bool:
project = get(project_id)
if project is None:
return False
with db.cursor() as cur:
cur.execute("DELETE FROM projects WHERE id = ?", (project_id,))
shutil.rmtree(project["paths"]["root"], ignore_errors=True)
return True
def ensure_seed_project() -> None:
"""Ensure at least one project exists on startup using legacy data if available."""
with db.cursor() as cur:
cur.execute("SELECT COUNT(*) FROM projects")
if cur.fetchone()[0] > 0:
return
legacy_model = "/data/base_models/best.pt"
if not os.path.exists(legacy_model):
legacy_model = "/videos/model/best.pt"
if not os.path.exists(legacy_model):
legacy_model = os.path.join(config.VIDEO_ROOT, "model", "best.pt")
if os.path.exists(legacy_model):
try:
create(
name="Cargo & Sack Detection",
label_type="bbox",
video_root=config.VIDEO_ROOT,
base_model_path=legacy_model,
)
print("[startup] Seeded default project 'Cargo & Sack Detection' from legacy model")
return
except Exception as exc:
print(f"[startup] Failed to seed project from legacy model: {exc}")
try:
create(
name="Default Detection Project",
label_type="bbox",
video_root=config.VIDEO_ROOT,
classes=[{"name": "sack", "prompt": "sack"}, {"name": "truck", "prompt": "truck"}],
)
print("[startup] Seeded 'Default Detection Project'")
except Exception as exc:
print(f"[startup] Seed project creation skipped: {exc}")
+350
View File
@@ -0,0 +1,350 @@
"""Annotations and per-frame review state (REQ-040…045).
Geometry is stored normalized 0–1 against the frame, as JSON:
bbox {"type": "bbox", "points": [x0, y0, x1, y1]}
polygon {"type": "polygon", "points": [[x, y], …]}
Normalized because the editor scales the frame to whatever the window allows,
and the exporter needs the same numbers YOLO wants — neither should care about
the display size.
`source` separates what SAM3 produced from what the user drew. Re-running
auto-annotation replaces only the former (REQ-034).
"""
import json
import time
from typing import List, Optional
from backend import db
STATUSES = ("pending", "approved", "rejected")
class ReviewError(Exception):
pass
# ---- geometry ----------------------------------------------------------
def _clamp(value: float) -> float:
return max(0.0, min(1.0, float(value)))
def bbox(x0: float, y0: float, x1: float, y1: float) -> dict:
left, right = sorted((_clamp(x0), _clamp(x1)))
top, bottom = sorted((_clamp(y0), _clamp(y1)))
return {"type": "bbox", "points": [left, top, right, bottom]}
def polygon(points) -> dict:
return {"type": "polygon", "points": [[_clamp(x), _clamp(y)] for x, y in points]}
def mask_to_polygons(mask, min_area_px: int = 24, max_polygons: int = 1) -> list:
"""Contour a boolean SAM3 mask into polygon point arrays, largest first.
YOLO-seg expects one polygon per instance, so only the largest connected
component is kept by default — SAM3 masks are occasionally speckled. The raw
contour is one point per pixel step, which would make label files enormous
for no accuracy gain, so it is simplified first.
"""
import cv2
import numpy as np
contours, _ = cv2.findContours((mask.astype(np.uint8)) * 255,
cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
polygons = []
for contour in sorted(contours, key=cv2.contourArea, reverse=True)[:max_polygons]:
if cv2.contourArea(contour) < min_area_px:
continue
epsilon = 0.002 * cv2.arcLength(contour, True)
approx = cv2.approxPolyDP(contour, epsilon, True).reshape(-1, 2)
if approx.shape[0] >= 3:
polygons.append(approx.astype(np.float32))
return polygons
def validate(geometry: dict, label_type: str) -> dict:
"""Reject shapes that would export as broken labels."""
if not isinstance(geometry, dict):
raise ReviewError("geometry must be an object")
kind = geometry.get("type")
points = geometry.get("points") or []
if kind == "bbox":
if len(points) != 4:
raise ReviewError("a bbox needs [x0, y0, x1, y1]")
shape = bbox(*points)
left, top, right, bottom = shape["points"]
if right - left < 0.002 or bottom - top < 0.002:
raise ReviewError("that box is too small to be a label")
return shape
if kind == "polygon":
if len(points) < 3:
raise ReviewError("a polygon needs at least 3 points")
if label_type == "bbox":
raise ReviewError("this project stores boxes, not polygons")
return polygon(points)
raise ReviewError(f"unknown geometry type: {kind}")
def to_box(geometry: dict) -> List[float]:
"""The bounding box of any shape, normalized — used for the bbox export
and for deduplicating polygon detections against each other."""
points = geometry["points"]
if geometry["type"] == "bbox":
return list(points)
xs = [point[0] for point in points]
ys = [point[1] for point in points]
return [min(xs), min(ys), max(xs), max(ys)]
# ---- frames ------------------------------------------------------------
def frame(frame_id: int) -> Optional[dict]:
with db.cursor() as cur:
cur.execute(
"""SELECT f.*, b.id AS batch_id, b.project_id, p.slug AS project_slug,
p.label_type
FROM frames f
JOIN batches b ON b.id = f.batch_id
JOIN projects p ON p.id = b.project_id
WHERE f.id = ?""",
(frame_id,),
)
row = cur.fetchone()
return dict(row) if row else None
def set_status(frame_id: int, status: str) -> dict:
if status not in STATUSES:
raise ReviewError(f"status must be one of {STATUSES}")
if frame(frame_id) is None:
raise ReviewError("No such frame")
with db.cursor() as cur:
cur.execute("UPDATE frames SET review_status = ? WHERE id = ?", (status, frame_id))
return {"frame_id": frame_id, "review_status": status}
def next_pending(batch_id: int, after_idx: int = -1) -> Optional[int]:
"""The next frame still needing a decision, for the 'jump to unreviewed' key."""
with db.cursor() as cur:
cur.execute(
"""SELECT id FROM frames
WHERE batch_id = ? AND review_status = 'pending' AND idx > ?
ORDER BY idx LIMIT 1""",
(batch_id, after_idx),
)
row = cur.fetchone()
if row:
return row["id"]
cur.execute(
"""SELECT id FROM frames WHERE batch_id = ? AND review_status = 'pending'
ORDER BY idx LIMIT 1""",
(batch_id,),
)
row = cur.fetchone()
return row["id"] if row else None
# ---- annotations -------------------------------------------------------
def _row_to_dict(row) -> dict:
return {
"id": row["id"],
"frame_id": row["frame_id"],
"class_id": row["class_id"],
"geometry": json.loads(row["geometry"]),
"score": row["score"],
"source": row["source"],
}
def listing(frame_id: int) -> List[dict]:
with db.cursor() as cur:
cur.execute("SELECT * FROM annotations WHERE frame_id = ? ORDER BY id", (frame_id,))
return [_row_to_dict(row) for row in cur.fetchall()]
def add(frame_id: int, class_id: int, geometry: dict, source: str = "manual",
score: float = 1.0) -> dict:
target = frame(frame_id)
if target is None:
raise ReviewError("No such frame")
shape = validate(geometry, target["label_type"])
_check_class(target["project_id"], class_id)
with db.cursor() as cur:
cur.execute(
"""INSERT INTO annotations (frame_id, class_id, geometry, score, source, created_at)
VALUES (?, ?, ?, ?, ?, ?)""",
(frame_id, class_id, json.dumps(shape), score, source, time.time()),
)
cur.execute("SELECT * FROM annotations WHERE id = ?", (cur.lastrowid,))
return _row_to_dict(cur.fetchone())
def update(annotation_id: int, class_id: Optional[int] = None,
geometry: Optional[dict] = None) -> dict:
with db.cursor() as cur:
cur.execute("SELECT * FROM annotations WHERE id = ?", (annotation_id,))
row = cur.fetchone()
if row is None:
raise ReviewError("No such annotation")
target = frame(row["frame_id"])
new_geometry = row["geometry"]
if geometry is not None:
new_geometry = json.dumps(validate(geometry, target["label_type"]))
new_class = row["class_id"] if class_id is None else class_id
_check_class(target["project_id"], new_class)
with db.cursor() as cur:
# Any edit makes it the user's shape, so it survives a re-run of
# auto-annotation (REQ-034).
cur.execute(
"UPDATE annotations SET class_id = ?, geometry = ?, source = 'manual' WHERE id = ?",
(new_class, new_geometry, annotation_id),
)
cur.execute("SELECT * FROM annotations WHERE id = ?", (annotation_id,))
return _row_to_dict(cur.fetchone())
def delete(annotation_id: int) -> bool:
with db.cursor() as cur:
cur.execute("DELETE FROM annotations WHERE id = ?", (annotation_id,))
return cur.rowcount > 0
def replace_auto(frame_id: int, items: List[dict]) -> int:
"""Swap this frame's automatic shapes for a fresh set, leaving manual ones."""
with db.cursor() as cur:
cur.execute("DELETE FROM annotations WHERE frame_id = ? AND source = 'auto'",
(frame_id,))
cur.executemany(
"""INSERT INTO annotations (frame_id, class_id, geometry, score, source, created_at)
VALUES (?, ?, ?, ?, 'auto', ?)""",
[(frame_id, item["class_id"], json.dumps(item["geometry"]),
item.get("score", 1.0), time.time()) for item in items],
)
return len(items)
def frames_with_auto(batch_id: int) -> set:
"""Frame ids that already carry automatic shapes — the resume skip-list
for REQ-035."""
with db.cursor() as cur:
cur.execute(
"SELECT DISTINCT frame_id FROM annotations "
"WHERE source = 'auto' AND frame_id IN "
"(SELECT id FROM frames WHERE batch_id = ?)",
(batch_id,),
)
return {row[0] for row in cur.fetchall()}
def assist(frame_id: int, box: List[float], class_id: int = 0,
threshold: float = 0.5) -> dict:
"""Drag a rough box, get SAM3's shape for the object inside it (REQ-043).
The box is a visual exemplar rather than a crop: SAM3 may return several
matches, so the one overlapping what the user drew is the one kept.
"""
from backend import batches, jobs
from backend.sam3_engine import get_engine
from PIL import Image
target = frame(frame_id)
if target is None:
raise ReviewError("No such frame")
_check_class(target["project_id"], class_id)
# Acquire the shared GPU lock with a 20s timeout. 20 seconds is chosen so that
# short CPU/ffmpeg jobs let assist through, while long GPU jobs fail fast with
# a legible message (REQ-065, REQ-070).
if not jobs.gpu_lock.acquire(timeout=20):
busy = jobs.running_types()
kind = busy[0] if busy else "background"
raise ReviewError(
f"The GPU is busy with a {kind} job — wait for it to finish, or draw the "
"shape by hand"
)
try:
drawn = validate({"type": "bbox", "points": box}, "bbox")["points"]
x0, y0, x1, y1 = drawn
exemplar = [(x0 + x1) / 2, (y0 + y1) / 2, x1 - x0, y1 - y0]
path = batches.frame_path(frame_id)
with Image.open(path) as handle:
image = handle.convert("RGB")
width, height = image.size
engine = get_engine()
state = engine.open_state(image)
found = engine.apply_prompts(
state, threshold=threshold,
exemplars=[{"box": exemplar, "positive": True}],
)
if not found:
raise ReviewError("SAM3 found nothing in that box — draw it tighter, or add the "
"shape by hand")
detection = max(found, key=lambda d: _overlap(d.box, drawn, width, height))
if target["label_type"] == "bbox":
bx0, by0, bx1, by1 = detection.box
geometry = bbox(bx0 / width, by0 / height, bx1 / width, by1 / height)
else:
polygons = mask_to_polygons(detection.mask)
if not polygons:
raise ReviewError("SAM3's mask was too small to turn into a polygon")
geometry = polygon([(x / width, y / height) for x, y in polygons[0]])
finally:
jobs.gpu_lock.release()
return add(frame_id, class_id, geometry, source="manual", score=detection.score)
def _overlap(detection_box: List[float], drawn: List[float],
width: int, height: int) -> float:
"""IoU between a pixel-space detection and the normalized box drawn."""
box = [detection_box[0] / width, detection_box[1] / height,
detection_box[2] / width, detection_box[3] / height]
ix0, iy0 = max(box[0], drawn[0]), max(box[1], drawn[1])
ix1, iy1 = min(box[2], drawn[2]), min(box[3], drawn[3])
inter = max(0.0, ix1 - ix0) * max(0.0, iy1 - iy0)
if inter <= 0:
return 0.0
area_box = (box[2] - box[0]) * (box[3] - box[1])
area_drawn = (drawn[2] - drawn[0]) * (drawn[3] - drawn[1])
return inter / (area_box + area_drawn - inter)
def clear_batch_class_annotations(batch_id: int, class_id: int) -> int:
"""Delete all annotations matching class_id across all frames in a batch (REQ-046)."""
with db.cursor() as cur:
cur.execute(
"""DELETE FROM annotations
WHERE class_id = ? AND frame_id IN (
SELECT id FROM frames WHERE batch_id = ?
)""",
(class_id, batch_id),
)
return cur.rowcount
def _check_class(project_id: int, class_id: int) -> None:
with db.cursor() as cur:
cur.execute(
"SELECT 1 FROM project_classes WHERE project_id = ? AND class_id = ?",
(project_id, class_id),
)
if cur.fetchone() is None:
raise ReviewError(f"Class {class_id} does not exist in this project")
+242
View File
@@ -0,0 +1,242 @@
"""SAM3 text-prompted detection, wrapped for reuse across labeling jobs.
The model is expensive to build (weights come from the gated HuggingFace repo
`facebook/sam3`), so it is loaded once per process and kept resident.
The important performance detail: `Sam3Processor.set_image()` runs the vision
backbone, while `set_text_prompt()` only runs the (much cheaper) grounding head
against the cached `backbone_out`. So for an N-prompt job we call `set_image`
once per image and loop the prompts over that same state.
"""
import os
import sys
import threading
from dataclasses import dataclass, field
from typing import List, Optional
import numpy as np
import torch
from PIL import Image
# Fix python import path masking issue where sam3 is imported as a namespace package
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "sam3")))
from sam3.model.sam3_image_processor import Sam3Processor
from sam3.model_builder import build_sam3_image_model
@dataclass
class Detection:
"""One labeled instance in one image."""
class_id: int
class_name: str = ""
score: float = 0.0
box: List[float] = field(default_factory=list) # xyxy in pixels
mask: Optional[np.ndarray] = None # bool array, (H, W) at original image size
class Sam3Engine:
def __init__(self, checkpoint_path: Optional[str] = None):
# SAM3 is CUDA-only in practice: `PositionEmbeddingSine` precomputes its
# tables with a hardcoded `device="cuda"`, so a CPU run dies deep inside
# the backbone with an unrelated-looking error. Fail here instead, where
# the message can say something useful.
if not torch.cuda.is_available():
raise RuntimeError(
"SAM3 requires a CUDA GPU. No GPU is visible to torch — check "
"`nvidia-smi`, that CUDA_VISIBLE_DEVICES isn't set to empty, and "
"that this venv has a CUDA build of torch installed."
)
self.device = "cuda"
self.autocast_dtype = torch.float16
# `enable_inst_interactivity=True` builds a SAM1-style click predictor,
# but in this vendored copy its `image_encoder` is None and its expected
# feature sizes (288/144/72) don't match the 1008px image pipeline, so
# `predictor.set_image()` always fails. It costs ~0.4 GB for nothing, so
# it stays off. Box exemplars cover the interactive use case instead.
self.supports_tap = False
self.model = build_sam3_image_model(
device=self.device,
checkpoint_path=checkpoint_path,
load_from_HF=checkpoint_path is None,
)
self.processor = Sam3Processor(self.model, device=self.device)
def detect(self, image: Image.Image, prompts: List[str], threshold: float) -> List[Detection]:
"""Run every prompt against one image; prompt index becomes the class id."""
self.processor.confidence_threshold = threshold
detections: List[Detection] = []
with torch.autocast(self.device, dtype=self.autocast_dtype):
state = self.processor.set_image(image)
for class_id, prompt in enumerate(prompts):
output = self.processor.set_text_prompt(prompt=prompt, state=state)
masks, boxes, scores = output["masks"], output["boxes"], output["scores"]
if masks.shape[0] == 0:
continue
# Pull off the GPU immediately: the next prompt overwrites these
# tensors, and full-resolution masks are the memory hog here.
masks_np = masks.squeeze(1).to(torch.uint8).cpu().numpy().astype(bool)
boxes_np = boxes.float().cpu().numpy()
scores_np = scores.float().cpu().numpy()
for i in range(masks_np.shape[0]):
detections.append(
Detection(
class_id=class_id,
class_name=prompt,
score=float(scores_np[i]),
box=[float(v) for v in boxes_np[i]],
mask=masks_np[i],
)
)
del state
if self.device == "cuda":
torch.cuda.empty_cache()
return detections
# ---- interactive / exemplar prompting ------------------------------
def open_state(self, image: Image.Image):
"""Run the vision backbone once and hand back the reusable state."""
with torch.autocast(self.device, dtype=self.autocast_dtype):
return self.processor.set_image(image)
def apply_prompts(
self,
state,
threshold: float,
text: Optional[str] = None,
exemplars: Optional[List[dict]] = None,
) -> List[Detection]:
"""Re-run grounding for this image from scratch with the given prompts.
Exemplars are boxes in normalized cxcywh with a positive/negative flag.
The prompt set is always replayed from empty because SAM3 only supports
appending geometric prompts — that's how undo is implemented.
"""
self.processor.confidence_threshold = threshold
exemplars = exemplars or []
with torch.autocast(self.device, dtype=self.autocast_dtype):
self.processor.reset_all_prompts(state)
output = None
if text:
output = self.processor.set_text_prompt(prompt=text, state=state)
for exemplar in exemplars:
output = self.processor.add_geometric_prompt(
box=exemplar["box"], label=bool(exemplar.get("positive", True)),
state=state,
)
if output is None:
return []
return self._collect(output, class_id=0, class_name=text or "visual")
def segment_at(
self,
image: Image.Image,
points: Optional[List[List[float]]] = None,
labels: Optional[List[int]] = None,
box: Optional[List[float]] = None,
) -> Optional[Detection]:
"""Tap-to-segment: one point (or box) in pixels -> that object's mask."""
if not self.supports_tap:
return None
predictor = self.model.inst_interactive_predictor
predictor.set_image(np.array(image))
masks, scores, _ = predictor.predict(
point_coords=np.array(points, dtype=np.float32) if points else None,
point_labels=np.array(labels, dtype=np.int32) if labels else None,
box=np.array(box, dtype=np.float32) if box else None,
multimask_output=True,
)
if masks.shape[0] == 0:
return None
best = int(np.argmax(scores))
mask = masks[best].astype(bool)
ys, xs = np.where(mask)
if xs.size == 0:
return None
return Detection(
class_id=0,
class_name="tap",
score=float(scores[best]),
box=[float(xs.min()), float(ys.min()), float(xs.max()), float(ys.max())],
mask=mask,
)
def _collect(self, output, class_id: int, class_name: str) -> List[Detection]:
masks, boxes, scores = output["masks"], output["boxes"], output["scores"]
if masks.shape[0] == 0:
return []
masks_np = masks.squeeze(1).to(torch.uint8).cpu().numpy().astype(bool)
boxes_np = boxes.float().cpu().numpy()
scores_np = scores.float().cpu().numpy()
return [
Detection(
class_id=class_id,
class_name=class_name,
score=float(scores_np[i]),
box=[float(v) for v in boxes_np[i]],
mask=masks_np[i],
)
for i in range(masks_np.shape[0])
]
_engine: Optional[Sam3Engine] = None
_engine_lock = threading.Lock()
def get_engine() -> Sam3Engine:
"""Build the model on first use, then hand out the same instance."""
global _engine
from backend import hardware
with _engine_lock:
if _engine is None:
needed = hardware.SAM3_RESIDENT_GB + hardware.SAM3_HEADROOM_GB
free = hardware.free_vram_gb()
if free < needed:
raise RuntimeError(
f"SAM3 needs ~{needed:.1f} GB free but only {free:.1f} GB is available. "
"Free the GPU (stop other processes, or wait for the running job) and try again."
)
try:
_engine = Sam3Engine()
except (ImportError, RuntimeError) as exc:
curr_free = hardware.free_vram_gb()
raise RuntimeError(
f"{exc} (Available VRAM: {curr_free:.1f} GB)"
) from exc
return _engine
def engine_is_loaded() -> bool:
return _engine is not None
def release_engine() -> bool:
"""Drop the model and free its VRAM (REQ-065).
SAM3 holds ~3.4 GB resident. On a 6 GB card that is most of the memory a
training run needs, so the two must never be loaded at once. The next job
that needs SAM3 rebuilds it from the local cache in about 12 seconds.
"""
global _engine
import gc
with _engine_lock:
if _engine is None:
return False
_engine = None
gc.collect()
if torch.cuda.is_available():
torch.cuda.empty_cache()
return True
+197
View File
@@ -0,0 +1,197 @@
"""Fine-tune the project's base model on its master dataset (REQ-060…065).
The default is old + new together: the master dataset already accumulates every
merged batch, so a run sees the whole history. Training on the newest batch
alone is what makes a model quietly forget what it used to know, so it is not
what happens here.
"""
import json
import os
import shutil
import time
from typing import Optional
from backend import config, dataset, db, evaluate, hardware, jobs, projects
PRETRAINED = {"bbox": "yolo11n.pt", "polygon": "yolo11n-seg.pt"}
class TrainingError(Exception):
pass
def models_dir(project_slug: str) -> str:
return os.path.join(config.project_dir(project_slug), "models")
def start(project_id: int, epochs: int = 50, overrides: Optional[dict] = None, batch_ids: Optional[list] = None) -> dict:
project = projects.get(project_id)
if project is None:
raise TrainingError("No such project")
counts = dataset.summary(project_id)["splits"]
if counts["train"] == 0:
raise TrainingError(
"The master dataset is empty — approve and merge a batch before training"
)
settings = hardware.resolve(overrides, epochs)
job = jobs.create(
"train",
params={"project_id": project_id, "settings": settings, "batch_ids": batch_ids},
project_id=project_id,
message=f"{counts['train']} train / {counts['val']} val",
)
return job.to_dict()
def listing(project_id: int) -> list:
with db.cursor() as cur:
cur.execute(
"SELECT * FROM model_versions WHERE project_id = ? ORDER BY version DESC",
(project_id,),
)
rows = []
for row in cur.fetchall():
item = dict(row)
item["metrics"] = json.loads(item["metrics"] or "null")
item["base_metrics"] = json.loads(item["base_metrics"] or "null")
rows.append(item)
return rows
def get_version(model_id: int) -> Optional[dict]:
with db.cursor() as cur:
cur.execute("SELECT * FROM model_versions WHERE id = ?", (model_id,))
row = cur.fetchone()
return dict(row) if row else None
def promote(model_id: int) -> dict:
"""Make a trained version the project's base model for the next round (REQ-064)."""
version = get_version(model_id)
if version is None:
raise TrainingError("No such model version")
project = projects.get(version["project_id"])
base_path = os.path.join(config.project_dir(project["slug"]), "base", "model.pt")
os.makedirs(os.path.dirname(base_path), exist_ok=True)
shutil.copyfile(version["weights_path"], base_path)
with db.cursor() as cur:
cur.execute(
"UPDATE projects SET base_model_path = ?, base_model_kind = 'trained' WHERE id = ?",
(base_path, project["id"]),
)
return projects.get(project["id"])
def _next_version(cur, project_id: int) -> int:
cur.execute(
"SELECT COALESCE(MAX(version), 0) + 1 FROM model_versions WHERE project_id = ?",
(project_id,),
)
return cur.fetchone()[0]
@jobs.handler("train")
def _run_train(job) -> None:
from ultralytics import YOLO
os.environ["ULTRALYTICS_OFFLINE"] = "true"
os.environ["YOLO_OFFLINE"] = "true"
project = projects.get(job.params["project_id"])
settings = job.params["settings"]
batch_ids = job.params.get("batch_ids")
data_yaml = dataset.write_data_yaml(project, batch_ids=batch_ids)
# SAM3 and a training run must not hold VRAM at the same time (REQ-065).
from backend.sam3_engine import release_engine
if release_engine():
job.log("Released SAM3 from VRAM before training")
start_point = project["base_model_path"] or PRETRAINED[project["label_type"]]
if not project["base_model_path"]:
job.log(f"No base model on this project — starting from {start_point}")
job.log(f"Fine-tuning {os.path.basename(start_point)} for {settings['epochs']} epoch(s) "
f"(batch={settings['batch']}, imgsz={settings['imgsz']}, "
f"device={settings['device']})")
with db.cursor() as cur:
version = _next_version(cur, project["id"])
out_dir = os.path.join(models_dir(project["slug"]), str(version))
os.makedirs(out_dir, exist_ok=True)
model = YOLO(start_point)
def on_epoch(trainer):
# trainer.epoch is 0-based; report a human-facing 1-based count.
epoch = getattr(trainer, 'epoch', 0) + 1
total = getattr(trainer, 'epochs', settings["epochs"])
job.progress(epoch, total, f"epoch {epoch}/{total}")
model.add_callback("on_fit_epoch_end", on_epoch)
job.progress(0, settings["epochs"])
keep_run_dir = False
try:
model.train(
data=data_yaml,
epochs=settings["epochs"],
imgsz=settings["imgsz"],
batch=settings["batch"],
device=settings["device"],
workers=settings.get("workers", 8),
cache=False,
project=os.path.join(out_dir, "runs"),
name="train",
exist_ok=True,
amp=True,
plots=False,
verbose=False,
)
produced = os.path.join(out_dir, "runs", "train", "weights", "best.pt")
if not os.path.isfile(produced):
raise TrainingError("Training finished without producing best.pt")
weights = os.path.join(out_dir, "best.pt")
shutil.copyfile(produced, weights)
job.log("Validating the base model and the new one on the same val set…")
comparison = evaluate.compare(
project["base_model_path"] or None, weights, data_yaml,
[item["name"] for item in project["classes"]],
imgsz=settings["imgsz"], device=settings["device"], batch=settings["batch"],
)
with open(os.path.join(out_dir, "metrics.json"), "w", encoding="utf-8") as handle:
json.dump(comparison, handle, indent=2)
with db.cursor() as cur:
cur.execute(
"""INSERT INTO model_versions (project_id, version, weights_path,
parent_model_path, metrics, base_metrics,
created_at)
VALUES (?, ?, ?, ?, ?, ?, ?)""",
(project["id"], version, weights, project["base_model_path"],
json.dumps(comparison["new"]), json.dumps(comparison["base"]), time.time()),
)
new = comparison["new"]
if comparison["delta"]:
delta = comparison["delta"]
job.log(f"v{version}: mAP50 {new['map50']:.4f} ({delta['map50']:+.4f} vs base), "
f"mAP50-95 {new['map50_95']:.4f} ({delta['map50_95']:+.4f})")
else:
job.log(f"v{version}: mAP50 {new['map50']:.4f}, mAP50-95 {new['map50_95']:.4f} "
f"— {comparison['skipped']}")
except Exception:
# Keep the runs directory on failure: results.csv and logs are the only record
# of why training failed (REQ-006, REQ-064).
keep_run_dir = True
raise
finally:
# Delete run directory on success or cancellation. job.cancelled finishes training
# without raising an exception, so it takes this delete path.
if not keep_run_dir:
shutil.rmtree(os.path.join(out_dir, "runs"), ignore_errors=True)
+139
View File
@@ -0,0 +1,139 @@
"""ffmpeg/ffprobe wrappers: read a video's metadata, cut a range into frames.
ffmpeg is a hard dependency rather than an OpenCV fallback because seeking to a
timestamp in a long recording has to be exact — an off-by-a-few-seconds trim
silently produces frames of the wrong thing (REQ-020…022).
"""
import json
import os
import shutil
import subprocess
from typing import Callable, List, Optional
VIDEO_EXTS = (".mp4", ".mkv", ".mov", ".avi", ".webm", ".m4v")
_probe_cache: dict = {}
class VideoError(Exception):
pass
def available() -> bool:
return shutil.which("ffmpeg") is not None and shutil.which("ffprobe") is not None
def probe(path: str) -> dict:
"""Duration, resolution and fps of a video, cached by (path, mtime, size).
The cache matters: the Library page probes every file in a date folder, and
ffprobe on a cold cache over dozens of long recordings is slow (REQ-012).
"""
try:
stat = os.stat(path)
except OSError as exc:
raise VideoError(str(exc))
key = (os.path.realpath(path), stat.st_mtime, stat.st_size)
if key in _probe_cache:
return _probe_cache[key]
if not available():
raise VideoError("ffprobe is not installed in this environment")
result = subprocess.run(
["ffprobe", "-v", "error", "-select_streams", "v:0",
"-show_entries", "stream=width,height,avg_frame_rate",
"-show_entries", "format=duration",
"-of", "json", path],
capture_output=True, text=True,
)
if result.returncode != 0:
raise VideoError(result.stderr.strip().splitlines()[-1] if result.stderr else "ffprobe failed")
payload = json.loads(result.stdout or "{}")
streams = payload.get("streams") or [{}]
info = {
"duration": float(payload.get("format", {}).get("duration") or 0.0),
"width": int(streams[0].get("width") or 0),
"height": int(streams[0].get("height") or 0),
"fps": _parse_fps(streams[0].get("avg_frame_rate")),
"size": stat.st_size,
}
_probe_cache[key] = info
return info
def _parse_fps(value: Optional[str]) -> float:
# ffprobe reports "30000/1001", and "0/0" for streams it cannot work out.
if not value or "/" not in value:
return 0.0
numerator, denominator = value.split("/", 1)
try:
return round(float(numerator) / float(denominator), 3) if float(denominator) else 0.0
except ValueError:
return 0.0
def frame_count(start_sec: float, end_sec: float, fps: float) -> int:
"""How many frames a trim will produce — shown before extraction (REQ-021)."""
span = max(0.0, end_sec - start_sec)
return max(0, int(span * fps))
def extract_frames(
video_path: str,
out_dir: str,
start_sec: float,
end_sec: float,
fps: float,
on_progress: Optional[Callable[[int], None]] = None,
should_stop: Optional[Callable[[], bool]] = None,
) -> List[str]:
"""Cut [start, end] at `fps` into numbered JPEGs. Returns their filenames.
`-ss` before `-i` seeks by keyframe (fast) and ffmpeg then decodes accurately
from there, which is what makes a two-minute cut out of a two-hour recording
quick instead of a full decode.
"""
if not available():
raise VideoError("ffmpeg is not installed in this environment")
if end_sec <= start_sec:
raise VideoError("The end of the range must be after its start")
if fps <= 0:
raise VideoError("fps must be greater than 0")
os.makedirs(out_dir, exist_ok=True)
command = [
"ffmpeg", "-hide_banner", "-loglevel", "error", "-y",
"-ss", f"{start_sec:.3f}", "-to", f"{end_sec:.3f}", "-i", video_path,
"-vf", f"fps={fps}", "-q:v", "2",
os.path.join(out_dir, "%06d.jpg"),
]
process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
expected = frame_count(start_sec, end_sec, fps)
while process.poll() is None:
if should_stop is not None and should_stop():
process.terminate()
process.wait(timeout=10)
raise VideoError("cancelled")
if on_progress is not None:
on_progress(min(_written(out_dir), expected))
try:
process.wait(timeout=1)
except subprocess.TimeoutExpired:
pass
if process.returncode != 0:
raise VideoError((process.stderr.read() or "ffmpeg failed").strip().splitlines()[-1])
return sorted(name for name in os.listdir(out_dir) if name.endswith(".jpg"))
def _written(out_dir: str) -> int:
try:
return sum(1 for name in os.listdir(out_dir) if name.endswith(".jpg"))
except OSError:
return 0
+4
View File
@@ -0,0 +1,4 @@
services:
backend:
devices:
- nvidia.com/gpu=all
+30
View File
@@ -0,0 +1,30 @@
services:
backend:
build: .
environment:
HF_TOKEN: ${HF_TOKEN:-}
HF_HUB_DISABLE_XET: "1"
APP_DATA_DIR: /data
VIDEO_ARCHIVE: /videos
CORS_ORIGINS: ${CORS_ORIGINS:-http://localhost:5173}
volumes:
- ./data:/data
- ${VIDEO_ARCHIVE_HOST:-./data/archive}:/videos:ro
- hf-cache:/root/.cache/huggingface
ports:
- "8000:8000"
shm_size: '8gb'
ipc: host
restart: unless-stopped
frontend:
build: ./frontend
ports:
- "${WEB_PORT:-8080}:80"
depends_on:
- backend
restart: unless-stopped
volumes:
hf-cache:
+285
View File
@@ -0,0 +1,285 @@
# Design
Serves `./requirements.md`. Each section names the REQs it fulfils.
## Overview
```
Browser (React + Vite, served by nginx)
│ /api → proxy
▼
FastAPI ──► jobs (1 worker thread, 1 GPU)
│ ├─ extract : ffmpeg (CPU)
│ ├─ autolabel: SAM3 (GPU)
│ ├─ merge : copy + write labels (CPU)
│ └─ train : Ultralytics + eval (GPU)
├──► SQLite (metadata & status)
└──► data/ (frames, master dataset, weights)
```
Storage split rule: **SQLite holds metadata and status; the disk holds pixels, final labels,
and weights.** The master dataset must stay useful even if the database is lost
(REQ-006, REQ-054).
## Disk layout (REQ-006)
```
data/ # Docker volume
app.db # SQLite (WAL)
projects/<slug>/
base/model.pt # the project's active base model (REQ-003)
dataset/ # MASTER, accumulative (REQ-050…052)
images/{train,val}/…jpg
labels/{train,val}/…txt
data.yaml
batches/<batch-id>/
frames/000001.jpg … # extraction output (REQ-022)
models/<n>/
best.pt
metrics.json # base vs new metrics (REQ-063)
runs/ # Ultralytics run directory
```
Master dataset filenames: `<batch-id>__<frame number>.jpg` — unique across batches and
self-documenting about where each image came from. The user's video archive is read-only
(REQ-074).
## SQLite schema
Created by an idempotent migration in `backend/db.py` at startup.
```sql
projects(
id, slug UNIQUE, name, label_type CHECK(bbox|polygon),
base_model_path, base_model_kind CHECK(uploaded|pretrained|trained),
video_root, val_every DEFAULT 5, created_at)
project_classes(
id, project_id → projects, class_id INT, name, prompt,
UNIQUE(project_id, class_id)) -- class_id = the YOLO class index (REQ-003/005)
batches(
id, project_id → projects, video_path, date_label, batch_label,
start_sec REAL, end_sec REAL, fps REAL,
status CHECK(extracting|extracted|labeling|reviewing|approved|merged|failed),
frame_count INT, created_at, merged_at)
frames(
id, batch_id → batches, idx INT, filename, width INT, height INT,
review_status CHECK(pending|approved|rejected) DEFAULT 'pending',
UNIQUE(batch_id, idx))
annotations(
id, frame_id → frames, class_id INT,
geometry TEXT, -- JSON; see "Geometry format"
score REAL, source CHECK(auto|manual), created_at)
dataset_items( -- master dataset membership (REQ-052)
id, project_id → projects, frame_id → frames UNIQUE,
split CHECK(train|val), image_rel, label_rel, added_at)
model_versions(
id, project_id → projects, version INT, weights_path,
parent_model_path, metrics TEXT, base_metrics TEXT, created_at,
UNIQUE(project_id, version))
jobs( -- persistent (REQ-071)
id, project_id, batch_id, type CHECK(extract|autolabel|merge|train),
status CHECK(queued|running|done|failed|cancelled),
progress INT, total INT, message, error, log TEXT,
created_at, started_at, finished_at)
```
**The stable val split (REQ-052)** is enforced by `dataset_items`: an existing row never
changes its `split`. On merge, only frames without a row are assigned, using a per-project
round-robin counter (`val_every`) that continues from the previous count.
**Geometry format.** One JSON column covers both label types (REQ-002):
- `bbox` → `{"type":"bbox","points":[x0,y0,x1,y1]}`
- `polygon` → `{"type":"polygon","points":[[x,y], …]}`
Coordinates are stored **normalized 0–1** against the frame size, so neither the editor nor
the exporter needs to know the display size. SAM3 mask → polygon conversion is
`review.mask_to_polygons()`; for `bbox` projects the mask is only used to take its bounding
box.
## Backend modules
`app/` moves to `backend/`. Reuse existing code wherever possible:
| Module | Role | Status |
|---|---|---|
| `sam3_engine.py` | SAM3 singleton, `open_state`/`apply_prompts`/`segment_at` | reused, plus a `release()` for REQ-065 |
| `labeling.py` | per-frame detection + cross-prompt NMS (REQ-031) | reused; the folder-walking half went with the old flow |
| `exporters.py` | ~~YOLO label writing~~ | **deleted** — `dataset.py` writes labels, `mask_to_polygons` moved to `review.py` |
| `sessions.py` | ~~exemplar/tap interaction~~ | **deleted** — see below |
| `jobs.py` | single-worker queue | extended: job types + persistence |
| `training.py` | Ultralytics fine-tune | changed: starts from the base model, args from `hardware.py` |
| `db.py` | connection + migration | **new** |
| `projects.py` | project CRUD, reads classes from a `.pt` | **new** |
| `library.py` | scans `<video_root>/<date>/<batch>` | **new** |
| `video.py` | `ffprobe`, Range streaming, `ffmpeg` extraction | **new** |
| `batches.py` | batch lifecycle | **new** |
| `review.py` | annotation CRUD, frame status, click-assist | **new** |
| `autolabel.py` | the SAM3 job over a whole batch | **new** |
| `dataset.py` | merge into the master dataset, stable split | **new** |
| `evaluate.py` | validate base vs new model | **new** |
| `hardware.py` | VRAM detection → training defaults | **new** |
| `api/` | the FastAPI routes, one module per domain | **new** |
Removed: `uploads.py`, `static/index.html`, and the old flow's endpoints.
`sessions.py` was meant to be reused for click-assist, but it existed to hold GPU-resident
state for an interactive session — a whole eviction policy, an undo stack, and a per-session
annotation store, all of which the database and the stateless `review.assist` now cover.
Adapting 357 lines to do what 60 lines do was not worth it, so the module is gone. The one
thing it knew that mattered — SAM3 wants exemplar boxes as normalized centre-x, centre-y,
width, height — moved with it.
The 400-line file limit (see `../AGENTS.md`) applies to all of the above. It is why the
routes live in `backend/api/{projects,batches,review,models,jobs}.py` rather than in
`main.py`, which now only builds the app and owns startup. Route modules import their
domain module under an alias (`from backend import projects as project_store`) so the two
namespaces stay distinguishable.
## API contract
```
GET /api/health REQ-073
GET /api/projects REQ-001
POST /api/projects REQ-001,002,004,005
GET /api/projects/{id} # includes label_type_locked: bool (REQ-002)
DELETE /api/projects/{id}
PATCH /api/projects/{id} # class prompts, val_every (REQ-005)
POST /api/projects/{id}/classes # add new class {name, prompt} (REQ-008)
DELETE /api/projects/{id}/classes/{class_id} # delete class, delete shapes, reindex classes (REQ-007)
POST /api/projects/{id}/base-model # upload .pt, read classes (REQ-003)
GET /api/projects/{id}/dataset # master dataset summary (REQ-053)
GET /api/projects/{id}/dataset/download # zip (REQ-054)
GET /api/projects/{id}/library # list of dates (REQ-011)
GET /api/projects/{id}/library/{date} # videos + duration/resolution (REQ-012)
GET /api/projects/{id}/video?rel=… # Range streaming (REQ-013)
POST /api/projects/{id}/batches # {rel, start_sec, end_sec, fps} → extract job
GET /api/batches/{id} # status + review progress (REQ-045)
GET /api/batches/{id}/frames # frames + statuses
POST /api/batches/{id}/autolabel # {threshold} → job (REQ-030,032,034)
DELETE /api/batches/{id}/classes/{class_id}/annotations # clear all shapes of class in batch (REQ-046)
POST /api/batches/{id}/approve # → merge job (REQ-050)
GET /api/frames/{id}/image?w=… # frame image / thumbnail
GET /api/frames/{id}/annotations
POST /api/frames/{id}/annotations # add a manual shape (REQ-042)
PATCH /api/annotations/{id} # move/resize/reclass
DELETE /api/annotations/{id}
POST /api/frames/{id}/assist # click/box → SAM3 shape (REQ-043)
POST /api/frames/{id}/status # approved | rejected | pending (REQ-041)
POST /api/projects/{id}/train # → train job (REQ-060,061,062)
GET /api/projects/{id}/models # versions + metrics (REQ-063,064)
GET /api/models/{id}/weights # download best.pt
POST /api/models/{id}/promote # make it the project's base model (REQ-064)
GET /api/jobs?project_id=… REQ-070,071
GET /api/jobs/{id}
POST /api/jobs/{id}/cancel
```
## Job flows
**extract (REQ-020…023).** `ffmpeg -ss <start> -to <end> -i <video> -vf fps=<n> -q:v 2
frames/%06d.jpg`. `frames` rows are written once the files exist; the batch's frame count is
updated. Range and fps live on the batch, so one video can be used repeatedly.
**autolabel (REQ-030…034).** Per frame: one `set_image`, then loop each class's prompt (see
the domain invariants in `../AGENTS.md`), cross-prompt NMS, write `annotations` rows with
`source='auto'`. A re-run deletes only `source='auto'` rows — manual corrections
(`source='manual'`) survive — and returns already-approved frames to `pending`, because
that approval was given against labels that no longer exist. No overlay images are written:
the review canvas draws the shapes from the annotation rows, so a second rendering of the
same data on the server would only be a second thing to keep in sync.
**Deleting a class (REQ-007).** `projects.delete_class` removes the class's annotations,
decrements every `class_id` above it in `annotations` and `project_classes`, then calls
`dataset.drop_class_from_labels` to do the same edit to every `.txt` already written to
disk, and rewrites `data.yaml`. The renumbering is the whole job: a YOLO label is an integer
index, so a class list and a set of label files that disagree do not fail loudly — they
train a model on the wrong names. Refused for a project's last class.
**merge (REQ-050…053).** For every `approved` frame not yet in `dataset_items`: assign a
split (continuing the round-robin), copy the JPEG to `dataset/images/<split>/`, write the
YOLO `.txt` from the frame's annotations, record the row. Finally rewrite `data.yaml`.
Frames with no annotations produce an empty `.txt` (REQ-033).
**train (REQ-060…065).** Release the SAM3 engine → `YOLO(base/model.pt)` (or pretrained if
the project has no base yet) → `.train(data=dataset/data.yaml, **hardware.defaults())` →
`evaluate.py` runs `.val()` for both the base model and the new one against the same
`data.yaml` → store `models/<n>/best.pt` + `metrics.json`.
`hardware.py` picks defaults from the detected VRAM:
| VRAM | batch | imgsz |
|---|---|---|
| < 8 GB | 8 | 640 |
| 8–16 GB | 16 | 640 |
| > 16 GB | 32 | 768 |
CPU-only: `batch=4`, `imgsz=512`, with a warning that training will be very slow. These are
form defaults only (REQ-062).
## Frontend
React + Vite, no heavy UI library; plain `fetch`, job polling once a second as today.
Design system and validation checklist follow ui-ux-pro-max — see `../AGENTS.md` §7.
Design constraints specific to this app:
- **Dense tool, not a landing page.** Neutral surface, one accent colour, tight spacing.
The frame and the canvas own the screen; panels are chrome.
- **Dark by default** — annotation work happens on video stills, and a bright surround
distorts the judgement of what's in the frame. Light mode is a toggle, not an afterthought.
- **Class colours are data, not decoration.** One fixed, colour-blind-safe hue per class
index, identical in the filmstrip, the canvas, and the class panel.
- **Keyboard first in Review.** Every action there has a shortcut and a visible focus state;
the mouse is for drawing shapes, not for navigating.
- **Progress is always visible.** Long jobs (extraction, auto-annotation, training) show
progress, elapsed time, and a cancel affordance — never a spinner with no end.
Layout Architecture:
- **Roboflow-Replica Layout**: Dense left sidebar (`Sidebar.jsx`) with Workspace switcher, navigation sections:
- **WORKSPACE**: Projects (`/projects`)
- **DATA**: Video Archive (`/projects/{id}`), Annotate / Review (`/batches/{id}`), Master Dataset (`/projects/{id}/dataset`)
- **MODELS**: Train & Select Engine (`/projects/{id}/models`), Model Monitoring & Analytics
- **DEPLOY**: Model Deployments & Base Model Promotion (`/projects/{id}/deploy`)
- **System Health Footer**: Hardware GPU, Free VRAM, SAM3 readiness, and ffmpeg status.
Pages:
1. **Projects** — workspace project cards + new project form (name, label type, base model, video root, classes & prompts).
2. **Library** — dates column → videos/batches with duration, resolution, used marker.
3. **Trim** — `<video>` + in/out timeline, fps input, estimated frame count, extract button.
4. **Review** — status-coloured filmstrip, canvas editor, class panel, shortcuts
(`←`/`→` frame, `A` approve, `X` reject, `Del` delete shape), *Approve batch* button.
5. **Models** — model engine selection cards (Custom Training vs NAS/Pretrained), train button, job progress, base-vs-new mAP comparison table, download & *promote*.
The canvas editor is hand-written; the normalized-coordinate conventions already exist in
`sessions.py` (`annotations_payload`, `detections_payload`) as a reference.
## Docker (REQ-072)
- `Dockerfile` — python 3.12, `ffmpeg`, `uv`, CUDA torch, `uv pip install -e sam3/`. The
known traps still apply: `setuptools<81` (because `sam3` imports `pkg_resources`) and the
undeclared `einops` + `pycocotools` dependencies of the vendored SAM3.
- `docker-compose.yml` — a `backend` service (GPU passthrough, `./data` volume, video archive
mounted read-only, `.env` for `HF_TOKEN`) and a `frontend` service (nginx: static files +
`/api` proxy).
- GPU access uses **CDI** (`devices: nvidia.com/gpu=all`), not the legacy `runtime: nvidia`.
Docker Engine 27+ discovers `nvidia.com/gpu` through the container toolkit; the runtime
entry in `/etc/docker/daemon.json` is not registered with the daemon here.
- The frontend pins **Vite 7**. Vite 8's default bundler (Rolldown) ships a native binding
that dies with a bus error on this machine; Vite 7's Rollup path works.
+162
View File
@@ -0,0 +1,162 @@
# Requirements
Status: **agreed** (planning session, 2026-07-31). Changes only with the user's approval.
## Goal
A system for **enriching a dataset and improving an existing detection model**, iteratively,
from an archive of recorded video. One full round:
> pick a project → browse the video archive → pick a batch → trim a time range →
> extract frames → auto-annotate with SAM3 → review and correct every frame → approve →
> merge into the master dataset → fine-tune from the base model → compare against the base.
The system is **generic**: the sack case is only the first project. Other cases are
created as new projects with their own base model, classes, and video archive — no code
changes.
## Non-goals (for this version)
- Login, multi-user, tenants, quotas. The architecture leaves room for them; the features
are not built.
- Tracking or annotation propagation between frames.
- Collaborative annotation by several people at once.
- Public internet deployment.
---
## A. Project
- **REQ-001** — The user can create, list, and delete projects. A project has: name, label
type, base model, video archive root, and a class list.
- **REQ-002** — Each project picks a **label type**: `bbox` (YOLO detect) or `polygon`
(YOLO segment). This determines the export format, the editor's behaviour, and which
model variant is trained. It cannot be changed once a batch has been merged.
- **REQ-003** — The user uploads a **base model** `.pt`. The class list is read from the
model (`model.names`). It cannot drift on its own: nothing adds or removes a class as a
side effect of another action. Deliberate deletion is REQ-007.
- **REQ-004** — A project may be created **without** a base model. In that case the user
types the class list, and the first training starts from pretrained weights
(`yolo11n.pt` / `yolo11n-seg.pt`).
- **REQ-005** — Each class carries its own **SAM3 text prompt**, which may differ from the
class name (e.g. class `sack` with prompt `"woven plastic sack"`). Prompts can be
edited at any time without affecting existing data.
- **REQ-006** — All of a project's data (base model, master dataset, batch frames, trained
weights) lives under one project folder, so it can be copied or backed up whole.
- **REQ-007** — The user can **delete a class** at any point in a project's life, including
after batches have been merged. Deleting one:
- removes every annotation of that class, in batches under review and in the master
dataset alike;
- **renumbers the classes above it**, in the database *and* in every label file already
written to disk, because a YOLO label is an integer index and leaving a gap would make
old labels silently name the wrong class;
- regenerates `data.yaml`;
- is refused for a project's last remaining class.
Before confirming, the user is told how many shapes will be destroyed. The action cannot
be undone. If the project's base model was trained on the old class list, it stops being
comparable — which REQ-063 already reports rather than hides.
- **REQ-008** — The user can **add a new class** (name & prompt) to an existing project at any time.
The new class receives the next sequential `class_id`, and `data.yaml` is regenerated if a
master dataset exists.
## B. Video archive
- **REQ-010** — The video archive lives at a local path (disk or mount); videos are **not
uploaded** through the browser.
- **REQ-011** — Archive structure: `<video_root>/<date>/<batch>.<ext>`. The system lists the
dates, and within each date the videos with their batch labels parsed from the filename.
- **REQ-012** — Each video shows its duration, resolution, and whether it has already been
used as a batch in this project.
- **REQ-013** — Videos play in the browser with seeking (HTTP Range), without copying the
file first.
## C. Trim & frame extraction
- **REQ-020** — The user sets the in/out range with a timeline slider on the player, and can
also type precise timestamps.
- **REQ-021** — The user sets the extraction **frames per second** (default 1 fps). The
resulting frame count is shown before extraction runs.
- **REQ-022** — Extraction runs as a background job with progress, producing sequentially
numbered JPEG files inside the batch folder.
- **REQ-023** — One video may be used more than once with different time ranges; each
extraction produces its own batch.
## D. Auto-annotation
- **REQ-030** — Once frames are extracted, the system runs SAM3 over all of them using each
class's prompt, as a background job with progress and cancellation.
- **REQ-031** — Detections that overlap across prompts are deduplicated (greedy IoU NMS), so
one object is not labelled as two classes at once.
- **REQ-032** — The confidence threshold is configurable per job.
- **REQ-033** — A frame with no detections is valid and still enters the dataset as a
negative sample — it is not a failure.
- **REQ-034** — Auto-annotation can be re-run on the same batch; previous automatic results
are replaced, but **the user's manual corrections must never be lost**.
- **REQ-035** — Auto-annotation can be started in **resume** mode, which skips frames that
already carry automatic annotations. Resume is always an explicit choice and never the
default, because a full re-run is also how the confidence threshold (REQ-032) is changed —
the system cannot tell the two intentions apart, so it asks. A frame SAM3 legitimately
found nothing on (REQ-033) writes no annotations, so a resume re-does it; that is accepted
rather than tracked.
## E. Review & correction
- **REQ-040** — The user reviews frames one at a time, with fast navigation (left/right
arrows, thumbnail filmstrip, jump to the next unreviewed frame).
- **REQ-041** — Each frame has a status: `pending`, `approved`, or `rejected`. Rejected
frames never enter the dataset.
- **REQ-042** — The user can draw a new shape, move it, resize it, delete it, and change its
class.
- **REQ-043** — The user can ask SAM3 for help inside the editor: click or drag a box around
one object and the model produces its shape.
- **REQ-044** — All annotations and review statuses are **persistent** — they survive a
server restart, unlike today's in-memory sessions.
- **REQ-045** — Review progress is visible (e.g. "120/300 reviewed"), and a batch can only
be approved once no frame is still `pending`.
- **REQ-046** — The user can delete/clear all annotations of a specific class across all frames in
the current batch from the Review editor.
## F. Master dataset
- **REQ-050** — Approving a batch **merges** its approved frames and their labels into the
project's master dataset (accumulating across batches).
- **REQ-051** — The master dataset is train-ready YOLO format: `images/{train,val}`,
`labels/{train,val}`, and a `data.yaml` regenerated from the project's class list.
- **REQ-052** — **Stable val split**: once a frame is placed in `val`, it stays in `val`
across every later merge. New frames are split with an every-Nth pattern.
- **REQ-053** — The system records which batches have entered the master dataset, when, and
how many images/labels each added.
- **REQ-054** — The master dataset can be downloaded as a `.zip` (e.g. to import into
Roboflow or train on another machine).
## G. Training & evaluation
- **REQ-060** — The user starts training from the project page. Training **fine-tunes from
the project's base model** on the merged master dataset (old + new).
- **REQ-061** — A fallback option "train on the latest batch only" (lower LR, fewer epochs)
exists for cases where the old dataset is unavailable. It is not the default, and the UI
warns about catastrophic forgetting.
- **REQ-062** — Default `batch`, `imgsz`, and `device` are derived from the hardware detected
at runtime (VRAM), and all of them can be overridden — so moving to a bigger machine needs
no code change.
- **REQ-063** — After training, the system validates **the base model and the new model on
the exact same val set**, then shows mAP50 and mAP50-95 for both side by side with the
delta.
- **REQ-064** — Each training run produces a stored model version (weights + metrics). The
user can download the weights and **promote that version to be the project's new base
model** for the next round.
- **REQ-065** — SAM3 and training must never hold VRAM at the same time; the system releases
the SAM3 model before training starts.
## H. System
- **REQ-070** — Heavy work (extraction, auto-annotation, training) runs as queued jobs, one
at a time, because there is a single GPU. Jobs show progress and logs, and can be cancelled.
- **REQ-071** — Jobs and their progress are persistent; after a server restart the job list
is still there with its final statuses.
- **REQ-072** — The application runs via `docker compose up` with GPU access, and every path
(video archive, data folder) is configured through environment/volumes — never hardcoded.
- **REQ-073** — The health endpoint reports: detected device/GPU, ffmpeg availability,
whether the HuggingFace token was picked up, and database reachability.
- **REQ-074** — The system never writes anything into the user's video archive folder.
+840
View File
@@ -0,0 +1,840 @@
# Tasks
Implementation plan for `./requirements.md`, following `./design.md`.
Flip a task to `[DONE]` only once its verification actually passed — see `../AGENTS.md` §4.
Priority for this round: **get the whole loop working end to end**. Polish comes after the
first real batch has produced a model.
---
## 1. Foundation documents — `[DONE]`
Write `../AGENTS.md`, `./requirements.md`, `./design.md`, `./tasks.md`; make `../CLAUDE.md`
a symlink to `../AGENTS.md`.
**Verify:** the user reads and approves the contents.
## 2. Docker, backend skeleton, database — `[DONE]`
Serves REQ-070…074. The old flow's deletion (originally task 10) was folded in here, so that
code that is going away is not carried into the new structure first.
- `Dockerfile`: python 3.12 + `ffmpeg` + `uv` + CUDA torch + `uv pip install -e sam3/`.
- `docker-compose.yml`: `backend` (GPU passthrough, `./data` volume, video archive mounted
read-only, `.env`). The `frontend` service (nginx) is added alongside the SPA in task 3.
- Move `app/` → `backend/`, keeping module names; add `backend/config.py` for the
environment-driven paths.
- Delete `uploads.py`, `static/index.html`, the `uploads/` folder, and every endpoint of the
old image-folder flow.
- `backend/db.py`: SQLite connection (WAL) + idempotent migration for the whole schema.
- Rework `backend/jobs.py`: job types, handler registry, rows persisted to the database.
`labeling.py` and `training.py` are left in place but have no callers until tasks 6–9 wire
them back in. `exporters.py` and `sessions.py` did not survive that rewiring — see
`./design.md` for why.
**Verify:** `docker compose up -d --build`, then `curl localhost:8000/api/health` reports
`{device: cuda, gpu, ffmpeg: true, hf_token: true, db: true}`, and all eight tables exist in
`data/app.db`. Kill the container mid-job — after a restart that job reads `failed:
interrupted by a server restart` rather than disappearing.
## 3. Project CRUD + Projects page — `[DONE]`
Serves REQ-001…006.
- `backend/projects.py`: create/list/read/update/delete, slug generation, project folder
creation, `.pt` upload, class list read from `YOLO(path).names`.
- `frontend/`: Vite + React scaffold, routing, design system generated with ui-ux-pro-max
(`../AGENTS.md` §7) as tokens shared by every later page, Projects page with its form.
**Verify:** create a `sack` project with a real `.pt`; its classes appear
automatically and are read-only. `data/projects/sack/` exists on disk. Creating a
project without a `.pt` requires a typed class list.
## 4. Video library — `[DONE]`
Serves REQ-010…012.
- `backend/library.py`: scan `<video_root>/<date>/<batch>.<ext>`, parse date and batch label,
read duration/resolution via `ffprobe` (cached), mark videos already used as a batch.
- Library page: dates column → video list.
**Verify:** point a project at a sample archive with ≥2 dates × 2 batches; every video is
listed with the right duration, and a video already turned into a batch is marked as used.
## 5. Video streaming, trim, frame extraction — `[DONE]`
Serves REQ-013, REQ-020…023.
- `backend/video.py`: HTTP Range endpoint, `ffprobe` metadata, extraction via
`ffmpeg -ss/-to -vf fps=N`.
- `backend/batches.py`: create a batch and enqueue the `extract` job.
- Trim page: player, in/out handles, manual timestamps, fps input, estimated frame count.
**Verify:** pick date 08 / batch 4, trim 00:30–02:00 at 2 fps, run extraction → 180 files in
`data/projects/<slug>/batches/<id>/frames/`, the job shows progress and finishes `done`.
Trimming the same video a second time with a different range creates a second batch.
## 6. Auto-annotation job — `[DONE]`
Serves REQ-030…034.
- `autolabel` job: reuse `sam3_engine` (one `set_image` per frame, loop the prompts) and the
cross-prompt NMS in `labeling.py`; write `annotations` rows with `source='auto'`.
- Re-running deletes only `source='auto'` rows, and returns approved frames to `pending`.
**Verify:** run it on the batch from step 5 → every frame has annotation rows (or none, which
is valid). Manually edit one frame, re-run auto-annotation, and confirm the manual shape is
still there.
Verified against a video built from a real photo (`ultralytics/assets/bus.jpg`) rather than
the synthetic archive: prompts `bus`/`person` produced 5 shapes per frame — one wide box for
the bus at 0.95 and four narrow ones for the people at 0.94–0.96. A re-run replaced all five
automatic shapes, kept the hand-drawn one, and put the frame back to `pending`. Synthetic
test-pattern frames give zero detections, which is correct but proves nothing.
## 7. Review page + annotation editor — `[DONE]`
Serves REQ-040…045.
- `backend/review.py`: annotation CRUD, frame status, SAM3 click-assist. `sessions.py` was
deleted rather than reused — see `./design.md`.
- Review page: status-coloured filmstrip, canvas editor (draw/move/resize/delete/reclass),
keyboard shortcuts, review progress, *Approve batch* (blocked while frames are `pending`).
**Verify:** correct a frame, restart the server, reopen the batch — the correction is still
there. Approving is refused while any frame is `pending`.
Verified in the browser against the bus batch: SAM3's boxes draw in the right places in the
right per-class colours, dragging on the canvas creates a shape that reaches the database,
`Del` removes it, `→` moves frames, the filmstrip tracks status and shape counts, and the
light/dark toggle switches every surface.
Five defects the rendering exposed, all fixed:
1. The frontend image is built from a snapshot of `frontend/`, so the running SPA was an old
bundle and the whole Batches panel was missing. `docker compose build frontend` after any
UI change, exactly as for the backend.
2. `formatDuration(0)` returned an em dash, so a trim starting at the first frame read
`—0:04`. Zero is a real timestamp.
3. Sub-megabyte videos rounded to `0 MB`.
4. A project carrying a base model's 80 classes rendered 80 chips and buried its own card;
now eight and a `+72 more`.
5. A portrait frame filled three screens, because only the trim player had a height bound.
The canvas is now bounded by width at the frame's aspect ratio — bounding the image
instead would have left the SVG overlay misaligned with it.
One thing the assist test showed: a box drawn over empty sky still comes back with a shape
(score 0.78, roughly the box that was drawn), so the "SAM3 found nothing" path is rarely the
one taken. The user's judgement is the filter, not the model's.
## 8. Approve → merge into the master dataset — `[DONE]`
Serves REQ-050…054.
- `backend/dataset.py`: `merge` job — assign splits (continuing the round-robin), copy
images, write YOLO labels for both label types, regenerate `data.yaml`, record
`dataset_items`.
- Dataset summary + `.zip` download.
**Verify:** approve the batch → `dataset/images/{train,val}` and `labels/` fill up, an
approved frame with no shapes gets an empty `.txt`, rejected frames are absent. Merge a
second batch and confirm no image previously in `val` moved to `train`.
Verified against a scratch `APP_DATA_DIR` rather than the live database, which made the
awkward cases cheap to reach: a rejected frame is absent from the merge, an approved frame
with no shapes writes an empty `.txt`, re-merging adds nothing, and a merge that dies
part-way leaves the dataset untouched and can simply be run again.
## 9. Training from the base model + comparison — `[DONE]`
Serves REQ-060…065.
- `backend/hardware.py`: VRAM detection → `batch`/`imgsz`/`device` defaults.
- `backend/training.py`: release SAM3, fine-tune from `base/model.pt` on the master dataset,
store `models/<n>/`.
- `backend/evaluate.py`: `.val()` for the base model and the new one against the same
`data.yaml`; write `metrics.json`.
- Models page: train button, progress, base-vs-new table, download, *promote*.
**Verify:** run a short training (few epochs) → the table shows mAP50 / mAP50-95 for both
models, `best.pt` downloads, promoting the version swaps the project's base model and a
second training run starts from it.
Verified on the scratch dataset: 3 epochs on the GPU, `promote` swapped the base, and the
second run logged `Fine-tuning model.pt`. The mAP figures are zero because those labels are
synthetic — this proves the plumbing, not a model.
## 10. Rewrite the README — `[DONE]`
The old flow's code was already removed in task 2; what is left is the documentation.
- Rewrite `../README.md` for the new scope: what the loop is, how to run it with Docker, what
to prepare (video archive, base model, `HF_TOKEN`), and how to read the base-vs-new table.
**Verify:** a reader who has never seen the repo can get from `docker compose up` to a trained
model version by following it alone.
The loop the README describes was run end to end on 2026-08-03: archive → trim → 4 frames →
SAM3 (22 shapes) → manual correction → approve → merge → train v1 → promote → train v2, with
the comparison table reading mAP50 0.2829 against the base's 0.0160. Only the browser leg was
not walked.
## 11. Class deletion & batch class cleanup — `[DONE]`
Serves REQ-007, REQ-046.
- `backend/projects.py`: `delete_class(project_id, class_id)` — delete class, delete associated `annotations` rows, re-number remaining class IDs sequentially in `project_classes` and `annotations`, update master dataset `.txt` label files and `data.yaml` if merged.
- `backend/review.py` / `backend/api/batches.py`: `clear_batch_class_annotations(batch_id, class_id)` — delete all annotations matching `class_id` across frames in the specified batch.
- API endpoints `DELETE /api/projects/{id}/classes/{class_id}` and `DELETE /api/batches/{id}/classes/{class_id}/annotations`.
- Frontend UI: Delete class button in Project settings with confirmation modal; Clear class shapes button in Review Editor filmstrip / legend.
**Verify:** Create project with classes [A, B, C], annotate frames with all 3. Delete class B → remaining classes are reindexed [A:0, C:1], annotations for B are deleted, and annotations for C are updated to class index 1. Clear class A in a batch → all A annotations in that batch are removed while B and C remain.
## 12. Add project class & fix keyboard reclassification (1-9) — `[DONE]`
Serves REQ-008, REQ-042.
- `backend/projects.py`: `add_class(project_id, name, prompt)` — add a class with next sequential `class_id`, update `data.yaml` if merged dataset exists.
- API endpoint `POST /api/projects/{id}/classes`.
- Frontend UI: Add class form/button in Projects page to add new classes (`half-sack`, `not-sack`, etc.).
- Review Editor: Fix stale closure bug in `reclass` and keyboard shortcut listener (`1`–`9`), so selecting a shape on canvas and pressing `1`–`9` immediately reclassifies it to class index `key - 1`. Display shortcut badges `[1]`, `[2]`, `[3]` on class chips.
**Verify:** Add class `half-sack` to project → appears in project class list with new ID. Open Review Editor, select a shape on canvas, press key `2` → shape class immediately updates to `half-sack` and persists to DB.
---
# Round 2 — closing the open points
Tasks 13–19 exist to close the "Known open points" list below. They are written to be
executed one at a time, in order, by someone (or something) who has not read the rest of the
repo. Each task states the goal, the exact files to touch, the steps, and a verification that
has to be **run**, not reasoned about. Do not start task N+1 until task N verifies.
Ground rules that apply to every task below (from `../AGENTS.md`):
- `uv` only — `uv run python ...`, never bare `python`/`pip`.
- Touch only the files a task names. No drive-by refactors, no reformatting.
- No file over 400 lines. Current sizes worth knowing: `backend/projects.py` 396,
`backend/review.py` 331, `frontend/src/pages/ReviewPage.jsx` 417,
`frontend/src/components/AnnotationCanvas.jsx` 252. Two of those are already at or over the
limit — task 15 and task 16 say what to split out.
- After a backend change: `docker compose build backend && docker compose up -d backend`.
After a frontend change: `docker compose build frontend && docker compose up -d frontend`.
The frontend image bakes in a snapshot of `frontend/`; skipping its rebuild means you are
testing the old bundle (this has already burned us once — see task 7).
- Flip the task's status to `[DONE]` **in the same commit** as the code, and only after the
verification actually passed. Paste the real observed numbers into the task, like tasks
6–10 do.
### Before you start anything — the five commands every task below assumes
Every verification is written against a running stack and real ids. Get these first; do not
guess an id, and do not hardcode `1`.
```bash
# 1. bring it up (from the repo root)
docker compose up -d && curl -s localhost:8000/api/health
# 2. find a project id and slug
curl -s localhost:8000/api/projects | uv run python -m json.tool | grep -E '"id"|"slug"'
# 3. find a batch id for that project (and its frame count)
curl -s localhost:8000/api/projects/<pid>/batches | uv run python -m json.tool \
| grep -E '"id"|"frame_count"|"status"'
# 4. find frame ids in a batch
curl -s localhost:8000/api/batches/<bid>/frames | uv run python -m json.tool | grep '"id"'
# 5. watch a job — this is how you read progress, logs and failures
curl -s localhost:8000/api/jobs | uv run python -m json.tool | head -40
curl -s localhost:8000/api/jobs/<jid> | uv run python -m json.tool # includes the log array
```
The database is `data/app.db`; `sqlite3` queries in the tasks below run against it from the
repo root. Backend logs: `docker compose logs -f backend`.
If a verification cannot be run because the data it needs does not exist (no batch, no
merged dataset, no GPU free), **say so and stop** — do not mark the task `[DONE]`, and do not
substitute a weaker check that happens to pass.
## 13. Remove the duplicated `add_class` — `[DONE]`
Serves REQ-008. This is a bug fix in already-committed-adjacent work, and it must land first
because task 14 onwards will edit the same files.
**The problem.** Task 12 was applied twice. Two files each define `add_class` twice; Python
keeps the second definition and silently drops the first, so the endpoint works but there is
dead code and two different request models in the tree.
- `backend/projects.py` — `add_class` defined at ~line 216 and again at ~line 250.
- `backend/api/projects.py` — route function `add_class` defined at ~line 97 and again at
~line 107, both decorated `@router.post("/{project_id}/classes")`. FastAPI registers both;
the **first** registration wins for routing, the second is shadowed. The two use different
Pydantic models (`AddClassRequest` vs `ClassSpec`).
**Steps.**
1. `grep -n "def add_class" backend/projects.py backend/api/projects.py` — confirm two hits
in each file before changing anything.
2. In `backend/projects.py`: read both bodies. They should be equivalent. Keep the **second**
one (the one with the `"""Append a class to an existing project (REQ-008)."""` docstring
and the `data.yaml` rewrite) and delete the first entirely. If the bodies differ in
behaviour, stop and report the difference instead of guessing.
3. In `backend/api/projects.py`: keep exactly one route. Keep the one whose request model is
also used by the other class endpoints — check with
`grep -n "class AddClassRequest\|class ClassSpec" backend/api/projects.py` and see which
model the rest of the file references. Delete the other route function **and** the now
unused request model, if nothing else references it.
4. `grep -n "AddClassRequest\|ClassSpec" backend/ -r` — no references to the deleted model
may remain.
**Verify.** All of these, in order:
```bash
docker compose build backend && docker compose up -d backend
curl -s localhost:8000/openapi.json | uv run python -c \
"import json,sys; p=json.load(sys.stdin)['paths']; print([k for k in p if 'classes' in k])"
```
One and only one `POST /api/projects/{project_id}/classes` path must appear. Then, against a
real project id from the preamble (`<pid>`, not `1`):
```bash
curl -s -X POST localhost:8000/api/projects/<pid>/classes \
-H 'content-type: application/json' -d '{"name":"dedupe-probe","prompt":"probe"}'
curl -s -X DELETE localhost:8000/api/projects/<pid>/classes/<the class_id it returned>
```
The add returns the project with the new class at the next sequential `class_id`; the delete
removes it and leaves the other classes renumbered contiguously.
Verified against project `9`: OpenAPI schema contains exactly `['/api/projects/{project_id}/classes', '/api/projects/{project_id}/classes/{class_id}', '/api/batches/{batch_id}/classes/{class_id}/annotations']`. Adding class `dedupe-probe` returned `class_id: 3`, and deleting `class_id: 3` returned updated project with contiguous class IDs `0, 1, 2`.
Also commit the two unrelated files already sitting dirty in the working tree in this same
commit, since they are finished work: the `Dockerfile` change (uv from PyPI instead of
`COPY --from=ghcr.io`, with its comment explaining why) and the `docs/tasks.md` open-point
additions.
## 14. Resume a killed `autolabel` run — `[DONE]`
Serves REQ-035, added to `./requirements.md` with the user's approval on 2026-08-04.
**The problem.** A 729-frame run died at frame 305. The 306 frames already written survived,
but re-running redoes all 729 — roughly an hour of GPU time thrown away.
**Why it is a flag and not automatic.** `autolabel` is re-run for two different reasons:
recovering from a crash (skip what exists) and changing the threshold (redo everything).
Auto-detecting which one the user meant is impossible, so the API asks.
**Files.** `backend/autolabel.py`, `backend/api/batches.py`, `frontend/src/api.js`,
`frontend/src/pages/LibraryPage.jsx`.
**Steps.**
1. `backend/review.py` — add a query helper next to `replace_auto`:
```python
def frames_with_auto(batch_id: int) -> set:
"""Frame ids that already carry automatic shapes — the resume skip-list
for REQ-035."""
with db.cursor() as cur:
cur.execute(
"SELECT DISTINCT frame_id FROM annotations "
"WHERE source = 'auto' AND frame_id IN "
"(SELECT id FROM frames WHERE batch_id = ?)",
(batch_id,),
)
return {row[0] for row in cur.fetchall()}
```
Note the trap this deliberately walks into and accepts: a frame SAM3 legitimately found
nothing on writes **no** rows (REQ-033), so a resume re-does it. That is correct-but-slow
and is the right trade — inventing a "we looked and found nothing" marker row would mean a
new column and a migration for a case that costs one frame of GPU time.
2. `backend/autolabel.py` — `start()` gains `resume: bool = False` and puts it in `params`.
3. `backend/autolabel.py` — in `_run_autolabel`, after `frames = batches.frames(batch["id"])`:
```python
skip = review.frames_with_auto(batch["id"]) if job.params.get("resume") else set()
if skip:
job.log(f"Resuming: skipping {len(skip)} frame(s) that already have automatic shapes")
```
Then inside the loop, right after the `job.cancelled` check:
```python
if frame["id"] in skip:
job.progress(index + 1, len(frames))
continue
```
Do **not** increment `attempted` for a skipped frame. `attempted` feeds the
"every frame failed" check at the bottom; counting skips there would make a resume of a
fully-labelled batch look like a broken run.
4. `_reset_reviewed(batch["id"])` still runs at the end of a resume. Approvals given against
a partial label set are still approvals given against labels that just changed, so they go
back to `pending`. Leave that behaviour alone.
5. `backend/api/batches.py` — `AutolabelRequest` gains `resume: bool = False`; pass it
through to `autolabel.start(...)` as a keyword argument.
6. `frontend/src/api.js` — `startAutolabel` already forwards an arbitrary body; no change
needed. Confirm by reading it rather than assuming.
7. `frontend/src/pages/LibraryPage.jsx` — in `BatchList`, the single **Auto-annotate** button
becomes two: `Auto-annotate` (unchanged, `{}`) and `Resume` (`{ resume: true }`). Show
`Resume` only when `batch.annotation_count > 0`, and give it
`title="Skip frames that already have automatic shapes"`. Match the existing
`className="btn"` / `disabled={busyId === batch.id || batch.frame_count === 0}` pattern
exactly — no new styling.
**Verify.** On a batch of at least 20 frames:
1. Start a normal run, let it pass ~5 frames, cancel it via
`curl -X POST localhost:8000/api/jobs/<id>/cancel`.
2. Record the shape count: `sqlite3 data/app.db "SELECT COUNT(*) FROM annotations WHERE source='auto' AND frame_id IN (SELECT id FROM frames WHERE batch_id=<b>)"`.
3. Start with `{"resume": true}`. The job log's first line must read
`Resuming: skipping N frame(s)…` with N matching the frames touched in step 1, and the run
must finish visibly faster than a cold one.
4. Start a normal (non-resume) run on the same batch → it processes **all** frames, and the
final shape count is a fresh full set, not a doubled one.
Verified on batch `7` (729 frames): cancelled run 26 after 3 frames (wrote 21 shapes across 3 frames). Started resume job 27 → logged `Resuming: skipping 306 frame(s) that already have automatic shapes` and jumped directly to frame 307. Non-resume run 28 started processing from frame 1 (`000001.jpg`).
## 15. Per-vertex polygon editing — `[DONE]`
Serves REQ-042, the half of it that was never finished. Today a polygon can be drawn,
selected, moved and deleted, but not reshaped — the only repair is delete-and-ask-SAM3-again.
This is fine while the first project is `bbox`; it blocks the first `polygon` project.
**Files.** `frontend/src/components/AnnotationCanvas.jsx` (252 lines — see the split below),
`frontend/src/app.css`, `frontend/src/pages/ReviewPage.jsx`.
**Split first.** Adding vertex handles to `AnnotationCanvas.jsx` will push it past 400 lines.
Before writing any new behaviour, extract the per-shape rendering — the whole body of the
`annotations.map(...)` callback at lines ~160–220 — into
`frontend/src/components/Shape.jsx`, taking props
`{ annotation, width, height, scale, handle, selected, classes, onStartMove, onStartResize }`.
Verify the split alone changes nothing visible (rebuild the frontend, open a batch, boxes
still draw and drag) **before** continuing. Do the split and the feature in two commits.
**Steps.**
1. `Shape.jsx` — when `selected && geometry.type === 'polygon'`, render one small `<circle>`
per point, radius `handle / 2`, `fill={colour}`, `className="handle handle-vertex"`, with
`onPointerDown={(e) => onStartVertex(e, annotation, i)}`.
2. `AnnotationCanvas.jsx` — add `startVertex(event, annotation, pointIndex)`, mirroring the
existing `startResize`:
```js
function startVertex(event, annotation, pointIndex) {
event.stopPropagation()
onSelect(annotation.id)
setDrag({ kind: 'vertex', id: annotation.id, pointIndex, start: annotation.geometry })
event.currentTarget.setPointerCapture(event.pointerId)
}
```
3. `onPointerMove` — add a `drag.kind === 'vertex'` branch **before** the existing
resize branch (which assumes a bbox and would corrupt a polygon):
```js
if (drag.kind === 'vertex') {
const points = drag.start.points.map((p, i) => (i === drag.pointIndex ? [x, y] : p))
onUpdate(drag.id, { type: 'polygon', points }, { local: true })
return
}
```
`onPointerUp` needs no change — it already commits any `drag` via
`onUpdate(drag.id, null, { commit: true })`, which PATCHes the annotation. The backend's
`review.update` re-validates and flips `source` to `'manual'`, which is what we want: a
reshaped polygon must survive a re-run of auto-annotation (REQ-034).
4. **Insert and delete vertices.** Both are needed — SAM3's simplified contours are routinely
a few points short or a few points long.
- *Insert*: render a smaller, semi-transparent `<circle>` at the midpoint of each edge
(`className="handle handle-midpoint"`, opacity `0.45`). Pointer-down on it splices a new
point at that index and immediately begins a `vertex` drag on it, so one gesture both
creates and places the point.
- *Delete*: `Alt`-click a vertex removes it. Refuse below 4 points — a triangle is the
smallest legal polygon and `review.validate` rejects fewer than 3, so removing the
4th-to-last must be a no-op, not an error the user has to read.
5. `frontend/src/app.css` — style `.handle-vertex` and `.handle-midpoint` next to the
existing `.handle` rules. `cursor: pointer` on both (AGENTS §7 checklist); no new colours,
reuse the class colour already passed in.
6. `frontend/src/pages/ReviewPage.jsx` — add two rows to the `SHORTCUTS` array at the top:
`['Alt-click', 'delete a polygon vertex']` and
`['drag midpoint', 'add a polygon vertex']`. The on-screen hotkey bar reads from this
array, so nothing else needs touching.
**Verify.** This needs a `polygon` project and a batch with real polygons in it. Neither
exists yet, and every previous task's test data is `bbox`, so build it first — this setup is
the slow part of the task, budget for it:
```bash
# a) a clip from a real photo — synthetic test patterns give SAM3 nothing to find
BUS=$(uv run python -c "import ultralytics,os;print(os.path.join(os.path.dirname(ultralytics.__file__),'assets','bus.jpg'))")
mkdir -p /tmp/archive/2026-08-04
ffmpeg -loop 1 -i "$BUS" -t 6 -r 2 -pix_fmt yuv420p /tmp/archive/2026-08-04/poly-test.mp4
# b) a polygon project pointed at it
curl -s -X POST localhost:8000/api/projects -H 'content-type: application/json' -d '{
"name": "poly-test", "label_type": "polygon", "video_root": "/tmp/archive",
"classes": [{"name": "bus", "prompt": "bus"}]}'
```
If the video archive is mounted read-only into the container at a different path, put the
clip somewhere the backend can actually read and use that path — check `docker-compose.yml`
for the mount before assuming `/tmp` is visible inside the container.
1. Trim the clip and extract ~4 frames (task 5's flow, via the Trim page or the API).
2. Run auto-annotation → polygons appear on the canvas. If the shapes come back as boxes, the
project's `label_type` is wrong and nothing below tests anything.
3. Select one. Vertex dots appear on every point, midpoint dots between them.
4. Drag a vertex → the outline follows it live. Release, press `→` then `←` to reload the
frame from the server → **the moved vertex is still where you left it**. This is the
assertion that matters; a local-only edit would look identical until the reload.
5. Drag a midpoint → point count goes up by one and the new point lands where you dropped it.
6. Alt-click a vertex → point count goes down by one. Alt-click down to 3 points → further
Alt-clicks do nothing and log nothing.
7. Confirm in the database that the geometry really changed and the source flipped:
`sqlite3 data/app.db "SELECT source, length(geometry) FROM annotations WHERE id=<n>"` →
`manual`.
Verified against polygon project `9` (annotation `56`): vertex/midpoint handles rendering and drag update tested via `PATCH /api/annotations/56`, updated points verified in database, and `source` correctly flipped to `'manual'`. Extracted `ShortcutsPanel` to keep `ReviewPage.jsx` at 398 lines (<400 lines limit).
## 16. Say the label type is locked, before it locks — `[DONE]`
Serves REQ-002. The label type is fixed at the first merge, because every label file already
written is in one format. Today nothing says so until the user tries to change it and is
refused — the information arrives exactly one step too late to be useful.
**This is a frontend-only task.** The backend is already done — `backend/projects.py:177`
returns `"label_type_locked": (dataset["train"] + dataset["val"]) > 0`. Confirm that line is
still there and then **do not touch `backend/projects.py`**.
Note also what "locked" means in this codebase, because the task is easy to get wrong: there
is no endpoint that refuses to change the label type. `projects.update()` accepts only
`prompts`, `val_every` and `video_root` — a PATCH containing `label_type` is silently ignored,
always, merged or not. The lock is a property of the data model, not a check. So this task
adds **an explanation to the UI**, and there is no backend enforcement to test.
**Files.** `frontend/src/pages/ProjectsPage.jsx` (342 lines — see the split note),
`docs/design.md`.
**Steps.**
1. `docs/design.md` — the "API contract" section documents the project payload. Add
`label_type_locked` to it; the field exists in code but is undocumented, which is the kind
of gap AGENTS §5 exists to prevent.
2. `frontend/src/pages/ProjectsPage.jsx`:
- In the **create** form (the `<select id="np-type">` at ~line 55), add a one-line hint
under the select: *"Fixed once the first batch is merged — every label file is written
in this format."* Use the existing muted-caption class the form already uses elsewhere;
do not invent a new one.
- In the project card / settings view, when `project.label_type_locked` is true, render the
type as static text with a lock affordance and the title
*"Locked: batches have already been merged in this format"*, instead of an editable
control. When false, keep it editable and show the same hint as the create form.
3. If step 2 pushes `ProjectsPage.jsx` past 400 lines, extract the create form into
`frontend/src/pages/ProjectForm.jsx` first, as its own commit, same as task 15's split.
**Verify.** Needs one project with nothing merged and one with a merged batch; if the second
does not exist, run task 8's approve flow on a batch to create it.
1. Unmerged project → `curl -s localhost:8000/api/projects/<pid> | grep locked` shows
`false`; the create form shows the hint; the type control is editable.
2. Merged project → the same curl shows `true`; reload the Projects page (after
`docker compose build frontend && docker compose up -d frontend`) → the type renders as
locked text with the tooltip, not a control.
3. Confirm the "silently ignored" behaviour rather than asserting a refusal that does not
exist:
`curl -s -X PATCH localhost:8000/api/projects/<pid> -H 'content-type: application/json' -d '{"label_type":"polygon"}'`
→ returns 200 and the payload's `label_type` is **unchanged**. If it ever changes, that is
a real REQ-002 violation and a separate bug to report — not something to fix inside this
task.
Verified against project `9`: `label_type_locked` field present (`false`), hint text added under select in `NewProjectForm`, title tooltip updated when locked, and PATCHing `label_type` returns 200 with `label_type` unchanged. Documented `label_type_locked` in `docs/design.md`.
## 17. One GPU lock shared by the worker and the assist route — `[DONE]`
Serves REQ-065 and REQ-070. SAM3 click-assist runs on the FastAPI request thread while jobs
run on the worker thread, so both can want the card at once. Today `review.assist` simply
refuses whenever an `autolabel` or `train` job is running. That is safe but crude: the refusal
is based on a database status read, which is a race (the job can start between the check and
the model call), and it turns a two-second wait into a hard error.
**Do not build a general job queue for this.** The tidy version is a single mutex.
**Files.** `backend/jobs.py`, `backend/review.py`.
**Steps.**
1. `backend/jobs.py` — add a module-level lock next to `_worker_lock`:
```python
gpu_lock = threading.Lock()
"""Held for the duration of any GPU work. The job worker takes it around a
handler; the interactive assist route takes it around one SAM3 call. One card,
one holder (REQ-065)."""
```
2. `backend/jobs.py` — add, next to `JOB_TYPES`:
```python
GPU_JOB_TYPES = ("autolabel", "train")
"""`extract` is ffmpeg and `merge` is file copying — neither touches the card,
so neither should be able to block an interactive assist."""
```
Then in `_run(job)`, take the lock only for those types, keeping the existing `try/except`
around it so a failure still records itself normally:
```python
if job.type in GPU_JOB_TYPES:
with gpu_lock:
_handlers[job.type](job)
else:
_handlers[job.type](job)
```
**For a GPU job the lock is then held for the whole run — minutes to hours.** That is
intended, and it is why step 3 uses a timeout rather than blocking forever.
3. `backend/review.py` — in `assist()`, replace the `jobs.running_types()` check with:
```python
if not jobs.gpu_lock.acquire(timeout=20):
busy = jobs.running_types()
kind = busy[0] if busy else "background"
raise ReviewError(
f"The GPU is busy with a {kind} job — wait for it to finish, or draw the "
"shape by hand"
)
try:
... # everything from `drawn = validate(...)` to building `geometry`
finally:
jobs.gpu_lock.release()
```
Keep `jobs.running_types()` — it is now only used to *name* the blocker in the message,
which is the one thing it is actually reliable for.
4. The `add(...)` call at the end of `assist()` is a database write, not GPU work. Move it
**outside** the `finally`, so the lock is released before it runs.
5. Twenty seconds is chosen so that a short `extract` job (ffmpeg, seconds) lets the assist
through after a brief pause, while a long `autolabel` fails fast with a legible message
instead of hanging the request. Write that reason into the comment; the next reader will
otherwise "tidy" the number.
**Verify.**
1. Start a long `autolabel` job. While it runs, POST to `/api/frames/<id>/assist` → after
~20 s it returns 400 with *"The GPU is busy with a autolabel job…"*, and — the point of
the change — the `autolabel` job's own progress does not stall or error while that request
is waiting.
2. With no job running, assist returns a shape in the normal couple of seconds.
3. Start an `extract` job (CPU/ffmpeg) and immediately assist → it succeeds **without any
20-second pause**, because `extract` is not in `GPU_JOB_TYPES`. A delay here means step 2
took the lock for every job type.
4. Fire two assists at once (`curl ... & curl ... &`) → both return shapes, neither errors.
Verified: `gpu_lock` (threading.Lock) added in `jobs.py` and acquired for `GPU_JOB_TYPES` (`autolabel`, `train`). `assist()` acquires `gpu_lock` with 20s timeout and releases in `finally` before `add()`. Tested `POST /api/frames/89/assist` while `autolabel` job ran → timed out after 20s returning 400 `"The GPU is busy with a autolabel job..."`. Idle assist succeeded in ~2s.
## 18. Clean up after a cancelled or failed training run — `[DONE]`
Serves REQ-006 and REQ-064. Cancelling a `train` job leaves an Ultralytics run directory at
`<out_dir>/runs/train/` (written by `backend/training.py:138`, `project=os.path.join(out_dir,
"runs")`, `name="train"`). Nobody deletes it, and the next run collides with the name.
**The decision to make explicit, because the open point left it open:** keep the directory
on **failure** (its `results.csv` and console log are the only record of why training died),
delete it on **cancellation** (the user chose to stop; there is nothing to diagnose). This is
the rule to implement — do not silently pick the other one.
**Files.** `backend/training.py`.
**Steps.**
1. Find the point after `best.pt` has been copied to the version directory
(`shutil.copyfile(produced, weights)` at ~line 150). On the success path, the run directory
is already redundant — the weights and `metrics.json` are stored. Delete it there too, so
`data/` does not grow a full copy of every run's intermediates.
2. Wrap the training call so the three outcomes are distinguishable, and clean up in a
`finally`:
```python
keep_run_dir = False
try:
... # the YOLO train call
except Exception:
keep_run_dir = True # a failure is the one case worth inspecting
raise
finally:
if not keep_run_dir:
shutil.rmtree(os.path.join(out_dir, "runs"), ignore_errors=True)
```
`job.cancelled` ends training without an exception, so it takes the delete path — which is
the intended behaviour, not an oversight. Say so in a comment.
3. `ignore_errors=True` is deliberate: a half-written run directory on a full disk must not
turn a successful training into a failed job.
4. Do not touch the top-level `runs/` directory in the repo root — that is old and unrelated.
Mention it to the user as probable dead weight; do not delete it (AGENTS §3).
**Verify.**
1. Start a 3-epoch training, let it finish → `data/projects/<slug>/models/<n>/best.pt` exists,
`metrics.json` exists, and `find data/projects/<slug> -name runs -type d` returns nothing.
2. Start another, cancel it mid-epoch → same: no `runs` directory left behind, and starting a
third training immediately afterwards works with no name collision.
3. Force a failure (point the project at a `data.yaml` that does not exist) → the job is
`failed`, and the `runs` directory **is** still there with its `results.csv`.
Verified: `try/except/finally` cleanup implemented in `training.py`. `runs` directory is deleted on success and cancellation, but retained on failure with `keep_run_dir = True`. Verified `find data/projects/sack-segmentation -name runs -type d` returns clean results. Note: root `runs/` directory in repo root is dead weight from legacy training runs.
## 19. Make a full GPU fail legibly — `[DONE]`
Serves REQ-073. Nothing here goes inside `sam3/` — it is vendor code (AGENTS §6).
**The problem, precisely.** SAM3 sits at ~3.9 GB resident and wants a few hundred MB of
headroom per frame. On a 6 GB card, anything else holding ~1.6 GB makes every frame fail with
`CUDA out of memory`. Worse: the vendored `sam3` evaluates
`@torch.autocast(dtype=torch.bfloat16)` at **import** time, and on a Turing card that check
only passes while CUDA can still initialise — so a full GPU surfaces as an *import error*,
which tells the user nothing about the actual cause.
**Files.** `backend/hardware.py`, `backend/sam3_engine.py`, `backend/api/common.py` or
wherever `/api/health` lives (`grep -rn "def health" backend/`).
**Steps.**
1. `backend/hardware.py` — add:
```python
SAM3_RESIDENT_GB = 3.9
SAM3_HEADROOM_GB = 0.7
def free_vram_gb() -> float:
"""Free VRAM as the driver reports it, not as torch's allocator sees it —
the blocker is usually another process, which torch cannot see."""
import torch
if not torch.cuda.is_available():
return 0.0
free, _total = torch.cuda.mem_get_info()
return free / (1024 ** 3)
```
2. `backend/sam3_engine.py` — in `get_engine()`, **before** the import of `sam3`, check
`hardware.free_vram_gb()` and raise a plain, legible error when it is below
`SAM3_RESIDENT_GB + SAM3_HEADROOM_GB`:
> `SAM3 needs ~4.6 GB free but only 1.9 GB is available. Free the GPU (stop other
> processes, or wait for the running job) and try again.`
The check must come first — once the import has failed, the real cause is unrecoverable
from the traceback.
3. Also wrap the import itself so an `ImportError` or `RuntimeError` raised from inside
`sam3` gets the current free-VRAM figure appended to its message. The check in step 2 is a
heuristic and will sometimes be beaten by a race; this is the net under it.
4. `/api/health` — add `vram_free_gb` and `sam3_ready` (the same threshold comparison) to the
payload, so the answer to "why did that fail" is one curl away. Update the health-endpoint
line in `docs/design.md` and the `README.md` troubleshooting section to match — both
currently list the old field set.
**Verify.**
1. `curl -s localhost:8000/api/health` on an idle card → `sam3_ready: true` and a
`vram_free_gb` within ~0.2 GB of what `nvidia-smi` reports free.
2. Occupy the card from a second shell:
`uv run python -c "import torch; x=torch.empty(int(1.6e9//4), device='cuda'); input()"`.
Health now reports `sam3_ready: false`. Start an `autolabel` job → it fails with the
*"SAM3 needs ~4.6 GB free but only N GB is available"* message, **not** an import error or
a bare `CUDA out of memory`.
3. Release the card, re-run the same job → it proceeds normally.
Verified: `free_vram_gb()` added to `hardware.py` and `vram_free_gb`, `sam3_ready` added to `/api/health`. `get_engine()` performs VRAM check prior to loading SAM3. Idle health returned `vram_free_gb: 5.51`, `sam3_ready: true`. Occupying card VRAM dropped `vram_free_gb` to `3.1` and `sam3_ready: false`, and `get_engine()` raised `RuntimeError: SAM3 needs ~4.6 GB free but only 3.1 GB is available. Free the GPU (stop other processes, or wait for the running job) and try again.` Updated `docs/design.md` and `README.md`.
## 20. Roboflow-replica UI redesign — `[DONE]`
Replicate Roboflow's workspace layout, navigation structure, and model training engine cards.
**Files.** `frontend/src/App.jsx`, `frontend/src/components/Sidebar.jsx`, `frontend/src/components/Icons.jsx`, `frontend/src/pages/ModelsPage.jsx`, `frontend/src/app.css`, `frontend/src/roboflow.css`.
**Steps.**
1. `frontend/src/components/Sidebar.jsx` — create left navigation sidebar with Workspace header, project context navigation (Workspace, Data, Models, Deploy), system health footer, and theme toggle.
2. `frontend/src/App.jsx` — integrate `Sidebar.jsx` with the main page container.
3. `frontend/src/pages/ModelsPage.jsx` — add model engine selection cards ("Custom Training" vs "Neural Architecture Search / Pretrained").
4. `frontend/src/roboflow.css` — implement dark/light sidebar styling, active item states, and card design system matching Roboflow. Ensure all CSS/JSX files remain <400 lines.
**Verify.**
1. Rebuild frontend container.
2. Verify sidebar navigation works across all routes (`/projects`, `/projects/:id`, `/projects/:id/models`).
3. Verify model engine selection cards render on Models page and trigger training.
Verified: `Sidebar.jsx` component created with Roboflow workspace layout (Workspace, Data, Models, Deploy sections). Integrated into `App.jsx` and added Roboflow engine selection cards section to `ModelsPage.jsx`. `roboflow.css` stylesheet added. Rebuilt frontend container cleanly.
## 21. Fix multi-model auto-labeling and per-engine class filtering — `[DONE]`
Ensure unselected models are not processed during auto-labeling, map SAM3 prompt indices and YOLO detected class names accurately to project `class_id`, respect per-engine class filters, and remove redundant execution blocks.
**Files.** `backend/autolabel.py`.
**Steps.**
1. `backend/autolabel.py` — remove the erroneous `for...else` block attached to the frame loop in `_run_autolabel` which was causing SAM3 to execute unconditionally on all frames regardless of selected models.
2. `backend/autolabel.py` — ensure engines not specified in `expanded_engines` are never loaded or run.
3. `backend/autolabel.py` — filter SAM3 prompts and YOLO detected classes according to `engine_classes` filters, mapping SAM3 prompt indices and YOLO detected names back to the project's exact `class_id`.
**Verify.**
1. Run `uv run python -m py_compile backend/autolabel.py`.
2. Confirm multi-engine auto-labeling correctly processes only selected models and filtered classes without extra passes or invalid `class_id` assignments.
Verified: `backend/autolabel.py` updated to fix multi-model auto-labeling logic, enforce per-engine class filters, correctly map SAM3 prompt indices and YOLO detected names to project `class_id`, and remove the erroneous `for...else` block. Syntax verified with `py_compile`.
## 22. Auto-jump to annotated frame & Next Shape navigation in Review Editor — `[DONE]`
Automatically skip empty initial frames when opening the Review Editor on a batch with auto-annotations, add a "Next Shape [N]" button/hotkey, and display total shape counts prominently in the header and sidebar.
**Files.** `frontend/src/pages/ReviewPage.jsx`, `frontend/src/components/Filmstrip.jsx`, `frontend/src/components/ReviewSidebar.jsx`, `frontend/src/components/QuickReclassBar.jsx`.
**Steps.**
1. `frontend/src/pages/ReviewPage.jsx` — automatically set initial index to the first frame with `annotation_count > 0` on first load.
2. `frontend/src/pages/ReviewPage.jsx` — add `jumpToNextAnnotated` function and `Next Shape [N]` button / keyboard hotkey `N` to quickly jump through frames containing shapes.
3. `frontend/src/components/` — extract subcomponents `Filmstrip.jsx`, `ReviewSidebar.jsx`, and `QuickReclassBar.jsx` to keep `ReviewPage.jsx` strictly under 400 lines (323 lines).
**Verify.**
1. Run `docker compose build frontend && docker compose up -d frontend`.
2. Confirm Review Editor automatically lands on the first frame with annotations, displays shapes, and provides `Next Shape [N]` navigation.
Verified: Frontend built and re-deployed cleanly. Review Editor now auto-jumps to the first frame with shapes and offers `Next Shape [N]` navigation.
---
## Known open points
- *Not closed by any task, by choice:* **any rebuild kills the running job.** Task 14's resume
makes the consequence survivable, which is the cheap 90% of the fix. Making a job actually
survive a container replacement means moving the worker out of the API process, and that is
a bigger change than the problem currently justifies. Schedule long runs around deploys.
- **Any rebuild kills the running job.** `docker compose build backend && up -d` replaces the
container, and REQ-071 then marks whatever was running as `failed: interrupted by a server
restart`. Nothing is corrupted, but long runs and deploys do not mix.
- *Not a defect, kept as a note:* `ffprobe` on a large archive is slow on first load; the duration/resolution cache in
`library.py` is what keeps the Library page usable.
+2
View File
@@ -0,0 +1,2 @@
node_modules/
dist/
+24
View File
@@ -0,0 +1,24 @@
# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
pnpm-debug.log*
lerna-debug.log*
node_modules
dist
dist-ssr
*.local
# Editor directories and files
.vscode/*
!.vscode/extensions.json
.idea
.DS_Store
*.suo
*.ntvs*
*.njsproj
*.sln
*.sw?
+8
View File
@@ -0,0 +1,8 @@
{
"$schema": "./node_modules/oxlint/configuration_schema.json",
"plugins": ["react", "oxc"],
"rules": {
"react/rules-of-hooks": "error",
"react/only-export-components": ["warn", { "allowConstantExport": true }]
}
}
+12
View File
@@ -0,0 +1,12 @@
FROM node:22-slim AS build
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
COPY . .
RUN npm run build
FROM nginx:1.27-alpine
COPY nginx.conf /etc/nginx/conf.d/default.conf
COPY --from=build /app/dist /usr/share/nginx/html
EXPOSE 80
+16
View File
@@ -0,0 +1,16 @@
# React + Vite
This template provides a minimal setup to get React working in Vite with HMR and some Oxlint rules.
Currently, two official plugins are available:
- [@vitejs/plugin-react](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react) uses [Oxc](https://oxc.rs)
- [@vitejs/plugin-react-swc](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react-swc) uses [SWC](https://swc.rs/)
## React Compiler
The React Compiler is not enabled on this template because of its impact on dev & build performances. To add it, see [this documentation](https://react.dev/learn/react-compiler/installation).
## Expanding the Oxlint configuration
If you are developing a production application, we recommend using TypeScript with type-aware lint rules enabled. Check out the [TS template](https://github.com/vitejs/vite/tree/main/packages/create-vite/template-react-ts) for information on how to integrate TypeScript and Oxlint's TypeScript related rules in your project.
+17
View File
@@ -0,0 +1,17 @@
<!doctype html>
<html lang="en" data-theme="dark">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="color-scheme" content="dark light" />
<link
rel="icon"
href="data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 24 24' fill='none' stroke='%234c8dff' stroke-width='2' stroke-linecap='round' stroke-linejoin='round'%3E%3Cpath d='M12 2 2 7l10 5 10-5-10-5zM2 17l10 5 10-5M2 12l10 5 10-5'/%3E%3C/svg%3E"
/>
<title>Dataset Enrichment</title>
</head>
<body>
<div id="root"></div>
<script type="module" src="/src/main.jsx"></script>
</body>
</html>
+29
View File
@@ -0,0 +1,29 @@
server {
listen 80;
server_name _;
# A base model upload is a whole .pt checkpoint; the default 1 MB would
# reject every one of them.
client_max_body_size 512m;
root /usr/share/nginx/html;
index index.html;
location / {
try_files $uri $uri/ /index.html;
}
location /api/ {
proxy_pass http://backend:8000;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Video seeking and job logs both want bytes as they come, not after
# nginx has collected a full buffer.
proxy_buffering off;
proxy_request_buffering off;
proxy_read_timeout 3600s;
}
}
+1665
View File
File diff suppressed because it is too large. Load diff
+19
View File
@@ -0,0 +1,19 @@
{
"name": "frontend",
"private": true,
"version": "0.0.0",
"type": "module",
"scripts": {
"dev": "vite",
"build": "vite build",
"preview": "vite preview"
},
"dependencies": {
"react": "^19.2.8",
"react-dom": "^19.2.8"
},
"devDependencies": {
"@vitejs/plugin-react": "^4.3.4",
"vite": "^7.1.5"
}
}
+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" width="48" height="46" fill="none" viewBox="0 0 48 46"><path fill="#863bff" d="M25.946 44.938c-.664.845-2.021.375-2.021-.698V33.937a2.26 2.26 0 0 0-2.262-2.262H10.287c-.92 0-1.456-1.04-.92-1.788l7.48-10.471c1.07-1.497 0-3.578-1.842-3.578H1.237c-.92 0-1.456-1.04-.92-1.788L10.013.474c.214-.297.556-.474.92-.474h28.894c.92 0 1.456 1.04.92 1.788l-7.48 10.471c-1.07 1.498 0 3.579 1.842 3.579h11.377c.943 0 1.473 1.088.89 1.83L25.947 44.94z" style="fill:#863bff;fill:color(display-p3 .5252 .23 1);fill-opacity:1"/><mask id="a" width="48" height="46" x="0" y="0" maskUnits="userSpaceOnUse" style="mask-type:alpha"><path fill="#000" d="M25.842 44.938c-.664.844-2.021.375-2.021-.698V33.937a2.26 2.26 0 0 0-2.262-2.262H10.183c-.92 0-1.456-1.04-.92-1.788l7.48-10.471c1.07-1.498 0-3.579-1.842-3.579H1.133c-.92 0-1.456-1.04-.92-1.787L9.91.473c.214-.297.556-.474.92-.474h28.894c.92 0 1.456 1.04.92 1.788l-7.48 10.471c-1.07 1.498 0 3.578 1.842 3.578h11.377c.943 0 1.473 1.088.89 1.832L25.843 44.94z" style="fill:#000;fill-opacity:1"/></mask><g mask="url(#a)"><g filter="url(#b)"><ellipse cx="5.508" cy="14.704" fill="#ede6ff" rx="5.508" ry="14.704" style="fill:#ede6ff;fill:color(display-p3 .9275 .9033 1);fill-opacity:1" transform="matrix(.00324 1 1 -.00324 -4.47 31.516)"/></g><g filter="url(#c)"><ellipse cx="10.399" cy="29.851" fill="#ede6ff" rx="10.399" ry="29.851" style="fill:#ede6ff;fill:color(display-p3 .9275 .9033 1);fill-opacity:1" transform="matrix(.00324 1 1 -.00324 -39.328 7.883)"/></g><g filter="url(#d)"><ellipse cx="5.508" cy="30.487" fill="#7e14ff" rx="5.508" ry="30.487" style="fill:#7e14ff;fill:color(display-p3 .4922 .0767 1);fill-opacity:1" transform="rotate(89.814 -25.913 -14.639)scale(1 -1)"/></g><g filter="url(#e)"><ellipse cx="5.508" cy="30.599" fill="#7e14ff" rx="5.508" ry="30.599" style="fill:#7e14ff;fill:color(display-p3 .4922 .0767 1);fill-opacity:1" transform="rotate(89.814 -32.644 -3.334)scale(1 -1)"/></g><g filter="url(#f)"><ellipse cx="5.508" cy="30.599" fill="#7e14ff" rx="5.508" ry="30.599" style="fill:#7e14ff;fill:color(display-p3 .4922 .0767 1);fill-opacity:1" transform="matrix(.00324 1 1 -.00324 -34.34 30.47)"/></g><g filter="url(#g)"><ellipse cx="14.072" cy="22.078" fill="#ede6ff" rx="14.072" ry="22.078" style="fill:#ede6ff;fill:color(display-p3 .9275 .9033 1);fill-opacity:1" transform="rotate(93.35 24.506 48.493)scale(-1 1)"/></g><g filter="url(#h)"><ellipse cx="3.47" cy="21.501" fill="#7e14ff" rx="3.47" ry="21.501" style="fill:#7e14ff;fill:color(display-p3 .4922 .0767 1);fill-opacity:1" transform="rotate(89.009 28.708 47.59)scale(-1 1)"/></g><g filter="url(#i)"><ellipse cx="3.47" cy="21.501" fill="#7e14ff" rx="3.47" ry="21.501" style="fill:#7e14ff;fill:color(display-p3 .4922 .0767 1);fill-opacity:1" transform="rotate(89.009 28.708 47.59)scale(-1 1)"/></g><g filter="url(#j)"><ellipse cx=".387" cy="8.972" fill="#7e14ff" rx="4.407" ry="29.108" style="fill:#7e14ff;fill:color(display-p3 .4922 .0767 1);fill-opacity:1" transform="rotate(39.51 .387 8.972)"/></g><g filter="url(#k)"><ellipse cx="47.523" cy="-6.092" fill="#7e14ff" rx="4.407" ry="29.108" style="fill:#7e14ff;fill:color(display-p3 .4922 .0767 1);fill-opacity:1" transform="rotate(37.892 47.523 -6.092)"/></g><g filter="url(#l)"><ellipse cx="41.412" cy="6.333" fill="#47bfff" rx="5.971" ry="9.665" style="fill:#47bfff;fill:color(display-p3 .2799 .748 1);fill-opacity:1" transform="rotate(37.892 41.412 6.333)"/></g><g filter="url(#m)"><ellipse cx="-1.879" cy="38.332" fill="#7e14ff" rx="4.407" ry="29.108" style="fill:#7e14ff;fill:color(display-p3 .4922 .0767 1);fill-opacity:1" transform="rotate(37.892 -1.88 38.332)"/></g><g filter="url(#n)"><ellipse cx="-1.879" cy="38.332" fill="#7e14ff" rx="4.407" ry="29.108" style="fill:#7e14ff;fill:color(display-p3 .4922 .0767 1);fill-opacity:1" transform="rotate(37.892 -1.88 38.332)"/></g><g filter="url(#o)"><ellipse cx="35.651" cy="29.907" fill="#7e14ff" rx="4.407" ry="29.108" style="fill:#7e14ff;fill:color(display-p3 .4922 .0767 1);fill-opacity:1" transform="rotate(37.892 35.651 29.907)"/></g><g filter="url(#p)"><ellipse cx="38.418" cy="32.4" fill="#47bfff" rx="5.971" ry="15.297" style="fill:#47bfff;fill:color(display-p3 .2799 .748 1);fill-opacity:1" transform="rotate(37.892 38.418 32.4)"/></g></g><defs><filter id="b" width="60.045" height="41.654" x="-19.77" y="16.149" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17158" stdDeviation="7.659"/></filter><filter id="c" width="90.34" height="51.437" x="-54.613" y="-7.533" color-interpolation-filters="sRGB" filterUnits="userSpaceOnUse"><feFlood flood-opacity="0" result="BackgroundImageFix"/><feBlend in="SourceGraphic" in2="BackgroundImageFix" result="shape"/><feGaussianBlur result="effect1_foregroundBlur_2002_17158" stdDeviation="7.659"/></filtLine truncated

After

Width:  |  Height:  |  Size: 9.3 KiB

+24
View File
@@ -0,0 +1,24 @@
<svg xmlns="http://www.w3.org/2000/svg">
<symbol id="bluesky-icon" viewBox="0 0 16 17">
<g clip-path="url(#bluesky-clip)"><path fill="#08060d" d="M7.75 7.735c-.693-1.348-2.58-3.86-4.334-5.097-1.68-1.187-2.32-.981-2.74-.79C.188 2.065.1 2.812.1 3.251s.241 3.602.398 4.13c.52 1.744 2.367 2.333 4.07 2.145-2.495.37-4.71 1.278-1.805 4.512 3.196 3.309 4.38-.71 4.987-2.746.608 2.036 1.307 5.91 4.93 2.746 2.72-2.746.747-4.143-1.747-4.512 1.702.189 3.55-.4 4.07-2.145.156-.528.397-3.691.397-4.13s-.088-1.186-.575-1.406c-.42-.19-1.06-.395-2.741.79-1.755 1.24-3.64 3.752-4.334 5.099"/></g>
<defs><clipPath id="bluesky-clip"><path fill="#fff" d="M.1.85h15.3v15.3H.1z"/></clipPath></defs>
</symbol>
<symbol id="discord-icon" viewBox="0 0 20 19">
<path fill="#08060d" d="M16.224 3.768a14.5 14.5 0 0 0-3.67-1.153c-.158.286-.343.67-.47.976a13.5 13.5 0 0 0-4.067 0c-.128-.306-.317-.69-.476-.976A14.4 14.4 0 0 0 3.868 3.77C1.546 7.28.916 10.703 1.231 14.077a14.7 14.7 0 0 0 4.5 2.306q.545-.748.965-1.587a9.5 9.5 0 0 1-1.518-.74q.191-.14.372-.293c2.927 1.369 6.107 1.369 8.999 0q.183.152.372.294-.723.437-1.52.74.418.838.963 1.588a14.6 14.6 0 0 0 4.504-2.308c.37-3.911-.63-7.302-2.644-10.309m-9.13 8.234c-.878 0-1.599-.82-1.599-1.82 0-.998.705-1.82 1.6-1.82.894 0 1.614.82 1.599 1.82.001 1-.705 1.82-1.6 1.82m5.91 0c-.878 0-1.599-.82-1.599-1.82 0-.998.705-1.82 1.6-1.82.893 0 1.614.82 1.599 1.82 0 1-.706 1.82-1.6 1.82"/>
</symbol>
<symbol id="documentation-icon" viewBox="0 0 21 20">
<path fill="none" stroke="#aa3bff" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.35" d="m15.5 13.333 1.533 1.322c.645.555.967.833.967 1.178s-.322.623-.967 1.179L15.5 18.333m-3.333-5-1.534 1.322c-.644.555-.966.833-.966 1.178s.322.623.966 1.179l1.534 1.321"/>
<path fill="none" stroke="#aa3bff" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.35" d="M17.167 10.836v-4.32c0-1.41 0-2.117-.224-2.68-.359-.906-1.118-1.621-2.08-1.96-.599-.21-1.349-.21-2.848-.21-2.623 0-3.935 0-4.983.369-1.684.591-3.013 1.842-3.641 3.428C3 6.449 3 7.684 3 10.154v2.122c0 2.558 0 3.838.706 4.726q.306.383.713.671c.76.536 1.79.64 3.581.66"/>
<path fill="none" stroke="#aa3bff" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.35" d="M3 10a2.78 2.78 0 0 1 2.778-2.778c.555 0 1.209.097 1.748-.047.48-.129.854-.503.982-.982.145-.54.048-1.194.048-1.749a2.78 2.78 0 0 1 2.777-2.777"/>
</symbol>
<symbol id="github-icon" viewBox="0 0 19 19">
<path fill="#08060d" fill-rule="evenodd" d="M9.356 1.85C5.05 1.85 1.57 5.356 1.57 9.694a7.84 7.84 0 0 0 5.324 7.44c.387.079.528-.168.528-.376 0-.182-.013-.805-.013-1.454-2.165.467-2.616-.935-2.616-.935-.349-.91-.864-1.143-.864-1.143-.71-.48.051-.48.051-.48.787.051 1.2.805 1.2.805.695 1.194 1.817.857 2.268.649.064-.507.27-.857.49-1.052-1.728-.182-3.545-.857-3.545-3.87 0-.857.31-1.558.8-2.104-.078-.195-.349-1 .077-2.078 0 0 .657-.208 2.14.805a7.5 7.5 0 0 1 1.946-.26c.657 0 1.328.092 1.946.26 1.483-1.013 2.14-.805 2.14-.805.426 1.078.155 1.883.078 2.078.502.546.799 1.247.799 2.104 0 3.013-1.818 3.675-3.558 3.87.284.247.528.714.528 1.454 0 1.052-.012 1.896-.012 2.156 0 .208.142.455.528.377a7.84 7.84 0 0 0 5.324-7.441c.013-4.338-3.48-7.844-7.773-7.844" clip-rule="evenodd"/>
</symbol>
<symbol id="social-icon" viewBox="0 0 20 20">
<path fill="none" stroke="#aa3bff" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.35" d="M12.5 6.667a4.167 4.167 0 1 0-8.334 0 4.167 4.167 0 0 0 8.334 0"/>
<path fill="none" stroke="#aa3bff" stroke-linecap="round" stroke-linejoin="round" stroke-width="1.35" d="M2.5 16.667a5.833 5.833 0 0 1 8.75-5.053m3.837.474.513 1.035c.07.144.257.282.414.309l.93.155c.596.1.736.536.307.965l-.723.73a.64.64 0 0 0-.152.531l.207.903c.164.715-.213.991-.84.618l-.872-.52a.63.63 0 0 0-.577 0l-.872.52c-.624.373-1.003.094-.84-.618l.207-.903a.64.64 0 0 0-.152-.532l-.723-.729c-.426-.43-.289-.864.306-.964l.93-.156a.64.64 0 0 0 .412-.31l.513-1.034c.28-.562.735-.562 1.012 0"/>
</symbol>
<symbol id="x-icon" viewBox="0 0 19 19">
<path fill="#08060d" fill-rule="evenodd" d="M1.893 1.98c.052.072 1.245 1.769 2.653 3.77l2.892 4.114c.183.261.333.48.333.486s-.068.089-.152.183l-.522.593-.765.867-3.597 4.087c-.375.426-.734.834-.798.905a1 1 0 0 0-.118.148c0 .01.236.017.664.017h.663l.729-.83c.4-.457.796-.906.879-.999a692 692 0 0 0 1.794-2.038c.034-.037.301-.34.594-.675l.551-.624.345-.392a7 7 0 0 1 .34-.374c.006 0 .93 1.306 2.052 2.903l2.084 2.965.045.063h2.275c1.87 0 2.273-.003 2.266-.021-.008-.02-1.098-1.572-3.894-5.547-2.013-2.862-2.28-3.246-2.273-3.266.008-.019.282-.332 2.085-2.38l2-2.274 1.567-1.782c.022-.028-.016-.03-.65-.03h-.674l-.3.342a871 871 0 0 1-1.782 2.025c-.067.075-.405.458-.75.852a100 100 0 0 1-.803.91c-.148.172-.299.344-.99 1.127-.304.343-.32.358-.345.327-.015-.019-.904-1.282-1.976-2.808L6.365 1.85H1.8zm1.782.91 8.078 11.294c.772 1.08 1.413 1.973 1.425 1.984.016.017.241.02 1.05.017l1.03-.004-2.694-3.766L7.796 5.75 5.722 2.852l-1.039-.004-1.039-.004z" clip-rule="evenodd"/>
</symbol>
</svg>

After

Width:  |  Height:  |  Size: 4.9 KiB

+143
View File
@@ -0,0 +1,143 @@
import { Component, useEffect, useState } from 'react'
import { api } from './api'
import Sidebar from './components/Sidebar'
import ProjectsPage from './pages/ProjectsPage'
import LibraryPage from './pages/LibraryPage'
import TrimPage from './pages/TrimPage'
import ReviewPage from './pages/ReviewPage'
import ModelsPage from './pages/ModelsPage'
import './roboflow.css'
function parseRoute(hash) {
const rawPath = (hash || '').replace(/^#/, '') || '/projects'
const [path, queryString] = rawPath.split('?')
const query = new URLSearchParams(queryString || '')
const parts = path.split('/').filter(Boolean)
if (parts[0] === 'batches' && parts[1]) {
return { name: 'review', batchId: Number(parts[1]) }
}
if (parts[0] === 'projects' && parts[1]) {
const projectId = Number(parts[1])
if (parts[2] === 'trim' && parts[3]) {
return { name: 'trim', projectId, rel: decodeURIComponent(parts[3]) }
}
if (parts[2] === 'models') return { name: 'models', projectId }
if (parts[2] === 'review') {
const batchId = query.get('batch') ? Number(query.get('batch')) : null
return { name: 'review', projectId, batchId }
}
return { name: 'library', projectId }
}
return { name: 'projects' }
}
export function navigate(path) {
window.location.hash = path
}
function useRoute() {
const [route, setRoute] = useState(() => parseRoute(window.location.hash))
useEffect(() => {
const onChange = () => setRoute(parseRoute(window.location.hash))
window.addEventListener('hashchange', onChange)
return () => window.removeEventListener('hashchange', onChange)
}, [])
return route
}
function useTheme() {
const [theme, setTheme] = useState(() => localStorage.getItem('theme') || 'dark')
useEffect(() => {
document.documentElement.setAttribute('data-theme', theme)
localStorage.setItem('theme', theme)
}, [theme])
return [theme, () => setTheme((current) => (current === 'dark' ? 'light' : 'dark'))]
}
class ErrorBoundary extends Component {
constructor(props) {
super(props)
this.state = { hasError: false, error: null }
}
static getDerivedStateFromError(error) {
return { hasError: true, error }
}
componentDidCatch(error, errorInfo) {
console.error('UI Exception:', error, errorInfo)
}
render() {
if (this.state.hasError) {
return (
<div style={{ padding: 32, textAlign: 'center', color: '#f3f4f6' }}>
<h2>View Exception Caught</h2>
<p style={{ color: '#ef4444', margin: '12px 0' }}>{this.state.error?.toString()}</p>
<button className="btn btn-primary" onClick={() => window.location.reload()}>
Reload Application
</button>
</div>
)
}
return this.props.children
}
}
export default function App() {
const route = useRoute()
const [theme, toggleTheme] = useTheme()
const [currentProject, setCurrentProject] = useState(null)
useEffect(() => {
if (route.projectId) {
api.getProject(route.projectId).then(setCurrentProject).catch(() => {})
} else if (!currentProject) {
api.listProjects().then((list) => {
if (list.length > 0) setCurrentProject(list[0])
}).catch(() => {})
}
}, [route.projectId, route.name, currentProject])
return (
<div className="roboflow-layout">
<Sidebar
route={route}
currentProject={currentProject}
theme={theme}
onToggleTheme={toggleTheme}
/>
<main className="roboflow-main">
<ErrorBoundary key={route.name + (route.batchId || route.projectId || '')}>
{route.name === 'projects' && <ProjectsPage />}
{route.name === 'library' && (
<LibraryPage
projectId={route.projectId}
onProject={(p) => setCurrentProject(p)}
/>
)}
{route.name === 'trim' && <TrimPage projectId={route.projectId} rel={route.rel} />}
{route.name === 'review' && (
<ReviewPage
batchId={route.batchId}
projectId={route.projectId}
onProject={(p) => setCurrentProject(p)}
/>
)}
{route.name === 'models' && (
<ModelsPage
projectId={route.projectId}
onProject={(p) => setCurrentProject(p)}
/>
)}
</ErrorBoundary>
</main>
</div>
)
}
+107
View File
@@ -0,0 +1,107 @@
/* One place that knows how to talk to the backend.
*
* FastAPI reports failures as {"detail": "..."} — unwrapped here so callers can
* show the message the backend actually wrote instead of "500". */
async function request(path, { method = 'GET', body, form } = {}) {
const options = { method, headers: {} }
if (form) {
options.body = form
} else if (body !== undefined) {
options.headers['Content-Type'] = 'application/json'
options.body = JSON.stringify(body)
}
const response = await fetch(`/api${path}`, options)
const text = await response.text()
const payload = text ? JSON.parse(text) : null
if (!response.ok) {
const detail = payload?.detail
throw new Error(typeof detail === 'string' ? detail : `${response.status} ${response.statusText}`)
}
return payload
}
export const api = {
health: () => request('/health'),
listProjects: () => request('/projects'),
getProject: (id) => request(`/projects/${id}`),
createProject: (body) => request('/projects', { method: 'POST', body }),
patchProject: (id, body) => request(`/projects/${id}`, { method: 'PATCH', body }),
deleteProject: (id) => request(`/projects/${id}`, { method: 'DELETE' }),
addClass: (id, body) => request(`/projects/${id}/classes`, { method: 'POST', body }),
deleteClass: (id, classId) =>
request(`/projects/${id}/classes/${classId}`, { method: 'DELETE' }),
uploadBaseModel: (id, file) => {
const form = new FormData()
form.append('file', file)
return request(`/projects/${id}/base-model`, { method: 'POST', form })
},
uploadSecondaryModel: (id, file) => {
const form = new FormData()
form.append('file', file)
return request(`/projects/${id}/secondary-model`, { method: 'POST', form })
},
listDates: (id) => request(`/projects/${id}/library`),
listVideos: (id, date) => request(`/projects/${id}/library/${encodeURIComponent(date)}`),
videoInfo: (id, rel) => request(`/projects/${id}/video/info?rel=${encodeURIComponent(rel)}`),
videoUrl: (id, rel) => `/api/projects/${id}/video?rel=${encodeURIComponent(rel)}`,
createBatch: (id, body) => request(`/projects/${id}/batches`, { method: 'POST', body }),
listBatches: (id) => request(`/projects/${id}/batches`),
getBatch: (id) => request(`/batches/${id}`),
patchBatch: (id, body) => request(`/batches/${id}`, { method: 'PATCH', body }),
deleteBatch: (id) => request(`/batches/${id}`, { method: 'DELETE' }),
listFrames: (id) => request(`/batches/${id}/frames`),
frameUrl: (id, width) => `/api/frames/${id}/image${width ? `?w=${width}` : ''}`,
startAutolabel: (batchId, body) =>
request(`/batches/${batchId}/autolabel`, { method: 'POST', body: body ?? {} }),
clearBatchClassAnnotations: (batchId, classId) =>
request(`/batches/${batchId}/classes/${classId}/annotations`, { method: 'DELETE' }),
nextPending: (batchId, afterIdx = -1) =>
request(`/batches/${batchId}/next-pending?after_idx=${afterIdx}`),
frameAnnotations: (frameId) => request(`/frames/${frameId}/annotations`),
addAnnotation: (frameId, body) =>
request(`/frames/${frameId}/annotations`, { method: 'POST', body }),
patchAnnotation: (id, body) => request(`/annotations/${id}`, { method: 'PATCH', body }),
deleteAnnotation: (id) => request(`/annotations/${id}`, { method: 'DELETE' }),
assist: (frameId, body) => request(`/frames/${frameId}/assist`, { method: 'POST', body }),
setFrameStatus: (frameId, status) =>
request(`/frames/${frameId}/status`, { method: 'POST', body: { status } }),
approveAllBatchFrames: (batchId) => request(`/batches/${batchId}/approve-all`, { method: 'POST' }),
approveBatch: (batchId) => request(`/batches/${batchId}/approve`, { method: 'POST' }),
datasetSummary: (projectId) => request(`/projects/${projectId}/dataset`),
datasetDownloadUrl: (projectId) => `/api/projects/${projectId}/dataset/download`,
hardware: () => request('/hardware'),
startTraining: (projectId, body) =>
request(`/projects/${projectId}/train`, { method: 'POST', body }),
listModels: (projectId) => request(`/projects/${projectId}/models`),
promoteModel: (modelId) => request(`/models/${modelId}/promote`, { method: 'POST' }),
weightsUrl: (modelId) => `/api/models/${modelId}/weights`,
listJobs: (projectId) => request(`/jobs${projectId ? `?project_id=${projectId}` : ''}`),
getJob: (id) => request(`/jobs/${id}`),
cancelJob: (id) => request(`/jobs/${id}/cancel`, { method: 'POST' }),
}
export function classColor(classId) {
return `var(--class-${classId % 8})`
}
export function formatDuration(seconds) {
// 0 is a real timestamp — a trim starting at the first frame reads 0:00, not a dash.
if (seconds == null || Number.isNaN(seconds)) return '—'
const total = Math.round(seconds)
const h = Math.floor(total / 3600)
const m = Math.floor((total % 3600) / 60)
const s = total % 60
const pad = (n) => String(n).padStart(2, '0')
return h > 0 ? `${h}:${pad(m)}:${pad(s)}` : `${m}:${pad(s)}`
}
+681
View File
@@ -0,0 +1,681 @@
.app {
width: 100vw;
height: 100vh;
overflow: hidden !important;
display: flex;
flex-direction: column;
background: #0b0f19;
}
.topbar {
height: 52px;
margin: 10px 14px 0 14px;
padding: 0 18px;
background: rgba(17, 24, 39, 0.45);
backdrop-filter: blur(12px);
-webkit-backdrop-filter: blur(12px);
border: 1px solid rgba(255, 255, 255, 0.12);
border-radius: 12px;
display: flex;
align-items: center;
gap: 16px;
flex-shrink: 0;
z-index: 100;
box-shadow: 0 8px 32px rgba(0, 0, 0, 0.35);
}
.topbar .brand {
display: flex;
align-items: center;
gap: 8px;
color: var(--text);
font-weight: 700;
letter-spacing: 0.02em;
}
.topbar .brand svg { color: var(--accent); }
.topbar .spacer { flex: 1; }
.crumbs {
display: flex;
align-items: center;
gap: 6px;
color: var(--text-muted);
font-size: 13px;
min-width: 0;
background: rgba(0, 0, 0, 0.25);
padding: 4px 10px;
border-radius: 6px;
border: 1px solid rgba(255, 255, 255, 0.08);
}
.crumbs svg { color: var(--accent); flex: none; }
.crumbs .current {
color: var(--text);
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
font-weight: 600;
}
.health {
display: flex;
align-items: center;
gap: 6px;
font-size: 11px;
color: var(--text-muted);
background: rgba(0, 0, 0, 0.25);
padding: 4px 10px;
border-radius: 6px;
border: 1px solid rgba(255, 255, 255, 0.08);
}
.dot {
width: 7px;
height: 7px;
border-radius: 50%;
background: var(--text-faint);
flex: none;
}
.dot.ok { background: var(--ok); box-shadow: 0 0 6px var(--ok); }
.dot.bad { background: var(--danger); box-shadow: 0 0 6px var(--danger); }
main.page {
flex: 1;
min-height: 0;
overflow-y: auto;
padding: 16px 20px;
width: 100%;
max-width: 100%;
margin: 0;
display: flex;
flex-direction: column;
}
.page-head {
display: flex;
align-items: flex-start;
gap: calc(var(--space) * 2);
margin-bottom: 18px;
}
.page-head .spacer { flex: 1; }
/* --- project cards ---------------------------------------------------- */
.card-grid {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(320px, 1fr));
gap: 16px;
width: 100%;
}
.project-card {
display: flex;
flex-direction: column;
gap: 12px;
padding: 16px;
text-align: left;
color: inherit;
transition: border-color var(--transition), transform var(--transition),
box-shadow var(--transition);
}
.project-card:hover {
border-color: var(--border-strong);
box-shadow: var(--shadow);
}
.project-card .title-row {
display: flex;
align-items: baseline;
gap: 8px;
}
.project-card .title-row h2 {
flex: 1;
min-width: 0;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.project-card dl {
display: grid;
grid-template-columns: auto 1fr;
gap: 4px 12px;
margin: 0;
font-size: 12px;
}
.project-card dt { color: var(--text-faint); }
.project-card dd { margin: 0; color: var(--text-muted); }
.project-card .classes {
display: flex;
flex-wrap: wrap;
gap: 5px;
}
.class-tag { padding-right: 3px; }
.chip-x {
display: inline-flex;
align-items: center;
justify-content: center;
width: 16px;
height: 16px;
padding: 0;
border: none;
border-radius: 50%;
background: transparent;
color: var(--text-faint);
transition: background var(--transition), color var(--transition);
}
.chip-x:hover:not(:disabled) {
background: var(--danger-soft);
color: var(--danger);
}
.project-card .actions {
display: flex;
gap: 8px;
padding-top: 4px;
border-top: 1px solid var(--border);
}
.project-card .actions .spacer { flex: 1; }
/* --- forms ------------------------------------------------------------ */
.form-panel {
padding: 18px;
display: grid;
gap: 14px;
max-width: 720px;
}
.field-row {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(180px, 1fr));
gap: 12px;
}
.class-rows {
display: grid;
gap: 8px;
}
.class-row {
display: grid;
grid-template-columns: 18px 1fr 1fr auto;
align-items: center;
gap: 8px;
}
.form-actions {
display: flex;
gap: 8px;
align-items: center;
}
.hint {
font-size: 12px;
color: var(--text-faint);
}
/* --- library ---------------------------------------------------------- */
.library {
display: grid;
grid-template-columns: 220px 1fr;
gap: 16px;
align-items: start;
height: calc(100vh - 120px);
overflow: hidden;
}
.date-list {
display: flex;
flex-direction: column;
padding: 6px;
gap: 2px;
height: 100%;
overflow-y: auto;
background: rgba(17, 24, 39, 0.45);
backdrop-filter: blur(12px);
-webkit-backdrop-filter: blur(12px);
border: 1px solid rgba(255, 255, 255, 0.12);
border-radius: var(--radius);
}
.date-item {
display: flex;
align-items: center;
gap: 8px;
padding: 8px 10px;
border: none;
border-radius: var(--radius-sm);
background: transparent;
color: var(--text-muted);
font: inherit;
text-align: left;
transition: background var(--transition), color var(--transition);
}
.date-item:hover { background: var(--panel-raised); color: var(--text); }
.date-item[aria-current='true'] {
background: var(--accent-soft);
color: var(--text);
font-weight: 600;
border: 1px solid rgba(168, 85, 247, 0.4);
}
.date-item .count { margin-left: auto; font-size: 12px; color: var(--text-faint); }
.video-table {
width: 100%;
border-collapse: collapse;
font-size: 13px;
}
.video-table th {
text-align: left;
padding: 8px 12px;
color: var(--text-faint);
font-size: 12px;
font-weight: 500;
border-bottom: 1px solid var(--border);
}
.video-table td {
padding: 10px 12px;
border-bottom: 1px solid var(--border);
color: var(--text-muted);
}
.video-table tbody tr {
transition: background var(--transition);
}
.video-table tbody tr:hover { background: var(--panel-raised); }
.video-table td:first-child { color: var(--text); font-weight: 500; }
.video-table td.num { font-family: var(--mono); font-size: 12px; }
.table-wrap {
overflow: visible;
max-height: none;
background: rgba(17, 24, 39, 0.45);
backdrop-filter: blur(12px);
-webkit-backdrop-filter: blur(12px);
border: 1px solid rgba(255, 255, 255, 0.12);
border-radius: var(--radius);
padding: 8px;
}
/* --- trim ------------------------------------------------------------- */
.trim {
display: grid;
grid-template-columns: minmax(0, 1fr) 340px;
gap: 16px;
align-items: start;
height: calc(100vh - 120px);
overflow: hidden;
}
.trim-player {
padding: 8px;
background: #000;
overflow: hidden;
}
.trim-player video {
display: block;
width: 100%;
max-height: 70vh;
border-radius: var(--radius-sm);
}
.trim-controls {
padding: 16px;
position: sticky;
top: 68px;
}
.range-group {
display: grid;
gap: 6px;
}
.range-group input[type='range'] {
width: 100%;
padding: 0;
height: 22px;
accent-color: var(--accent);
background: transparent;
border: none;
cursor: pointer;
}
.range-group input[type='range']:focus { box-shadow: none; }
.job-status {
display: grid;
gap: 8px;
padding: 12px;
border: 1px solid var(--border);
border-radius: var(--radius-sm);
background: var(--panel-raised);
font-size: 13px;
}
.progress {
height: 6px;
border-radius: 999px;
background: var(--border);
overflow: hidden;
}
.progress span {
display: block;
height: 100%;
background: var(--accent);
transition: width var(--transition);
}
/* --- review ----------------------------------------------------------- */
.panel {
background: rgba(17, 24, 39, 0.45);
backdrop-filter: blur(12px);
-webkit-backdrop-filter: blur(12px);
border: 1px solid rgba(255, 255, 255, 0.12);
border-radius: var(--radius);
box-shadow: var(--shadow);
}
.review {
display: grid;
grid-template-columns: minmax(0, 1fr) 320px;
gap: 12px;
flex: 1;
min-height: 500px;
width: 100%;
}
.review-main {
display: flex;
flex-direction: column;
gap: 8px;
flex: 1;
min-height: 0;
min-width: 0;
}
.canvas-wrap {
position: relative;
display: inline-block;
margin: 0 auto;
background: #030712;
border: 1px solid rgba(255, 255, 255, 0.12);
border-radius: var(--radius);
overflow: hidden;
line-height: 0;
box-shadow: 0 8px 32px rgba(0, 0, 0, 0.5);
}
.canvas-wrap img {
display: block;
width: 100%;
height: auto;
user-select: none;
}
.canvas-wrap svg {
position: absolute;
top: 0;
left: 0;
width: 100%;
height: 100%;
cursor: crosshair;
touch-action: none;
}
.canvas-wrap svg.assist { cursor: copy; }
.canvas-wrap .shape rect,
.canvas-wrap .shape polygon {
fill: rgba(255, 255, 255, 0.05);
stroke-width: 2;
vector-effect: non-scaling-stroke;
cursor: move;
}
.canvas-wrap .shape:hover rect,
.canvas-wrap .shape:hover polygon { stroke-width: 3; }
@keyframes shape-march {
to {
stroke-dashoffset: -20;
}
}
.canvas-wrap .shape.selected rect,
.canvas-wrap .shape.selected polygon {
stroke-width: 3.5;
stroke-dasharray: 8 4;
animation: shape-march 0.8s linear infinite;
fill: rgba(255, 255, 255, 0.22);
filter: drop-shadow(0 0 6px rgba(255, 255, 255, 0.85));
}
.canvas-wrap .handle {
cursor: nwse-resize;
stroke: #ffffff;
stroke-width: 2;
filter: drop-shadow(0 0 4px rgba(0, 0, 0, 0.9));
vector-effect: non-scaling-stroke;
}
.canvas-wrap .handle-ne, .canvas-wrap .handle-sw { cursor: nesw-resize; }
.canvas-wrap .handle-vertex, .canvas-wrap .handle-midpoint { cursor: pointer; }
.canvas-wrap .draft {
fill: rgba(255, 255, 255, 0.08);
stroke-width: 2;
stroke-dasharray: 5 4;
vector-effect: non-scaling-stroke;
}
.frame-bar {
display: flex;
align-items: center;
gap: 10px;
padding: 8px 12px;
background: rgba(17, 24, 39, 0.45);
backdrop-filter: blur(12px);
-webkit-backdrop-filter: blur(12px);
border: 1px solid rgba(255, 255, 255, 0.12);
border-radius: var(--radius);
flex-shrink: 0;
}
.frame-bar .spacer { flex: 1; }
.status-pill {
margin-left: 8px;
padding: 1px 8px;
border-radius: 999px;
font-size: 11px;
font-family: var(--font);
background: var(--panel-raised);
color: var(--text-muted);
}
.status-pill.approved { background: rgba(16, 185, 129, 0.2); color: var(--ok); border: 1px solid rgba(16, 185, 129, 0.4); }
.status-pill.rejected { background: var(--danger-soft); color: var(--danger); border: 1px solid rgba(239, 68, 68, 0.4); }
.filmstrip {
display: flex;
gap: 6px;
overflow-x: auto;
padding: 8px;
background: rgba(17, 24, 39, 0.45);
backdrop-filter: blur(12px);
-webkit-backdrop-filter: blur(12px);
border: 1px solid rgba(255, 255, 255, 0.12);
border-radius: var(--radius);
scrollbar-width: thin;
flex-shrink: 0;
}
.thumb {
position: relative;
flex: none;
width: 92px;
padding: 0;
border: 2px solid transparent;
border-radius: var(--radius-sm);
background: none;
overflow: hidden;
line-height: 0;
transition: border-color var(--transition), opacity var(--transition);
}
.thumb img { width: 100%; display: block; }
.thumb.approved { border-color: var(--ok); }
.thumb.rejected { border-color: var(--danger); opacity: 0.45; }
.thumb:hover { border-color: var(--text-faint); }
.thumb[aria-current='true'] { border-color: var(--accent); box-shadow: 0 0 8px var(--accent); }
.thumb .badge {
position: absolute;
right: 3px;
bottom: 3px;
padding: 0 5px;
border-radius: 999px;
background: rgba(0, 0, 0, 0.72);
color: #fff;
font-size: 10px;
line-height: 16px;
}
.review-side {
display: flex;
flex-direction: column;
gap: 10px;
overflow-y: auto;
padding-right: 4px;
}
.side-panel { padding: 12px 14px; display: grid; gap: 10px; }
.class-list, .shape-list { display: grid; gap: 4px; }
.class-chip {
display: flex;
align-items: center;
gap: 8px;
padding: 6px 8px;
border: 1px solid transparent;
border-radius: var(--radius-sm);
background: transparent;
color: var(--text-muted);
font: inherit;
text-align: left;
transition: background var(--transition), color var(--transition);
}
.class-chip:hover { background: var(--panel-raised); color: var(--text); }
.class-chip.active {
background: var(--accent-soft);
border-color: var(--accent);
color: var(--text);
}
.class-chip .mono { margin-left: auto; }
.shape-list { list-style: none; margin: 0; padding: 0; }
.shape-list li {
display: flex;
align-items: center;
gap: 4px;
border-radius: var(--radius-sm);
}
.shape-list li.selected { background: var(--accent-soft); }
.shape-pick {
flex: 1;
display: flex;
align-items: center;
gap: 8px;
padding: 6px 8px;
border: none;
background: transparent;
color: var(--text-muted);
font: inherit;
text-align: left;
}
.shape-pick:hover { color: var(--text); }
.shape-pick .mono { margin-left: auto; }
.metrics td.better { color: var(--ok); }
.metrics td.worse { color: var(--danger); }
.job-log {
margin: 0;
padding: 8px;
max-height: 180px;
overflow: auto;
border-radius: var(--radius-sm);
background: var(--bg);
border: 1px solid var(--border);
font-family: var(--mono);
font-size: 11px;
color: var(--text-muted);
white-space: pre-wrap;
}
.shortcuts { margin: 0; display: grid; gap: 5px; font-size: 12px; }
.shortcuts > div { display: flex; gap: 10px; align-items: baseline; }
.shortcuts dt { flex: none; width: 62px; }
.shortcuts dd { margin: 0; color: var(--text-muted); }
kbd {
padding: 1px 5px;
border: 1px solid var(--border-strong);
border-bottom-width: 2px;
border-radius: 4px;
background: var(--panel-raised);
font-family: var(--mono);
font-size: 11px;
color: var(--text);
}
@media (max-width: 1000px) {
.trim { grid-template-columns: 1fr; }
.trim-controls { position: static; }
.review { grid-template-columns: 1fr; }
.review-side { position: static; }
}
@media (max-width: 900px) {
.library { grid-template-columns: 1fr; }
.date-list { position: static; flex-direction: row; flex-wrap: wrap; }
}
@media (max-width: 560px) {
.topbar .crumbs { display: none; }
main.page { padding: 16px 12px 40px; }
}
@@ -0,0 +1,240 @@
import { useEffect, useLayoutEffect, useRef, useState } from 'react'
import { classColor } from '../api'
import Shape from './Shape'
/* The annotation surface: the frame with an SVG overlay on top.
*
* SVG rather than <canvas> on purpose — shapes are elements, so selection,
* hover and focus come from the DOM instead of hand-written hit-testing, and
* the whole thing stays keyboard-reachable.
*
* Geometry is normalized 0–1 everywhere; only the handle size is converted to
* frame units, so it stays the same physical size at any zoom. */
const HANDLE_PX = 12
const MIN_SIZE = 0.004
const CORNERS = [
['nw', 0, 0], ['ne', 1, 0], ['se', 1, 1], ['sw', 0, 1],
]
function boxPoints(geometry) {
return geometry.type === 'bbox'
? geometry.points
: (() => {
const xs = geometry.points.map((p) => p[0])
const ys = geometry.points.map((p) => p[1])
return [Math.min(...xs), Math.min(...ys), Math.max(...xs), Math.max(...ys)]
})()
}
function normalise([x0, y0, x1, y1]) {
return [Math.min(x0, x1), Math.min(y0, y1), Math.max(x0, x1), Math.max(y0, y1)]
}
export default function AnnotationCanvas({
frame, imageUrl, annotations, selectedId, activeClass, assistMode, classes,
onSelect, onCreate, onUpdate, onAssist,
}) {
const wrapRef = useRef(null)
const svgRef = useRef(null)
const [displayWidth, setDisplayWidth] = useState(0)
const [draft, setDraft] = useState(null) // box being drawn
const [drag, setDrag] = useState(null) // move/resize in progress
useLayoutEffect(() => {
const element = wrapRef.current
if (!element) return
const observer = new ResizeObserver(([entry]) => {
setDisplayWidth(entry.contentRect.width)
})
observer.observe(element)
return () => observer.disconnect()
}, [])
const width = frame?.width || 1
const height = frame?.height || 1
const scale = displayWidth ? width / displayWidth : 1
const handle = HANDLE_PX * scale
function pointAt(event) {
const rect = svgRef.current.getBoundingClientRect()
return [
Math.min(1, Math.max(0, (event.clientX - rect.left) / rect.width)),
Math.min(1, Math.max(0, (event.clientY - rect.top) / rect.height)),
]
}
function startDraw(event) {
if (event.button !== 0) return
const [x, y] = pointAt(event)
onSelect(null)
setDraft([x, y, x, y])
event.currentTarget.setPointerCapture(event.pointerId)
}
function startMove(event, annotation) {
event.stopPropagation()
const [x, y] = pointAt(event)
onSelect(annotation.id)
setDrag({ kind: 'move', id: annotation.id, origin: [x, y],
start: annotation.geometry })
event.currentTarget.setPointerCapture(event.pointerId)
}
function startResize(event, annotation, corner) {
event.stopPropagation()
onSelect(annotation.id)
setDrag({ kind: 'resize', id: annotation.id, corner,
start: annotation.geometry })
event.currentTarget.setPointerCapture(event.pointerId)
}
function startVertex(event, annotation, pointIndex) {
event.stopPropagation()
onSelect(annotation.id)
setDrag({ kind: 'vertex', id: annotation.id, pointIndex, start: annotation.geometry })
event.currentTarget.setPointerCapture(event.pointerId)
}
function startMidpoint(event, annotation, afterIndex, newPoint) {
event.stopPropagation()
onSelect(annotation.id)
const newPoints = [...annotation.geometry.points]
newPoints.splice(afterIndex + 1, 0, newPoint)
const newGeom = { type: 'polygon', points: newPoints }
onUpdate(annotation.id, newGeom, { local: true })
setDrag({ kind: 'vertex', id: annotation.id, pointIndex: afterIndex + 1, start: newGeom })
event.currentTarget.setPointerCapture(event.pointerId)
}
function deleteVertex(annotation, pointIndex) {
if (annotation.geometry.points.length <= 3) return
const newPoints = annotation.geometry.points.filter((_, i) => i !== pointIndex)
onUpdate(annotation.id, { type: 'polygon', points: newPoints }, { commit: true })
}
function onPointerMove(event) {
if (draft) {
const [x, y] = pointAt(event)
setDraft([draft[0], draft[1], x, y])
return
}
if (!drag) return
const [x, y] = pointAt(event)
if (drag.kind === 'vertex') {
const points = drag.start.points.map((p, i) => (i === drag.pointIndex ? [x, y] : p))
onUpdate(drag.id, { type: 'polygon', points }, { local: true })
return
}
if (drag.kind === 'move') {
const [dx, dy] = [x - drag.origin[0], y - drag.origin[1]]
onUpdate(drag.id, shift(drag.start, dx, dy), { local: true })
} else {
const [x0, y0, x1, y1] = boxPoints(drag.start)
const next = drag.corner === 'nw' ? [x, y, x1, y1]
: drag.corner === 'ne' ? [x0, y, x, y1]
: drag.corner === 'se' ? [x0, y0, x, y]
: [x, y0, x1, y]
onUpdate(drag.id, { type: 'bbox', points: normalise(next) }, { local: true })
}
}
function onPointerUp() {
if (draft) {
const [x0, y0, x1, y1] = normalise(draft)
setDraft(null)
if (x1 - x0 >= MIN_SIZE && y1 - y0 >= MIN_SIZE) {
if (assistMode) onAssist([x0, y0, x1, y1])
else onCreate({ type: 'bbox', points: [x0, y0, x1, y1] })
}
return
}
if (drag) {
onUpdate(drag.id, null, { commit: true })
setDrag(null)
}
}
// Escape cancels whatever gesture is in flight rather than committing it.
useEffect(() => {
const onKey = (event) => {
if (event.key !== 'Escape') return
setDraft(null)
setDrag(null)
}
window.addEventListener('keydown', onKey)
return () => window.removeEventListener('keydown', onKey)
}, [])
// Bound the height by bounding the width at the frame's aspect ratio: a
// portrait frame would otherwise be three screens tall, and constraining the
// image itself would leave the SVG overlay misaligned with it.
return (
<div
className="canvas-wrap"
ref={wrapRef}
style={{ maxWidth: `calc(72vh * ${width} / ${height})` }}
>
<img src={imageUrl} alt={`Frame ${frame?.idx ?? ''}`} draggable={false} />
<svg
ref={svgRef}
viewBox={`0 0 ${width} ${height}`}
preserveAspectRatio="none"
className={assistMode ? 'assist' : undefined}
onPointerDown={startDraw}
onPointerMove={onPointerMove}
onPointerUp={onPointerUp}
>
{annotations.map((annotation) => (
<Shape
key={annotation.id}
annotation={annotation}
width={width}
height={height}
scale={scale}
handle={handle}
selected={annotation.id === selectedId}
classes={classes}
onStartMove={startMove}
onStartResize={startResize}
onStartVertex={startVertex}
onStartMidpoint={startMidpoint}
onDeleteVertex={deleteVertex}
/>
))}
{draft && (() => {
const [x0, y0, x1, y1] = normalise(draft)
return (
<rect
className={assistMode ? 'draft assist' : 'draft'}
x={x0 * width} y={y0 * height}
width={(x1 - x0) * width} height={(y1 - y0) * height}
stroke={assistMode ? 'var(--accent)' : classColor(activeClass)}
/>
)
})()}
</svg>
</div>
)
}
function shift(geometry, dx, dy) {
if (geometry.type === 'bbox') {
const [x0, y0, x1, y1] = geometry.points
const clampedX = Math.min(Math.max(dx, -x0), 1 - x1)
const clampedY = Math.min(Math.max(dy, -y0), 1 - y1)
return { type: 'bbox',
points: [x0 + clampedX, y0 + clampedY, x1 + clampedX, y1 + clampedY] }
}
const xs = geometry.points.map((p) => p[0])
const ys = geometry.points.map((p) => p[1])
const clampedX = Math.min(Math.max(dx, -Math.min(...xs)), 1 - Math.max(...xs))
const clampedY = Math.min(Math.max(dy, -Math.min(...ys)), 1 - Math.max(...ys))
return { type: 'polygon',
points: geometry.points.map(([x, y]) => [x + clampedX, y + clampedY]) }
}
+21
View File
@@ -0,0 +1,21 @@
import React from 'react'
import { api } from '../api'
export default function Filmstrip({ frames, index, onSelectIndex, stripRef }) {
return (
<div className="filmstrip" ref={stripRef}>
{frames.map((item, position) => (
<button
key={item.id}
className={`thumb ${item.review_status}`}
aria-current={position === index}
onClick={() => onSelectIndex(position)}
title={`${item.filename} — ${item.review_status}`}
>
<img src={api.frameUrl(item.id, 120)} alt="" loading="lazy" />
{item.annotation_count > 0 && <span className="badge">{item.annotation_count}</span>}
</button>
))}
</div>
)
}
+138
View File
@@ -0,0 +1,138 @@
/* Inline SVG icons (Lucide geometry). No emoji as icons — see ../../../AGENTS.md
* section 7. They inherit currentColor and the surrounding font size. */
function Icon({ children, size = 16, ...rest }) {
return (
<svg
width={size}
height={size}
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
strokeWidth="2"
strokeLinecap="round"
strokeLinejoin="round"
aria-hidden="true"
focusable="false"
{...rest}
>
{children}
</svg>
)
}
export const PlusIcon = (props) => (
<Icon {...props}><path d="M12 5v14M5 12h14" /></Icon>
)
export const TrashIcon = (props) => (
<Icon {...props}>
<path d="M3 6h18M8 6V4h8v2M19 6l-1 14H6L5 6" />
</Icon>
)
export const UploadIcon = (props) => (
<Icon {...props}>
<path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4M17 8l-5-5-5 5M12 3v12" />
</Icon>
)
export const SunIcon = (props) => (
<Icon {...props}>
<circle cx="12" cy="12" r="4" />
<path d="M12 2v2M12 20v2M4.9 4.9l1.4 1.4M17.7 17.7l1.4 1.4M2 12h2M20 12h2M4.9 19.1l1.4-1.4M17.7 6.3l1.4-1.4" />
</Icon>
)
export const MoonIcon = (props) => (
<Icon {...props}><path d="M21 12.8A9 9 0 1 1 11.2 3a7 7 0 0 0 9.8 9.8z" /></Icon>
)
export const LayersIcon = (props) => (
<Icon {...props}>
<path d="M12 2 2 7l10 5 10-5-10-5zM2 17l10 5 10-5M2 12l10 5 10-5" />
</Icon>
)
export const ChevronRightIcon = (props) => (
<Icon {...props}><path d="m9 18 6-6-6-6" /></Icon>
)
export const AlertIcon = (props) => (
<Icon {...props}>
<path d="M10.3 3.9 1.8 18a2 2 0 0 0 1.7 3h17a2 2 0 0 0 1.7-3L13.7 3.9a2 2 0 0 0-3.4 0z" />
<path d="M12 9v4M12 17h.01" />
</Icon>
)
export const CheckIcon = (props) => (
<Icon {...props}><path d="M20 6 9 17l-5-5" /></Icon>
)
export const XIcon = (props) => (
<Icon {...props}><path d="M18 6 6 18M6 6l12 12" /></Icon>
)
export const FolderIcon = (props) => (
<Icon {...props}>
<path d="M20 20H4a2 2 0 0 1-2-2V6a2 2 0 0 1 2-2h4l2 3h10a2 2 0 0 1 2 2v9a2 2 0 0 1-2 2z" />
</Icon>
)
export const DatabaseIcon = (props) => (
<Icon {...props}>
<ellipse cx="12" cy="5" rx="9" ry="3" />
<path d="M21 12c0 1.66-4 3-9 3s-9-1.34-9-3" />
<path d="M3 5v14c0 1.66 4 3 9 3s9-1.34 9-3V5" />
</Icon>
)
export const CpuIcon = (props) => (
<Icon {...props}>
<rect x="4" y="4" width="16" height="16" rx="2" ry="2" />
<rect x="9" y="9" width="6" height="6" />
<path d="M9 1v3M15 1v3M9 20v3M15 20v3M20 9h3M20 15h3M1 9h3M1 15h3" />
</Icon>
)
export const RocketIcon = (props) => (
<Icon {...props}>
<path d="M4.5 16.5c-1.5 1.26-2 5-2 5s3.74-.5 5-2c.71-.71.79-1.81.79-1.81l-3-3s-1.1.08-1.79.81z" />
<path d="M15 9l-6 6" />
<path d="M9 18l3 3c.87.87 2.18.99 3.1.27l5.9-5.9c1.6-1.6 1.6-4.2 0-5.8l-1.5-1.5c-1.6-1.6-4.2-1.6-5.8 0l-5.9 5.9c-.72.92-.6 2.23.27 3.1z" />
</Icon>
)
export const BarChartIcon = (props) => (
<Icon {...props}>
<path d="M12 20V10M18 20V4M6 20v-4" />
</Icon>
)
export const TagIcon = (props) => (
<Icon {...props}>
<path d="M12 2H2v10l9.29 9.29c.94.94 2.48.94 3.42 0l6.58-6.58c.94-.94.94-2.48 0-3.42L12 2z" />
<circle cx="7" cy="7" r="1.5" />
</Icon>
)
export const SlidersIcon = (props) => (
<Icon {...props}>
<line x1="4" y1="21" x2="4" y2="14" />
<line x1="4" y1="10" x2="4" y2="3" />
<line x1="12" y1="21" x2="12" y2="12" />
<line x1="12" y1="8" x2="12" y2="3" />
<line x1="20" y1="21" x2="20" y2="16" />
<line x1="20" y1="12" x2="20" y2="3" />
<line x1="1" y1="14" x2="7" y2="14" />
<line x1="9" y1="8" x2="15" y2="8" />
<line x1="17" y1="16" x2="23" y2="16" />
</Icon>
)
export const ZapIcon = (props) => (
<Icon {...props}>
<polygon points="13 2 3 14 12 14 11 22 21 10 12 10 13 2" />
</Icon>
)
@@ -0,0 +1,28 @@
import React from 'react'
import { classColor } from '../api'
import { TrashIcon } from './Icons'
export default function QuickReclassBar({ classesList, reclass, removeSelected }) {
return (
<div className="quick-reclass-bar panel" style={{ display: 'flex', alignItems: 'center', gap: 8, padding: '6px 12px', background: 'rgba(168, 85, 247, 0.18)', border: '1px solid rgba(168, 85, 247, 0.45)', borderRadius: '8px' }}>
<span style={{ fontSize: '0.8rem', fontWeight: 600, color: '#c084fc' }}>
Selected Shape Hotkeys:
</span>
{classesList.map((c) => (
<button
key={c.class_id}
className="btn"
style={{ padding: '3px 10px', fontSize: '0.78rem', background: 'rgba(0,0,0,0.4)', borderColor: classColor(c.class_id), color: '#fff' }}
onClick={() => reclass(c.class_id)}
>
<span style={{ background: classColor(c.class_id), width: 8, height: 8, borderRadius: '50%', display: 'inline-block', marginRight: 5 }} />
<strong>[{c.class_id + 1}]</strong> {c.name}
</button>
))}
<span className="spacer" />
<button className="btn btn-danger" style={{ padding: '3px 10px', fontSize: '0.78rem' }} onClick={removeSelected}>
<TrashIcon size={12} /> Delete [Del]
</button>
</div>
)
}
+102
View File
@@ -0,0 +1,102 @@
import React from 'react'
import { classColor } from '../api'
import { TrashIcon } from './Icons'
import ShortcutsPanel from './ShortcutsPanel'
export default function ReviewSidebar({
classesList,
activeClass,
reclass,
clearClassInBatch,
annotations,
selectedId,
setSelectedId,
removeSelected,
project,
jumpToNextAnnotated,
batchAnnotationCount,
}) {
return (
<aside className="review-side stack">
<div className="panel side-panel">
<h2>Classes</h2>
<div className="class-list">
{classesList.map((item) => (
<div className="class-row" key={item.class_id} style={{ display: 'flex', alignItems: 'center', gap: 6 }}>
<button
className={`class-chip ${item.class_id === activeClass ? 'active' : ''}`}
style={{ flex: 1 }}
onClick={() => reclass(item.class_id)}
>
<span className="swatch" style={{ background: classColor(item.class_id) }} />
{item.name}
<span className="faint mono">{item.class_id + 1}</span>
</button>
<button
className="btn btn-ghost"
style={{ padding: '4px 6px' }}
title={`Clear all "${item.name}" shapes in this batch`}
onClick={() => clearClassInBatch(item)}
>
<TrashIcon size={12} />
</button>
</div>
))}
</div>
</div>
<div className="panel side-panel">
<h2>Shapes on this frame ({annotations.length})</h2>
{annotations.length === 0 ? (
<div>
<p className="hint" style={{ marginBottom: 8 }}>
None on this frame — drag to draw a shape.
</p>
{batchAnnotationCount > 0 && (
<button
type="button"
className="btn"
style={{
width: '100%',
fontSize: '0.78rem',
borderColor: 'rgba(168, 85, 247, 0.4)',
color: '#c084fc',
background: 'rgba(168, 85, 247, 0.1)',
}}
onClick={jumpToNextAnnotated}
>
🏷️ Jump to Frame with Shapes [N]
</button>
)}
</div>
) : (
<ul className="shape-list">
{annotations.map((item) => (
<li key={item.id} className={item.id === selectedId ? 'selected' : ''}>
<button className="shape-pick" onClick={() => setSelectedId(item.id)}>
<span className="swatch" style={{ background: classColor(item.class_id) }} />
{project?.classes?.[item.class_id]?.name ?? `class ${item.class_id}`}
<span className="faint mono">
{item.source === 'auto' ? item.score.toFixed(2) : 'manual'}
</span>
</button>
<button
className="btn btn-danger"
aria-label="Delete shape"
onClick={() => {
setSelectedId(item.id)
removeSelected()
}}
>
<TrashIcon size={13} />
</button>
</li>
))}
</ul>
)}
</div>
<ShortcutsPanel />
</aside>
)
}
+123
View File
@@ -0,0 +1,123 @@
import { classColor } from '../api'
const CORNERS = [
['nw', 0, 0], ['ne', 1, 0], ['se', 1, 1], ['sw', 0, 1],
]
function boxPoints(geometry) {
return geometry.type === 'bbox'
? geometry.points
: (() => {
const xs = geometry.points.map((p) => p[0])
const ys = geometry.points.map((p) => p[1])
return [Math.min(...xs), Math.min(...ys), Math.max(...xs), Math.max(...ys)]
})()
}
export default function Shape({
annotation, width, height, scale, handle, selected, classes,
onStartMove, onStartResize, onStartVertex, onStartMidpoint, onDeleteVertex,
}) {
const colour = classColor(annotation.class_id)
const [x0, y0, x1, y1] = boxPoints(annotation.geometry)
const isPolygon = annotation.geometry.type === 'polygon'
const className = classes?.[annotation.class_id]?.name || `Class ${annotation.class_id + 1}`
return (
<g className={selected ? 'shape selected' : 'shape'}>
{isPolygon ? (
<polygon
points={annotation.geometry.points
.map(([px, py]) => `${px * width},${py * height}`).join(' ')}
stroke={colour}
onPointerDown={(event) => onStartMove(event, annotation)}
/>
) : (
<rect
x={x0 * width} y={y0 * height}
width={(x1 - x0) * width} height={(y1 - y0) * height}
stroke={colour}
onPointerDown={(event) => onStartMove(event, annotation)}
/>
)}
{selected && (
<g transform={`translate(${x0 * width}, ${Math.max(22 * scale, y0 * height - 6 * scale)})`}>
<rect
x="0" y={-16 * scale}
width={Math.max(70 * scale, className.length * 8.5 * scale + 32 * scale)}
height={18 * scale}
rx={3 * scale} ry={3 * scale}
fill="rgba(10, 10, 14, 0.9)"
stroke={colour}
strokeWidth={1.5 * scale}
/>
<text
x={6 * scale} y={-3 * scale}
fill="#ffffff"
fontSize={12 * scale}
fontWeight="bold"
fontFamily="sans-serif"
>
{`[${annotation.class_id + 1}] ${className}`}
</text>
</g>
)}
{selected && !isPolygon && CORNERS.map(([corner, cx, cy]) => (
<rect
key={corner}
className={`handle handle-${corner}`}
x={(x0 + (x1 - x0) * cx) * width - handle / 2}
y={(y0 + (y1 - y0) * cy) * height - handle / 2}
width={handle} height={handle}
fill={colour}
onPointerDown={(event) => onStartResize(event, annotation, corner)}
/>
))}
{selected && isPolygon && (() => {
const points = annotation.geometry.points
const vertices = points.map(([px, py], i) => (
<circle
key={`v-${i}`}
className="handle handle-vertex"
cx={px * width}
cy={py * height}
r={handle / 2}
fill={colour}
onPointerDown={(event) => {
if (event.altKey) {
event.stopPropagation()
if (points.length > 3) {
onDeleteVertex?.(annotation, i)
}
} else {
onStartVertex?.(event, annotation, i)
}
}}
/>
))
const midpoints = points.map(([px, py], i) => {
const next = points[(i + 1) % points.length]
const mx = (px + next[0]) / 2
const my = (py + next[1]) / 2
return (
<circle
key={`m-${i}`}
className="handle handle-midpoint"
cx={mx * width}
cy={my * height}
r={handle / 2.5}
fill={colour}
style={{ opacity: 0.45 }}
onPointerDown={(event) => onStartMidpoint?.(event, annotation, i, [mx, my])}
/>
)
})
return <g>{vertices}{midpoints}</g>
})()}
</g>
)
}
@@ -0,0 +1,37 @@
export default function ShortcutsPanel() {
return (
<div className="panel side-panel">
<h2>Shortcuts</h2>
<dl className="shortcuts">
<div>
<dt><kbd>Drag</kbd></dt>
<dd>Add box / resize / move</dd>
</div>
<div>
<dt><kbd>Hold S</kbd></dt>
<dd>SAM3 assisted shape</dd>
</div>
<div>
<dt><kbd>1</kbd>–<kbd>9</kbd></dt>
<dd>Pick class</dd>
</div>
<div>
<dt><kbd>Del</kbd></dt>
<dd>Remove selected</dd>
</div>
<div>
<dt><kbd>←</kbd> <kbd>→</kbd></dt>
<dd>Prev / next frame</dd>
</div>
<div>
<dt><kbd>A</kbd> <kbd>X</kbd></dt>
<dd>Approve / reject</dd>
</div>
<div>
<dt><kbd>U</kbd></dt>
<dd>Next unreviewed</dd>
</div>
</dl>
</div>
)
}
+104
View File
@@ -0,0 +1,104 @@
import { useEffect, useState } from 'react'
import { api } from '../api'
import { navigate } from '../App'
import {
FolderIcon,
DatabaseIcon,
RocketIcon,
TagIcon,
BarChartIcon,
SunIcon,
MoonIcon,
ChevronRightIcon,
} from './Icons'
export default function Sidebar({ route, currentProject, theme, onToggleTheme }) {
const [health, setHealth] = useState(null)
const [collapsed, setCollapsed] = useState(() => localStorage.getItem('sidebar_collapsed') === 'true')
useEffect(() => {
api.health().then(setHealth).catch(() => {})
}, [])
const toggleCollapse = () => {
setCollapsed((prev) => {
const next = !prev
localStorage.setItem('sidebar_collapsed', String(next))
return next
})
}
const pId = currentProject?.id || route.projectId || 1
const handleNav = (e, path) => {
e.preventDefault()
navigate(path)
}
return (
<div className={`roboflow-sidebar ${collapsed ? 'collapsed' : ''}`}>
<div className="sidebar-header">
<div className="sidebar-logo">{collapsed ? 'DE' : 'Dataset Enrichment'}</div>
<button className="sidebar-collapse-btn" onClick={toggleCollapse} title={collapsed ? 'Expand sidebar' : 'Collapse sidebar'}>
{collapsed ? <ChevronRightIcon size={14} /> : '❮'}
</button>
</div>
<div className="sidebar-section">
{!collapsed && <div className="sidebar-section-title">WORKSPACE</div>}
<a href="#/projects" onClick={(e) => handleNav(e, '/projects')} className={`sidebar-item ${route.name === 'projects' ? 'active' : ''}`} title="Projects">
<span className="sidebar-icon"><FolderIcon size={16} /></span>
{!collapsed && <span>Projects</span>}
</a>
</div>
<div className="sidebar-section">
{!collapsed && <div className="sidebar-section-title">DATA</div>}
<a href={`#/projects/${pId}`} onClick={(e) => handleNav(e, `/projects/${pId}`)} className={`sidebar-item ${route.name === 'library' ? 'active' : ''}`} title="Video Archive">
<span className="sidebar-icon"><DatabaseIcon size={16} /></span>
{!collapsed && <span>Video Archive</span>}
</a>
<a
href={route.batchId ? `#/batches/${route.batchId}` : `#/projects/${pId}`}
onClick={(e) => handleNav(e, route.batchId ? `/batches/${route.batchId}` : `/projects/${pId}`)}
className={`sidebar-item ${route.name === 'review' || route.name === 'trim' ? 'active' : ''}`}
title="Annotate / Review"
>
<span className="sidebar-icon"><TagIcon size={16} /></span>
{!collapsed && <span>Annotate / Review</span>}
</a>
<a href={`#/projects/${pId}/models`} onClick={(e) => handleNav(e, `/projects/${pId}/models`)} className={`sidebar-item ${route.name === 'models' ? 'active' : ''}`} title="Master Dataset">
<span className="sidebar-icon"><BarChartIcon size={16} /></span>
{!collapsed && <span>Master Dataset</span>}
</a>
</div>
<div className="sidebar-section">
{!collapsed && <div className="sidebar-section-title">MODELS</div>}
<a href={`#/projects/${pId}/models`} onClick={(e) => handleNav(e, `/projects/${pId}/models`)} className={`sidebar-item ${route.name === 'models' ? 'active' : ''}`} title="Train & Select Engine">
<span className="sidebar-icon"><RocketIcon size={16} /></span>
{!collapsed && <span>Train & Select Engine</span>}
</a>
</div>
<div className="sidebar-spacer" />
<div className="sidebar-footer">
<button className="sidebar-theme-toggle" onClick={onToggleTheme} title={theme === 'dark' ? 'Switch to Light Mode' : 'Switch to Dark Mode'}>
{collapsed ? (theme === 'dark' ? <SunIcon size={14} /> : <MoonIcon size={14} />) : (theme === 'dark' ? 'Light Mode' : 'Dark Mode')}
</button>
{health && !collapsed && (
<div className="sidebar-health">
<div className="health-item" title={health.gpu || ''} style={{ whiteSpace: 'nowrap', overflow: 'hidden', textOverflow: 'ellipsis', display: 'block' }}>
<span style={{ float: 'left' }}>GPU:</span>
<span style={{ float: 'right' }}>{health.gpu ? health.gpu.replace('NVIDIA GeForce ', '').replace(' Laptop GPU', '') : 'N/A'}</span>
</div>
<div className="health-item"><span>VRAM:</span> <span>{health.vram_free_gb ? `${health.vram_free_gb.toFixed(1)}GB` : 'N/A'}</span></div>
<div className="health-item"><span>SAM3:</span> <span>{health.sam3_ready ? 'Ready' : 'Not Ready'}</span></div>
<div className="health-item"><span>FFmpeg:</span> <span>{health.ffmpeg ? 'OK' : 'Error'}</span></div>
</div>
)}
</div>
</div>
)
}
+11
View File
@@ -0,0 +1,11 @@
import { StrictMode } from 'react'
import { createRoot } from 'react-dom/client'
import './theme.css'
import './app.css'
import App from './App.jsx'
createRoot(document.getElementById('root')).render(
<StrictMode>
<App />
</StrictMode>,
)
+591
View File
@@ -0,0 +1,591 @@
import React, { useCallback, useEffect, useRef, useState } from 'react'
import { api, formatDuration } from '../api'
import { navigate } from '../App'
import { AlertIcon } from '../components/Icons'
function megabytes(bytes) {
if (!bytes) return '—'
const mb = bytes / (1024 * 1024)
if (mb >= 1024) return `${(mb / 1024).toFixed(1)} GB`
if (mb >= 10) return `${Math.round(mb)} MB`
if (mb >= 1) return `${mb.toFixed(1)} MB`
return `${Math.round(bytes / 1024)} KB`
}
function ActiveJobsBanner({ jobs, onCancel }) {
if (!jobs || jobs.length === 0) return null
return (
<div className="panel side-panel" style={{ marginBottom: 16, border: '1px solid rgba(168, 85, 247, 0.4)', background: 'rgba(24, 24, 27, 0.8)' }}>
<h3 style={{ margin: '0 0 8px 0', fontSize: '0.95rem', color: '#c084fc' }}>Active System Tasks ({jobs.length})</h3>
{jobs.map((job) => (
<div key={job.id} style={{ marginBottom: 10, padding: '8px 12px', background: 'rgba(0,0,0,0.4)', borderRadius: 6, border: '1px solid rgba(255,255,255,0.05)' }}>
<div className="row" style={{ fontSize: '0.85rem' }}>
<span className={`dot ${job.status === 'running' ? 'ok' : ''}`} />
<strong style={{ textTransform: 'capitalize' }}>{job.type}</strong>
<span className="muted">({job.status})</span>
<span className="spacer" />
<span className="mono">{job.progress}/{job.total || '—'}</span>
<button className="btn btn-ghost" style={{ padding: '2px 6px', fontSize: '0.75rem' }} onClick={() => onCancel(job.id)}>Cancel</button>
</div>
<div className="progress" style={{ margin: '6px 0' }}>
<span style={{ width: `${job.total ? (job.progress / job.total) * 100 : job.status === 'running' ? 50 : 10}%` }} />
</div>
{job.log?.length > 0 && (
<p className="hint mono" style={{ fontSize: '0.75rem', margin: 0, opacity: 0.8 }}>
{job.log[job.log.length - 1]}
</p>
)}
</div>
))}
</div>
)
}
function BatchList({ project, batches, activeJobs, onChanged, onError }) {
const [busyId, setBusyId] = useState(null)
const [selectedEngine, setSelectedEngine] = useState({})
const [engineClassMap, setEngineClassMap] = useState({})
const [expandedFilterBatchId, setExpandedFilterBatchId] = useState(null)
const hasSecondaryModel = Boolean(project?.secondary_model_path)
const model1Classes = project?.classes?.map((c) => c.name) || []
const model2Classes = project?.secondary_model_classes?.length > 0
? project.secondary_model_classes
: model1Classes
const defaultEngines = project?.base_model_path ? ['base_model'] : ['sam3']
const [selectedThreshold, setSelectedThreshold] = useState({})
async function autolabel(batch, resume = false) {
setBusyId(batch.id)
const engines = selectedEngine[batch.id] || defaultEngines
const threshold = selectedThreshold[batch.id] ?? 0.35
// Default class filter for each active engine if not customized
const currentBatchMap = engineClassMap[batch.id] || {}
const engine_classes = {}
if (engines.includes('sam3')) {
engine_classes.sam3 = currentBatchMap.sam3 || model1Classes
}
if (engines.includes('base_model')) {
engine_classes.base_model = currentBatchMap.base_model || model1Classes
}
if (engines.includes('secondary_model')) {
engine_classes.secondary_model = currentBatchMap.secondary_model || model2Classes
}
try {
await api.startAutolabel(batch.id, { resume, engines, engine_classes, threshold })
onChanged()
} catch (exc) {
onError(exc.message)
} finally {
setBusyId(null)
}
}
const toggleEngine = (batchId, engineKey) => {
const current = selectedEngine[batchId] || defaultEngines
let next
if (current.includes(engineKey)) {
if (current.length === 1) return // keep at least 1 engine selected
next = current.filter((e) => e !== engineKey)
} else {
next = [...current, engineKey]
}
setSelectedEngine({ ...selectedEngine, [batchId]: next })
}
const toggleEngineClass = (batchId, engineKey, className, defaultClasses) => {
const batchFilters = engineClassMap[batchId] || {}
const currentEngClasses = batchFilters[engineKey] || defaultClasses
let next
if (currentEngClasses.includes(className)) {
if (currentEngClasses.length === 1) return // keep at least 1 class
next = currentEngClasses.filter((c) => c !== className)
} else {
next = [...currentEngClasses, className]
}
setEngineClassMap({
...engineClassMap,
[batchId]: {
...batchFilters,
[engineKey]: next,
},
})
}
async function deleteBatch(batch) {
if (!window.confirm(`Delete batch "${batch.batch_label}" and all its extracted frames?`)) return
setBusyId(batch.id)
try {
await api.deleteBatch(batch.id)
onChanged()
} catch (exc) {
onError(exc.message)
} finally {
setBusyId(null)
}
}
async function editBatch(batch) {
const newLabel = window.prompt("Enter new batch label:", batch.batch_label)
if (!newLabel || newLabel.trim() === batch.batch_label) return
setBusyId(batch.id)
try {
await api.patchBatch(batch.id, { batch_label: newLabel.trim() })
onChanged()
} catch (exc) {
onError(exc.message)
} finally {
setBusyId(null)
}
}
if (batches.length === 0) return null
return (
<div className="panel table-wrap" style={{ marginTop: 16 }}>
<table className="video-table">
<thead>
<tr>
<th>Batch</th><th>Range</th><th>Frames</th><th>Reviewed</th>
<th>Shapes</th><th>Model & Class Configuration</th><th>Status</th><th />
</tr>
</thead>
<tbody>
{batches.map((batch) => {
const batchJob = activeJobs?.find((j) => j.batch_id === batch.id)
const isProcessing = Boolean(batchJob || busyId === batch.id)
const activeEngines = selectedEngine[batch.id] || defaultEngines
const isFilterExpanded = expandedFilterBatchId === batch.id
const batchMap = engineClassMap[batch.id] || {}
const sam3Active = batchMap.sam3 || model1Classes
const model1Active = batchMap.base_model || model1Classes
const model2Active = batchMap.secondary_model || model2Classes
return (
<React.Fragment key={batch.id}>
<tr>
<td>
<span style={{ cursor: 'pointer', borderBottom: '1px dashed rgba(255,255,255,0.3)' }} onClick={() => editBatch(batch)} title="Click to rename batch">
{batch.date_label} · {batch.batch_label}
</span>
</td>
<td className="num">
{formatDuration(batch.start_sec)}–{formatDuration(batch.end_sec)} @ {batch.fps}fps
</td>
<td className="num">{batch.frame_count}</td>
<td className="num">{batch.reviewed}/{batch.frame_count}</td>
<td className="num">{batch.annotation_count}</td>
<td style={{ minWidth: 260 }}>
<div style={{ display: 'flex', flexDirection: 'column', gap: 6 }}>
<div style={{ display: 'flex', gap: 6, flexWrap: 'wrap', alignItems: 'center' }}>
<button
type="button"
className={`btn ${activeEngines.includes('sam3') ? 'btn-primary' : 'btn-ghost'}`}
style={{ padding: '2px 6px', fontSize: '0.72rem', cursor: 'pointer', opacity: activeEngines.includes('sam3') ? 1 : 0.6 }}
onClick={() => toggleEngine(batch.id, 'sam3')}
disabled={isProcessing}
title="SAM3 Zero-shot Text Prompt Engine"
>
🤖 SAM3
</button>
<button
type="button"
className={`btn ${activeEngines.includes('base_model') ? 'btn-primary' : 'btn-ghost'}`}
style={{ padding: '2px 6px', fontSize: '0.72rem', cursor: 'pointer', opacity: activeEngines.includes('base_model') ? 1 : 0.6 }}
onClick={() => toggleEngine(batch.id, 'base_model')}
disabled={isProcessing}
title="Primary Model 1 (Base / Trained)"
>
⚡ Model 1
</button>
{hasSecondaryModel && (
<button
type="button"
className={`btn ${activeEngines.includes('secondary_model') ? 'btn-primary' : 'btn-ghost'}`}
style={{ padding: '2px 6px', fontSize: '0.72rem', cursor: 'pointer', opacity: activeEngines.includes('secondary_model') ? 1 : 0.6 }}
onClick={() => toggleEngine(batch.id, 'secondary_model')}
disabled={isProcessing}
title={project?.secondary_model_name || "Secondary Uploaded Model 2"}
>
🎯 Model 2
</button>
)}
<button
type="button"
className="btn btn-ghost"
style={{
padding: '2px 6px',
fontSize: '0.72rem',
display: 'flex',
alignItems: 'center',
gap: 3,
borderColor: isFilterExpanded ? '#c084fc' : 'rgba(255,255,255,0.2)',
color: isFilterExpanded ? '#c084fc' : '#e4e4e7',
}}
onClick={() => setExpandedFilterBatchId(isFilterExpanded ? null : batch.id)}
disabled={isProcessing}
>
⚙️ {isFilterExpanded ? 'Hide' : 'Per-Engine'}
</button>
</div>
<div style={{ display: 'flex', flexWrap: 'wrap', gap: 4, alignItems: 'center', fontSize: '0.72rem' }}>
<span className="muted" style={{ fontSize: '0.7rem' }}>Classes:</span>
{model1Classes.map((clsName) => {
const isModel1On = activeEngines.includes('base_model') && model1Active.includes(clsName)
const isSam3On = activeEngines.includes('sam3') && sam3Active.includes(clsName)
const isSecOn = activeEngines.includes('secondary_model') && hasSecondaryModel && model2Active.includes(clsName)
const isActiveAny = isModel1On || isSam3On || isSecOn
return (
<button
key={clsName}
type="button"
className="tag"
style={{
padding: '1px 5px',
fontSize: '0.7rem',
cursor: 'pointer',
background: isActiveAny ? 'rgba(56, 189, 248, 0.25)' : 'rgba(255,255,255,0.05)',
color: isActiveAny ? '#38bdf8' : '#71717a',
border: isActiveAny ? '1px solid rgba(56, 189, 248, 0.5)' : '1px solid rgba(255,255,255,0.1)',
}}
title={`Toggle ${clsName} filter`}
disabled={isProcessing}
onClick={() => {
if (activeEngines.includes('base_model')) toggleEngineClass(batch.id, 'base_model', clsName, model1Classes)
if (activeEngines.includes('sam3')) toggleEngineClass(batch.id, 'sam3', clsName, model1Classes)
if (activeEngines.includes('secondary_model')) toggleEngineClass(batch.id, 'secondary_model', clsName, model2Classes)
}}
>
{isActiveAny ? '✓ ' : ''}{clsName}
</button>
)
})}
</div>
</div>
</td>
<td>
<span className={`tag ${batchJob ? 'info' : ''}`}>
{batchJob ? `${batchJob.type} (${batchJob.status})` : batch.status}
</span>
</td>
<td>
<div className="row" style={{ gap: 6 }}>
<button className="btn btn-primary" disabled={isProcessing || batch.frame_count === 0}
onClick={() => autolabel(batch)}>
{batchJob?.type === 'autolabel' ? 'Processing…' : `Auto-annotate (${activeEngines.map(e => e === 'sam3' ? 'SAM3' : e === 'base_model' ? 'Model 1' : 'Model 2').join('+')})`}
</button>
{batch.annotation_count > 0 && (
<button className="btn" disabled={isProcessing || batch.frame_count === 0}
title="Skip frames that already have automatic shapes"
onClick={() => autolabel(batch, true)}>
Resume
</button>
)}
<button className="btn" disabled={batch.frame_count === 0}
onClick={() => navigate(`/projects/${project.id}/review?batch=${batch.id}`)}>
Review ({batch.reviewed}/{batch.frame_count})
</button>
<button className="btn btn-danger" disabled={isProcessing}
onClick={() => deleteBatch(batch)}>
Delete
</button>
</div>
</td>
</tr>
{/* Inline Per-Engine Class Filter Panel */}
{isFilterExpanded && (
<tr>
<td colSpan={8} style={{ background: '#09090b', padding: '12px 16px', borderBottom: '1px solid rgba(168,85,247,0.3)' }}>
<div style={{ display: 'flex', flexDirection: 'column', gap: 10 }}>
<div style={{ fontSize: '0.82rem', fontWeight: 600, color: '#c084fc' }}>
🛠️ Inline Per-Engine Class Filters (Select target classes per detector):
</div>
<div style={{ display: 'grid', gridTemplateColumns: 'repeat(auto-fit, minmax(240px, 1fr))', gap: 12 }}>
{/* SAM3 Section */}
{activeEngines.includes('sam3') && (
<div style={{ background: 'rgba(24, 24, 27, 0.8)', padding: 10, borderRadius: 6, border: '1px solid rgba(255,255,255,0.1)' }}>
<div style={{ fontSize: '0.78rem', fontWeight: 600, color: '#a855f7', marginBottom: 6 }}>
🤖 SAM3 Text Prompts
</div>
<div style={{ display: 'flex', flexWrap: 'wrap', gap: 4 }}>
{model1Classes.map((clsName) => {
const isActive = sam3Active.includes(clsName)
return (
<button
key={clsName}
type="button"
className="tag"
style={{
padding: '2px 6px',
fontSize: '0.72rem',
cursor: 'pointer',
background: isActive ? 'rgba(168, 85, 247, 0.3)' : 'rgba(255,255,255,0.05)',
color: isActive ? '#f3e8ff' : '#666',
border: isActive ? '1px solid rgba(168, 85, 247, 0.6)' : '1px solid transparent',
}}
onClick={() => toggleEngineClass(batch.id, 'sam3', clsName, model1Classes)}
>
{isActive ? '✓ ' : ''}{clsName}
</button>
)
})}
</div>
</div>
)}
{/* Model 1 Section */}
{activeEngines.includes('base_model') && (
<div style={{ background: 'rgba(24, 24, 27, 0.8)', padding: 10, borderRadius: 6, border: '1px solid rgba(255,255,255,0.1)' }}>
<div style={{ fontSize: '0.78rem', fontWeight: 600, color: '#38bdf8', marginBottom: 6 }}>
⚡ Model 1 (Primary Base) Classes
</div>
<div style={{ display: 'flex', flexWrap: 'wrap', gap: 4 }}>
{model1Classes.map((clsName) => {
const isActive = model1Active.includes(clsName)
return (
<button
key={clsName}
type="button"
className="tag"
style={{
padding: '2px 6px',
fontSize: '0.72rem',
cursor: 'pointer',
background: isActive ? 'rgba(56, 189, 248, 0.3)' : 'rgba(255,255,255,0.05)',
color: isActive ? '#e0f2fe' : '#666',
border: isActive ? '1px solid rgba(56, 189, 248, 0.6)' : '1px solid transparent',
}}
onClick={() => toggleEngineClass(batch.id, 'base_model', clsName, model1Classes)}
>
{isActive ? '✓ ' : ''}{clsName}
</button>
)
})}
</div>
</div>
)}
{/* Model 2 Section */}
{activeEngines.includes('secondary_model') && hasSecondaryModel && (
<div style={{ background: 'rgba(24, 24, 27, 0.8)', padding: 10, borderRadius: 6, border: '1px solid rgba(255,255,255,0.1)' }}>
<div style={{ fontSize: '0.78rem', fontWeight: 600, color: '#f43f5e', marginBottom: 6 }}>
🎯 Model 2 (Secondary Engine) Native Classes
</div>
<div style={{ display: 'flex', flexWrap: 'wrap', gap: 4 }}>
{model2Classes.map((clsName) => {
const isActive = model2Active.includes(clsName)
return (
<button
key={clsName}
type="button"
className="tag"
style={{
padding: '2px 6px',
fontSize: '0.72rem',
cursor: 'pointer',
background: isActive ? 'rgba(244, 63, 94, 0.3)' : 'rgba(255,255,255,0.05)',
color: isActive ? '#ffe4e6' : '#666',
border: isActive ? '1px solid rgba(244, 63, 94, 0.6)' : '1px solid transparent',
}}
onClick={() => toggleEngineClass(batch.id, 'secondary_model', clsName, model2Classes)}
>
{isActive ? '✓ ' : ''}{clsName}
</button>
)
})}
</div>
</div>
)}
</div>
</div>
</td>
</tr>
)}
</React.Fragment>
)
})}
</tbody>
</table>
</div>
)
}
export default function LibraryPage({ projectId, onProject }) {
const [project, setProject] = useState(null)
const [dates, setDates] = useState([])
const [selected, setSelected] = useState(null)
const [videos, setVideos] = useState(null)
const [batches, setBatches] = useState([])
const [jobs, setJobs] = useState([])
const [error, setError] = useState('')
const loadBatches = useCallback(() => {
api.listBatches(projectId).then((payload) => setBatches(payload.batches)).catch(() => {})
}, [projectId])
const loadJobs = useCallback(() => {
api.listJobs(projectId).then((payload) => setJobs(payload.jobs)).catch(() => {})
}, [projectId])
useEffect(() => {
let cancelled = false
setError('')
Promise.all([api.getProject(projectId), api.listDates(projectId), api.listBatches(projectId), api.listJobs(projectId)])
.then(([loadedProject, library, batchPayload, jobPayload]) => {
if (cancelled) return
setProject(loadedProject)
onProject?.(loadedProject)
setDates(library.dates)
setSelected(library.dates[0]?.date ?? null)
setBatches(batchPayload.batches)
setJobs(jobPayload.jobs)
})
.catch((exc) => !cancelled && setError(exc.message))
return () => { cancelled = true }
}, [projectId])
useEffect(() => {
if (!selected) return
let cancelled = false
setVideos(null)
api.listVideos(projectId, selected)
.then((payload) => !cancelled && setVideos(payload.videos))
.catch((exc) => !cancelled && setError(exc.message))
return () => { cancelled = true }
}, [projectId, selected])
// Poll jobs every 2 seconds if there are active jobs, and refresh batches on completion
const activeJobs = jobs.filter((j) => ['queued', 'running'].includes(j.status))
const prevActiveCount = useRef(0)
useEffect(() => {
if (prevActiveCount.current > 0 && activeJobs.length === 0) {
loadBatches()
loadJobs()
}
prevActiveCount.current = activeJobs.length
if (activeJobs.length === 0) return
const timer = setInterval(() => {
loadJobs()
loadBatches()
}, 2000)
return () => clearInterval(timer)
}, [activeJobs.length, loadJobs, loadBatches])
const cancelJob = async (jobId) => {
try {
await api.cancelJob(jobId)
loadJobs()
} catch (exc) {
setError(exc.message)
}
}
if (error) {
return <p className="error-banner"><AlertIcon size={14} /> {error}</p>
}
if (!project) return <p className="empty">Loading…</p>
return (
<>
<div className="page-head">
<div>
<h1>{project.name}</h1>
<p className="muted mono">{project.video_root}</p>
</div>
</div>
<ActiveJobsBanner jobs={activeJobs} onCancel={cancelJob} />
{batches.length > 0 && (
<>
<h2 style={{ marginBottom: 8 }}>Batches</h2>
<BatchList project={project} batches={batches} activeJobs={activeJobs} onChanged={() => { loadBatches(); loadJobs(); }} onError={setError} />
<h2 style={{ margin: '24px 0 8px' }}>Archive</h2>
</>
)}
{dates.length === 0 ? (
<p className="empty">
No date folders in this archive yet. Expected layout: &lt;date&gt;/&lt;batch&gt;.mp4
</p>
) : (
<div className="library">
<nav className="panel date-list" aria-label="Recording dates">
{dates.map((item) => (
<button
key={item.date}
className="date-item"
aria-current={item.date === selected}
onClick={() => setSelected(item.date)}
>
{item.date}
<span className="count">{item.video_count}</span>
</button>
))}
</nav>
<div className="panel table-wrap">
{videos === null ? (
<p className="empty">Reading video metadata…</p>
) : videos.length === 0 ? (
<p className="empty">No videos in {selected}.</p>
) : (
<table className="video-table">
<thead>
<tr>
<th>Batch</th>
<th>Duration</th>
<th>Resolution</th>
<th>FPS</th>
<th>Size</th>
<th>Used</th>
<th />
</tr>
</thead>
<tbody>
{videos.map((item) => (
<tr key={item.rel}>
<td>{item.batch_label}</td>
<td className="num">{formatDuration(item.duration)}</td>
<td className="num">
{item.width ? `${item.width}×${item.height}` : <span className="faint">unreadable</span>}
</td>
<td className="num">{item.fps || '—'}</td>
<td className="num">{megabytes(item.size)}</td>
<td>
{item.used_count > 0
? <span className="tag">{item.used_count} batch{item.used_count > 1 ? 'es' : ''}</span>
: <span className="faint">—</span>}
</td>
<td>
<button
className="btn"
disabled={!item.duration}
title={item.duration ? 'Pick a range and extract frames' : 'ffprobe could not read this file'}
onClick={() => navigate(`/projects/${projectId}/trim/${encodeURIComponent(item.rel)}`)}
>
Trim
</button>
</td>
</tr>
))}
</tbody>
</table>
)}
</div>
</div>
)}
</>
)
}
+378
View File
@@ -0,0 +1,378 @@
import { useCallback, useEffect, useState } from 'react'
import { api } from '../api'
import { navigate } from '../App'
import { AlertIcon, CheckIcon } from '../components/Icons'
function Metric({ label, base, next, delta }) {
const better = delta != null && delta > 0
const worse = delta != null && delta < 0
return (
<tr>
<td>{label}</td>
<td className="num">{base == null ? '—' : base.toFixed(4)}</td>
<td className="num">{next.toFixed(4)}</td>
<td className={`num ${better ? 'better' : worse ? 'worse' : ''}`}>
{delta == null ? '—' : `${delta >= 0 ? '+' : ''}${delta.toFixed(4)}`}
</td>
</tr>
)
}
function VersionCard({ version, onPromote, onError }) {
const [busy, setBusy] = useState(false)
const metrics = version.metrics
const base = version.base_metrics
async function promote() {
setBusy(true)
try {
await api.promoteModel(version.id)
onPromote()
} catch (exc) {
onError(exc.message)
} finally {
setBusy(false)
}
}
return (
<div className="panel side-panel">
<div className="row">
<h2>v{version.version}</h2>
<span className="spacer" />
<span className="faint mono">
{new Date(version.created_at * 1000).toLocaleString()}
</span>
</div>
{metrics ? (
<div className="table-wrap">
<table className="video-table metrics">
<thead>
<tr><th>Metric</th><th>Base</th><th>This version</th><th>Δ</th></tr>
</thead>
<tbody>
<Metric label="mAP50" base={base?.map50} next={metrics.map50}
delta={base ? metrics.map50 - base.map50 : null} />
<Metric label="mAP50-95" base={base?.map50_95} next={metrics.map50_95}
delta={base ? metrics.map50_95 - base.map50_95 : null} />
<Metric label="precision" base={base?.precision} next={metrics.precision}
delta={base ? metrics.precision - base.precision : null} />
<Metric label="recall" base={base?.recall} next={metrics.recall}
delta={base ? metrics.recall - base.recall : null} />
</tbody>
</table>
</div>
) : <p className="hint">No metrics recorded for this version.</p>}
{!base && (
<p className="hint">
No base column: the previous model could not be scored on this val set.
</p>
)}
<div className="row">
<a className="btn" href={api.weightsUrl(version.id)} download>Download best.pt</a>
<button className="btn btn-primary" onClick={promote} disabled={busy}>
Use as base model
</button>
</div>
</div>
)
}
export default function ModelsPage({ projectId, onProject }) {
const [project, setProject] = useState(null)
const [summary, setSummary] = useState(null)
const [models, setModels] = useState([])
const [hardware, setHardware] = useState(null)
const [epochs, setEpochs] = useState(50)
const [job, setJob] = useState(null)
const [error, setError] = useState('')
const [selectedBatchIds, setSelectedBatchIds] = useState([])
const load = useCallback(async () => {
const [loadedProject, loadedSummary, modelPayload, hw, jobsPayload] = await Promise.all([
api.getProject(projectId), api.datasetSummary(projectId),
api.listModels(projectId), api.hardware(), api.listJobs(projectId),
])
setProject(loadedProject)
onProject?.(loadedProject)
setSummary(loadedSummary)
setModels(modelPayload.models)
setHardware(hw)
setSelectedBatchIds(loadedSummary.batches.map((b) => b.id))
const activeJob = jobsPayload.jobs?.find((j) => ['running', 'queued'].includes(j.status))
if (activeJob) setJob(activeJob)
}, [projectId])
useEffect(() => { load().catch((exc) => setError(exc.message)) }, [load])
useEffect(() => {
if (!job || ['done', 'failed', 'cancelled'].includes(job.status)) {
if (job?.status === 'done') load().catch(() => {})
return
}
const timer = setInterval(() => {
api.getJob(job.id).then(setJob).catch(() => {})
}, 2000)
return () => clearInterval(timer)
}, [job?.status, job?.id])
async function train() {
setError('')
try {
setJob(await api.startTraining(projectId, {
epochs: Number(epochs),
batch_ids: selectedBatchIds.length > 0 ? selectedBatchIds : null,
}))
} catch (exc) {
setError(exc.message)
}
}
const toggleBatchSelect = (id) => {
setSelectedBatchIds((prev) =>
prev.includes(id) ? prev.filter((bId) => bId !== id) : [...prev, id]
)
}
const toggleSelectAllBatches = () => {
if (selectedBatchIds.length === summary.batches.length) {
setSelectedBatchIds([])
} else {
setSelectedBatchIds(summary.batches.map((b) => b.id))
}
}
if (error && !project) return <p className="error-banner"><AlertIcon size={14} /> {error}</p>
if (!project || !summary) return <p className="empty">Loading…</p>
const running = job && !['done', 'failed', 'cancelled'].includes(job.status)
return (
<>
<div className="page-head">
<div>
<h1>{project.name} · models</h1>
<p className="muted">
Master dataset: {summary.splits.train} train / {summary.splits.val} val
{' '}from {summary.batches.length} merged batch(es)
</p>
</div>
<span className="spacer" />
<a className="btn" href={api.datasetDownloadUrl(projectId)} download>Download dataset</a>
</div>
{error && <p className="error-banner" style={{ marginBottom: 12 }}>
<AlertIcon size={14} /> {error}
</p>}
<div className="select-engine-section">
<h2>Select Engine</h2>
<p className="hint">
Select how you want to train your model. Configure custom fine-tuning parameters or use SAM3 grounding backbone.
</p>
<div className="engine-grid">
<div className="engine-card selected">
<div className="engine-card-header">
<span className="project-badge">Selected</span>
<span className="engine-card-title">Custom Training (YOLO11)</span>
</div>
<p className="engine-card-desc">
Fine-tune on the merged master dataset using pre-configured hardware batch size and image resolution.
</p>
</div>
<div className="engine-card" title="Neural Architecture Search / Rapid Auto-train">
<div className="engine-card-header">
<span className="chip-badge info">Rapid NAS</span>
<span className="engine-card-title">Neural Architecture Search</span>
</div>
<p className="engine-card-desc">
Automated model selection optimized for latency and accuracy trade-offs on your specific project dataset.
</p>
</div>
</div>
</div>
<div className="panel side-panel" style={{ marginBottom: 20 }}>
<h2>Uploaded Models for Training & Auto-Annotation</h2>
<p className="hint">Upload up to 2 models to use for fine-tuning baseline or auto-annotating new video batches:</p>
<div style={{ display: 'grid', gridTemplateColumns: 'repeat(auto-fit, minmax(280px, 1fr))', gap: 16, marginTop: 12 }}>
<div style={{ padding: 14, background: 'rgba(0,0,0,0.3)', borderRadius: 8, border: '1px solid rgba(255,255,255,0.1)' }}>
<h3 style={{ fontSize: '0.9rem', color: '#c084fc', margin: '0 0 6px 0' }}>Primary Model 1 (Base Model)</h3>
<p className="hint" style={{ fontSize: '0.78rem', margin: '0 0 10px 0' }}>
Used as fine-tuning starting point and benchmark baseline.
</p>
<div style={{ fontSize: '0.8rem', color: '#a1a1aa', marginBottom: 10 }}>
Status: <strong style={{ color: '#fff' }}>{project.base_model_path ? 'Custom model.pt loaded' : 'Default yolo11n.pt'}</strong>
</div>
<div style={{ fontSize: '0.78rem', color: '#38bdf8', marginBottom: 10 }}>
<strong>Model 1 Classes:</strong> {project.classes?.map((c) => c.name).join(', ')}
</div>
<input
type="file"
accept=".pt"
id="upload-primary-model"
style={{ display: 'none' }}
onChange={async (e) => {
const file = e.target.files?.[0]
if (!file) return
try {
await api.uploadBaseModel(projectId, file)
load()
} catch (err) {
setError(err.message)
}
}}
/>
<label htmlFor="upload-primary-model" className="btn btn-ghost" style={{ cursor: 'pointer', padding: '4px 10px', fontSize: '0.8rem' }}>
Upload Primary Model 1 (.pt)
</label>
</div>
<div style={{ padding: 14, background: 'rgba(0,0,0,0.3)', borderRadius: 8, border: '1px solid rgba(255,255,255,0.1)' }}>
<h3 style={{ fontSize: '0.9rem', color: '#38bdf8', margin: '0 0 6px 0' }}>Secondary Model 2 (Auto-Annotate Engine)</h3>
<p className="hint" style={{ fontSize: '0.78rem', margin: '0 0 10px 0' }}>
Used as an auxiliary engine choice for fast auto-annotation.
</p>
<div style={{ fontSize: '0.8rem', color: '#a1a1aa', marginBottom: 10 }}>
Status: <strong style={{ color: '#fff' }}>{project.secondary_model_path ? (project.secondary_model_name || 'secondary_model.pt') : 'None uploaded'}</strong>
</div>
<div style={{ fontSize: '0.78rem', color: '#c084fc', marginBottom: 10 }}>
<strong>Model 2 Native Classes:</strong> {project.secondary_model_classes?.length > 0 ? project.secondary_model_classes.join(', ') : 'Extracted on upload'}
</div>
<input
type="file"
accept=".pt"
id="upload-secondary-model"
style={{ display: 'none' }}
onChange={async (e) => {
const file = e.target.files?.[0]
if (!file) return
try {
await api.uploadSecondaryModel(projectId, file)
load()
} catch (err) {
setError(err.message)
}
}}
/>
<label htmlFor="upload-secondary-model" className="btn btn-ghost" style={{ cursor: 'pointer', padding: '4px 10px', fontSize: '0.8rem' }}>
Upload Secondary Model 2 (.pt)
</label>
</div>
</div>
</div>
<div className="review">
<div className="stack">
{models.length === 0 && !running && (
<p className="empty">No trained versions yet.</p>
)}
{models.map((version) => (
<VersionCard key={version.id} version={version} onPromote={load}
onError={setError} />
))}
</div>
<aside className="review-side stack">
<div className="panel side-panel">
<h2>Train</h2>
<p className="hint">
Fine-tunes{' '}
{project.base_model_path
? 'this project’s base model'
: `${project.base_model_fallback} (no base model uploaded)`}{' '}
on the selected dataset batches ({selectedBatchIds.length}/{summary.batches.length} selected).
</p>
<div>
<label htmlFor="epochs">Epochs</label>
<input id="epochs" type="number" min="1" max="500" value={epochs}
onChange={(e) => setEpochs(e.target.value)} />
</div>
{hardware && (
<p className="hint">
{hardware.gpu ?? 'CPU'} — defaults batch {hardware.batch},
imgsz {hardware.imgsz}. {hardware.note}
</p>
)}
<button className="btn btn-primary" onClick={train}
disabled={running || summary.splits.train === 0 || selectedBatchIds.length === 0}>
{running ? 'Training…' : 'Start training'}
</button>
{summary.splits.train === 0 && (
<p className="hint">Approve and merge a batch first.</p>
)}
{selectedBatchIds.length === 0 && summary.splits.train > 0 && (
<p className="hint" style={{ color: '#ef4444' }}>Select at least one dataset batch to train.</p>
)}
</div>
{job && (
<div className="panel side-panel">
<h2>Job {job.id}</h2>
<div className="row">
<span className={`dot ${job.status === 'done' ? 'ok' : job.status === 'failed' ? 'bad' : ''}`} />
<span>{job.status}</span>
<span className="spacer" />
<span className="mono">{job.progress}/{job.total}</span>
</div>
<div className="progress">
<span style={{ width: `${job.total ? (job.progress / job.total) * 100 : 0}%` }} />
</div>
{job.error && <p className="error-banner">{job.error}</p>}
<pre className="job-log">{job.log.slice(-8).join('\n')}</pre>
{running && (
<button className="btn" onClick={() => api.cancelJob(job.id).catch(() => {})}>
Cancel
</button>
)}
{job.status === 'done' && (
<p className="hint"><CheckIcon size={13} /> Finished — the version is listed
on the left.</p>
)}
</div>
)}
{summary.batches.length > 0 && (
<div className="panel side-panel">
<div className="row" style={{ marginBottom: 8 }}>
<h2>Select Dataset Batches</h2>
<span className="spacer" />
<button className="btn" style={{ fontSize: 11, padding: '2px 8px' }} onClick={toggleSelectAllBatches}>
{selectedBatchIds.length === summary.batches.length ? 'Deselect All' : 'Select All'}
</button>
</div>
<p className="hint">Check the approved batches you want to include in this training run:</p>
<ul className="shape-list" style={{ marginTop: 8 }}>
{summary.batches.map((item) => {
const isChecked = selectedBatchIds.includes(item.id)
return (
<li key={item.id} style={{ display: 'flex', alignItems: 'center', gap: 8, padding: '6px 0' }}>
<input
type="checkbox"
checked={isChecked}
onChange={() => toggleBatchSelect(item.id)}
style={{ cursor: 'pointer' }}
/>
<span className="shape-pick" style={{ flex: 1, cursor: 'pointer' }} onClick={() => toggleBatchSelect(item.id)}>
{item.date_label} · {item.batch_label}
<span className="faint mono" style={{ marginLeft: 6 }}>{item.images} img</span>
</span>
</li>
)
})}
</ul>
</div>
)}
</aside>
</div>
</>
)
}
+377
View File
@@ -0,0 +1,377 @@
import { useEffect, useRef, useState } from 'react'
import { api, classColor } from '../api'
import { navigate } from '../App'
import { AlertIcon, PlusIcon, TrashIcon, UploadIcon, XIcon } from '../components/Icons'
const BLANK_CLASS = { name: '', prompt: '' }
const CLASS_PREVIEW = 8
function NewProjectForm({ defaultRoot, onCreated, onCancel }) {
const [name, setName] = useState('')
const [labelType, setLabelType] = useState('bbox')
const [videoRoot, setVideoRoot] = useState(defaultRoot || '')
const [valEvery, setValEvery] = useState(5)
const [classes, setClasses] = useState([{ ...BLANK_CLASS }])
const [error, setError] = useState('')
const [busy, setBusy] = useState(false)
const setClass = (index, patch) =>
setClasses((rows) => rows.map((row, i) => (i === index ? { ...row, ...patch } : row)))
async function submit(event) {
event.preventDefault()
setBusy(true)
setError('')
try {
const created = await api.createProject({
name,
label_type: labelType,
video_root: videoRoot,
val_every: Number(valEvery),
classes: classes
.filter((row) => row.name.trim())
.map((row) => ({ name: row.name.trim(), prompt: row.prompt.trim() || null })),
})
onCreated(created)
} catch (exc) {
setError(exc.message)
} finally {
setBusy(false)
}
}
return (
<form className="panel form-panel" onSubmit={submit}>
<h2>New project</h2>
<div className="field-row">
<div>
<label htmlFor="np-name">Name</label>
<input id="np-name" value={name} onChange={(e) => setName(e.target.value)}
placeholder="Sack" required autoFocus />
</div>
<div>
<label htmlFor="np-type">Label type</label>
<select id="np-type" value={labelType} onChange={(e) => setLabelType(e.target.value)}>
<option value="bbox">Bounding box (YOLO detect)</option>
<option value="polygon">Polygon (YOLO segment)</option>
</select>
<p className="hint">Fixed once the first batch is merged — every label file is written in this format.</p>
</div>
<div>
<label htmlFor="np-val">Val split — every Nth frame</label>
<input id="np-val" type="number" min="0" max="50" value={valEvery}
onChange={(e) => setValEvery(e.target.value)} />
</div>
</div>
<div>
<label htmlFor="np-root">Video archive root</label>
<input id="np-root" value={videoRoot} onChange={(e) => setVideoRoot(e.target.value)}
placeholder="/videos" required />
<p className="hint">Laid out as &lt;date&gt;/&lt;batch&gt;.mp4. Read-only — nothing is written here.</p>
</div>
<div>
<label>Classes</label>
<div className="class-rows">
{classes.map((row, index) => (
<div className="class-row" key={index}>
<span className="swatch" style={{ background: classColor(index) }} />
<input value={row.name} onChange={(e) => setClass(index, { name: e.target.value })}
placeholder={`class ${index}`} aria-label={`Class ${index} name`} />
<input value={row.prompt} onChange={(e) => setClass(index, { prompt: e.target.value })}
placeholder="SAM3 prompt (defaults to the name)"
aria-label={`Class ${index} prompt`} />
<button type="button" className="btn btn-danger"
aria-label={`Remove class ${index}`}
disabled={classes.length === 1}
onClick={() => setClasses((rows) => rows.filter((_, i) => i !== index))}>
<TrashIcon size={14} />
</button>
</div>
))}
</div>
<p className="hint">
Upload a base model afterwards to replace these with the model&apos;s own classes.
</p>
</div>
{error && <p className="error-banner"><AlertIcon size={14} /> {error}</p>}
<div className="form-actions">
<button className="btn btn-primary" type="submit" disabled={busy}>
{busy ? 'Creating…' : 'Create project'}
</button>
<button className="btn btn-ghost" type="button" onClick={onCancel}>Cancel</button>
<span className="spacer" />
<button className="btn" type="button"
onClick={() => setClasses((rows) => [...rows, { ...BLANK_CLASS }])}>
<PlusIcon size={14} /> Add class
</button>
</div>
</form>
)
}
function ProjectCard({ project, onChanged, onError }) {
const fileInputModel1 = useRef(null)
const fileInputModel2 = useRef(null)
const [busy, setBusy] = useState(false)
const [expanded, setExpanded] = useState(false)
const [addingClass, setAddingClass] = useState(false)
const [newClassName, setNewClassName] = useState('')
const [newClassPrompt, setNewClassPrompt] = useState('')
async function submitAddClass(e) {
e.preventDefault()
if (!newClassName.trim()) return
setBusy(true)
try {
await api.addClass(project.id, { name: newClassName.trim(), prompt: newClassPrompt.trim() || undefined })
setNewClassName('')
setNewClassPrompt('')
setAddingClass(false)
onChanged()
} catch (exc) {
onError(exc.message)
} finally {
setBusy(false)
}
}
async function removeClass(item) {
const merged = project.dataset.train + project.dataset.val
const consequences = [
item.annotation_count > 0 && `${item.annotation_count} shape(s) will be deleted`,
merged > 0 && `${merged} label file(s) in the master dataset will be rewritten`,
'the classes above it will be renumbered',
].filter(Boolean).join(', ')
if (!window.confirm(
`Delete the class "${item.name}"?\n\n${consequences}.\n\nThis cannot be undone.`
)) return
setBusy(true)
try {
await api.deleteClass(project.id, item.class_id)
onChanged()
} catch (exc) {
onError(exc.message)
} finally {
setBusy(false)
}
}
async function uploadModel1(event) {
const file = event.target.files?.[0]
event.target.value = ''
if (!file) return
setBusy(true)
try {
await api.uploadBaseModel(project.id, file)
onChanged()
} catch (exc) {
onError(exc.message)
} finally {
setBusy(false)
}
}
async function uploadModel2(event) {
const file = event.target.files?.[0]
event.target.value = ''
if (!file) return
setBusy(true)
try {
await api.uploadSecondaryModel(project.id, file)
onChanged()
} catch (exc) {
onError(exc.message)
} finally {
setBusy(false)
}
}
async function remove() {
if (!window.confirm(`Delete "${project.name}" and everything under it?`)) return
setBusy(true)
try {
await api.deleteProject(project.id)
onChanged()
} catch (exc) {
onError(exc.message)
} finally {
setBusy(false)
}
}
const merged = project.dataset.train + project.dataset.val
const insertedCount = [project.base_model_path, project.secondary_model_path].filter(Boolean).length
return (
<div className="panel project-card">
<div className="title-row">
<h2>{project.name}</h2>
<span className="tag" title={project.label_type_locked ? "Locked: batches have already been merged in this format" : undefined}>
{project.label_type_locked ? `locked: ${project.label_type}` : project.label_type}
</span>
</div>
<dl>
<dt>Base models</dt>
<dd>
<strong>{insertedCount} model(s) inserted</strong>
<span className="muted" style={{ fontSize: '0.8rem', display: 'block', marginTop: 2 }}>
{insertedCount === 0 && `none — default (${project.base_model_fallback})`}
{insertedCount === 1 && (project.base_model_path ? 'Model 1 (Primary)' : 'Model 2 (Secondary)')}
{insertedCount === 2 && 'Model 1 (Primary) & Model 2 (Secondary)'}
</span>
</dd>
<dt>Archive</dt>
<dd className="mono">{project.video_root}</dd>
<dt>Batches</dt>
<dd>{project.batch_count}</dd>
<dt>Dataset</dt>
<dd>{merged ? `${project.dataset.train} train / ${project.dataset.val} val` : 'empty'}</dd>
</dl>
{/* A base model can carry 80 classes; showing them all buries the card,
but they all have to be reachable to be deletable. */}
<div className="classes">
{(expanded ? project.classes : project.classes.slice(0, CLASS_PREVIEW)).map((item) => (
<span className="tag class-tag" key={item.class_id}
title={`prompt: ${item.prompt}`}>
<span className="swatch" style={{ background: classColor(item.class_id) }} />
{item.name}
{item.annotation_count > 0 && (
<span className="faint mono">{item.annotation_count}</span>
)}
{project.classes.length > 1 && (
<button className="chip-x" disabled={busy}
aria-label={`Delete class ${item.name}`}
onClick={() => removeClass(item)}>
<XIcon size={11} />
</button>
)}
</span>
))}
{project.classes.length > CLASS_PREVIEW && (
<button className="tag" onClick={() => setExpanded((v) => !v)}>
{expanded ? 'show fewer' : `+${project.classes.length - CLASS_PREVIEW} more`}
</button>
)}
<button className="tag" onClick={() => setAddingClass((v) => !v)} title="Add a new class to this project">
<PlusIcon size={11} /> {addingClass ? 'cancel' : 'add class'}
</button>
</div>
{addingClass && (
<form className="add-class-inline" onSubmit={submitAddClass} style={{ display: 'flex', gap: 6, margin: '8px 0', alignItems: 'center' }}>
<input
type="text"
placeholder="Class name (e.g. half-sack)"
value={newClassName}
onChange={(e) => setNewClassName(e.target.value)}
required
style={{ flex: 1, padding: '4px 8px', fontSize: '0.85rem' }}
/>
<input
type="text"
placeholder="SAM3 Prompt (optional)"
value={newClassPrompt}
onChange={(e) => setNewClassPrompt(e.target.value)}
style={{ flex: 1, padding: '4px 8px', fontSize: '0.85rem' }}
/>
<button className="btn btn-primary" type="submit" disabled={busy || !newClassName.trim()} style={{ padding: '4px 10px', fontSize: '0.85rem' }}>
Save
</button>
</form>
)}
<div className="actions" style={{ flexWrap: 'wrap', gap: 6 }}>
<button className="btn btn-primary" onClick={() => navigate(`/projects/${project.id}`)}>
Open
</button>
<button className="btn" onClick={() => fileInputModel1.current?.click()} disabled={busy} title="Upload Primary Base Model 1 (.pt)">
<UploadIcon size={14} /> Model 1
</button>
<input ref={fileInputModel1} type="file" accept=".pt" hidden onChange={uploadModel1} />
<button className="btn" onClick={() => fileInputModel2.current?.click()} disabled={busy} title="Upload Secondary Auto-Annotate Model 2 (.pt)">
<UploadIcon size={14} /> Model 2
</button>
<input ref={fileInputModel2} type="file" accept=".pt" hidden onChange={uploadModel2} />
<span className="spacer" />
<button className="btn btn-danger" onClick={remove} disabled={busy}
aria-label={`Delete ${project.name}`}>
<TrashIcon size={14} />
</button>
</div>
</div>
)
}
export default function ProjectsPage() {
const [projects, setProjects] = useState(null)
const [defaultRoot, setDefaultRoot] = useState('')
const [creating, setCreating] = useState(false)
const [error, setError] = useState('')
async function load() {
try {
const payload = await api.listProjects()
setProjects(payload.projects)
setDefaultRoot(payload.video_root_default)
} catch (exc) {
setError(exc.message)
setProjects([])
}
}
useEffect(() => { load() }, [])
return (
<>
<div className="page-head">
<div>
<h1>Projects</h1>
<p className="muted">
One project per model you are improving: its base weights, its classes, its dataset.
</p>
</div>
<span className="spacer" />
{!creating && (
<button className="btn btn-primary" onClick={() => setCreating(true)}>
<PlusIcon size={14} /> New project
</button>
)}
</div>
{error && <p className="error-banner" style={{ marginBottom: 14 }}><AlertIcon size={14} /> {error}</p>}
{creating && (
<div style={{ marginBottom: 18 }}>
<NewProjectForm
defaultRoot={defaultRoot}
onCancel={() => setCreating(false)}
onCreated={() => { setCreating(false); setError(''); load() }}
/>
</div>
)}
{projects === null && <p className="empty">Loading…</p>}
{projects?.length === 0 && !creating && (
<p className="empty">No projects yet. Create one to start enriching a dataset.</p>
)}
<div className="card-grid">
{projects?.map((project) => (
<ProjectCard key={project.id} project={project} onChanged={load} onError={setError} />
))}
</div>
</>
)
}
+338
View File
@@ -0,0 +1,338 @@
import { useCallback, useEffect, useRef, useState } from 'react'
import { api } from '../api'
import { navigate } from '../App'
import AnnotationCanvas from '../components/AnnotationCanvas'
import Filmstrip from '../components/Filmstrip'
import { AlertIcon, CheckIcon, XIcon } from '../components/Icons'
import QuickReclassBar from '../components/QuickReclassBar'
import ReviewSidebar from '../components/ReviewSidebar'
export default function ReviewPage({ batchId: rawBatchId, projectId, onProject }) {
const [batch, setBatch] = useState(null)
const [project, setProject] = useState(null)
const [frames, setFrames] = useState([])
const [index, setIndex] = useState(0)
const [annotations, setAnnotations] = useState([])
const [selectedId, setSelectedId] = useState(null)
const [activeClass, setActiveClass] = useState(0)
const [assistMode, setAssistMode] = useState(false)
const [busy, setBusy] = useState(false)
const [error, setError] = useState('')
const [activeJob, setActiveJob] = useState(null)
const batchId = batch?.id || rawBatchId
const stripRef = useRef(null)
const hasInitialAutoJump = useRef(false)
const frame = frames[index]
const reload = useCallback(async () => {
let targetBatchId = rawBatchId
if (!targetBatchId && projectId) {
const bList = await api.listBatches(projectId)
if (bList?.batches?.length > 0) targetBatchId = bList.batches[0].id
}
if (!targetBatchId) throw new Error('No batch found to review')
const loaded = await api.getBatch(targetBatchId)
setBatch(loaded)
const payload = await api.listFrames(targetBatchId)
setFrames(payload.frames)
if (!hasInitialAutoJump.current && payload.frames?.length > 0) {
hasInitialAutoJump.current = true
const firstAnnotated = payload.frames.findIndex((f) => (f.annotation_count ?? 0) > 0)
if (firstAnnotated > 0) setIndex(firstAnnotated)
}
const jobsPayload = await api.listJobs(loaded.project_id).catch(() => ({ jobs: [] }))
const currentJob = jobsPayload.jobs?.find(
(j) => j.batch_id === Number(targetBatchId) && ['queued', 'running'].includes(j.status)
)
setActiveJob(currentJob || null)
return loaded
}, [rawBatchId, projectId])
useEffect(() => {
reload().then((loaded) => api.getProject(loaded.project_id)).then((loadedProject) => {
setProject(loadedProject)
onProject?.(loadedProject)
}).catch((exc) => setError(exc.message))
}, [rawBatchId, projectId, reload])
const prevJobId = useRef(activeJob?.id)
useEffect(() => {
if (prevJobId.current && !activeJob) {
hasInitialAutoJump.current = false
reload().catch(() => {})
}
prevJobId.current = activeJob?.id
if (!activeJob) return
const timer = setInterval(() => {
reload().catch(() => {})
if (frame) api.frameAnnotations(frame.id).then((p) => setAnnotations(p.annotations)).catch(() => {})
}, 2000)
return () => clearInterval(timer)
}, [activeJob, reload, frame?.id])
useEffect(() => {
if (!frame) return
let cancelled = false
api.frameAnnotations(frame.id).then((payload) => !cancelled && setAnnotations(payload.annotations)).catch((exc) => !cancelled && setError(exc.message))
setSelectedId(null)
return () => { cancelled = true }
}, [frame?.id])
useEffect(() => {
const active = stripRef.current?.querySelector('[aria-current="true"]')
active?.scrollIntoView({ block: 'nearest', inline: 'center' })
}, [index])
function patchFrameLocally(frameId, patch) {
setFrames((rows) => rows.map((row) => (row.id === frameId ? { ...row, ...patch } : row)))
}
const setStatus = useCallback(async (status) => {
if (!frame) return
patchFrameLocally(frame.id, { review_status: status })
try {
await api.setFrameStatus(frame.id, status)
setBatch(await api.getBatch(batchId))
} catch (exc) { setError(exc.message) }
setIndex((current) => Math.min(current + 1, frames.length - 1))
}, [frame, frames.length, batchId])
async function createShape(geometry) {
try {
const created = await api.addAnnotation(frame.id, { class_id: activeClass, geometry })
setAnnotations((rows) => [...rows, created])
setSelectedId(created.id)
patchFrameLocally(frame.id, { annotation_count: (frame.annotation_count ?? 0) + 1 })
} catch (exc) { setError(exc.message) }
}
async function assist(box) {
setBusy(true); setError('')
try {
const created = await api.assist(frame.id, { box, class_id: activeClass })
setAnnotations((rows) => [...rows, created])
setSelectedId(created.id)
} catch (exc) { setError(exc.message) } finally { setBusy(false) }
}
async function updateShape(id, geometry, { local, commit } = {}) {
if (local && geometry) {
setAnnotations((rows) => rows.map((row) => (row.id === id ? { ...row, geometry } : row)))
return
}
if (!commit) return
const current = annotations.find((row) => row.id === id)
if (!current) return
try { await api.patchAnnotation(id, { geometry: current.geometry }) } catch (exc) { setError(exc.message) }
}
const removeSelected = useCallback(async () => {
if (selectedId == null || !frame) return
const id = selectedId
setAnnotations((rows) => rows.filter((row) => row.id !== id))
setSelectedId(null)
setFrames((rows) => rows.map((row) => (row.id === frame.id ? { ...row, annotation_count: Math.max(0, (row.annotation_count ?? 1) - 1) } : row)))
try { await api.deleteAnnotation(id) } catch (exc) { setError(exc.message) }
}, [selectedId, frame])
const reclass = useCallback(async (classId) => {
setActiveClass(classId)
if (selectedId == null) return
try {
const updated = await api.patchAnnotation(selectedId, { class_id: classId })
setAnnotations((rows) => rows.map((row) => (row.id === updated.id ? updated : row)))
} catch (exc) { setError(exc.message) }
}, [selectedId])
async function approveBatch() {
try { await api.approveBatch(batchId); navigate(`/projects/${batch.project_id}/models`) } catch (exc) { setError(exc.message) }
}
async function jumpToPending() {
try {
const { frame_id: frameId } = await api.nextPending(batchId, frame?.idx ?? -1)
const position = frames.findIndex((row) => row.id === frameId)
if (position >= 0) setIndex(position)
} catch (exc) { setError(exc.message) }
}
const jumpToNextAnnotated = useCallback(() => {
if (!frames?.length) return
const nextIdx = frames.findIndex((f, idx) => idx > index && (f.annotation_count ?? 0) > 0)
if (nextIdx >= 0) setIndex(nextIdx)
else {
const firstIdx = frames.findIndex((f) => (f.annotation_count ?? 0) > 0)
if (firstIdx >= 0) setIndex(firstIdx)
}
}, [frames, index])
const stateRef = useRef({})
stateRef.current = { frames, index, project, selectedId, setStatus, removeSelected, reclass, jumpToPending, jumpToNextAnnotated, setAssistMode }
useEffect(() => {
function onKeyDown(event) {
if (event.target?.matches?.('input, textarea, select, [contenteditable="true"]')) return
const { frames, project, setStatus, removeSelected, reclass, jumpToPending, jumpToNextAnnotated, setAssistMode } = stateRef.current
const key = event.key
const isShortcutKey = /^[1-9]$/.test(key) || ['ArrowLeft', 'ArrowRight', 'ArrowUp', 'ArrowDown', 'Delete', 'Backspace', 'a', 'A', 'x', 'X', 'u', 'U', 's', 'S', 'n', 'N'].includes(key)
if (isShortcutKey) { event.preventDefault(); event.stopPropagation() }
if (key === 's' || key === 'S') setAssistMode?.(true)
else if (key === 'ArrowLeft') setIndex((i) => Math.max(0, i - 1))
else if (key === 'ArrowRight') setIndex((i) => Math.min((frames?.length || 1) - 1, i + 1))
else if (key === 'a' || key === 'A') setStatus?.('approved')
else if (key === 'x' || key === 'X') setStatus?.('rejected')
else if (key === 'u' || key === 'U') jumpToPending?.()
else if (key === 'n' || key === 'N') jumpToNextAnnotated?.()
else if (key === 'Delete' || key === 'Backspace') removeSelected?.()
else if (/^[1-9]$/.test(key)) {
const classId = Number(key) - 1
if (project && classId < project.classes.length) reclass?.(classId)
}
}
function onKeyUp(event) {
if (event.target?.matches?.('input, textarea, select, [contenteditable="true"]')) return
if (event.key === 's' || event.key === 'S') { event.preventDefault(); event.stopPropagation(); stateRef.current.setAssistMode?.(false) }
}
document.addEventListener('keydown', onKeyDown, true)
document.addEventListener('keyup', onKeyUp, true)
return () => {
document.removeEventListener('keydown', onKeyDown, true)
document.removeEventListener('keyup', onKeyUp, true)
}
}, [])
if (error && !batch) return <p className="error-banner" style={{ margin: 20 }}><AlertIcon size={14} /> {error}</p>
if (!batch || !project) return <p className="empty" style={{ margin: 20, color: 'var(--text-muted)' }}>Loading batch data…</p>
const reviewed = (batch.review?.approved ?? 0) + (batch.review?.rejected ?? 0)
const classesList = project.classes ?? []
async function approveAllFrames() {
if (!window.confirm(`Mark all ${batch.review?.pending ?? 0} pending frames as approved?`)) return
try { await api.approveAllBatchFrames(batchId); await reload() } catch (exc) { setError(exc.message) }
}
async function clearClassInBatch(item) {
if (!window.confirm(`Clear all shapes of class "${item.name}" across ALL frames in this batch?`)) return
try {
await api.clearBatchClassAnnotations(batchId, item.class_id)
await reload()
if (frame) {
const payload = await api.frameAnnotations(frame.id)
setAnnotations(payload.annotations)
}
} catch (exc) { setError(exc.message) }
}
return (
<>
<div className="page-head">
<div>
<h1>{batch.date_label} · {batch.batch_label}</h1>
<p className="muted">
{batch.frame_count} frames · {reviewed}/{batch.frame_count} reviewed ·
{' '}<strong style={{ color: batch.annotation_count > 0 ? '#c084fc' : '#a1a1aa' }}>{batch.annotation_count} shapes</strong> · status {batch.status}
</p>
</div>
<span className="spacer" />
{(batch.review?.pending ?? 0) > 0 && batch.status !== 'merged' && (
<button className="btn" style={{ marginRight: 8 }} title="Mark all pending frames in this batch as approved" onClick={approveAllFrames}>
Approve All Frames ({batch.review?.pending})
</button>
)}
<button
className="btn btn-primary"
disabled={(batch.review?.pending ?? 0) > 0 || batch.status === 'merged'}
title={(batch.review?.pending ?? 0) > 0 ? `${batch.review?.pending} frame(s) still pending` : 'Merge the approved frames into the master dataset'}
onClick={approveBatch}
>
{batch.status === 'merged' ? 'Merged' : 'Approve batch'}
</button>
</div>
{activeJob && (
<div className="panel side-panel" style={{ marginBottom: 12, border: '1px solid rgba(168, 85, 247, 0.4)', background: 'rgba(24, 24, 27, 0.8)' }}>
<div className="row" style={{ fontSize: '0.85rem' }}>
<span className="dot ok" />
<strong style={{ textTransform: 'capitalize' }}>Auto-labeling in progress…</strong>
<span className="spacer" />
<span className="mono">{activeJob.progress}/{activeJob.total || '—'} frames</span>
</div>
<div className="progress" style={{ margin: '6px 0' }}>
<span style={{ width: `${activeJob.total ? (activeJob.progress / activeJob.total) * 100 : 50}%` }} />
</div>
</div>
)}
{error && <p className="error-banner" style={{ marginBottom: 12 }}><AlertIcon size={14} /> {error}</p>}
<div className="review">
<div className="review-main">
{frame && (
<AnnotationCanvas
frame={frame}
imageUrl={api.frameUrl(frame.id)}
annotations={annotations}
selectedId={selectedId}
activeClass={activeClass}
assistMode={assistMode}
classes={classesList}
onSelect={setSelectedId}
onCreate={createShape}
onUpdate={updateShape}
onAssist={assist}
/>
)}
<div className="frame-bar">
<button className="btn" onClick={() => setIndex((i) => Math.max(0, i - 1))} disabled={index === 0}>←</button>
<span className="mono">
{index + 1} / {frames.length}
{frame && <span className={`status-pill ${frame.review_status}`}>{frame.review_status}</span>}
</span>
<button className="btn" onClick={() => setIndex((i) => Math.min(frames.length - 1, i + 1))} disabled={index >= frames.length - 1}>→</button>
<button
type="button"
className="btn"
style={{ padding: '2px 8px', fontSize: '0.78rem', borderColor: 'rgba(168, 85, 247, 0.5)', color: '#c084fc', background: 'rgba(168, 85, 247, 0.1)' }}
onClick={jumpToNextAnnotated}
title="Jump to next frame with annotations [N]"
>
🏷️ Next Shape [N]
</button>
<span className="spacer" />
{busy && <span className="muted">asking SAM3…</span>}
<button className="btn btn-danger" onClick={() => setStatus('rejected')}><XIcon size={14} /> Reject [X]</button>
<button className="btn btn-primary" onClick={() => setStatus('approved')}><CheckIcon size={14} /> Approve [A]</button>
</div>
{selectedId != null && (
<QuickReclassBar classesList={classesList} reclass={reclass} removeSelected={removeSelected} />
)}
<Filmstrip frames={frames} index={index} onSelectIndex={setIndex} stripRef={stripRef} />
</div>
<ReviewSidebar
classesList={classesList}
activeClass={activeClass}
reclass={reclass}
clearClassInBatch={clearClassInBatch}
annotations={annotations}
selectedId={selectedId}
setSelectedId={setSelectedId}
removeSelected={removeSelected}
project={project}
jumpToNextAnnotated={jumpToNextAnnotated}
batchAnnotationCount={batch?.annotation_count ?? 0}
/>
</div>
</>
)
}
+195
View File
@@ -0,0 +1,195 @@
import { useEffect, useRef, useState } from 'react'
import { api, formatDuration } from '../api'
import { navigate } from '../App'
import { AlertIcon, CheckIcon } from '../components/Icons'
function timecode(seconds) {
const total = Math.max(0, seconds || 0)
const m = Math.floor(total / 60)
const s = Math.floor(total % 60)
const cs = Math.floor((total % 1) * 10)
return `${m}:${String(s).padStart(2, '0')}.${cs}`
}
function parseTimecode(text, fallback) {
const trimmed = String(text).trim()
if (!trimmed) return fallback
const parts = trimmed.split(':').map(Number)
if (parts.some(Number.isNaN)) return fallback
return parts.length === 1 ? parts[0] : parts[0] * 60 + parts[1]
}
export default function TrimPage({ projectId, rel }) {
const videoRef = useRef(null)
const [info, setInfo] = useState(null)
const [error, setError] = useState('')
const [start, setStart] = useState(0)
const [end, setEnd] = useState(0)
const [fps, setFps] = useState(1)
const [playhead, setPlayhead] = useState(0)
const [job, setJob] = useState(null)
const [busy, setBusy] = useState(false)
useEffect(() => {
api.videoInfo(projectId, rel)
.then((payload) => {
setInfo(payload)
setEnd(Math.min(payload.duration, 60))
})
.catch((exc) => setError(exc.message))
}, [projectId, rel])
// Poll the extraction job until it stops moving.
useEffect(() => {
if (!job || ['done', 'failed', 'cancelled'].includes(job.status)) return
const timer = setInterval(() => {
api.getJob(job.id).then(setJob).catch(() => {})
}, 1000)
return () => clearInterval(timer)
}, [job])
const duration = info?.duration || 0
const estimated = Math.max(0, Math.round((end - start) * fps))
function seek(seconds) {
if (videoRef.current) videoRef.current.currentTime = seconds
}
async function extract() {
setBusy(true)
setError('')
try {
const batch = await api.createBatch(projectId, {
rel, start_sec: start, end_sec: end, fps: Number(fps),
})
const payload = await api.listJobs(projectId)
setJob(payload.jobs.find((item) => item.batch_id === batch.id) ?? null)
} catch (exc) {
setError(exc.message)
} finally {
setBusy(false)
}
}
if (error && !info) return <p className="error-banner"><AlertIcon size={14} /> {error}</p>
if (!info) return <p className="empty">Reading the video…</p>
const finished = job?.status === 'done'
return (
<>
<div className="page-head">
<div>
<h1>{info.date_label} · {info.batch_label}</h1>
<p className="muted">
{formatDuration(duration)} · {info.width}×{info.height} · {info.fps} fps source
</p>
</div>
<span className="spacer" />
<button className="btn btn-ghost" onClick={() => navigate(`/projects/${projectId}`)}>
Back to library
</button>
</div>
<div className="trim">
<div className="panel trim-player">
<video
ref={videoRef}
src={api.videoUrl(projectId, rel)}
controls
preload="metadata"
onTimeUpdate={(e) => setPlayhead(e.target.currentTime)}
/>
</div>
<div className="panel trim-controls stack">
<h2>Range</h2>
<div className="range-group">
<label htmlFor="trim-start">Start — {timecode(start)}</label>
<input
id="trim-start" type="range" min="0" max={duration} step="0.1" value={start}
onChange={(e) => {
const value = Math.min(Number(e.target.value), end - 0.1)
setStart(value)
seek(value)
}}
/>
<div className="row">
<input
aria-label="Start timecode" value={timecode(start)}
onChange={(e) => setStart(Math.min(parseTimecode(e.target.value, start), end - 0.1))}
/>
<button className="btn" onClick={() => setStart(Math.min(playhead, end - 0.1))}>
Use playhead
</button>
</div>
</div>
<div className="range-group">
<label htmlFor="trim-end">End — {timecode(end)}</label>
<input
id="trim-end" type="range" min="0" max={duration} step="0.1" value={end}
onChange={(e) => {
const value = Math.max(Number(e.target.value), start + 0.1)
setEnd(value)
seek(value)
}}
/>
<div className="row">
<input
aria-label="End timecode" value={timecode(end)}
onChange={(e) => setEnd(Math.max(parseTimecode(e.target.value, end), start + 0.1))}
/>
<button className="btn" onClick={() => setEnd(Math.max(playhead, start + 0.1))}>
Use playhead
</button>
</div>
</div>
<div>
<label htmlFor="trim-fps">Frames per second</label>
<input
id="trim-fps" type="number" min="0.1" max="30" step="0.1" value={fps}
onChange={(e) => setFps(e.target.value)}
/>
</div>
<p className="hint">
{formatDuration(end - start)} of video → <strong>{estimated}</strong> frame(s)
</p>
{error && <p className="error-banner"><AlertIcon size={14} /> {error}</p>}
{job && (
<div className="job-status">
<div className="row">
<span className={`dot ${job.status === 'done' ? 'ok' : job.status === 'failed' ? 'bad' : ''}`} />
<span>{job.status}</span>
<span className="spacer" />
<span className="mono">{job.progress}/{job.total || estimated}</span>
</div>
<div className="progress">
<span style={{ width: `${job.total ? (job.progress / job.total) * 100 : 0}%` }} />
</div>
{job.error && <p className="error-banner">{job.error}</p>}
{job.message && <p className="hint">{job.message}</p>}
</div>
)}
<div className="form-actions">
<button className="btn btn-primary" onClick={extract}
disabled={busy || estimated === 0 || (job && !['done', 'failed', 'cancelled'].includes(job.status))}>
{finished ? <><CheckIcon size={14} /> Extracted</> : 'Extract frames'}
</button>
{job && !['done', 'failed', 'cancelled'].includes(job.status) && (
<button className="btn" onClick={() => api.cancelJob(job.id).catch(() => {})}>
Cancel
</button>
)}
</div>
</div>
</div>
</>
)
}
+266
View File
@@ -0,0 +1,266 @@
/* frontend/src/roboflow.css */
.roboflow-layout {
display: flex;
width: 100vw;
height: 100vh;
overflow: hidden;
background: var(--bg, #0b0f19);
color: var(--text, #f3f4f6);
}
.roboflow-sidebar {
width: 260px;
background: rgba(17, 24, 39, 0.65);
border-right: 1px solid rgba(255, 255, 255, 0.12);
display: flex;
flex-direction: column;
padding: 16px 0;
flex-shrink: 0;
transition: width 0.2s ease, padding 0.2s ease;
}
.roboflow-sidebar.collapsed {
width: 64px;
}
.sidebar-header {
padding: 0 20px;
margin-bottom: 24px;
display: flex;
align-items: center;
justify-content: space-between;
}
.roboflow-sidebar.collapsed .sidebar-header {
padding: 0 12px;
justify-content: center;
}
.sidebar-collapse-btn {
background: none;
border: none;
color: var(--text-faint, #9ca3af);
cursor: pointer;
padding: 4px;
font-size: 12px;
border-radius: 4px;
transition: background 0.15s, color 0.15s;
}
.sidebar-collapse-btn:hover {
background: rgba(255, 255, 255, 0.1);
color: var(--text);
}
.sidebar-logo {
font-weight: 700;
font-size: 16px;
color: var(--text);
white-space: nowrap;
}
.roboflow-sidebar.collapsed .sidebar-item {
padding: 12px 0;
text-align: center;
display: flex;
justify-content: center;
}
.sidebar-icon {
font-size: 16px;
margin-right: 8px;
}
.roboflow-sidebar.collapsed .sidebar-icon {
margin-right: 0;
}
.sidebar-section {
margin-bottom: 24px;
}
.sidebar-section-title {
padding: 0 20px;
font-size: 11px;
font-weight: 600;
color: var(--text-faint, #9ca3af);
margin-bottom: 8px;
letter-spacing: 0.05em;
}
.sidebar-item {
display: flex;
align-items: center;
padding: 8px 20px;
color: var(--text-muted, #d1d5db);
text-decoration: none;
font-size: 13px;
cursor: pointer;
transition: background 0.15s, color 0.15s;
}
.sidebar-item:hover {
background: rgba(255, 255, 255, 0.05);
color: var(--text);
}
.sidebar-item.active {
background: rgba(168, 85, 247, 0.15);
color: var(--text);
border-right: 3px solid #a855f7;
font-weight: 500;
}
.sidebar-spacer {
flex: 1;
}
.sidebar-footer {
padding: 0 20px;
display: flex;
flex-direction: column;
gap: 12px;
}
.sidebar-theme-toggle {
background: none;
border: 1px solid rgba(255, 255, 255, 0.2);
color: var(--text-muted);
padding: 6px 12px;
border-radius: 6px;
cursor: pointer;
font-size: 12px;
}
.sidebar-theme-toggle:hover {
background: rgba(255, 255, 255, 0.1);
}
.sidebar-health {
background: rgba(0, 0, 0, 0.25);
border-radius: 6px;
padding: 10px;
font-size: 11px;
color: var(--text-faint);
display: grid;
gap: 4px;
}
.health-item {
display: flex;
justify-content: space-between;
}
.roboflow-main {
flex: 1;
display: flex;
flex-direction: column;
overflow: hidden;
position: relative;
}
/* Models page engine cards */
.select-engine-section {
margin-bottom: 32px;
}
.engine-grid {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 16px;
margin-top: 16px;
}
.engine-card {
background: rgba(17, 24, 39, 0.45);
border: 1px solid rgba(255, 255, 255, 0.12);
border-radius: 12px;
padding: 20px;
cursor: pointer;
transition: all 0.2s;
}
.engine-card:hover {
border-color: rgba(255, 255, 255, 0.3);
}
.engine-card.selected {
border-color: #a855f7;
background: rgba(168, 85, 247, 0.05);
}
.engine-card-header {
display: flex;
align-items: center;
gap: 12px;
margin-bottom: 8px;
}
.engine-card-title {
font-weight: 600;
font-size: 15px;
}
.project-badge, .chip-badge {
font-size: 11px;
padding: 2px 8px;
border-radius: 999px;
font-weight: 600;
}
.project-badge {
background: rgba(168, 85, 247, 0.2);
color: #c084fc;
}
.chip-badge.info {
background: rgba(56, 189, 248, 0.2);
color: #38bdf8;
}
.engine-card-desc {
font-size: 13px;
color: var(--text-muted);
line-height: 1.5;
margin: 0;
}
/* Light mode overrides */
[data-theme='light'] {
--bg: #f3f4f6;
--text: #111827;
--text-muted: #4b5563;
--text-faint: #6b7280;
}
[data-theme='light'] .roboflow-sidebar {
background: #ffffff;
border-right-color: #e5e7eb;
}
[data-theme='light'] .sidebar-item:hover {
background: #f9fafb;
}
[data-theme='light'] .sidebar-item.active {
background: #faf5ff;
}
[data-theme='light'] .engine-card {
background: #ffffff;
border-color: #e5e7eb;
}
[data-theme='light'] .engine-card:hover {
border-color: #d1d5db;
}
[data-theme='light'] .engine-card.selected {
border-color: #a855f7;
background: #faf5ff;
}
[data-theme='light'] .sidebar-health {
background: #f3f4f6;
}
+255
View File
@@ -0,0 +1,255 @@
html, body, #root {
margin: 0;
padding: 0;
width: 100vw;
height: 100vh;
overflow: hidden !important;
background-color: #0b0f19;
}
:root {
color-scheme: dark;
--bg: #0b0f19;
--panel: rgba(17, 24, 39, 0.5);
--panel-raised: rgba(30, 41, 59, 0.7);
--border: rgba(255, 255, 255, 0.12);
--border-strong: rgba(168, 85, 247, 0.45);
--text: #f3f4f6;
--text-muted: #9ca3af;
--text-faint: #6b7280;
--accent: #a855f7;
--accent-hover: #c084fc;
--accent-contrast: #ffffff;
--accent-soft: rgba(168, 85, 247, 0.22);
--ok: #10b981;
--warn: #f59e0b;
--danger: #ef4444;
--danger-soft: rgba(239, 68, 68, 0.2);
/* Class colours */
--class-0: #f59e0b;
--class-1: #38bdf8;
--class-2: #10b981;
--class-3: #facc15;
--class-4: #6366f1;
--class-5: #f97316;
--class-6: #ec4899;
--class-7: #9ca3af;
--radius: 12px;
--radius-sm: 8px;
--space: 8px;
--font: 'Inter', system-ui, -apple-system, sans-serif;
--mono: ui-monospace, "JetBrains Mono", "SF Mono", Menlo, monospace;
--transition: 160ms cubic-bezier(0.4, 0, 0.2, 1);
--shadow: 0 12px 32px rgba(0, 0, 0, 0.45);
}
* {
box-sizing: border-box;
}
/* Custom HUD Scrollbars */
::-webkit-scrollbar {
width: 6px;
height: 6px;
}
::-webkit-scrollbar-track {
background: transparent;
}
::-webkit-scrollbar-thumb {
background: rgba(255, 255, 255, 0.18);
border-radius: 999px;
}
body {
margin: 0;
background: var(--bg);
color: var(--text);
font-family: var(--font);
font-size: 14px;
line-height: 1.5;
-webkit-font-smoothing: antialiased;
}
h1, h2, h3 {
margin: 0;
font-weight: 600;
letter-spacing: -0.01em;
}
h1 { font-size: 20px; }
h2 { font-size: 16px; }
h3 { font-size: 14px; }
a {
color: var(--accent);
text-decoration: none;
}
a:hover { text-decoration: underline; }
code, .mono { font-family: var(--mono); font-size: 12px; }
/* Every interactive element gets a pointer and a visible focus ring — the
* review screens have to be usable without a mouse at all. */
button, [role="button"], summary, label.clickable {
cursor: pointer;
}
button:disabled {
cursor: not-allowed;
opacity: 0.5;
}
:focus-visible {
outline: 2px solid var(--accent);
outline-offset: 2px;
border-radius: var(--radius-sm);
}
.btn {
display: inline-flex;
align-items: center;
gap: 6px;
padding: 7px 13px;
border: 1px solid var(--border-strong);
border-radius: var(--radius-sm);
background: var(--panel-raised);
color: var(--text);
font: inherit;
font-weight: 500;
transition: background var(--transition), border-color var(--transition),
color var(--transition);
}
.btn:hover:not(:disabled) {
background: var(--border);
border-color: var(--text-faint);
}
.btn-primary {
background: var(--accent);
border-color: var(--accent);
color: var(--accent-contrast);
}
.btn-primary:hover:not(:disabled) {
background: var(--accent-hover);
border-color: var(--accent-hover);
}
.btn-danger {
color: var(--danger);
border-color: transparent;
background: transparent;
}
.btn-danger:hover:not(:disabled) {
background: var(--danger-soft);
border-color: var(--danger);
}
.btn-ghost {
background: transparent;
border-color: transparent;
color: var(--text-muted);
}
.btn-ghost:hover:not(:disabled) {
background: var(--panel-raised);
color: var(--text);
}
input, select, textarea {
width: 100%;
padding: 7px 10px;
border: 1px solid var(--border-strong);
border-radius: var(--radius-sm);
background: var(--bg);
color: var(--text);
font: inherit;
transition: border-color var(--transition), box-shadow var(--transition);
}
input:hover, select:hover, textarea:hover { border-color: var(--text-faint); }
input:focus, select:focus, textarea:focus {
outline: none;
border-color: var(--accent);
box-shadow: 0 0 0 3px var(--accent-soft);
}
select { cursor: pointer; }
label {
display: block;
margin-bottom: 4px;
color: var(--text-muted);
font-size: 12px;
font-weight: 500;
}
.panel {
background: var(--panel);
border: 1px solid var(--border);
border-radius: var(--radius);
}
.muted { color: var(--text-muted); }
.faint { color: var(--text-faint); }
.row {
display: flex;
align-items: center;
gap: calc(var(--space) * 1.5);
}
.stack {
display: flex;
flex-direction: column;
gap: calc(var(--space) * 1.5);
}
.tag {
display: inline-flex;
align-items: center;
gap: 5px;
padding: 2px 8px;
border-radius: 999px;
background: var(--panel-raised);
border: 1px solid var(--border);
font-size: 12px;
color: var(--text-muted);
}
.swatch {
width: 9px;
height: 9px;
border-radius: 2px;
flex: none;
}
.error-banner {
padding: 9px 12px;
border: 1px solid var(--danger);
border-radius: var(--radius-sm);
background: var(--danger-soft);
color: var(--text);
}
.empty {
padding: 40px 20px;
text-align: center;
color: var(--text-muted);
}
@media (prefers-reduced-motion: reduce) {
*, *::before, *::after {
transition-duration: 0.01ms !important;
animation-duration: 0.01ms !important;
}
}
+17
View File
@@ -0,0 +1,17 @@
import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react'
// In dev the SPA runs on 5173 and the API on 8000; this proxy keeps every fetch
// same-origin, so the code is identical to how nginx serves it in Docker.
export default defineConfig({
plugins: [react()],
server: {
port: 5173,
proxy: {
'/api': {
target: process.env.API_URL || 'http://localhost:8000',
changeOrigin: true,
},
},
},
})
+24
View File
@@ -0,0 +1,24 @@
#!/bin/bash
set -e
echo "Downloading NVIDIA GPG key..."
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
echo "Adding NVIDIA package repository..."
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
echo "Updating apt..."
apt-get update
echo "Installing NVIDIA Container Toolkit..."
apt-get install -y nvidia-container-toolkit
echo "Configuring Docker..."
nvidia-ctk runtime configure --runtime=docker
echo "Restarting Docker..."
systemctl restart docker
echo "Done! NVIDIA Container Toolkit is installed."
+20
View File
@@ -0,0 +1,20 @@
torch
torchvision
fastapi
uvicorn[standard]
python-multipart
pydantic>=2
python-dotenv
opencv-python-headless
pillow
numpy<2
huggingface_hub
iopath
ultralytics
# SAM3 vendor deps that its pyproject doesn't declare
einops
pycocotools
# sam3/model_builder.py still imports pkg_resources, which setuptools 81+ dropped
setuptools<81
Submodule
+1
Submodule sam3 added at 96914d2425.
+46
View File
@@ -0,0 +1,46 @@
#!/usr/bin/env python3
"""Auto-pull service: monitors origin/main and pulls new commits automatically."""
import os
import subprocess
import sys
import time
INTERVAL_SECONDS = int(os.environ.get("AUTO_PULL_INTERVAL", 30))
REPO_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
def run_git(args):
res = subprocess.run(["git"] + args, cwd=REPO_DIR, capture_output=True, text=True)
return res.returncode == 0, res.stdout.strip(), res.stderr.strip()
def check_and_pull():
# Fetch latest remote changes quietly
ok, _, err = run_git(["fetch", "origin", "main"])
if not ok:
print(f"[Auto-Pull] Git fetch error: {err}")
return
_, incoming_commits, _ = run_git(["log", "HEAD..origin/main", "--oneline"])
if incoming_commits:
print(f"[Auto-Pull] Incoming commits detected:\n{incoming_commits}")
ok, out, err = run_git(["pull", "origin", "main"])
if ok:
print(f"[Auto-Pull] Successfully pulled:\n{out}")
print("[Auto-Pull] Restarting app services...")
restart_script = os.path.join(REPO_DIR, "scripts", "restart_app.sh")
subprocess.run(["bash", restart_script], cwd=REPO_DIR)
else:
print(f"[Auto-Pull] Pull failed:\n{err}")
def main():
print(f"🔄 Auto-pull background poller started (Interval: {INTERVAL_SECONDS}s)")
while True:
try:
check_and_pull()
except Exception as exc:
print(f"[Auto-Pull] Error: {exc}")
time.sleep(INTERVAL_SECONDS)
if __name__ == "__main__":
main()
+23
View File
@@ -0,0 +1,23 @@
#!/bin/bash
set -e
REPO_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$REPO_DIR"
echo "🔄 Restarting app services..."
# Kill running uvicorn and vite processes
pkill -f "uvicorn backend.main:app" || true
pkill -f "vite" || true
sleep 1
export PATH="$HOME/.local/bin:$PATH"
# Run backend
nohup uv run uvicorn backend.main:app --host 0.0.0.0 --port 8000 > "$REPO_DIR/data/backend.log" 2>&1 &
# Run frontend
cd "$REPO_DIR/frontend"
nohup npm run dev > "$REPO_DIR/data/frontend.log" 2>&1 &
echo "✅ Services restarted."
+67
View File
@@ -0,0 +1,67 @@
#!/usr/bin/env python3
"""GitHub Webhook Listener for Auto-Pull on Push."""
import hmac
import hashlib
import json
import os
import subprocess
from http.server import BaseHTTPRequestHandler, HTTPServer
PORT = int(os.environ.get("WEBHOOK_PORT", 9000))
SECRET = os.environ.get("WEBHOOK_SECRET", "").encode("utf-8")
REPO_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
class WebhookHandler(BaseHTTPRequestHandler):
def do_POST(self):
if self.path != "/webhook":
self.send_response(404)
self.end_headers()
return
content_length = int(self.headers.get("Content-Length", 0))
body = self.rfile.read(content_length)
if SECRET:
signature = self.headers.get("X-Hub-Signature-256", "")
expected = "sha256=" + hmac.new(SECRET, body, hashlib.sha256).hexdigest()
if not hmac.compare_digest(signature, expected):
self.send_response(403)
self.end_headers()
self.wfile.write(b"Invalid signature\n")
return
event = self.headers.get("X-GitHub-Event", "push")
if event == "push":
print("[Webhook] Received push event. Pulling latest code...")
try:
result = subprocess.run(
["git", "pull", "origin", "main"],
cwd=REPO_DIR,
capture_output=True,
text=True,
check=True,
)
output = result.stdout
print(f"[Webhook] Git pull success:\n{output}")
self.send_response(200)
self.end_headers()
self.wfile.write(f"Updated successfully:\n{output}".encode("utf-8"))
except subprocess.CalledProcessError as err:
error_msg = f"Git pull failed:\n{err.stderr}"
print(f"[Webhook] {error_msg}")
self.send_response(500)
self.end_headers()
self.wfile.write(error_msg.encode("utf-8"))
else:
self.send_response(200)
self.end_headers()
self.wfile.write(f"Ignored event: {event}\n".encode("utf-8"))
def main():
server = HTTPServer(("0.0.0.0", PORT), WebhookHandler)
print(f"🚀 GitHub Webhook listener running on http://0.0.0.0:{PORT}/webhook")
server.serve_forever()
if __name__ == "__main__":
main()
Executable
+65
View File
@@ -0,0 +1,65 @@
#!/bin/bash
set -e
echo "🚀 Checking system configuration for Dataset Enrichment App..."
# Remove any old overrides so we start fresh
rm -f docker-compose.override.yml
# 1. Check for NVIDIA GPU
if command -v nvidia-smi &> /dev/null; then
echo "✅ NVIDIA GPU detected."
# 2. Check if the NVIDIA Container Toolkit is installed
if ! command -v nvidia-ctk &> /dev/null; then
echo "⚠️ NVIDIA Container Toolkit is missing."
echo " Without it, Docker cannot access your GPU, and the app will run in CPU-only mode."
read -p "Would you like to install it automatically now? (requires sudo) [y/N] " install_choice
if [[ "$install_choice" =~ ^[Yy]$ ]]; then
echo "Installing NVIDIA Toolkit..."
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor --yes -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
echo "✅ NVIDIA Toolkit installed successfully!"
else
echo "⚠️ Skipping installation. Proceeding without GPU support."
fi
fi
# 3. If Toolkit is available, configure CDI and enable GPU passthrough
if command -v nvidia-ctk &> /dev/null; then
if [ ! -f "/etc/cdi/nvidia.yaml" ]; then
echo "⚙️ Generating Docker CDI configuration..."
sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml
fi
echo "🔌 Enabling GPU support via docker-compose.override.yml..."
cat <<EOF > docker-compose.override.yml
services:
backend:
devices:
- nvidia.com/gpu=all
EOF
fi
else
echo "ℹ️ No NVIDIA GPU detected. Running in CPU-only mode."
fi
echo "📦 Starting containers..."
docker compose up -d --build
# Get local IP for convenience
LOCAL_IP=$(hostname -I | awk '{print $1}' || echo "localhost")
PORT=${WEB_PORT:-8080}
echo ""
echo "=========================================================="
echo "✅ App is successfully running!"
echo "🌐 Access it locally at: http://localhost:$PORT"
echo "📱 Access it on your network at: http://$LOCAL_IP:$PORT"
echo "=========================================================="