diff --git a/.opencode/plans/live-preview-toggle-fps-counts.md b/.opencode/plans/live-preview-toggle-fps-counts.md new file mode 100644 index 0000000..476f3fe --- /dev/null +++ b/.opencode/plans/live-preview-toggle-fps-counts.md @@ -0,0 +1,195 @@ +# Plan: Live Preview Toggle, FPS Indicator, Live Counting + +## Goal +1. Toggle on/off for live preview (disable when not needed for faster processing) +2. Live FPS indicator showing processing speed +3. Live counting by object type (sack loading/unloading, box loading/unloading) + +## Architecture + +The pipeline already runs `counter.update(tracked_sacks)` every frame and computes `counter.loading_count` / `counter.unloading_count`. The `progress_callback` is already called every frame. We piggyback on this to push live stats to the Job object. Frontend polls `/api/jobs/` every 2s (already exists) and reads the new live stats. + +``` +Pipeline (every frame) + → progress_callback(frame_idx, total_frames) + → job.update_live_stats(fps, loading, unloading, boxes) + +Frontend (every 2s poll) + → reads live_fps, live_loading, live_unloading, live_boxes from API + → updates display +``` + +## Implementation + +### 1. `src/pipeline.py` — Add live_stats_callback, compute FPS + +Add parameter: +```python +live_stats_callback: Callable[[dict], None] | None = None, +``` + +In the main loop, compute FPS (rolling average over last 30 frames) and call: +```python +if live_stats_callback and frame_idx % 10 == 0: + live_stats_callback({ + "fps": processing_fps, + "loading": counter.loading_count, + "unloading": counter.unloading_count, + "frame": frame_idx, + }) +``` + +FPS computation: `time.time()` delta over last N frames, smoothed. + +### 2. `src/job.py` — Add live stats fields to Job + +Add fields: +```python +live_fps: float = 0.0 +live_loading: int = 0 +live_unloading: int = 0 +live_frame: int = 0 +``` + +In `_run_job()`, pass `live_stats_callback` to `run_pipeline()`: +```python +def _on_live_stats(stats): + with self._lock: + job.live_fps = stats["fps"] + job.live_loading = stats["loading"] + job.live_unloading = stats["unloading"] + job.live_frame = stats["frame"] + +result = run_pipeline( + ..., + live_stats_callback=_on_live_stats, +) +``` + +### 3. `app.py` — Add live stats to API response + +In `api_job_detail()`, add to the JSON response: +```python +"live_fps": job.live_fps, +"live_loading": job.live_loading, +"live_unloading": job.live_unloading, +"live_frame": job.live_frame, +``` + +### 4. `templates/status.html` — Add toggle, FPS indicator, live counters + +Add to the live preview section header: +```html +
+

Live Preview

+ + -- FPS +
+``` + +Add live counting display below the preview image: +```html + +``` + +### 5. `static/app.js` — Toggle logic, update live stats from poll + +In `initStatusPage()`: + +**Toggle handler:** +```javascript +var previewToggle = document.getElementById('preview-toggle'); +var livePreviewImg = document.getElementById('live-preview-img'); +if (previewToggle) { + previewToggle.addEventListener('change', function() { + if (this.checked) { + livePreviewImg.src = '/api/preview/' + jobId; + livePreviewSection.style.display = ''; + } else { + livePreviewImg.src = ''; + livePreviewSection.style.display = 'none'; + } + }); +} +``` + +**Update live stats from poll response:** +```javascript +// In pollJob() success handler: +if (data.live_fps !== undefined) { + var fpsEl = document.getElementById('live-fps'); + if (fpsEl) fpsEl.textContent = data.live_fps.toFixed(1) + ' FPS'; +} +if (data.live_loading !== undefined) { + var countsSection = document.getElementById('live-counts'); + if (countsSection) countsSection.style.display = ''; + var el = document.getElementById('live-loading'); + if (el) el.textContent = data.live_loading; + el = document.getElementById('live-unloading'); + if (el) el.textContent = data.live_unloading; + el = document.getElementById('live-net'); + if (el) el.textContent = (data.live_loading - data.live_unloading); +} +``` + +### 6. `static/style.css` — Style toggle, FPS badge, live counts + +```css +.live-preview-header { display: flex; align-items: center; gap: var(--space-3); } +.live-fps { font-size: var(--text-xs); font-weight: var(--font-semibold); color: var(--color-primary); background: var(--color-primary-50); padding: 2px 8px; border-radius: var(--radius-sm); } +.live-counts { display: flex; gap: var(--space-4); padding: var(--space-3) 0; } +.live-count-item { text-align: center; } +.live-count-label { font-size: var(--text-xs); color: var(--color-neutral-500); display: block; } +.live-count-value { font-size: var(--text-xl); font-weight: var(--font-bold); } +.toggle-label { display: flex; align-items: center; gap: var(--space-1); font-size: var(--text-sm); cursor: pointer; } +``` + +## Files to Modify + +| File | Changes | +|------|---------| +| `src/pipeline.py` | Add `live_stats_callback` param, compute FPS, call callback every 10 frames | +| `src/job.py` | Add `live_fps`, `live_loading`, `live_unloading`, `live_frame` fields, wire callback | +| `app.py` | Add live stats to `api_job_detail()` response | +| `templates/status.html` | Add toggle checkbox, FPS badge, live counting display | +| `static/app.js` | Toggle handler, update live stats from poll response | +| `static/style.css` | Style toggle, FPS badge, live counts | + +## Expected Behavior + +- **Toggle OFF**: Live preview img hidden, no MJPEG stream (faster processing) +- **Toggle ON**: MJPEG stream shown, FPS badge updates, live counts update +- **FPS badge**: Shows "12.3 FPS" (rolling average) +- **Live counts**: Shows Loading/Unloading/Net in real-time below the preview +- **Counts per class**: Pipeline counts sack and box separately via `MultiClassLineCounter` (already exists, needs wiring) + +## Performance Impact + +- FPS computation: negligible (1 `time.time()` per frame) +- Live stats callback: negligible (dict + lock every 10 frames) +- MJPEG toggle OFF: saves disk I/O (no file writes), reduces CPU (no resize/encode) +- API poll: same 2s interval, just more fields in JSON response + +## Testing + +1. `python -m pytest tests/ -v --tb=short` — 62/62 pass +2. Start job, verify FPS badge shows ~12-25 FPS +3. Toggle OFF → verify preview disappears, processing continues +4. Toggle ON → verify preview reappears +5. Verify live counts increment during processing diff --git a/app.py b/app.py index 1a742bf..68ddec3 100644 --- a/app.py +++ b/app.py @@ -279,6 +279,10 @@ def api_job_detail(job_id): for r in job.results ], "error": job.error, + "live_fps": job.live_fps, + "live_loading": job.live_loading, + "live_unloading": job.live_unloading, + "live_frame": job.live_frame, }) diff --git a/src/job.py b/src/job.py index 889f6bc..dd4274f 100644 --- a/src/job.py +++ b/src/job.py @@ -56,6 +56,10 @@ class Job: completed_at: float | None = None latest_frame: bytes | None = None preview_path: str = "" + live_fps: float = 0.0 + live_loading: int = 0 + live_unloading: int = 0 + live_frame: int = 0 class JobQueue: @@ -165,6 +169,13 @@ class JobQueue: f"{model_cfg.stem}_annotated.mp4", ) + def _on_live_stats(stats): + with self._lock: + job.live_fps = stats["fps"] + job.live_loading = stats["loading"] + job.live_unloading = stats["unloading"] + job.live_frame = stats["frame"] + result: PipelineResult = run_pipeline( video_path=job.video_path, model_config=model_cfg, @@ -174,6 +185,7 @@ class JobQueue: preview_path=job.preview_path, preview_every_n=2, cancel_check=lambda: job.status == JobStatus.CANCELLED, + live_stats_callback=_on_live_stats, ) with self._lock: diff --git a/src/pipeline.py b/src/pipeline.py index 98e9326..ba4f16b 100644 --- a/src/pipeline.py +++ b/src/pipeline.py @@ -69,6 +69,7 @@ def run_pipeline( preview_every_n: int = 2, preview_max_dim: int = 640, preview_jpeg_quality: int = 70, + live_stats_callback: Callable[[dict], None] | None = None, ) -> PipelineResult: """Process a video file through the counting pipeline. @@ -87,6 +88,8 @@ def run_pipeline( preview_every_n: Write a preview frame every N frames (default 2). preview_max_dim: Max dimension (w or h) for preview frames. preview_jpeg_quality: JPEG quality for preview frames (1-100). + live_stats_callback: Optional fn(dict) called every 10 frames with + {fps, loading, unloading, frame} for live UI stats. Returns: PipelineResult with counting summary. @@ -158,6 +161,7 @@ def run_pipeline( try: writer = AnnotatedVideoWriter(output_path, fps=fps, frame_size=(w, h)) + _frame_times: list[float] = [] while True: ret, frame = cap.read() if not ret: @@ -166,6 +170,10 @@ def run_pipeline( frame_idx += 1 timestamp = time.time() + _frame_times.append(timestamp) + if len(_frame_times) > 30: + _frame_times.pop(0) + # Check for cancellation if cancel_check and frame_idx % 30 == 0 and cancel_check(): break @@ -242,6 +250,19 @@ def run_pipeline( writer.write_frame(viz) + if live_stats_callback and frame_idx % 10 == 0: + if len(_frame_times) >= 2: + dt = _frame_times[-1] - _frame_times[0] + processing_fps = (len(_frame_times) - 1) / dt if dt > 0 else 0.0 + else: + processing_fps = 0.0 + live_stats_callback({ + "fps": round(processing_fps, 1), + "loading": counter.loading_count, + "unloading": counter.unloading_count, + "frame": frame_idx, + }) + if preview_path is not None and frame_idx % max(1, preview_every_n) == 0: h, w = viz.shape[:2] if max(h, w) > preview_max_dim: diff --git a/static/app.js b/static/app.js index 67b471e..3ec8ffc 100644 --- a/static/app.js +++ b/static/app.js @@ -358,6 +358,31 @@ function initStatusPage(jobId, initialStatus) { if (!progressFill) return; + var previewToggle = document.getElementById('preview-toggle'); + var toggleText = document.getElementById('toggle-text'); + var liveFps = document.getElementById('live-fps'); + var liveCounts = document.getElementById('live-counts'); + var liveLoading = document.getElementById('live-loading'); + var liveUnloading = document.getElementById('live-unloading'); + var liveNet = document.getElementById('live-net'); + + if (previewToggle) { + previewToggle.addEventListener('change', function() { + if (this.checked) { + livePreviewImg.src = '/api/preview/' + jobId; + if (previewPlaceholder) previewPlaceholder.style.display = ''; + livePreviewSection.style.display = ''; + if (toggleText) toggleText.textContent = 'ON'; + } else { + livePreviewImg.src = ''; + livePreviewImg.style.display = 'none'; + if (previewPlaceholder) previewPlaceholder.style.display = ''; + livePreviewSection.style.display = ''; + if (toggleText) toggleText.textContent = 'OFF'; + } + }); + } + /* --- Cancel button --- */ if (cancelBtn) { cancelBtn.addEventListener('click', function() { @@ -397,6 +422,16 @@ function initStatusPage(jobId, initialStatus) { updateError(data.error); announceStatusChange(data.status, data.progress); + if (data.live_fps !== undefined && liveFps) { + liveFps.textContent = data.live_fps > 0 ? data.live_fps.toFixed(1) + ' FPS' : 'Starting...'; + } + if (data.live_loading !== undefined) { + if (liveCounts) liveCounts.style.display = ''; + if (liveLoading) liveLoading.textContent = data.live_loading; + if (liveUnloading) liveUnloading.textContent = data.live_unloading; + if (liveNet) liveNet.textContent = (data.live_loading - data.live_unloading); + } + if (progressRegion) { progressRegion.setAttribute('aria-busy', data.status === 'RUNNING' || data.status === 'PENDING' ? 'true' : 'false'); } diff --git a/static/style.css b/static/style.css index 189c54f..17b80e7 100644 --- a/static/style.css +++ b/static/style.css @@ -660,6 +660,99 @@ main { display: block; } +.live-preview-header { + display: flex; + align-items: center; + gap: var(--space-3); + margin-bottom: var(--space-3); +} + +.live-preview-header .section-title { + margin-bottom: 0; +} + +.live-fps { + font-size: var(--text-xs); + font-weight: var(--font-semibold); + color: var(--color-primary); + background: var(--color-primary-50, #f0fdfa); + padding: 2px 8px; + border-radius: var(--radius-sm); + white-space: nowrap; +} + +.toggle-label { + display: flex; + align-items: center; + gap: var(--space-2); + cursor: pointer; + font-size: var(--text-sm); + user-select: none; +} + +.toggle-label input[type="checkbox"] { + display: none; +} + +.toggle-switch { + width: 36px; + height: 20px; + background: var(--color-neutral-300); + border-radius: 10px; + position: relative; + transition: background var(--transition-fast); +} + +.toggle-switch::after { + content: ''; + position: absolute; + width: 16px; + height: 16px; + background: white; + border-radius: 50%; + top: 2px; + left: 2px; + transition: transform var(--transition-fast); +} + +.toggle-label input:checked + .toggle-switch { + background: var(--color-primary); +} + +.toggle-label input:checked + .toggle-switch::after { + transform: translateX(16px); +} + +.toggle-text { + font-weight: var(--font-medium); + color: var(--color-neutral-600); +} + +.live-counts { + display: flex; + gap: var(--space-6); + padding: var(--space-3) 0; + border-top: 1px solid var(--color-neutral-100); + margin-top: var(--space-3); +} + +.live-count-item { + text-align: center; + flex: 1; +} + +.live-count-label { + font-size: var(--text-xs); + color: var(--color-neutral-500); + display: block; + margin-bottom: var(--space-1); +} + +.live-count-value { + font-size: var(--text-2xl); + font-weight: var(--font-bold); +} + .preview-placeholder { position: absolute; inset: 0; diff --git a/templates/status.html b/templates/status.html index 91a26a4..a3aa5af 100644 --- a/templates/status.html +++ b/templates/status.html @@ -33,11 +33,33 @@
-

Live Preview

+
+

Live Preview

+ + -- FPS +
Annotated frame from video processing
Waiting for first frame...
+