feedmill-recounter/.opencode/plans/mjpeg-live-preview.md

# Plan: Switch Live Preview from WebSocket+base64 to MJPEG Streaming

## Goal
Replace the WebSocket+base64 live preview with MJPEG streaming (same approach as the original karung-counting project) for faster, smoother preview with less overhead.

## Why MJPEG is Faster

| Factor | WebSocket+base64 | MJPEG |
|--------|-----------------|-------|
| **Frame size** | JPEG + 33% base64 bloat + JSON wrapper | Raw JPEG bytes |
| **Transport** | JSON over WebSocket | Binary HTTP stream |
| **Decode** | JS sets img.src per frame (triggers decode) | Browser native `<img>` MJPEG decode |
| **Code** | Queue + thread + SocketIO emit + JS handler | File write + HTTP generator |
| **Dependencies** | flask-socketio, JS socket.io client | None (stdlib only) |

## Architecture

```
Pipeline (pipeline.py)
  → write JPEG to /tmp/feedmill_preview_{job_id}.jpg (atomic tmp+replace)

Flask endpoint (app.py)
  GET /api/preview/<job_id>
  → reads file in tight loop, yields multipart/x-mixed-replace MJPEG stream

Frontend (status.html)
  <img src="/api/preview/{job_id}" />
  → browser handles MJPEG natively, no JS needed
```

## Implementation Plan

### Phase 1: Pipeline — Write to file instead of Queue

**Modify `src/pipeline.py`:**
- Add `preview_path: str | None = None` parameter
- Replace queue put with atomic file write:
```python
if preview_path is not None and frame_idx % max(1, preview_every_n) == 0:
    h, w = viz.shape[:2]
    if max(h, w) > preview_max_dim:
        scale = preview_max_dim / max(h, w)
        preview_viz = cv2.resize(viz, (int(w * scale), int(h * scale)))
    else:
        preview_viz = viz
    tmp_path = preview_path + ".tmp.jpg"
    cv2.imwrite(tmp_path, preview_viz, [cv2.IMWRITE_JPEG_QUALITY, preview_jpeg_quality])
    os.replace(tmp_path, preview_path)
```
- Remove `preview_queue` parameter (deprecated)

### Phase 2: Job Queue — Compute preview path, remove broadcaster

**Modify `src/job.py`:**
- Remove `preview_queue`, `_preview_thread`, `_preview_stop` fields
- Remove `start_preview_broadcaster()` and `stop_preview_broadcaster()` methods
- Add `preview_path: str` field (computed in `__init__`):
```python
preview_path: str = ""  # set in add_job to /tmp/feedmill_preview_{job_id}.jpg
```
- In `add_job()`, set `job.preview_path = f"/tmp/feedmill_preview_{job.job_id}.jpg"`
- In `_run_job()`, pass `preview_path=job.preview_path` to `run_pipeline()`
- In cleanup, delete the preview file:
```python
try:
    os.remove(job.preview_path)
except OSError:
    pass
```

### Phase 3: Flask — Add MJPEG endpoint, remove SocketIO preview

**Modify `app.py`:**
- Add MJPEG endpoint:
```python
import time as _time

@app.route("/api/preview/<job_id>")
def api_preview(job_id):
    job = job_queue.get_job(job_id)
    if job is None:
        return jsonify({"error": "not found"}), 404
    
    preview_path = job.preview_path
    if not preview_path:
        return jsonify({"error": "no preview path"}), 404
    
    def generate():
        consecutive_fails = 0
        MAX_FAILS = 30
        while True:
            try:
                with open(preview_path, "rb") as f:
                    jpeg = f.read()
                consecutive_fails = 0
                yield (b"--frame\r\n"
                       b"Content-Type: image/jpeg\r\n\r\n" + jpeg + b"\r\n")
            except FileNotFoundError:
                consecutive_fails += 1
                if consecutive_fails >= MAX_FAILS:
                    return
                _time.sleep(1.0)
                continue
            except Exception:
                consecutive_fails += 1
                if consecutive_fails >= MAX_FAILS:
                    return
                _time.sleep(0.5)
                continue
            _time.sleep(0.05)
    return Response(generate(), mimetype="multipart/x-mixed-replace; boundary=frame")
```
- Remove SocketIO handlers for preview (`on_connect`, `on_disconnect`, `on_join_job`, `on_leave_job`)
- Remove `job.start_preview_broadcaster()` calls from upload routes
- Remove `flask_socketio` import and `socketio` init (if no longer needed)
- Keep `socketio.run()` in `main()` OR switch to `app.run()` if SocketIO is fully removed

### Phase 4: Frontend — Replace SocketIO with MJPEG img tag

**Modify `templates/status.html`:**
- Change `<img>` src to MJPEG endpoint:
```html
<img id="live-preview-img" class="live-preview-img" 
     src="/api/preview/{{ job.job_id }}"
     alt="Annotated frame from video processing" />
```
- Remove `preview-placeholder` div (MJPEG auto-shows when frames arrive)

**Modify `static/app.js`:**
- Remove SocketIO code:
```javascript
// DELETE: var socket = io();
// DELETE: socket.on('connect', ...)
// DELETE: socket.on('preview_frame', ...)
// DELETE: window.addEventListener('beforeunload', ...)
```
- Remove `updateLivePreview()` function (no longer called)

### Phase 5: Cleanup — Remove SocketIO if fully unused

**Modify `app.py`:**
- Remove `from flask_socketio import SocketIO, join_room, leave_room`
- Remove `socketio = SocketIO(...)` line
- Change `socketio.run(app, ...)` to `app.run(host, port, debug)`

**Modify `pyproject.toml`:**
- Remove `flask-socketio` and `eventlet` from optional deps

**Modify `templates/base.html`:**
- Remove SocketIO CDN script tag

## Files to Modify

| File | Changes |
|------|---------|
| `src/pipeline.py` | Replace `preview_queue` with `preview_path` file write |
| `src/job.py` | Compute `preview_path`, remove broadcaster thread/queue |
| `app.py` | Add MJPEG endpoint, remove SocketIO handlers, use `app.run()` |
| `templates/status.html` | Add `src="/api/preview/{job_id}"` to img tag |
| `templates/base.html` | Remove SocketIO CDN script |
| `static/app.js` | Remove SocketIO code and `updateLivePreview()` |
| `pyproject.toml` | Remove `flask-socketio`, `eventlet` deps |

## Expected Performance

| Metric | Before (WebSocket) | After (MJPEG) |
|--------|-------------------|---------------|
| Frame size | ~50-80 KB (base64+JSON) | ~30-50 KB (raw JPEG) |
| Transport overhead | 33% base64 + JSON | 0% |
| Decode latency | JS img.src assignment | Native browser MJPEG |
| Dependencies | flask-socketio, eventlet, socket.io CDN | None |
| JS complexity | ~20 lines SocketIO code | 0 lines |

## Testing

1. Run `python -m pytest tests/ -v --tb=short` — 62/62 pass
2. Start web UI, upload video, start job
3. Verify MJPEG stream loads at `http://localhost:9000/api/preview/{job_id}`
4. Verify live preview updates smoothly in browser
5. Verify cleanup: preview file deleted after job completes

## Rollout

1. Implement all changes
2. Test locally
3. Commit and push
4. Restart web UI