feedmill-recounter/.opencode/plans/mjpeg-live-preview.md

Plan: Switch Live Preview from WebSocket+base64 to MJPEG Streaming

Goal

Replace the WebSocket+base64 live preview with MJPEG streaming (same approach as the original karung-counting project) for faster, smoother preview with less overhead.

Why MJPEG is Faster

Factor	WebSocket+base64	MJPEG
Frame size	JPEG + 33% base64 bloat + JSON wrapper	Raw JPEG bytes
Transport	JSON over WebSocket	Binary HTTP stream
Decode	JS sets img.src per frame (triggers decode)	Browser native <img> MJPEG decode
Code	Queue + thread + SocketIO emit + JS handler	File write + HTTP generator
Dependencies	flask-socketio, JS socket.io client	None (stdlib only)

Architecture

Pipeline (pipeline.py)
  → write JPEG to /tmp/feedmill_preview_{job_id}.jpg (atomic tmp+replace)

Flask endpoint (app.py)
  GET /api/preview/<job_id>
  → reads file in tight loop, yields multipart/x-mixed-replace MJPEG stream

Frontend (status.html)
  <img src="/api/preview/{job_id}" />
  → browser handles MJPEG natively, no JS needed

Implementation Plan

Phase 1: Pipeline — Write to file instead of Queue

Modify src/pipeline.py:

• Add preview_path: str | None = None parameter
• Replace queue put with atomic file write:

if preview_path is not None and frame_idx % max(1, preview_every_n) == 0:
    h, w = viz.shape[:2]
    if max(h, w) > preview_max_dim:
        scale = preview_max_dim / max(h, w)
        preview_viz = cv2.resize(viz, (int(w * scale), int(h * scale)))
    else:
        preview_viz = viz
    tmp_path = preview_path + ".tmp.jpg"
    cv2.imwrite(tmp_path, preview_viz, [cv2.IMWRITE_JPEG_QUALITY, preview_jpeg_quality])
    os.replace(tmp_path, preview_path)

• Remove preview_queue parameter (deprecated)

Phase 2: Job Queue — Compute preview path, remove broadcaster

Modify src/job.py:

• Remove preview_queue, _preview_thread, _preview_stop fields
• Remove start_preview_broadcaster() and stop_preview_broadcaster() methods
• Add preview_path: str field (computed in __init__):

preview_path: str = ""  # set in add_job to /tmp/feedmill_preview_{job_id}.jpg

• In add_job(), set job.preview_path = f"/tmp/feedmill_preview_{job.job_id}.jpg"
• In _run_job(), pass preview_path=job.preview_path to run_pipeline()
• In cleanup, delete the preview file:

try:
    os.remove(job.preview_path)
except OSError:
    pass

Phase 3: Flask — Add MJPEG endpoint, remove SocketIO preview

Modify app.py:

• Add MJPEG endpoint:

import time as _time

@app.route("/api/preview/<job_id>")
def api_preview(job_id):
    job = job_queue.get_job(job_id)
    if job is None:
        return jsonify({"error": "not found"}), 404
    
    preview_path = job.preview_path
    if not preview_path:
        return jsonify({"error": "no preview path"}), 404
    
    def generate():
        consecutive_fails = 0
        MAX_FAILS = 30
        while True:
            try:
                with open(preview_path, "rb") as f:
                    jpeg = f.read()
                consecutive_fails = 0
                yield (b"--frame\r\n"
                       b"Content-Type: image/jpeg\r\n\r\n" + jpeg + b"\r\n")
            except FileNotFoundError:
                consecutive_fails += 1
                if consecutive_fails >= MAX_FAILS:
                    return
                _time.sleep(1.0)
                continue
            except Exception:
                consecutive_fails += 1
                if consecutive_fails >= MAX_FAILS:
                    return
                _time.sleep(0.5)
                continue
            _time.sleep(0.05)
    return Response(generate(), mimetype="multipart/x-mixed-replace; boundary=frame")

• Remove SocketIO handlers for preview (on_connect, on_disconnect, on_join_job, on_leave_job)
• Remove job.start_preview_broadcaster() calls from upload routes
• Remove flask_socketio import and socketio init (if no longer needed)
• Keep socketio.run() in main() OR switch to app.run() if SocketIO is fully removed

Phase 4: Frontend — Replace SocketIO with MJPEG img tag

Modify templates/status.html:

• Change <img> src to MJPEG endpoint:

<img id="live-preview-img" class="live-preview-img" 
     src="/api/preview/{{ job.job_id }}"
     alt="Annotated frame from video processing" />

• Remove preview-placeholder div (MJPEG auto-shows when frames arrive)

Modify static/app.js:

• Remove SocketIO code:

// DELETE: var socket = io();
// DELETE: socket.on('connect', ...)
// DELETE: socket.on('preview_frame', ...)
// DELETE: window.addEventListener('beforeunload', ...)

• Remove updateLivePreview() function (no longer called)

Phase 5: Cleanup — Remove SocketIO if fully unused

Modify app.py:

• Remove from flask_socketio import SocketIO, join_room, leave_room
• Remove socketio = SocketIO(...) line
• Change socketio.run(app, ...) to app.run(host, port, debug)

Modify pyproject.toml:

• Remove flask-socketio and eventlet from optional deps

Modify templates/base.html:

• Remove SocketIO CDN script tag

Files to Modify

File	Changes
src/pipeline.py	Replace preview_queue with preview_path file write
src/job.py	Compute preview_path, remove broadcaster thread/queue
app.py	Add MJPEG endpoint, remove SocketIO handlers, use app.run()
templates/status.html	Add src="/api/preview/{job_id}" to img tag
templates/base.html	Remove SocketIO CDN script
static/app.js	Remove SocketIO code and updateLivePreview()
pyproject.toml	Remove flask-socketio, eventlet deps

Expected Performance

Metric	Before (WebSocket)	After (MJPEG)
Frame size	~50-80 KB (base64+JSON)	~30-50 KB (raw JPEG)
Transport overhead	33% base64 + JSON	0%
Decode latency	JS img.src assignment	Native browser MJPEG
Dependencies	flask-socketio, eventlet, socket.io CDN	None
JS complexity	~20 lines SocketIO code	0 lines

Testing

1. Run python -m pytest tests/ -v --tb=short — 62/62 pass
2. Start web UI, upload video, start job
3. Verify MJPEG stream loads at http://localhost:9000/api/preview/{job_id}
4. Verify live preview updates smoothly in browser
5. Verify cleanup: preview file deleted after job completes

Rollout

1. Implement all changes
2. Test locally
3. Commit and push
4. Restart web UI