STOP: host-wide /proc sweep guarantees zero RECOUNT_CMD processes remain

AGENTS.md

@@ -47,6 +47,7 @@ PYTHONNOUSERSITE=1 venv/bin/python recounting_dashboard_upload.py
 - `counter_live.py` (TensorRT) is Jetson-only and won't work on RK3588.
 - **Reset flow**: `POST /api/reset` → deletes state JSON + touches `{SHM_DIR}/.reset`. `batch_store._check_reset_signal()` watches this marker and clears in-memory state on next crossing.
 - **Recount continue flow** (upload dashboard, port 5003): `POST /api/load-mp4 {path}` kills the old process, spawns `RECOUNT_CMD` (loads + pauses), and POSTs `/api/reset` to the recount node. `POST /api/start-recount` (no body, requires a loaded file) touches `{SHM_DIR}/.continue`; the C++ process consumes that marker to begin counting. The dashboard never clears the marker.
+- **Recount STOP sweep**: `POST /api/stop-recount` kills the tracked process group (`killpg` TERM→5s→KILL) then sweeps the host via `/proc` for any process whose argv matches the `RECOUNT_CMD` binary name (basename of the first token, so it also catches orphans from a dashboard restart); TERM→`_SWEEP_GRACE_SEC`→KILL with `_SWEEP_MAX_RETRIES` retries, all under `_recount_lock`. Returns `{success, killed, remaining}`; `remaining` is 0 in the normal path, and survivors (e.g. D-state) are logged rather than force-hanging.
 - **Upload filename contract**: uploads are expected as `batch_XX_YYYYMMDD_HHmmSS.mp4` (XX = batch number, timestamp = date). `recounting_dashboard_upload.py` parses both to query the live API's `/api/day-detail/<YYYY-MM-DD>` for the recorded count.
 - **`RECOUNT_CMD`** in the upload dashboard is a template with a `{path}` placeholder, e.g. `bytetrack-counter-cpp config.env --source {path}`; the uploaded file path is `shlex.quote()`d before substitution, and the command is spawned by **LOAD MP4** (loads + pauses until `.continue` is touched).
 - **Cross-dashboard header links**: `UPLOAD_DASHBOARD_URL` (live → upload dashboard, `counter_dashboard.py`) and `LIVE_DASHBOARD_URL` (upload → live dashboard, `recounting_dashboard_upload.py`); empty value hides the header link.

SPEC.md

@@ -126,7 +126,7 @@ api: GET /api/uploads → list uploaded MP4s sorted by mtime desc → [ {name, p
 api: POST /api/delete-upload {path} → delete uploaded MP4 (path restricted to UPLOAD_DIR)
 api: POST /api/load-mp4 {path} → kill old process, spawn RECOUNT_CMD (path shlex-quoted) paused, then reset recount API
 api: POST /api/start-recount → require loaded file, touch {SHM_DIR}/.continue to signal C++ process to begin counting
-api: POST /api/stop-recount → kill recount process + clear current file
+api: POST /api/stop-recount → killpg tracked group + host-wide /proc sweep of RECOUNT_CMD binary (TERM→grace→KILL, verify loop); returns {success, killed, remaining}
 api: GET /api/state → {streaming, file, file_name, stream_url, batch_number, batch_date}
 api: GET /api/proxy-stream → proxy MJPEG from RECOUNT_API_URL:/api/live-video (same-origin)
 api: GET /api/live-progress → proxy to LIVE_API_URL:/api/current-batch
@@ -162,6 +162,7 @@ V20: reset → delete STATE_FILE + touch {SHM_DIR}/.reset; batch_store._check_re
 V21: upload filename batch_XX_YYYYMMDD_HHmmSS.mp4 → batch# + date parsed; batch-result queries /api/day-detail/<YYYY-MM-DD> on live API
 V22: preview auto-on when recount running — enablePreview() on START RECOUNT + applyState streaming branch; manual toggle overrides; img.onerror retries cache-busted src while streaming
 V23: .continue marker flow — LOAD MP4 spawns RECOUNT_CMD paused; START RECOUNT touches {SHM_DIR}/.continue; C++ process consumes marker to begin counting; dashboard never clears it (LOAD alone does not count)
+V24: STOP → killpg tracked group then host-wide /proc sweep of RECOUNT_CMD binary (basename of first token); sweep runs under _recount_lock, TERM→_SWEEP_GRACE_SEC→KILL, bounded retries; returns {success, killed, remaining} with remaining==0 in normal path; survivors (D-state) logged, never force-hang
 ```
 
 ## §T — Tasks
@@ -186,6 +187,7 @@ T15|x|add upload recounting dashboard — MP4 upload, progress bar, RECOUNT_CMD
 T16|x|add cross-dashboard header links — UPLOAD_DASHBOARD_URL (live→upload) & LIVE_DASHBOARD_URL (upload→live)|I.config
 T17|x|auto-on live preview when recount running — enablePreview() on START + applyState; onerror retry while streaming|V22
 T18|x|split recount flow — /api/load-mp4 spawns paused RECOUNT_CMD, /api/start-recount touches {SHM_DIR}/.continue; C++ consumes marker|V23
+T19|x|hard-kill STOP — sweep all RECOUNT_CMD-binary processes via /proc, verify remaining==0, report killed/remaining|V24
 ```
 
 ## §B — Bugs

recounting_dashboard_upload.py

@@ -48,6 +48,9 @@ RECOUNT_CMD = os.getenv("RECOUNT_CMD", "bytetrack-counter config.env --source {p
 SHM_DIR = os.getenv("SHM_DIR", "/dev/shm/bytetrack-counter")
 _CONTINUE_FLAG = Path(f"{SHM_DIR}/.continue")
 
+_SWEEP_GRACE_SEC = 5
+_SWEEP_MAX_RETRIES = 3
+
 _http_session = requests.Session()
 _http_session.timeout = 3
 
@@ -105,6 +108,60 @@ def _stop_recount_locked():
         pass
 
 
+def _recount_binary_name():
+    if not RECOUNT_CMD:
+        return "bytetrack-counter-cpp"
+    try:
+        first = shlex.split(RECOUNT_CMD)[0]
+    except (ValueError, IndexError):
+        return "bytetrack-counter-cpp"
+    return Path(first).name or "bytetrack-counter-cpp"
+
+
+def _iter_matching_pids(binary):
+    for entry in os.listdir("/proc"):
+        if not entry.isdigit():
+            continue
+        try:
+            with open(f"/proc/{entry}/cmdline", "rb") as f:
+                raw = f.read().split(b"\x00")
+        except (FileNotFoundError, PermissionError, ProcessLookupError):
+            continue
+        args = [t.decode(errors="ignore") for t in raw if t]
+        for token in args:
+            for piece in token.split():
+                if Path(piece).name == binary:
+                    yield int(entry)
+                    break
+            else:
+                continue
+            break
+
+
+def _sweep_recount_processes(binary):
+    killed = 0
+    for _ in range(_SWEEP_MAX_RETRIES):
+        pids = list(_iter_matching_pids(binary))
+        if not pids:
+            break
+        for pid in pids:
+            if pid not in list(_iter_matching_pids(binary)):
+                continue
+            try:
+                os.kill(pid, signal.SIGTERM)
+                killed += 1
+            except (ProcessLookupError, PermissionError):
+                pass
+        time.sleep(_SWEEP_GRACE_SEC)
+        for pid in list(_iter_matching_pids(binary)):
+            try:
+                os.kill(pid, signal.SIGKILL)
+            except (ProcessLookupError, PermissionError):
+                pass
+        time.sleep(1)
+    return killed, len(list(_iter_matching_pids(binary)))
+
+
 def _folder_date(filepath, base):
     rel = Path(filepath).resolve().relative_to(base.resolve())
     parts = rel.parts
@@ -262,7 +319,13 @@ def stop_recount():
     with _recount_lock:
         _stop_recount_locked()
         _current_file = None
-    return jsonify({"success": True})
+        killed, remaining = _sweep_recount_processes(_recount_binary_name())
+    stamp = datetime.now().strftime('%Y-%m-%d %H:%M:%S')
+    if remaining:
+        print(f"[{stamp}] Recount stop: killed {killed}, remaining {remaining} (SIGKILL could not reap)")
+    else:
+        print(f"[{stamp}] Recount stop: killed {killed}, remaining {remaining}")
+    return jsonify({"success": True, "killed": killed, "remaining": remaining})
 
 
 @app.route("/api/state")