# Architecture ## Pipeline overview (v3, `src/main.py`) ``` Frame → TruckDetect → ROI → SackTrack → Stabilize → Filter → Count → Overlay/CSV ``` `run()` processes **every frame** (`src/main.py:111`). Truck detection is the exception: it runs every **15 frames** (`TRUCK_DET_INTERVAL`, `src/main.py:130`) because the truck moves slowly and that model is heavy. All components are wired by dependency injection in `build_pipeline()` (`src/main.py:27`) and communicate through the DTOs/protocols in `src/interfaces.py` (`Detection`, `FrameResult`, `StreamSource`, `Detector`, `Tracker`, `Counter`, `BatchManager`). Per-frame sequence (`src/main.py:132-202`): 1. **Truck detect (every 15th frame)** — `TruckDetector.detect(frame)` → `TruckROITracker.update(trucks)` returns smoothed `TruckROI` (or holds last ROI). 2. **Sync counting line to ROI** — `counter.line_y/x_start/x_end = roi.*`, so the line follows the truck instead of staying fixed. 3. **Batch update (every 15th frame)** — `batch_mgr.update(truck_present, …)`. 4. **Track → Stabilize → Filter → Count** (only while a batch is active): `ByteTrackTracker.update` → `BboxStabilizer.update` → `_filter_sacks_in_roi` (keep sacks whose centroid-X is inside ROI) → `LineCrossCounter.update` → `CSVLogger.log_event` per crossing. 5. **Overlay** — `DashboardOverlay.draw(...)` → `cv2.imshow`. ## Modules ### Streaming (`src/streaming.py`) - `VideoFileSource(path)` — offline testing / validation. - `RTSPSource(url)` — live camera via FFmpeg backend (`cv2.CAP_FFMPEG`). - Common interface: `open() / read() / release()`, `fps`, `frame_size`. ### Detection (`src/detection.py`) - `SackDetector` — YOLO **segmentation** model; `_parse` keeps only `class_name == "sack"` and attaches the mask. (Docstring notes persons are visible to the model but dropped.) - `TruckDetector` — YOLO detection model; keeps all classes (single-class `truck` model). - Both are single-responsibility; new model types are added as new classes. ### Tracking (`src/tracking.py`) - `ByteTrackTracker` — `YOLO.track(persist=True, tracker=cfg/tracker.yaml)` (Ultralytics FastTrack, occlusion-aware: Kalman rollback on occlusion, enlarged search region, re-ID). - `_parse` keeps classes `("sack", "truck")` and attaches `track_id` (`boxes.id`). - `reset()` reloads the model weights — called on batch boundaries. ### Stabilizer (`src/stabilizer.py`) Per-track-ID fixes for worker occlusion / flicker: 1. **Jitter** (10–50 px jumps) → EMA on bbox (`ema_alpha=0.35`). 2. **Dropout at the line** (worker blocks sack 5–10 frames) → hold last smoothed bbox for `max_hold_frames=10`, with 0.85 confidence decay. 3. **Height expansion spike** (worker body merges into sack box) → clamp to `smooth_h * 1.5`. 4. **Height shrinkage** (worker covers sack bottom) → clamp to `smooth_h * 0.70`. ### Truck ROI (`src/truck_roi.py`) - `_pick_main_truck`: largest detection whose center-X falls in the lane (`0.35–0.80` of frame width). - EMA smoothing (`alpha=0.15`); holds last ROI for 5 missed updates (~3 s), then clears. - Counting line = truck **top edge** + `LINE_OFFSET_PX` (= 0). ### Counting (`src/counting.py`) `LineCrossCounter` tracks the **top edge (`y1`)** of each stabilized sack bbox against a zone band `[line_y − margin, line_y + margin]` (default margin 20 px): - Track states per ID: `above` (`y1 < upper`) / `below` (`y1 > lower`) / hold in band. - History is latched forever (`has_been_above/below`) so a crossing is caught even if the track jumps over the line between low-FPS frames — no exact crossing frame needed. - **Loading** = ever-above now below; **Unloading** = ever-below now above. - 3-layer dedup: (1) must have been on the opposite side first, (2) spatial radius 30 px / 3 s window, (3) one count per track-ID per direction. - Detections with centroid-X outside `[line_x_start, line_x_end]` are skipped. - `reset()` on every new batch. ### Batch lifecycle (`src/batch.py`) 4-state machine (`BatchState`): ``` IDLE ──truck in area──▶ TRUCK_STABILIZING ──stable 5 s──▶ COUNTING_SACKS ▲ │ truck gone 3 s │ ▲ │ └────────▶ IDLE │ │ sacks resume │ │ │ │ 10 s no sack activity ▼ │ │ WAITING_FOR_ACTIVITY │ (batch OPEN) └────────────────── truck leaves ─────────────────────────────────┘ (batch finalized → BatchRecord → history) ``` - `update_truck(truck_detected, centroid, ts)` drives IDLE / STABILIZING / WAITING. - `update_sacks(crossing_event, sacks_in_area, ts, …)` drives COUNTING / WAITING; activity resumes the same batch instead of opening a new one. - `update(...)` is a backward-compatible shim combining both. - `on_batch_start(id, ts)` / `on_batch_end(BatchRecord)` callbacks — `main.py` uses them to reset counter/stabilizer/ROI and to write the CSV row. - Tunables: `stabilize_seconds=5`, `stabilize_threshold_px=15`, `sack_idle_timeout=10`, `min_batch_duration=30`, `truck_gone_tolerance=3`. ### Dashboard overlay (`src/dashboard.py`) Draws onto the frame: truck ROI box + confidence, counting-zone band + center line, sack boxes with `track_id` + confidence and a green **top-edge (y1) trigger marker**, stats panel (Loading / Unloading / Net / last-3-batch history), and a bottom state bar (IDLE / stabilizing + progress / counting + duration-idle / waiting). ### Config & logging - `src/config_loader.py` — canonical loader: `config.yaml` (stream, models, counting, batch, **do**, output, camera) + `.env` (secrets/deployment only) + `zones.json` polygons + `cfg/tracker.yaml` reference. Model modes are data (`models.modes`); missing file falls back to `.env` + legacy defaults. - `src/do_batch.py` — pure helpers for DO-gated batches (mode validation, plate grouping, start gates, net/discard both-nets rule, unit classify, retention). - `src/do_ocr.py` — `extract_do_fields(image, engine)` for `tesseract`|`paddle`|`none` (dashboard upload only; never in the predict loop). - `src/config.py` — **deprecated** frozen v3 `Config` (different key names); do not add keys here. - `src/logger.py` — `CSVLogger` appends `batch_summary.csv` (`batch_id,start,end,duration,loading,unloading,net`) and `sack_events.csv` (`timestamp,batch_id,sack_crossed,T-,direction`). ## Production pipeline (`predict.py`) Same stage order as v3 but implemented with shapely polygons (`zones.json`: pallet/truck/counting), visual-similarity + Re-ID registries, static-frame debounce, SQLite persistence, and live-frame publishing to `/dev/shm` for the dashboard. Uses the repo `src/` modules (imports at `predict.py`). Zero CLI flags = systemd behaviour (`config.yaml` + `.env`); dev flags (`--source`, `--config`, `--model-mode`, `--no-dashboard`, …) are additive overrides — see `predict.py --help` and `docs/scripts.md`. (`init_db()` and the `http.server` imports are dead code — persistence goes through `finalize_batch`/`save_active_batch_state`, dashboard integration is file-based.) ### Batch modes (three-way) `batch_mode.json` `mode` (default seed `batch.default_mode: auto`): | Mode | Open/close | Notes | |---|---|---| | `auto` | existing truck FSM | no DO APIs; start/stop manual → 409 | | `do_manual` | operator + staged DO photos | DO gates in dashboard start only | | `manual` | operator buttons (legacy) | no DO gate | `predict.py` reads mode every frame: `auto` → FSM branch; any other valid value → operator/state-file branch (persists `count`, `box_count`, `box_unloading`, **`unloading`**). Finalize discard = **both nets 0** (`net_sack = loading − unloading`, `net_box = box_loading − box_unloading`). DO pipeline (dashboard): smartphone upload → OCR draft → edit/stage → start gates → active batch → stop-preview soft-warn → stop → `batches` + DO columns. ERD: [`../ERD.md`](../ERD.md). UI plan: [`docs/operator-tiles-history-plan.md`](operator-tiles-history-plan.md). ## Retired (`archive/`, not imported) - **`predict_new.py`** — duo-model state machine (`WAITING_FOR_TRUCK → COUNTING_SACKS → TRUCK_LEAVING`) with hardcoded default zones, perspective filtering and HUD. - **`rpo_iki/`** — alternate geometric engine (`count.py`: `LineCounter` with approach depth, burst/cooldown dedup, blind-truck overlay) + `predict_rpo_iki.py` runner.