andrew
f621ae86fb
ci / smoke (push) Canceled after 0s
Big docked truck bbox rests 1-2px past detection_polygon bottom edge, so 100%-containment made truck_in_area flicker False -> batch 15 split (6+276) on 2026-09-29 while the truck never left. v4-best.engine detected it at conf 0.92-0.97 in every replayed frame (no model miss, no retraining). - truck gate: frac_inside >= 0.5 instead of contains(bbox) - batch.truck_gone_tolerance_seconds (new, 30) authoritative; drop the hardcoded batch_mgr._truck_gone_tolerance = 30.0 override; batch.timeout_seconds documented as sack-idle pause only - [TRUCK] truck_in_area transition debug log
167 lines
9.2 KiB
Markdown
167 lines
9.2 KiB
Markdown
# Architecture
|
||
|
||
## Pipeline overview (v3, `src/main.py`)
|
||
|
||
```
|
||
Frame → TruckDetect → ROI → SackTrack → Stabilize → Filter → Count → Overlay/CSV
|
||
```
|
||
|
||
`run()` processes **every frame** (`src/main.py:111`). Truck detection is the exception:
|
||
it runs every **15 frames** (`TRUCK_DET_INTERVAL`, `src/main.py:130`) because the truck
|
||
moves slowly and that model is heavy. All components are wired by dependency injection
|
||
in `build_pipeline()` (`src/main.py:27`) and communicate through the DTOs/protocols in
|
||
`src/interfaces.py` (`Detection`, `FrameResult`, `StreamSource`, `Detector`, `Tracker`,
|
||
`Counter`, `BatchManager`).
|
||
|
||
Per-frame sequence (`src/main.py:132-202`):
|
||
|
||
1. **Truck detect (every 15th frame)** — `TruckDetector.detect(frame)` →
|
||
`TruckROITracker.update(trucks)` returns smoothed `TruckROI` (or holds last ROI).
|
||
2. **Sync counting line to ROI** — `counter.line_y/x_start/x_end = roi.*`, so the line
|
||
follows the truck instead of staying fixed.
|
||
3. **Batch update (every 15th frame)** — `batch_mgr.update(truck_present, …)`.
|
||
4. **Track → Stabilize → Filter → Count** (only while a batch is active):
|
||
`ByteTrackTracker.update` → `BboxStabilizer.update` →
|
||
`_filter_sacks_in_roi` (keep sacks whose centroid-X is inside ROI) →
|
||
`LineCrossCounter.update` → `CSVLogger.log_event` per crossing.
|
||
5. **Overlay** — `DashboardOverlay.draw(...)` → `cv2.imshow`.
|
||
|
||
## Modules
|
||
|
||
### Streaming (`src/streaming.py`)
|
||
- `VideoFileSource(path)` — offline testing / validation.
|
||
- `RTSPSource(url)` — live camera via FFmpeg backend (`cv2.CAP_FFMPEG`).
|
||
- Common interface: `open() / read() / release()`, `fps`, `frame_size`.
|
||
|
||
### Detection (`src/detection.py`)
|
||
- `SackDetector` — YOLO **segmentation** model; `_parse` keeps only `class_name == "sack"`
|
||
and attaches the mask. (Docstring notes persons are visible to the model but dropped.)
|
||
- `TruckDetector` — YOLO detection model; keeps all classes (single-class `truck` model).
|
||
- Both are single-responsibility; new model types are added as new classes.
|
||
|
||
### Tracking (`src/tracking.py`)
|
||
- `ByteTrackTracker` — `YOLO.track(persist=True, tracker=cfg/tracker.yaml)` (Ultralytics
|
||
FastTrack, occlusion-aware: Kalman rollback on occlusion, enlarged search region, re-ID).
|
||
- `_parse` keeps classes `("sack", "truck")` and attaches `track_id` (`boxes.id`).
|
||
- `reset()` reloads the model weights — called on batch boundaries.
|
||
|
||
### Stabilizer (`src/stabilizer.py`)
|
||
Per-track-ID fixes for worker occlusion / flicker:
|
||
1. **Jitter** (10–50 px jumps) → EMA on bbox (`ema_alpha=0.35`).
|
||
2. **Dropout at the line** (worker blocks sack 5–10 frames) → hold last smoothed bbox
|
||
for `max_hold_frames=10`, with 0.85 confidence decay.
|
||
3. **Height expansion spike** (worker body merges into sack box) → clamp to
|
||
`smooth_h * 1.5`.
|
||
4. **Height shrinkage** (worker covers sack bottom) → clamp to `smooth_h * 0.70`.
|
||
|
||
### Truck ROI (`src/truck_roi.py`)
|
||
- `_pick_main_truck`: largest detection whose center-X falls in the lane
|
||
(`0.35–0.80` of frame width).
|
||
- EMA smoothing (`alpha=0.15`); holds last ROI for 5 missed updates (~3 s), then clears.
|
||
- Counting line = truck **top edge** + `LINE_OFFSET_PX` (= 0).
|
||
|
||
### Counting (`src/counting.py`)
|
||
`LineCrossCounter` tracks the **top edge (`y1`)** of each stabilized sack bbox against a
|
||
zone band `[line_y − margin, line_y + margin]` (default margin 20 px):
|
||
|
||
- Track states per ID: `above` (`y1 < upper`) / `below` (`y1 > lower`) / hold in band.
|
||
- History is latched forever (`has_been_above/below`) so a crossing is caught even if
|
||
the track jumps over the line between low-FPS frames — no exact crossing frame needed.
|
||
- **Loading** = ever-above now below; **Unloading** = ever-below now above.
|
||
- 3-layer dedup: (1) must have been on the opposite side first, (2) spatial radius
|
||
30 px / 3 s window, (3) one count per track-ID per direction.
|
||
- Detections with centroid-X outside `[line_x_start, line_x_end]` are skipped.
|
||
- `reset()` on every new batch.
|
||
|
||
### Batch lifecycle (`src/batch.py`)
|
||
4-state machine (`BatchState`):
|
||
|
||
```
|
||
IDLE ──truck in area──▶ TRUCK_STABILIZING ──stable 5 s──▶ COUNTING_SACKS
|
||
▲ │ truck gone 3 s │ ▲
|
||
│ └────────▶ IDLE │ │ sacks resume
|
||
│ │ │
|
||
│ 10 s no sack activity ▼ │
|
||
│ WAITING_FOR_ACTIVITY
|
||
│ (batch OPEN)
|
||
└────────────────── truck leaves ─────────────────────────────────┘
|
||
(batch finalized → BatchRecord → history)
|
||
```
|
||
|
||
- `update_truck(truck_detected, centroid, ts)` drives IDLE / STABILIZING / WAITING.
|
||
- `update_sacks(crossing_event, sacks_in_area, ts, …)` drives COUNTING / WAITING;
|
||
activity resumes the same batch instead of opening a new one.
|
||
- `update(...)` is a backward-compatible shim combining both.
|
||
- `on_batch_start(id, ts)` / `on_batch_end(BatchRecord)` callbacks — `main.py` uses them
|
||
to reset counter/stabilizer/ROI and to write the CSV row.
|
||
- Tunables: `stabilize_seconds=5`, `stabilize_threshold_px=15`, `sack_idle_timeout=10`,
|
||
`min_batch_duration=30`, `truck_gone_tolerance=3` (constructor defaults).
|
||
Live values from `predict.py`: `stabilize_seconds=0`, `min_batch_duration=5`,
|
||
`sack_idle_timeout` ← `config.yaml batch.timeout_seconds`,
|
||
`truck_gone_tolerance` ← `batch.truck_gone_tolerance_seconds` (both overridden
|
||
by `--batch-timeout` for dev runs).
|
||
|
||
### Dashboard overlay (`src/dashboard.py`)
|
||
Draws onto the frame: truck ROI box + confidence, counting-zone band + center line,
|
||
sack boxes with `track_id` + confidence and a green **top-edge (y1) trigger marker**,
|
||
stats panel (Loading / Unloading / Net / last-3-batch history), and a bottom state bar
|
||
(IDLE / stabilizing + progress / counting + duration-idle / waiting).
|
||
|
||
### Config & logging
|
||
- `src/config_loader.py` — canonical loader: `config.yaml` (stream, models,
|
||
counting, batch, **do**, output, camera) + `.env` (secrets/deployment only) +
|
||
`zones.json` polygons + `cfg/tracker.yaml` reference. Model modes are data
|
||
(`models.modes`); missing file falls back to `.env` + legacy defaults.
|
||
- `src/do_batch.py` — pure helpers for DO-gated batches (mode validation, plate
|
||
grouping, start gates, net/discard both-nets rule, unit classify, retention).
|
||
- `src/do_ocr.py` — `extract_do_fields(image, engine)` for `rapid`|`tesseract`|`paddle`|`none` (default `rapid`)
|
||
(dashboard upload only; never in the predict loop).
|
||
- `src/config.py` — **deprecated** frozen v3 `Config` (different key names);
|
||
do not add keys here.
|
||
- `src/logger.py` — `CSVLogger` appends `batch_summary.csv`
|
||
(`batch_id,start,end,duration,loading,unloading,net`) and `sack_events.csv`
|
||
(`timestamp,batch_id,sack_crossed,T-<id>,direction`). **Wired into
|
||
`predict.py`**: one `log_event` per crossing in both event loops, one
|
||
`log_batch` per kept batch (`finalize_batch` + manual dashboard stop);
|
||
files live in `output.dir` next to `jetson_counter.db`, all calls
|
||
best-effort (warn on failure, never crash the pipeline).
|
||
|
||
## Production pipeline (`predict.py`)
|
||
|
||
Same stage order as v3 but implemented with shapely polygons (`zones.json`:
|
||
pallet/truck/counting), visual-similarity + Re-ID registries, static-frame debounce,
|
||
SQLite persistence, and live-frame publishing to `/dev/shm` for the dashboard. Uses the
|
||
repo `src/` modules (imports at `predict.py`). Zero CLI flags = systemd behaviour
|
||
(`config.yaml` + `.env`); dev flags (`--source`, `--config`, `--model-mode`,
|
||
`--no-dashboard`, …) are additive overrides — see
|
||
`predict.py --help` and `docs/scripts.md`. (`init_db()` and the `http.server` imports
|
||
are dead code — persistence goes through `finalize_batch`/`save_active_batch_state`,
|
||
dashboard integration is file-based.)
|
||
|
||
### Batch modes (three-way)
|
||
|
||
`batch_mode.json` `mode` (default seed `batch.default_mode: auto`):
|
||
|
||
| Mode | Open/close | Notes |
|
||
|---|---|---|
|
||
| `auto` | existing truck FSM | no DO APIs; start/stop manual → 409 |
|
||
| `do_manual` | operator + staged DO photos | DO gates in dashboard start only |
|
||
| `manual` | operator buttons (legacy) | no DO gate |
|
||
|
||
`predict.py` reads mode every frame: `auto` → FSM branch; any other valid value
|
||
→ operator/state-file branch (persists `count`, `box_count`, `box_unloading`,
|
||
**`unloading`**). Finalize discard = **both gross counts 0** (`count == 0` and
|
||
`box_loading == 0`). Stored nets: `net_sack = loading − unloading`,
|
||
`net_box = box_loading − box_unloading`.
|
||
|
||
DO pipeline (dashboard): smartphone upload → OCR draft → edit/stage → start
|
||
gates → active batch → stop-preview soft-warn → stop → `batches` + DO columns.
|
||
ERD: [`../ERD.md`](../ERD.md). UI plan: [`docs/operator-tiles-history-plan.md`](operator-tiles-history-plan.md).
|
||
|
||
## Retired (`archive/`, not imported)
|
||
|
||
- **`predict_new.py`** — duo-model state machine (`WAITING_FOR_TRUCK → COUNTING_SACKS →
|
||
TRUCK_LEAVING`) with hardcoded default zones, perspective filtering and HUD.
|
||
- **`rpo_iki/`** — alternate geometric engine (`count.py`: `LineCounter` with approach
|
||
depth, burst/cooldown dedup, blind-truck overlay) + `predict_rpo_iki.py` runner.
|