karung-counting-feedmill-semarang/docs/architecture.md

Architecture

Pipeline overview (v3, src/main.py)

Frame → TruckDetect → ROI → SackTrack → Stabilize → Filter → Count → Overlay/CSV

run() processes every frame (src/main.py:111). Truck detection is the exception: it runs every 15 frames (TRUCK_DET_INTERVAL, src/main.py:130) because the truck moves slowly and that model is heavy. All components are wired by dependency injection in build_pipeline() (src/main.py:27) and communicate through the DTOs/protocols in src/interfaces.py (Detection, FrameResult, StreamSource, Detector, Tracker, Counter, BatchManager).

Per-frame sequence (src/main.py:132-202):

1. Truck detect (every 15th frame) — TruckDetector.detect(frame) → TruckROITracker.update(trucks) returns smoothed TruckROI (or holds last ROI).
2. Sync counting line to ROI — counter.line_y/x_start/x_end = roi.*, so the line follows the truck instead of staying fixed.
3. Batch update (every 15th frame) — batch_mgr.update(truck_present, …).
4. Track → Stabilize → Filter → Count (only while a batch is active): ByteTrackTracker.update → BboxStabilizer.update → _filter_sacks_in_roi (keep sacks whose centroid-X is inside ROI) → LineCrossCounter.update → CSVLogger.log_event per crossing.
5. Overlay — DashboardOverlay.draw(...) → cv2.imshow.

Modules

Streaming (src/streaming.py)

• VideoFileSource(path) — offline testing / validation.
• RTSPSource(url) — live camera via FFmpeg backend (cv2.CAP_FFMPEG).
• Common interface: open() / read() / release(), fps, frame_size.

Detection (src/detection.py)

• SackDetector — YOLO segmentation model; _parse keeps only class_name == "sack" and attaches the mask. (Docstring notes persons are visible to the model but dropped.)
• TruckDetector — YOLO detection model; keeps all classes (single-class truck model).
• Both are single-responsibility; new model types are added as new classes.

Tracking (src/tracking.py)

• ByteTrackTracker — YOLO.track(persist=True, tracker=cfg/tracker.yaml) (Ultralytics FastTrack, occlusion-aware: Kalman rollback on occlusion, enlarged search region, re-ID).
• _parse keeps classes ("sack", "truck") and attaches track_id (boxes.id).
• reset() reloads the model weights — called on batch boundaries.

Stabilizer (src/stabilizer.py)

Per-track-ID fixes for worker occlusion / flicker:

1. Jitter (10–50 px jumps) → EMA on bbox (ema_alpha=0.35).
2. Dropout at the line (worker blocks sack 5–10 frames) → hold last smoothed bbox for max_hold_frames=10, with 0.85 confidence decay.
3. Height expansion spike (worker body merges into sack box) → clamp to smooth_h * 1.5.
4. Height shrinkage (worker covers sack bottom) → clamp to smooth_h * 0.70.

Truck ROI (src/truck_roi.py)

• _pick_main_truck: largest detection whose center-X falls in the lane (0.35–0.80 of frame width).
• EMA smoothing (alpha=0.15); holds last ROI for 5 missed updates (~3 s), then clears.
• Counting line = truck top edge + LINE_OFFSET_PX (= 0).

Counting (src/counting.py)

LineCrossCounter tracks the top edge (y1) of each stabilized sack bbox against a zone band [line_y − margin, line_y + margin] (default margin 20 px):

• Track states per ID: above (y1 < upper) / below (y1 > lower) / hold in band.
• History is latched forever (has_been_above/below) so a crossing is caught even if the track jumps over the line between low-FPS frames — no exact crossing frame needed.
• Loading = ever-above now below; Unloading = ever-below now above.
• 3-layer dedup: (1) must have been on the opposite side first, (2) spatial radius 30 px / 3 s window, (3) one count per track-ID per direction.
• Detections with centroid-X outside [line_x_start, line_x_end] are skipped.
• reset() on every new batch.

Batch lifecycle (src/batch.py)

4-state machine (BatchState):

IDLE ──truck in area──▶ TRUCK_STABILIZING ──stable 5 s──▶ COUNTING_SACKS
 ▲                          │ truck gone 3 s                    │  ▲
 │                          └────────▶ IDLE                     │  │ sacks resume
 │                                                             │  │
 │                              10 s no sack activity          ▼  │
 │                                                     WAITING_FOR_ACTIVITY
 │                                                        (batch OPEN)
 └────────────────── truck leaves ─────────────────────────────────┘
                    (batch finalized → BatchRecord → history)

• update_truck(truck_detected, centroid, ts) drives IDLE / STABILIZING / WAITING.
• update_sacks(crossing_event, sacks_in_area, ts, …) drives COUNTING / WAITING; activity resumes the same batch instead of opening a new one.
• update(...) is a backward-compatible shim combining both.
• on_batch_start(id, ts) / on_batch_end(BatchRecord) callbacks — main.py uses them to reset counter/stabilizer/ROI and to write the CSV row.
• Tunables: stabilize_seconds=5, stabilize_threshold_px=15, sack_idle_timeout=10, min_batch_duration=30, truck_gone_tolerance=3.

Dashboard overlay (src/dashboard.py)

Draws onto the frame: truck ROI box + confidence, counting-zone band + center line, sack boxes with track_id + confidence and a green top-edge (y1) trigger marker, stats panel (Loading / Unloading / Net / last-3-batch history), and a bottom state bar (IDLE / stabilizing + progress / counting + duration-idle / waiting).

Config & logging

• src/config_loader.py — canonical loader: config.yaml (stream, models, counting, batch, do, output, camera) + .env (secrets/deployment only) + zones.json polygons + cfg/tracker.yaml reference. Model modes are data (models.modes); missing file falls back to .env + legacy defaults.
• src/do_batch.py — pure helpers for DO-gated batches (mode validation, plate grouping, start gates, net/discard both-nets rule, unit classify, retention).
• src/do_ocr.py — extract_do_fields(image, engine) for rapid|tesseract|paddle|none (default rapid) (dashboard upload only; never in the predict loop).
• src/config.py — deprecated frozen v3 Config (different key names); do not add keys here.
• src/logger.py — CSVLogger appends batch_summary.csv (batch_id,start,end,duration,loading,unloading,net) and sack_events.csv (timestamp,batch_id,sack_crossed,T-<id>,direction).

Production pipeline (predict.py)

Same stage order as v3 but implemented with shapely polygons (zones.json: pallet/truck/counting), visual-similarity + Re-ID registries, static-frame debounce, SQLite persistence, and live-frame publishing to /dev/shm for the dashboard. Uses the repo src/ modules (imports at predict.py). Zero CLI flags = systemd behaviour (config.yaml + .env); dev flags (--source, --config, --model-mode, --no-dashboard, …) are additive overrides — see predict.py --help and docs/scripts.md. (init_db() and the http.server imports are dead code — persistence goes through finalize_batch/save_active_batch_state, dashboard integration is file-based.)

Batch modes (three-way)

batch_mode.json mode (default seed batch.default_mode: auto):

Mode	Open/close	Notes
auto	existing truck FSM	no DO APIs; start/stop manual → 409
do_manual	operator + staged DO photos	DO gates in dashboard start only
manual	operator buttons (legacy)	no DO gate

predict.py reads mode every frame: auto → FSM branch; any other valid value → operator/state-file branch (persists count, box_count, box_unloading, unloading). Finalize discard = both nets 0 (net_sack = loading − unloading, net_box = box_loading − box_unloading).

DO pipeline (dashboard): smartphone upload → OCR draft → edit/stage → start gates → active batch → stop-preview soft-warn → stop → batches + DO columns. ERD: ../ERD.md. UI plan: docs/operator-tiles-history-plan.md.

Retired (archive/, not imported)

• predict_new.py — duo-model state machine (WAITING_FOR_TRUCK → COUNTING_SACKS → TRUCK_LEAVING) with hardcoded default zones, perspective filtering and HUD.
• rpo_iki/ — alternate geometric engine (count.py: LineCounter with approach depth, burst/cooldown dedup, blind-truck overlay) + predict_rpo_iki.py runner.