feat: add engine auto-recompilation, mortality detection, multi-execution batching, and dashboard updates
- Add engine_utils for TensorRT compatibility verification, auto-recompilation from .pt models, and YAML auto-updates - Add mortality detection pipeline (mortality.py, test_run_mortality.sh, mortality_config.yaml) - Add multi-execution batch modes (parallel_processes, tensor_batching, hybrid) in batch_runner.py - Add daily test run automation scripts and video processing runners - Add dashboard REST API, live stream endpoints, and web UI templates - Clean up git tracking by ignoring __pycache__, .pyc, and build artifacts
This commit is contained in:
proitlab committed
1 parent
c05f3d98d0
commit
010f6e1493
97 files changed
+3361
-643
No files matched your search
@@ -1,11 +1,240 @@
|
||||
# Counter
|
||||
## Pakai virtual env di /media/jetson/DATA/karung-sukawarna/venv
|
||||
# Quick Start Guide
|
||||
|
||||
alias chicken-counter='PYTHONPATH={fullpath git clone folder} /media/jetson/DATA/karung-sukawarna/venv/bin/python -m chicken_counter.cli'
|
||||
This guide gets a new developer up and running from scratch.
|
||||
|
||||
chicken-counter batch --config configs/cycle7_batch.yaml --date 2026-06-10 --no-video --progress-bar
|
||||
---
|
||||
|
||||
# Dashboard
|
||||
## Pakai virtual env di /media/jetson/DATA/karung-sukawarna/venv
|
||||
source /media/jetson/DATA/karung-sukawarna/venv/bin/source
|
||||
python dashboard.py --port 8080
|
||||
## 1. Prerequisites
|
||||
|
||||
- Python 3.10+
|
||||
- `ffmpeg` on PATH (for video compression)
|
||||
- NVIDIA GPU + CUDA drivers (optional but recommended for inference speed)
|
||||
|
||||
---
|
||||
|
||||
## 2. First-Time Setup
|
||||
|
||||
```bash
|
||||
# Clone / copy the project folder, then enter it
|
||||
cd chicken-counting-sukawarna-det
|
||||
|
||||
# Create a virtual environment and install all dependencies
|
||||
python3 -m venv venv
|
||||
venv/bin/pip install --upgrade pip
|
||||
venv/bin/pip install -r requirements.txt
|
||||
```
|
||||
|
||||
> **Note**: If moving the project from another machine, always recreate the venv.
|
||||
> Do NOT copy the `venv/` folder — it contains absolute paths baked in from the source machine.
|
||||
|
||||
---
|
||||
|
||||
## 3. Project Layout
|
||||
|
||||
```text
|
||||
chicken-counting-sukawarna-det/
|
||||
├── configs/
|
||||
│ ├── cycle7_batch_optimized.yaml ← Main batch processing config
|
||||
│ ├── cycle7_batch.yaml ← Alternate batch config
|
||||
│ ├── mortality_config.yaml ← Mortality (carcass) detection config
|
||||
│ ├── cameras/example_camera.yaml ← Single-camera run config template
|
||||
│ └── trackers/botsort_chicken.yaml ← BoT-SORT tracker settings
|
||||
├── db/
|
||||
│ └── chicken_counts.db ← SQLite database (auto-created)
|
||||
├── models/ ← Place your .pt / .onnx / .engine files here
|
||||
│ ├── chicken-detection-model-v26n-300e-best-2026-05-02-NEW.onnx
|
||||
│ └── chicken-detection-model-v26n-seg-300e-best-2026-04-18.pt ← Used by mortality
|
||||
├── src/chicken_counter/ ← Main Python package
|
||||
├── templates/ ← Dashboard HTML
|
||||
├── dashboard.py ← Live API + Dashboard server
|
||||
├── start_dashboard.sh ← Portable dashboard launcher ← USE THIS
|
||||
├── test_run_mortality.sh ← Run mortality detection
|
||||
├── test_run.sh ← Run batch processing all dates
|
||||
└── requirements.txt
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Model Setup
|
||||
|
||||
Put your model files in the `models/` directory.
|
||||
|
||||
| Purpose | File |
|
||||
| :--- | :--- |
|
||||
| Batch video counting | `chicken-detection-model-v26n-300e-best-2026-05-02-NEW.onnx` (speed) or `.pt` (accuracy) |
|
||||
| Mortality detection | `chicken-detection-model-v26n-seg-300e-best-2026-04-18.pt` |
|
||||
|
||||
Update `configs/mortality_config.yaml` and `configs/cycle7_batch_optimized.yaml` if using different filenames.
|
||||
|
||||
---
|
||||
|
||||
## 5. Running the Systems
|
||||
|
||||
### A — Batch Video Processing (Daily Chicken Count)
|
||||
|
||||
Place input videos under the `VIDEOS` folder adjacent to the project:
|
||||
|
||||
```text
|
||||
../VIDEOS/cycle7/kandang-atas/
|
||||
2026-06-18/
|
||||
kandang_1_camera_1_2026-06-18_120056.mp4
|
||||
kandang_1_camera_2_2026-06-18_120456.mp4
|
||||
...
|
||||
```
|
||||
|
||||
Then run:
|
||||
|
||||
```bash
|
||||
# Run all dates (multi-process mode)
|
||||
./test_run.sh
|
||||
|
||||
# Run all dates (Tensor Batching mode)
|
||||
./test_run_tensor_batch.sh
|
||||
|
||||
# Run all dates (Hybrid mode: Threaded CPU + Batched GPU)
|
||||
./test_run_hybrid.sh
|
||||
|
||||
# Run a specific date (4 cameras in parallel)
|
||||
PYTHONPATH=src venv/bin/python -m chicken_counter.cli batch \
|
||||
--config configs/cycle7_batch_optimized.yaml \
|
||||
--date 2026-06-18
|
||||
|
||||
# Run a specific date using Model-Level Tensor Batching
|
||||
PYTHONPATH=src venv/bin/python -m chicken_counter.cli batch \
|
||||
--config configs/cycle7_batch_optimized.yaml \
|
||||
--date 2026-06-18 \
|
||||
--mode tensor_batching
|
||||
|
||||
# Run a specific date using Hybrid Execution Mode
|
||||
PYTHONPATH=src venv/bin/python -m chicken_counter.cli batch \
|
||||
--config configs/cycle7_batch_optimized.yaml \
|
||||
--date 2026-06-18 \
|
||||
--mode hybrid
|
||||
|
||||
# Run with video output enabled
|
||||
./test_run_video_2026-06-18.sh
|
||||
```
|
||||
|
||||
Output is saved in `../VIDEOS/cycle7/kandang-atas/2026-06-18/output/`.
|
||||
|
||||
---
|
||||
|
||||
### B — Mortality Detection (Carcass Photo Scanning)
|
||||
|
||||
Place input photos in `../VIDEOS/cycle7/kandang-atas/mortality/`.
|
||||
|
||||
```bash
|
||||
# Run with default config
|
||||
./test_run_mortality.sh
|
||||
|
||||
# Override confidence threshold
|
||||
CONF=0.65 ./test_run_mortality.sh
|
||||
|
||||
# Run on a specific image
|
||||
./test_run_mortality.sh /path/to/photo.jpg
|
||||
```
|
||||
|
||||
Output annotated images are saved as `output_<original_name>.jpg` in the same directory.
|
||||
A `mortality_report.json` is also saved there with full detection data.
|
||||
|
||||
#### Key config options in `configs/mortality_config.yaml`
|
||||
|
||||
| Setting | Description |
|
||||
| :--- | :--- |
|
||||
| `conf` | Detection confidence threshold (0.0–1.0) |
|
||||
| `iou` | IoU NMS threshold |
|
||||
| `min_box_area_px` | Minimum bounding box area in pixels |
|
||||
| `dedupe_radius_px` | Centroid deduplication radius in pixels |
|
||||
| `two_pass` | Enable 2x Detect & Refine pipeline |
|
||||
| `classes` | `[0]` = chicken only; ignores background/text/equipment |
|
||||
|
||||
---
|
||||
|
||||
### C — Dashboard & API Server
|
||||
|
||||
```bash
|
||||
# Start the dashboard (auto-discovers mortality directory)
|
||||
./start_dashboard.sh
|
||||
|
||||
# Custom port and directories
|
||||
PORT=9090 ./start_dashboard.sh
|
||||
|
||||
# Multiple mortality directories
|
||||
MORTALITY_DIRS="/path/to/mortality1,/path/to/mortality2" ./start_dashboard.sh
|
||||
```
|
||||
|
||||
Open in browser: **http://localhost:8080**
|
||||
|
||||
Available API endpoints:
|
||||
|
||||
| Endpoint | Description |
|
||||
| :--- | :--- |
|
||||
| `GET /api/status` | Live system status, active counting cameras & latest date |
|
||||
| `GET /api/cameras` | Live camera list from `/dev/shm` |
|
||||
| `GET /api/db/summary` | Total chickens, days, hours across all dates |
|
||||
| `GET /api/db/history` | Per-date summary, newest first |
|
||||
| `GET /api/db/date/<YYYY-MM-DD>` | Per-camera breakdown for a specific date |
|
||||
| `GET /api/db/camera/<id>` | History for a specific camera (CC1, CC2...) |
|
||||
| `GET /api/db/location/<name>` | Summary and history for a location |
|
||||
| `GET /api/mortality/latest` | Latest mortality detection report (JSON) |
|
||||
| `GET /api/mortality/history` | All mortality reports, newest first |
|
||||
| `GET /api/mortality/image/<filename>` | Serve annotated output JPEG by filename |
|
||||
| `GET /shm/<cam>/stats.json` | Live pipeline stats for a running camera |
|
||||
| `GET /shm/<cam>/frame.jpg` | Live frame snapshot from a running camera |
|
||||
|
||||
See `API.md` for full response schemas.
|
||||
|
||||
---
|
||||
|
||||
### D — Install as a System Service (Auto-Start)
|
||||
|
||||
```bash
|
||||
# Copy the service file and adjust WorkingDirectory / User if needed
|
||||
sudo cp chicken-dashboard.service /etc/systemd/system/
|
||||
|
||||
# Enable and start
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable chicken-dashboard
|
||||
sudo systemctl start chicken-dashboard
|
||||
|
||||
# Check status
|
||||
sudo systemctl status chicken-dashboard
|
||||
```
|
||||
|
||||
The service reads `start_dashboard.sh`, so it also auto-discovers the mortality directory.
|
||||
|
||||
---
|
||||
|
||||
## 6. Database
|
||||
|
||||
Results are automatically written to `db/chicken_counts.db` when `db_path` is set in the batch YAML. To store results manually from a JSON report:
|
||||
|
||||
```bash
|
||||
PYTHONPATH=src venv/bin/python store_results.py \
|
||||
../VIDEOS/cycle7/kandang-atas/2026-06-18/output/counts_2026-06-18.json \
|
||||
--location kandang-atas \
|
||||
--db db/chicken_counts.db
|
||||
```
|
||||
|
||||
Export to Excel:
|
||||
|
||||
```bash
|
||||
PYTHONPATH=src venv/bin/python export_excel_report.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Moving the Project to Another Machine
|
||||
|
||||
1. Delete the `venv/` folder before copying:
|
||||
```bash
|
||||
rm -rf venv/
|
||||
```
|
||||
2. Copy the entire project folder to the new machine.
|
||||
3. Update the `WorkingDirectory` and `ExecStart` in `chicken-dashboard.service` to the new path.
|
||||
4. Recreate the venv on the new machine:
|
||||
```bash
|
||||
python3 -m venv venv
|
||||
venv/bin/pip install -r requirements.txt
|
||||
```
|
||||
5. All configs and Python code use relative paths and will work without any other changes.
|
||||
Reference in new issue
Block a user