Files
feedmill-auto-label/docs/annotation-policy.md
T
asus 5c7c122105 feat: add counting bench, triage, and dataset modules
This commit includes major additions and updates to the frontend and backend architectures, introducing new dataset management, live counting features, batch processing, and triage logic. Includes new UI pages, components, and API routes.
2026-08-14 16:28:52 +07:00

3.4 KiB
Raw Blame History

Annotation policy — sack counting

Why this document exists: the model is not the product, the count is. Every rule below is derived from how algoritma-batch/src/counting.py turns boxes into counts. Change the counter and this policy has to change with it.

The one fact that drives everything

LineCrossCounter counts on y1 — the top edge of the box (counting.py, "Uses y1 (top edge) of the stabilized sack bounding box"). A track is counted when it was seen ABOVE the zone and later BELOW it.

So the top edge of every box you draw is a measurement instrument. Anything that moves y1 for a reason other than the sack moving is a counting error you baked into the dataset.

The rule

Situation Annotate? Box
Fully visible Yes Visible extent
Partly occluded, top edge visible Yes Visible extent
Top edge hidden (head, arm, another sack over the top) No —
Under ~40% visible No —
Fully hidden Never —

If you cannot see where the sack's top edge is, do not annotate it.

Reasoning: a head over the top of a sack drags y1 down 50–100 px. The sack reads as lower than it is and can trip above → below early — a phantom count caused by annotation, not by the model. Leaving it unannotated costs nothing, because cfg/tracker.yaml sets track_buffer: 60 (~2.4 s at 25 fps) specifically to "survive worker occlusion". The tracker coasts through the gap and keeps the ID.

Never annotate a sack you cannot see. Labeling invisible objects teaches the model to hallucinate, which produces ghost tracks and overcounting.

Consistency beats correctness

Either occlusion convention can work. Mixing them cannot. A model trained on "sometimes we box occluded sacks, sometimes not" learns to fire at unstable confidence on ambiguous evidence — which is how you get ID switches (overcount) and dropped tracks (undercount) at the same time. Pick the table above and apply it identically, every session.

Don't let triage delete the hard examples

SAM3's confidence correlates with occlusion: low score usually means partly hidden, not wrong. A blanket "ignore the weakest 10%" rule therefore deletes exactly the occluded examples and trains a model that only knows easy, fully-visible sacks. It will then fail where workers stand — which is where the counting line is.

Use triage to remove boxes that are wrong:

  • slivers and long thin masks (low aspect ratio)
  • duplicates of the same sack
  • background objects that are not sacks
  • boxes so tiny they cannot be a sack at this camera distance

Do not use it to remove boxes that are merely hard. Check the crop grid before saving any ignore rule — if the crops show real sacks, the threshold is too aggressive.

"Half sack" is usually "occluded sack"

A sack that looks half-sized on screen is most often a full sack that is partly hidden, or one further from the camera. Both are normal sacks. Only split a second class if the crop grid shows a genuinely different object, and remember that box area tracks distance from the camera at least as much as it tracks object size.

Before trusting any of this

There is no ground truth yet. Hand-count 2–3 videos per camera and store the numbers, so a retrained model can be measured against something real. Until then, mAP moves are not evidence that the count improved — and the count is the product.