fhanyuh
caf8e98378
No content changes: git diff --ignore-all-space over these files is empty. The churn came from editing on Windows against a repo checked out with LF.
Product-scan validation images — frozen benchmark set
This is the actual Validation Set the accuracy harness scores
(accuracy-check-scan.mts's getImagePath() points here, not at
../product-test-images/). It exists so re-running the harness always grades
the exact same images — the live-intake folder can keep growing from new
/manual-label-scan drops without silently shifting the benchmark underfoot.
Naming: each file is <index> <no_sku>.<ext> (e.g. 1 11110059.jpeg),
where <index> is just this file's stable position in the set — it carries
no other meaning. product_manual_labels.json's ground-truth entries for
these images use this same filename.
Do not hand-edit this folder. It's fully generated by
node scripts/freeze-validation-set.mjs (run from backend/), which copies
every flat (non-training) entry out of product_manual_labels.json from
../product-test-images/, renames it, and rewrites those entries'
filename fields to match. To add a new SKU/photo to the benchmark:
label it in the live-intake folder first (see that folder's README), then
re-run the freeze script.
79 images as of 2026-07-14. 5 of them (SKUs 12010801, 12012504, 12130504,
13050101, 15040102) are flagged TRAINED-ON in product_manual_labels.json's
notes field — their only available source photo (from an external
research-sam3 segmentation project) was already used to train the
classifier (as a SAM3 crop + augmentations), so they are not a clean
held-out test. Their per-image scores will read as memorization, not real
generalization, until a fresh, never-trained-on photo is dropped for those
SKUs. The other 74 are genuinely held out.