feat(backend): scan-product accuracy 66.2% -> 79.7% + frozen validation benchmark
Accuracy work on the 79-image product-scan validation set (user goal: 90%): - classify_ocr_server.py: 0/90/180/270-degree expiry-date search (stops at first hit, 0-degree fallback); classification decoupled onto the upright image (rotated frames regressed DINOv2 -6pts until this); cross-line date stitching; tiled full-res OCR pass (defeats the 4000px downscale that killed small inkjet dates); VL-pipeline expiry fallback with keyword-anchored anti-hallucination guard; VL text lines merged into text_lines + VL SKU retry. Visualization endpoints removed entirely (Visual/Spotting grids - unused by frontend, 3x per-scan GPU cost). - product-scan.ts: coverage-normalized OCR-evidence re-ranking of DINOv2 top-K (tuned offline: +8/-0 on top-1 misses), re-ranked class mapped to sku_master by SKU prefix; classifier timeout 90s->240s for fallback paths. - Frozen benchmark: product-test-images-fixed/ (79 renamed images) + freeze/seed/build-undetected/capture/experiment scripts; labels trimmed to the 79 validation entries (training rows kept in .bak-with-training); 5 TRAINED-ON SKUs replaced with fresh held-out photos. - manual-label-scan page: shows last batch-test AI prediction under every field by default (new /api/product-scan-results); serves the fixed folder; fixed total hydration failure via allowedDevOrigins 127.0.0.1. - Measured (all-79, zero failures): sku/name 87.3%, expiry 64.6%, overall 79.7%. Tiles/VL-evidence/VL-SKU deployed but not yet batch-measured. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Gr6HH7JrdsXX8AARejQboM
This commit is contained in:
1 parent
19f1facf9b
commit
e76ccb60a6
156 files changed
+17150
-1405
No files matched your search
@@ -0,0 +1,59 @@
|
||||
// One-off capture: hit /api/scan-pfm for every image in
|
||||
// sources/product-test-images-fixed/ and save the full text-level response
|
||||
// (classification all_probabilities, OCR text_lines, extracted fields,
|
||||
// possibleMatches) minus base64 image blobs to
|
||||
// sources/product_scan_fullcap.json. This lets classification re-ranking
|
||||
// experiments run offline against ground truth in seconds instead of
|
||||
// re-running the 11-minute GPU batch per iteration.
|
||||
// Usage: node scripts/capture-scan-responses.mjs [baseUrl]
|
||||
import fs from "fs";
|
||||
import path from "path";
|
||||
|
||||
const BASE_URL = process.argv[2] || process.env.ACCURACY_BASE_URL || "http://127.0.0.1:3000";
|
||||
const FIXED_DIR = path.join("sources", "product-test-images-fixed");
|
||||
const OUT_PATH = path.join("sources", "product_scan_fullcap.json");
|
||||
|
||||
const files = fs.readdirSync(FIXED_DIR)
|
||||
.filter((f) => /\.(jpe?g|png|webp)$/i.test(f))
|
||||
.sort((a, b) => parseInt(a) - parseInt(b));
|
||||
|
||||
const results = [];
|
||||
for (const filename of files) {
|
||||
const b64 = "data:image/jpeg;base64," +
|
||||
fs.readFileSync(path.join(FIXED_DIR, filename), "base64");
|
||||
process.stdout.write(`Capturing ${filename}... `);
|
||||
try {
|
||||
const res = await fetch(`${BASE_URL}/api/scan-pfm`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ image_base64: b64 }),
|
||||
signal: AbortSignal.timeout(120_000)
|
||||
});
|
||||
if (!res.ok) throw new Error(`HTTP ${res.status}`);
|
||||
const d = await res.json();
|
||||
results.push({
|
||||
filename,
|
||||
classification: {
|
||||
top1_name: d.classification?.top1_name,
|
||||
top1_confidence: d.classification?.top1_confidence,
|
||||
method: d.classification?.method,
|
||||
all_probabilities: (d.classification?.all_probabilities || []).slice(0, 15)
|
||||
},
|
||||
ocr: {
|
||||
text_lines: d.ocr?.text_lines || [],
|
||||
extracted_sku: d.ocr?.extracted_sku ?? null,
|
||||
extracted_product_name: d.ocr?.extracted_product_name ?? null,
|
||||
extracted_expired_date: d.ocr?.extracted_expired_date ?? null,
|
||||
expired_source_line: d.ocr?.expired_source_line ?? null
|
||||
},
|
||||
possibleMatches: d.possibleMatches || []
|
||||
});
|
||||
console.log("ok");
|
||||
} catch (err) {
|
||||
console.log(`FAILED (${err.message})`);
|
||||
results.push({ filename, error: err.message });
|
||||
}
|
||||
}
|
||||
|
||||
fs.writeFileSync(OUT_PATH, JSON.stringify(results, null, 2), "utf8");
|
||||
console.log(`\nWrote ${results.length} captures to ${OUT_PATH}`);
|
||||
Reference in new issue
Block a user