Rafhan Mazaya FathurrahmanandClaude Fable 5
e76ccb60a6
Accuracy work on the 79-image product-scan validation set (user goal: 90%): - classify_ocr_server.py: 0/90/180/270-degree expiry-date search (stops at first hit, 0-degree fallback); classification decoupled onto the upright image (rotated frames regressed DINOv2 -6pts until this); cross-line date stitching; tiled full-res OCR pass (defeats the 4000px downscale that killed small inkjet dates); VL-pipeline expiry fallback with keyword-anchored anti-hallucination guard; VL text lines merged into text_lines + VL SKU retry. Visualization endpoints removed entirely (Visual/Spotting grids - unused by frontend, 3x per-scan GPU cost). - product-scan.ts: coverage-normalized OCR-evidence re-ranking of DINOv2 top-K (tuned offline: +8/-0 on top-1 misses), re-ranked class mapped to sku_master by SKU prefix; classifier timeout 90s->240s for fallback paths. - Frozen benchmark: product-test-images-fixed/ (79 renamed images) + freeze/seed/build-undetected/capture/experiment scripts; labels trimmed to the 79 validation entries (training rows kept in .bak-with-training); 5 TRAINED-ON SKUs replaced with fresh held-out photos. - manual-label-scan page: shows last batch-test AI prediction under every field by default (new /api/product-scan-results); serves the fixed folder; fixed total hydration failure via allowedDevOrigins 127.0.0.1. - Measured (all-79, zero failures): sku/name 87.3%, expiry 64.6%, overall 79.7%. Tiles/VL-evidence/VL-SKU deployed but not yet batch-measured. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Gr6HH7JrdsXX8AARejQboM
60 lines
2.4 KiB
JavaScript
60 lines
2.4 KiB
JavaScript
// One-off capture: hit /api/scan-pfm for every image in
|
|
// sources/product-test-images-fixed/ and save the full text-level response
|
|
// (classification all_probabilities, OCR text_lines, extracted fields,
|
|
// possibleMatches) minus base64 image blobs to
|
|
// sources/product_scan_fullcap.json. This lets classification re-ranking
|
|
// experiments run offline against ground truth in seconds instead of
|
|
// re-running the 11-minute GPU batch per iteration.
|
|
// Usage: node scripts/capture-scan-responses.mjs [baseUrl]
|
|
import fs from "fs";
|
|
import path from "path";
|
|
|
|
const BASE_URL = process.argv[2] || process.env.ACCURACY_BASE_URL || "http://127.0.0.1:3000";
|
|
const FIXED_DIR = path.join("sources", "product-test-images-fixed");
|
|
const OUT_PATH = path.join("sources", "product_scan_fullcap.json");
|
|
|
|
const files = fs.readdirSync(FIXED_DIR)
|
|
.filter((f) => /\.(jpe?g|png|webp)$/i.test(f))
|
|
.sort((a, b) => parseInt(a) - parseInt(b));
|
|
|
|
const results = [];
|
|
for (const filename of files) {
|
|
const b64 = "data:image/jpeg;base64," +
|
|
fs.readFileSync(path.join(FIXED_DIR, filename), "base64");
|
|
process.stdout.write(`Capturing ${filename}... `);
|
|
try {
|
|
const res = await fetch(`${BASE_URL}/api/scan-pfm`, {
|
|
method: "POST",
|
|
headers: { "Content-Type": "application/json" },
|
|
body: JSON.stringify({ image_base64: b64 }),
|
|
signal: AbortSignal.timeout(120_000)
|
|
});
|
|
if (!res.ok) throw new Error(`HTTP ${res.status}`);
|
|
const d = await res.json();
|
|
results.push({
|
|
filename,
|
|
classification: {
|
|
top1_name: d.classification?.top1_name,
|
|
top1_confidence: d.classification?.top1_confidence,
|
|
method: d.classification?.method,
|
|
all_probabilities: (d.classification?.all_probabilities || []).slice(0, 15)
|
|
},
|
|
ocr: {
|
|
text_lines: d.ocr?.text_lines || [],
|
|
extracted_sku: d.ocr?.extracted_sku ?? null,
|
|
extracted_product_name: d.ocr?.extracted_product_name ?? null,
|
|
extracted_expired_date: d.ocr?.extracted_expired_date ?? null,
|
|
expired_source_line: d.ocr?.expired_source_line ?? null
|
|
},
|
|
possibleMatches: d.possibleMatches || []
|
|
});
|
|
console.log("ok");
|
|
} catch (err) {
|
|
console.log(`FAILED (${err.message})`);
|
|
results.push({ filename, error: err.message });
|
|
}
|
|
}
|
|
|
|
fs.writeFileSync(OUT_PATH, JSON.stringify(results, null, 2), "utf8");
|
|
console.log(`\nWrote ${results.length} captures to ${OUT_PATH}`);
|