andrew/pfm-ocr
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
48334ded8fa8c6988034a7c603ffc373c3c3b82d
pfm-ocr/backend/sources/product-test-images-undetected/manifest.md
T
Rafhan Mazaya FathurrahmanandClaude Fable 5 e76ccb60a6 feat(backend): scan-product accuracy 66.2% -> 79.7% + frozen validation benchmark 
Accuracy work on the 79-image product-scan validation set (user goal: 90%):
- classify_ocr_server.py: 0/90/180/270-degree expiry-date search (stops at
  first hit, 0-degree fallback); classification decoupled onto the upright
  image (rotated frames regressed DINOv2 -6pts until this); cross-line date
  stitching; tiled full-res OCR pass (defeats the 4000px downscale that
  killed small inkjet dates); VL-pipeline expiry fallback with
  keyword-anchored anti-hallucination guard; VL text lines merged into
  text_lines + VL SKU retry. Visualization endpoints removed entirely
  (Visual/Spotting grids - unused by frontend, 3x per-scan GPU cost).
- product-scan.ts: coverage-normalized OCR-evidence re-ranking of DINOv2
  top-K (tuned offline: +8/-0 on top-1 misses), re-ranked class mapped to
  sku_master by SKU prefix; classifier timeout 90s->240s for fallback paths.
- Frozen benchmark: product-test-images-fixed/ (79 renamed images) +
  freeze/seed/build-undetected/capture/experiment scripts; labels trimmed to
  the 79 validation entries (training rows kept in .bak-with-training);
  5 TRAINED-ON SKUs replaced with fresh held-out photos.
- manual-label-scan page: shows last batch-test AI prediction under every
  field by default (new /api/product-scan-results); serves the fixed folder;
  fixed total hydration failure via allowedDevOrigins 127.0.0.1.
- Measured (all-79, zero failures): sku/name 87.3%, expiry 64.6%, overall
  79.7%. Tiles/VL-evidence/VL-SKU deployed but not yet batch-measured.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Gr6HH7JrdsXX8AARejQboM
2026-07-14 19:55:17 +07:00

13 KiB
Raw Blame History

Undetected / mismatched images

Generated 2026-07-14T08:45:37.890Z from the accuracy-check-scan.mts detail dump. 42 of 79 Validation Set images had at least one wrong/missing field or failed to scan.

Image Field OK? Expected Predicted
1 11110059 [FAIL expiry_date].jpeg no_sku ✓ 11110059 11110059
1 11110059 [FAIL expiry_date].jpeg nama_item ✓ CEKER BERKUKU FROZEN PACK 1 KG(*) CEKER BERKUKU FROZEN PACK 1 KG(*)
1 11110059 [FAIL expiry_date].jpeg expiry_date ✗ 13/06/2027 (empty — not detected)
2 11140051 [FAIL expiry_date].jpeg no_sku ✓ 11140051 11140051
2 11140051 [FAIL expiry_date].jpeg nama_item ✓ AMPELA FROZEN PACK 1 KG(*) AMPELA FROZEN PACK 1 KG(*)
2 11140051 [FAIL expiry_date].jpeg expiry_date ✗ 26/11/2026 (empty — not detected)
5 11660050 [FAIL no_sku,nama_item,expiry_date].jpeg no_sku ✗ 11660050 11140051
5 11660050 [FAIL no_sku,nama_item,expiry_date].jpeg nama_item ✗ PAHA BAWAH (1 KG)(*) AMPELA FROZEN PACK 1 KG(*)
5 11660050 [FAIL no_sku,nama_item,expiry_date].jpeg expiry_date ✗ 22/06/2027 (empty — not detected)
6 11710051 [FAIL expiry_date].jpeg no_sku ✓ 11710051 11710051
6 11710051 [FAIL expiry_date].jpeg nama_item ✓ DADA UTUH (1 KG)(*) DADA UTUH (1 KG)(*)
6 11710051 [FAIL expiry_date].jpeg expiry_date ✗ 09/05/2027 (empty — not detected)
7 11818300 [FAIL no_sku,nama_item,expiry_date].jpeg no_sku ✗ 11818300 20120115
7 11818300 [FAIL no_sku,nama_item,expiry_date].jpeg nama_item ✗ CP-BEBEK GORENG 400GR/PAC FIESTA RACIK AYAM GORENG 20 GR/PAC
7 11818300 [FAIL no_sku,nama_item,expiry_date].jpeg expiry_date ✗ 10/01/2027 11/1/01
9 11959937 [FAIL expiry_date].jpeg no_sku ✓ 11959937 11959937
9 11959937 [FAIL expiry_date].jpeg nama_item ✓ SATE AYAM FRESHMART 360 GR (PAC) SATE AYAM FRESHMART 360 GR (PAC)
9 11959937 [FAIL expiry_date].jpeg expiry_date ✗ 29/11/2026 (empty — not detected)
11 12010115 [FAIL expiry_date].jpeg no_sku ✓ 12010115 12010115
11 12010115 [FAIL expiry_date].jpeg nama_item ✓ FIESTA NUGGET ZOO 400 GR/PAC FIESTA NUGGET ZOO 400 GR/PAC
11 12010115 [FAIL expiry_date].jpeg expiry_date ✗ 01/12/2026 00/22/26
15 12010127 [FAIL expiry_date].jpeg no_sku ✓ 12010127 12010127
15 12010127 [FAIL expiry_date].jpeg nama_item ✓ FIESTA SPICY NUGGET 400 GR/PAC FIESTA SPICY NUGGET 400 GR/PAC
15 12010127 [FAIL expiry_date].jpeg expiry_date ✗ 09/12/2027 (empty — not detected)
16 12010509 [FAIL expiry_date].jpeg no_sku ✓ 12010509 12010509
16 12010509 [FAIL expiry_date].jpeg nama_item ✓ CHAMP CRUNCHY NUGGET 450 GR/PAC CHAMP CRUNCHY NUGGET 450 GR/PAC
16 12010509 [FAIL expiry_date].jpeg expiry_date ✗ 15/04/2027 (empty — not detected)
17 12010515 [FAIL no_sku,nama_item].jpeg no_sku ✗ 12010515 12012502
17 12010515 [FAIL no_sku,nama_item].jpeg nama_item ✗ CHAMP KOIN KOMBINASI 450 GR/PAC AKUMO CHICKEN NUGGET 500 GR
17 12010515 [FAIL no_sku,nama_item].jpeg expiry_date ✓ 15/04/2027 15/04/2027
20 12012501 [FAIL no_sku,nama_item].jpeg no_sku ✗ 12012501 12032502
20 12012501 [FAIL no_sku,nama_item].jpeg nama_item ✗ AKUMO CHICKEN NAGET 250 GR AKUMO CHICKEN STIK 500 GR
20 12012501 [FAIL no_sku,nama_item].jpeg expiry_date ✓ 21/05/2027 21/05/2027
21 12012503 [FAIL expiry_date].jpeg no_sku ✓ 12012503 12012503
21 12012503 [FAIL expiry_date].jpeg nama_item ✓ AKUMO CHICKEN NUGGET 1000 GR AKUMO CHICKEN NUGGET 1000 GR
21 12012503 [FAIL expiry_date].jpeg expiry_date ✗ 17/06/2027 (empty — not detected)
22 12012505 [FAIL no_sku,nama_item,expiry_date].jpeg no_sku ✗ 12012505 13010524
22 12012505 [FAIL no_sku,nama_item,expiry_date].jpeg nama_item ✗ AKUMO KOIN 400 GR/PAC CHAMP SSG JUMBO BAKAR 500 GR/PAC (NEW)
22 12012505 [FAIL no_sku,nama_item,expiry_date].jpeg expiry_date ✗ 16/11/2026 (empty — not detected)
25 12030403 [FAIL no_sku,nama_item].jpeg no_sku ✗ 12030403 12010519
25 12030403 [FAIL no_sku,nama_item].jpeg nama_item ✗ GOLDEN FIESTA STIKIE W/ SWEET CHILLI SAUCE 500GR CHAMP NUGGET STICK 900 GR/PAC
25 12030403 [FAIL no_sku,nama_item].jpeg expiry_date ✓ 07/04/2027 07/04/2027
27 12040101 [FAIL no_sku,nama_item].jpeg no_sku ✗ 12040101 12010117
27 12040101 [FAIL no_sku,nama_item].jpeg nama_item ✗ FIESTA SCHNITZEL 400 GR/PAC FIESTA NUGGET HAPPY STAR 400 GR/PAC
27 12040101 [FAIL no_sku,nama_item].jpeg expiry_date ✓ 09/10/2026 09/10/2026
28 12040102 [FAIL expiry_date].jpeg no_sku ✓ 12040102 12040102
28 12040102 [FAIL expiry_date].jpeg nama_item ✓ FIESTA CRISPY BUBBLE KATSU 400 GR/PAC FIESTA CRISPY BUBBLE KATSU 400 GR/PAC
28 12040102 [FAIL expiry_date].jpeg expiry_date ✗ 12/04/2027 (empty — not detected)
31 12080101 [FAIL no_sku,nama_item].jpeg no_sku ✗ 12080101 12010111
31 12080101 [FAIL no_sku,nama_item].jpeg nama_item ✗ FIESTA SPICY CHICK 400 GR/PAC FIESTA CRISPY BUBBLE 400 GR/PAC
31 12080101 [FAIL no_sku,nama_item].jpeg expiry_date ✓ 26/04/2027 26/04/2027
32 12130102 [FAIL expiry_date].jpeg no_sku ✓ 12130102 12130102
32 12130102 [FAIL expiry_date].jpeg nama_item ✓ FIESTA CRISPY BURGER 360 GR (NEW) FIESTA CRISPY BURGER 360 GR (NEW)
32 12130102 [FAIL expiry_date].jpeg expiry_date ✗ 27/04/2027 09/06/2027
36 12240103 [FAIL no_sku,nama_item,expiry_date].jpeg no_sku ✗ 12240103 12010111
36 12240103 [FAIL no_sku,nama_item,expiry_date].jpeg nama_item ✗ FIESTA YAKINIKU R/BITES 400 GR/PAC FIESTA CRISPY BUBBLE 400 GR/PAC
36 12240103 [FAIL no_sku,nama_item,expiry_date].jpeg expiry_date ✗ 07/05/2027 (empty — not detected)
37 13010111 [FAIL expiry_date].jpeg no_sku ✓ 13010111 13010111
37 13010111 [FAIL expiry_date].jpeg nama_item ✓ FIESTA SOSIS BRATWURST 300 GR FIESTA SOSIS BRATWURST 300 GR
37 13010111 [FAIL expiry_date].jpeg expiry_date ✗ 25/03/2027 (empty — not detected)
38 13010116 [FAIL expiry_date].jpeg no_sku ✓ 13010116 13010116
38 13010116 [FAIL expiry_date].jpeg nama_item ✓ FIESTA SSG ORIGINAL 300 GR FIESTA SSG ORIGINAL 300 GR
38 13010116 [FAIL expiry_date].jpeg expiry_date ✗ 23/06/2027 (empty — not detected)
41 13010122 [FAIL no_sku,nama_item,expiry_date].jpeg no_sku ✗ 13010122 17210109
41 13010122 [FAIL no_sku,nama_item,expiry_date].jpeg nama_item ✗ FIESTA RTG SAUSAGE WITH HOT LAVA 60G FIESTA RTS B/BULGOGI 300GR/PAC
41 13010122 [FAIL no_sku,nama_item,expiry_date].jpeg expiry_date ✗ (empty) 20/10/2026
42 13010123 [FAIL expiry_date].jpeg no_sku ✓ 13010123 13010123
42 13010123 [FAIL expiry_date].jpeg nama_item ✓ FIESTA RTG SAUSAGE WITH CHEESE LAVA 60G FIESTA RTG SAUSAGE WITH CHEESE LAVA 60G
42 13010123 [FAIL expiry_date].jpeg expiry_date ✗ 25/10/2026 (empty — not detected)
44 13010518 [FAIL no_sku,nama_item].jpeg no_sku ✗ 13010518 13010524
44 13010518 [FAIL no_sku,nama_item].jpeg nama_item ✗ CHAMP SSG JUMBO BAKAR 500 GR/PAC CHAMP SSG JUMBO BAKAR 500 GR/PAC (NEW)
44 13010518 [FAIL no_sku,nama_item].jpeg expiry_date ✓ (empty) (empty — not detected)
47 13030501 [FAIL no_sku,nama_item].jpeg no_sku ✗ 13030501 1195008A
47 13030501 [FAIL no_sku,nama_item].jpeg nama_item ✗ CHAMP CHICK MEATBALL 200 GR RTC CHICKEN KALASAN 400 GR (PAC)
47 13030501 [FAIL no_sku,nama_item].jpeg expiry_date ✓ (empty) (empty — not detected)
48 13070506 [FAIL expiry_date].jpeg no_sku ✓ 13070506 13070506
48 13070506 [FAIL expiry_date].jpeg nama_item ✓ CHAMP FRANKFURTER SSG 375GR CHAMP FRANKFURTER SSG 375GR
48 13070506 [FAIL expiry_date].jpeg expiry_date ✗ 13/03/2027 (empty — not detected)
50 15010101 [FAIL expiry_date].jpeg no_sku ✓ 15010101 15010101
50 15010101 [FAIL expiry_date].jpeg nama_item ✓ FIESTA SHOESTRING 500 GR FIESTA SHOESTRING 500 GR
50 15010101 [FAIL expiry_date].jpeg expiry_date ✗ 04/06/2027 (empty — not detected)
51 15010102 [FAIL no_sku,nama_item,expiry_date].jpeg no_sku ✗ 15010102 15010107
51 15010102 [FAIL no_sku,nama_item,expiry_date].jpeg nama_item ✗ FIESTA SHOESTRING 1000 GR FIESTA FRENCH F SHOESTRING INSTITUSI 2KG
51 15010102 [FAIL no_sku,nama_item,expiry_date].jpeg expiry_date ✗ 05/03/2027 (empty — not detected)
52 15010107 [FAIL no_sku,nama_item,expiry_date].jpeg no_sku ✗ 15010107 15020101
52 15010107 [FAIL no_sku,nama_item,expiry_date].jpeg nama_item ✗ FIESTA FRENCH F SHOESTRING INSTITUSI 2KG FIESTA STRAIGHT CUT 500 GR
52 15010107 [FAIL no_sku,nama_item,expiry_date].jpeg expiry_date ✗ 17/06/2027 (empty — not detected)
53 15020101 [FAIL no_sku,nama_item].jpeg no_sku ✗ 15020101 12010115
53 15020101 [FAIL no_sku,nama_item].jpeg nama_item ✗ FIESTA STRAIGHT CUT 500 GR FIESTA NUGGET ZOO 400 GR/PAC
53 15020101 [FAIL no_sku,nama_item].jpeg expiry_date ✓ 19/06/2027 19/06/2027
56 15030102 [FAIL no_sku,nama_item].jpeg no_sku ✗ 15030102 20120115
56 15030102 [FAIL no_sku,nama_item].jpeg nama_item ✗ FIESTA CRINKLE CUT 1000 GR FIESTA RACIK AYAM GORENG 20 GR/PAC
56 15030102 [FAIL no_sku,nama_item].jpeg expiry_date ✓ 07/04/2027 07/04/2027
58 16060114 [FAIL expiry_date].jpeg no_sku ✓ 16060114 16060114
58 16060114 [FAIL expiry_date].jpeg nama_item ✓ FIESTA GYOZA 180 GR (NEW) FIESTA GYOZA 180 GR (NEW)
58 16060114 [FAIL expiry_date].jpeg expiry_date ✗ 18/05/2027 (empty — not detected)
60 17210106 [FAIL no_sku,nama_item].jpeg no_sku ✗ 17210106 17200109
60 17210106 [FAIL no_sku,nama_item].jpeg nama_item ✗ FIESTA RTS B/YAKINIKU 300GR/PAC FIESTA RTS C/TERIYAKI 300GR/PAC
60 17210106 [FAIL no_sku,nama_item].jpeg expiry_date ✓ 19/05/2027 19/05/2027
63 17210109 [FAIL expiry_date].jpeg no_sku ✓ 17210109 17210109
63 17210109 [FAIL expiry_date].jpeg nama_item ✓ FIESTA RTS B/BULGOGI 300GR/PAC FIESTA RTS B/BULGOGI 300GR/PAC
63 17210109 [FAIL expiry_date].jpeg expiry_date ✗ 18/05/2027 (empty — not detected)
65 20120102 [FAIL no_sku,nama_item].jpeg no_sku ✗ 20120102 17210109
65 20120102 [FAIL no_sku,nama_item].jpeg nama_item ✗ FIESTA T/B AYAM GORENG 80 GR FIESTA RTS B/BULGOGI 300GR/PAC
65 20120102 [FAIL no_sku,nama_item].jpeg expiry_date ✓ (empty) (empty — not detected)
66 20120105 [FAIL expiry_date].jpeg no_sku ✓ 20120105 20120105
66 20120105 [FAIL expiry_date].jpeg nama_item ✓ FIESTA T/B SERBAGUNA  80 GR FIESTA T/B SERBAGUNA  80 GR
66 20120105 [FAIL expiry_date].jpeg expiry_date ✗ 09/03/2027 (empty — not detected)
68 20120116 [FAIL expiry_date].jpeg no_sku ✓ 20120116 20120116
68 20120116 [FAIL expiry_date].jpeg nama_item ✓ FIESTA RACIK NASI GORENG 20 GR/PAC FIESTA RACIK NASI GORENG 20 GR/PAC
68 20120116 [FAIL expiry_date].jpeg expiry_date ✗ 13/10/2026 (empty — not detected)
69 21000123 [FAIL no_sku,nama_item,expiry_date].jpeg no_sku ✗ 21000123 21000126
69 21000123 [FAIL no_sku,nama_item,expiry_date].jpeg nama_item ✗ FIESTA RICE W/GEPREK CHICKEN 320GR/PAC NEW FIESTA CHICK RENDANG W RICE 320GR (PAC)
69 21000123 [FAIL no_sku,nama_item,expiry_date].jpeg expiry_date ✗ 30/04/2027 (empty — not detected)
71 21000130 [FAIL no_sku,nama_item,expiry_date].jpeg no_sku ✗ 21000130 21000126
71 21000130 [FAIL no_sku,nama_item,expiry_date].jpeg nama_item ✗ NEW FIESTA RICE W/C CHEESE BULDAK 320GR (PAC) NEW FIESTA CHICK RENDANG W RICE 320GR (PAC)
71 21000130 [FAIL no_sku,nama_item,expiry_date].jpeg expiry_date ✗ 09/04/2027 04/05/2026
74 21200107 [FAIL expiry_date].jpeg no_sku ✓ 21200107 21200107
74 21200107 [FAIL expiry_date].jpeg nama_item ✓ NEW FIESTA SPAGHETTI CARBONARA 300GR (PAC) NEW FIESTA SPAGHETTI CARBONARA 300GR (PAC)
74 21200107 [FAIL expiry_date].jpeg expiry_date ✗ 20/05/2027 (empty — not detected)
75 12010801 [FAIL expiry_date].jpeg no_sku ✓ 12010801 12010801
75 12010801 [FAIL expiry_date].jpeg nama_item ✓ OKEY NUGGET 500GR OKEY NUGGET 500GR
75 12010801 [FAIL expiry_date].jpeg expiry_date ✗ 02/12/2027 02/02/2027
76 12012504 [FAIL no_sku,nama_item].jpeg no_sku ✗ 12012504 12012502
76 12012504 [FAIL no_sku,nama_item].jpeg nama_item ✗ AKUMO COIN 200 GR/PAC AKUMO CHICKEN NUGGET 500 GR
76 12012504 [FAIL no_sku,nama_item].jpeg expiry_date ✓ 05/03/2027 05/03/2027
Reference in new issue View Git Blame Copy Permalink