pfm-ocr/screenshots/v2/WORKFLOW.md

Prima Mart Scanner — App Workflow & Screenshot Reference

Captured 2026-07-10 on an Android emulator (Pixel-class, 1080×2400), running the current main branch build (commit ada6488) against the real, live backend stack (Postgres + Next.js gateway + PaddleOCR/vLLM pipeline + DINOv2 product classifier). Login used a real account (WH_JPDMGN1, store "PM PADEMANGAN") created specifically for this capture with zero prior documents, so every screen below — including the OCR extraction, the product classification, the PDF, and the two failure states — is a genuine backend response, not mocked or hand-edited data.

Confidentiality note: the DO photo used (backend/sources/test-images/do-001.jpg) and product photo (foto-kemasan-v2/...) are the client's own real reference images, already committed to the repo as ground-truth/training data (see backend/CLAUDE.md). They're fine for an internal presentation about this same client's own app, but avoid re-publishing them outside that context.

All 32 screenshots referenced below live in this folder (screenshots/v2/).

---

1. App at a glance

• Purpose: in-store app for Prima Fresh Mart store staff (petugas toko) on duty at each store — not delivery drivers. A staff member photographs the DO when a delivery arrives and confirms receipt themselves; each login account is bound to exactly one store (role: store). Two independent scanning modes, switched from the drawer:
  • DO Scan — photograph a paper Delivery Order → OCR extracts header/items → operator reviews/corrects → confirms → PDF receipt.
  • Product Scan — photograph a product package → AI identifies the SKU + expiry date → operator confirms.
• Stack: Flutter client (this repo root) talking to a Next.js API gateway, which fronts a PaddleOCR-VL-1.6/vLLM pipeline (DO documents) and a DINOv2-similarity + YOLO-fallback classifier (products), backed by Postgres.
• Auth: real JWT login (POST /api/v1/auth/login), bcrypt-hashed passwords, token attached as Authorization: Bearer <token> on every /api/v1/* call.
• Offline resilience: every capture goes into a local Hive-backed "pending documents" queue immediately, independent of network state, and is retried automatically; nothing is lost on an app kill mid-upload.

---

2. Screen-by-screen walkthrough

2.1 Splash & Login

Screenshot	What it shows
01-splash-screen.png	Launch splash — logo only, ~1s, while AppConfig.initializeApiBaseUrl() races a LAN probe against the public ngrok fallback to decide which backend URL to use for the rest of the session.
02-login-empty.png	Empty login form: ID dari Admin (username), Kata Sandi (password), Masuk (submit) button. No self-serve signup — copy at the bottom reads "Hubungi admin pusat untuk mendapatkan kredensial akun" (contact central admin for credentials).
02b-login-filled.png	Fields filled with real credentials before submit.
04-login-error-wrong-password.png	Real 401 Unauthorized: Invalid username or password banner — this is the backend's actual bcrypt-compare rejection, surfaced verbatim by lib/core/network/api_exception.dart, not a canned client-side message.
03-location-permission-dialog.png	Real Android OS permission prompt (not app UI) — the app requests precise location immediately after login because every uploaded document is geotagged.

Button/field function:

• Masuk: POST /api/v1/auth/login. On success stores the JWT + profile (username, store name/address) and routes to /camera. On failure shows the red banner in place; no navigation.

2.2 Camera home + navigation drawer

Screenshot	What it shows
04b-camera-home-do-mode.png	DO Scan home. Illustration card with 3 photo-quality tips, then Ambil Foto Kamera (take photo), Pilih Dari Galeri (pick from gallery), Riwayat Dokumen (history) text link.
09-camera-home-product-mode.png	Same layout, Product Scan copy/icon/tips (SKU/expiry visibility, centering, lighting).
05-drawer-menu-do-mode.png	Hamburger menu → drawer. Shows the logged-in store ("Prima Fresh Mart" / "PM PADEMANGAN" — pulled from the JWT profile, not hardcoded), the Mode toggle pill, Home, History, Profile, Notifications, Settings, Help, and Logout.
08-drawer-mode-toggle-product.png	Mode pill after tapping Product Scan — turns solid green; DO Scan turns solid orange when active (see AppConfig.doModeColor — this orange/green pairing is reused everywhere DO vs. Product state is shown: tab switcher, document-card labels).
06-drawer-help-dialog.png	Help opens a static "Tentang Prima Mart" (About) dialog with company boilerplate — no network call.
07-drawer-notimplemented-snackbar.png	Profile, Notifications, Settings are stubbed: each just closes the drawer and shows a "feature coming soon!" snackbar. Real, working items are only Home/History/Mode toggle/Logout.

Button/field function:

• Mode toggle (scanModeProvider, a global Riverpod state): switches both the camera-home copy/icon and which endpoint scan_mode is tagged with on upload (scan_mode: "DO" or "Product" form field). It's the single source of truth also read by the History tab switcher.
• Ambil Foto Kamera: opens the OS camera via image_picker (ImageSource.camera), downscaled to 2048×2048 @ 85% JPEG quality.
• Pilih Dari Galeri: opens the OS photo picker (ImageSource.gallery); if the picked file has EXIF GPS tags, those are used instead of the live GPS fix.
• Riwayat Dokumen / History: both go to /documents.
• Logout: see §2.6.

2.3 Photo confirmation + real blur detection

Screenshot	What it shows
10-image-preview-blur-check.png	Real DO photo just picked, blur-checked live on-device: Ketajaman Foto (sharpness) 75.6 / 100, below the 80 minimum → red "Foto Terdeteksi Blur!" warning. The button row still lets the store staff upload anyway (Ambil Ulang / retake vs. Unggah Dokumen / upload).
21-product-image-preview.png	Product photo, sharper framing → 100.0 / 100, green "Kualitas Foto Baik".

Algorithm — on-device blur detection (lib/features/camera/blur_detector.dart, runs in a background isolate so the UI doesn't jank):

1. Decode the photo, downscale to 300px wide.
2. Crop the center 60% (ignores table/background, focuses on the document/label text).
3. Gaussian-blur radius 1 (denoise sensor/shake noise) → grayscale.
4. Compute a 3×3 discrete Laplacian at every pixel (4×center − left − right − top − bottom) and take its variance across the image — the classic "variance of Laplacian" sharpness metric: a sharp, high-contrast edge produces large Laplacian swings; a blurred photo produces a low, flat variance.
5. Scale that raw variance (0–500 raw → 0–100 shown) and flag isBlur = score < 80.

This never leaves the phone — it's a pure client-side gate, purely advisory (the upload button stays enabled either way).

2.4 DO Scan — real upload → OCR → review → confirm → PDF

Screenshot	What it shows
11-history-pending-upload.png	Immediately after tapping Unggah Dokumen: the document lands in the local pending queue and shows "Sedang memproses dokumen…" with a progress bar, under the History screen's DO Scan tab.
11b-document-processing-failed-timeout.png	Real failure state, captured live: mid-capture the host machine's Wi-Fi dropped, so the app's own poll loop genuinely timed out and surfaced "Gagal Memproses — Koneksi timeout. Silakan coba lagi." — not a simulated error.
12-document-parsed-ready-to-confirm.png	Same document after the network was restored and the OCR pipeline actually finished: "Ketuk untuk dikonfirmasi" (tap to confirm) with a green check.
13-editor-real-ocr-result-top.png / 14-editor-real-ocr-result-items.png	The Tinjau & Edit Data (review & edit) screen, fully populated by the real vLLM/PaddleOCR extraction: Tanggal, No PO/SO/DO, Kepada Yth, Order Untuk, Alamat, Plat Truk all correctly read off the photo, plus a 5-row Barang (items) list where each OCR'd item description was matched against the SKU master table (e.g. "Griller Size 2" → 11310014 — AYAM SIZE 2 FROZEN (0.8-0.9)KG(*)).
14b-editor-simpan-tapped-silent-validation.png / 15-editor-validation-error.png	Real bug/behavior found live: "Nama Driver" (driver name) was blank because the photo only shows a signature, not a printed name. Tapping Simpan & Konfirmasi silently does nothing the first time (Flutter's Form.validate() returns false and the button's onPressed just returns) — only a subsequent screen inspection reveals the actual inline red "Nama Driver harus diisi" error already drawn under the field. This is worth flagging to the team: there's no scroll-to-error or SnackBar telling the user why nothing happened.
16-editor-confirmation-dialog.png / 17-editor-confirmation-dialog-filled.png	After all fields are valid, Simpan & Konfirmasi opens a second, explicit "Konfirmasi Pengiriman" dialog — a receiver name field + a mandatory "Saya setuju…" checkbox. The Konfirmasi button stays disabled/greyed until both are filled. This is the newest feature (last commit, "confirmation-gated documents") — it exists specifically so a document can't be silently marked delivered without someone at the store physically acknowledging it.
18-document-saved-success.png / 19-history-do-tab-populated.png	Real save success: green "Dokumen Berhasil Disimpan!" snackbar, and the History → DO Scan list now shows the confirmed card — receiver name, PO number, date, item count, and a green Terkonfirmasi (confirmed) badge.
20-pdf-preview.png	Tapping the print icon on the card generates a "TANDA TERIMA PENGIRIMAN" (delivery receipt) PDF on-device and opens the real Android print dialog — header, item table, and both signature lines (driver + store receiver name) are populated from the same confirmed data.

Algorithm — upload & poll (lib/features/documents/pending_documents_provider.dart):

1. addDocument() writes a PendingDocument to the Hive-backed local queue immediately (status uploading) — survives an app kill.
2. POST /documents/upload (multipart: image + GPS + scan_mode). On success the item moves to status processing and a poll loop starts.
3. Poll loop: GET /documents/:id every 2 seconds, up to 130 attempts (~4.3 minutes). Each response's parseStatus is interpreted by poll_outcome.dart: "done" → success, "failed" → surfaced error, anything else (including missing/legacy) → keep polling.
4. On app relaunch, any item still processing/uploading in the queue automatically resumes polling/uploading — nothing is silently dropped.
5. document_sync_merge.dart ensures that if a local edit's PUT sync failed, the local corrected copy still wins over a stale server copy on the next history refresh, instead of the correction disappearing.

2.5 Product Scan — real DINOv2 classification + OCR expiry extraction

Screenshot	What it shows
22-history-product-pending-upload.png	Product photo uploading, under History → Product Scan tab.
23-product-parsed-ready-to-confirm.png	Classification finished — ready to review.
24-product-editor-real-result.png	Review Hasil Produk screen: AI Crop Region / Preview (the cropped region the classifier actually scored), AI Product Classification — real result 12010119 — FIESTA NUGGET CHEESE 123 400 GR/PAC at 72.5% confidence (this is the exact SKU of the photo used) — and AI OCR Expiry Extraction — Batch 1: 23/09/2026, tagged "Tanggal terdeteksi otomatis dari OCR" (date auto-detected from OCR), i.e. read directly off the package's printed expiry text, not guessed.
25-product-saved-success.png / 26-history-product-tab-populated.png	Real save success — green "Hasil Pemindaian Produk Berhasil Disimpan!" snackbar, and the History card shows the confirmed SKU name, Batch 1, and the expiry date with a Terkonfirmasi badge.

Algorithm — product classification (backend config/classify_ocr_server.py, see backend/docs/scan-product.md for full detail):

1. Primary: DINOv2 similarity search. A dinov2_vits14 vision transformer embeds the query photo (224² resize, ImageNet normalization) into a 384-dim vector; this is dot-producted (cosine similarity) against a precomputed index of 118 reference photos across 16 known SKU classes; each class's score = the max similarity among its own reference photos. These are not softmax probabilities — they're cosine similarities, typically clustered high (0.4–0.9), so they should be read relatively ("this is clearly the best match") rather than as an absolute confidence.
2. Fallback: YOLO classifier (yolo26n-cls, fine-tuned, 83.3% top-1 val accuracy) only kicks in if the DINOv2 index/model is unavailable — its scores are real softmax probabilities.
3. In parallel, PaddleOCR reads every text line off the package and regex-extracts a candidate SKU number, product name, and expiry date; the Flutter editor only shows a batch-expiry option when a real date was actually found (an empty OCR result means "no date found," not a fabricated placeholder — see the code comment in product_editor_data_logic.dart).
4. The gateway also independently ranks the whole sku_master table by Levenshtein similarity between each nama_item and the classifier's top-1 name, returning the top 5 candidates — this is the dropdown list the operator can override in Pilih Produk / SKU.
5. If the classifier is entirely unreachable, the screen shows an explicit "Gagal Memuat Klasifikasi Produk" retry state rather than fabricating a fake result (a deliberate fix — see G7 in docs/api-contract-map.md).

2.6 Logout (real unsynced-queue warning)

Screenshot	What it shows
27-logout-confirm-dialog.png	Tapping Logout while a document is still sitting in the local pending queue (unsynced) triggers a real, data-driven warning: "Ada Dokumen Belum Tersinkron" — "Ada 1 dokumen…" (the count comes straight from pendingDocumentsProvider's live length), with Batal (cancel) / Ya, Logout (yes, logout anyway). If the queue is empty, logout proceeds immediately with no dialog.
28-logged-out-back-to-login.png	After confirming, the JWT + local Hive cache are cleared and the app routes back to the login screen.

---

3. Real vs. simulated — what this capture proves

Everything above was produced by actually driving the shipped app against the live backend, with the emulator's network genuinely dropping and recovering mid-session (captured as 11b-document-processing-failed-timeout.png) and a genuine client-side validation gap discovered live (14b/15). Nothing was staged by editing local state or mocking a response — every OCR field, every classification score, every PDF value traces back to a real POST/GET against the Postgres-backed gateway.

4. Suggested presentation flow

1. Login (§2.1) → drawer/mode-toggle (§2.2) as the "here's the app" opener.
2. DO Scan end-to-end (§2.4) as the core value story: photo → real OCR → human review/correction → confirmation gate → printable receipt.
3. Product Scan (§2.5) as the second capability: photo → AI SKU ID + auto expiry read.
4. Close on the resilience story: blur gate, offline-safe queue, the real timeout screenshot, and the confirmation-gate/unsynced-logout guardrails — these are the "we thought about failure modes" slides.