pfm-ocr/screenshots/v2/WORKFLOW.md

# Prima Mart Scanner — App Workflow & Screenshot Reference

Captured 2026-07-10 on an Android emulator (Pixel-class, 1080×2400), running the
**current `main` branch build** (commit `ada6488`) against the **real, live
backend stack** (Postgres + Next.js gateway + PaddleOCR/vLLM pipeline + DINOv2
product classifier). Login used a real account (`WH_JPDMGN1`, store "PM
PADEMANGAN") created specifically for this capture with **zero prior
documents**, so every screen below — including the OCR extraction, the product
classification, the PDF, and the two failure states — is a **genuine backend
response**, not mocked or hand-edited data.

> **Confidentiality note:** the DO photo used (`backend/sources/test-images/do-001.jpg`)
> and product photo (`foto-kemasan-v2/...`) are the client's own real reference
> images, already committed to the repo as ground-truth/training data (see
> `backend/CLAUDE.md`). They're fine for an internal presentation about this
> same client's own app, but avoid re-publishing them outside that context.

All 32 screenshots referenced below live in this folder (`screenshots/v2/`).

---

## 1. App at a glance

- **Purpose**: in-store app for Prima Fresh Mart **store staff (petugas toko)
  on duty at each store** — not delivery drivers. A staff member photographs
  the DO when a delivery arrives and confirms receipt themselves; each login
  account is bound to exactly one store (`role: store`). Two independent
  scanning modes, switched from the drawer:
  - **DO Scan** — photograph a paper Delivery Order → OCR extracts header/items → operator reviews/corrects → confirms → PDF receipt.
  - **Product Scan** — photograph a product package → AI identifies the SKU + expiry date → operator confirms.
- **Stack**: Flutter client (this repo root) talking to a Next.js API gateway,
  which fronts a PaddleOCR-VL-1.6/vLLM pipeline (DO documents) and a
  DINOv2-similarity + YOLO-fallback classifier (products), backed by Postgres.
- **Auth**: real JWT login (`POST /api/v1/auth/login`), bcrypt-hashed
  passwords, token attached as `Authorization: Bearer <token>` on every
  `/api/v1/*` call.
- **Offline resilience**: every capture goes into a local Hive-backed "pending
  documents" queue immediately, independent of network state, and is retried
  automatically; nothing is lost on an app kill mid-upload.

---

## 2. Screen-by-screen walkthrough

### 2.1 Splash & Login

| Screenshot | What it shows |
|---|---|
| `01-splash-screen.png` | Launch splash — logo only, ~1s, while `AppConfig.initializeApiBaseUrl()` races a LAN probe against the public ngrok fallback to decide which backend URL to use for the rest of the session. |
| `02-login-empty.png` | Empty login form: **ID dari Admin** (username), **Kata Sandi** (password), **Masuk** (submit) button. No self-serve signup — copy at the bottom reads "Hubungi admin pusat untuk mendapatkan kredensial akun" (contact central admin for credentials). |
| `02b-login-filled.png` | Fields filled with real credentials before submit. |
| `04-login-error-wrong-password.png` | Real `401 Unauthorized: Invalid username or password` banner — this is the backend's actual bcrypt-compare rejection, surfaced verbatim by `lib/core/network/api_exception.dart`, not a canned client-side message. |
| `03-location-permission-dialog.png` | Real Android OS permission prompt (not app UI) — the app requests **precise location** immediately after login because every uploaded document is geotagged. |

**Button/field function:**
- **Masuk**: `POST /api/v1/auth/login`. On success stores the JWT + profile (username, store name/address) and routes to `/camera`. On failure shows the red banner in place; no navigation.

### 2.2 Camera home + navigation drawer

| Screenshot | What it shows |
|---|---|
| `04b-camera-home-do-mode.png` | DO Scan home. Illustration card with 3 photo-quality tips, then **Ambil Foto Kamera** (take photo), **Pilih Dari Galeri** (pick from gallery), **Riwayat Dokumen** (history) text link. |
| `09-camera-home-product-mode.png` | Same layout, Product Scan copy/icon/tips (SKU/expiry visibility, centering, lighting). |
| `05-drawer-menu-do-mode.png` | Hamburger menu → drawer. Shows the logged-in store ("Prima Fresh Mart" / "PM PADEMANGAN" — pulled from the JWT profile, not hardcoded), the **Mode** toggle pill, **Home**, **History**, **Profile**, **Notifications**, **Settings**, **Help**, and **Logout**. |
| `08-drawer-mode-toggle-product.png` | Mode pill after tapping **Product Scan** — turns solid green; **DO Scan** turns solid orange when active (see `AppConfig.doModeColor` — this orange/green pairing is reused everywhere DO vs. Product state is shown: tab switcher, document-card labels). |
| `06-drawer-help-dialog.png` | **Help** opens a static "Tentang Prima Mart" (About) dialog with company boilerplate — no network call. |
| `07-drawer-notimplemented-snackbar.png` | **Profile**, **Notifications**, **Settings** are stubbed: each just closes the drawer and shows a "feature coming soon!" snackbar. Real, working items are only Home/History/Mode toggle/Logout. |

**Button/field function:**
- **Mode toggle** (`scanModeProvider`, a global Riverpod state): switches both the camera-home copy/icon and which endpoint scan_mode is tagged with on upload (`scan_mode: "DO"` or `"Product"` form field). It's the single source of truth also read by the History tab switcher.
- **Ambil Foto Kamera**: opens the OS camera via `image_picker` (`ImageSource.camera`), downscaled to 2048×2048 @ 85% JPEG quality.
- **Pilih Dari Galeri**: opens the OS photo picker (`ImageSource.gallery`); if the picked file has EXIF GPS tags, those are used instead of the live GPS fix.
- **Riwayat Dokumen** / **History**: both go to `/documents`.
- **Logout**: see §2.6.

### 2.3 Photo confirmation + real blur detection

| Screenshot | What it shows |
|---|---|
| `10-image-preview-blur-check.png` | Real DO photo just picked, blur-checked **live on-device**: Ketajaman Foto (sharpness) **75.6 / 100**, below the 80 minimum → red **"Foto Terdeteksi Blur!"** warning. The button row still lets the store staff upload anyway (**Ambil Ulang** / retake vs. **Unggah Dokumen** / upload). |
| `21-product-image-preview.png` | Product photo, sharper framing → **100.0 / 100**, green **"Kualitas Foto Baik"**. |

**Algorithm — on-device blur detection** (`lib/features/camera/blur_detector.dart`, runs in a background isolate so the UI doesn't jank):
1. Decode the photo, downscale to 300px wide.
2. Crop the center 60% (ignores table/background, focuses on the document/label text).
3. Gaussian-blur radius 1 (denoise sensor/shake noise) → grayscale.
4. Compute a 3×3 discrete Laplacian at every pixel (`4×center − left − right − top − bottom`) and take its **variance across the image** — the classic "variance of Laplacian" sharpness metric: a sharp, high-contrast edge produces large Laplacian swings; a blurred photo produces a low, flat variance.
5. Scale that raw variance (0–500 raw → 0–100 shown) and flag `isBlur = score < 80`.

This never leaves the phone — it's a pure client-side gate, purely advisory (the upload button stays enabled either way).

### 2.4 DO Scan — real upload → OCR → review → confirm → PDF

| Screenshot | What it shows |
|---|---|
| `11-history-pending-upload.png` | Immediately after tapping **Unggah Dokumen**: the document lands in the local pending queue and shows **"Sedang memproses dokumen…"** with a progress bar, under the History screen's **DO Scan** tab. |
| `11b-document-processing-failed-timeout.png` | **Real failure state**, captured live: mid-capture the host machine's Wi-Fi dropped, so the app's own poll loop genuinely timed out and surfaced **"Gagal Memproses — Koneksi timeout. Silakan coba lagi."** — not a simulated error. |
| `12-document-parsed-ready-to-confirm.png` | Same document after the network was restored and the OCR pipeline actually finished: **"Ketuk untuk dikonfirmasi"** (tap to confirm) with a green check. |
| `13-editor-real-ocr-result-top.png` / `14-editor-real-ocr-result-items.png` | The **Tinjau & Edit Data** (review & edit) screen, fully populated by the real vLLM/PaddleOCR extraction: Tanggal, No PO/SO/DO, Kepada Yth, Order Untuk, Alamat, Plat Truk all correctly read off the photo, plus a 5-row **Barang** (items) list where each OCR'd item description was matched against the SKU master table (e.g. "Griller Size 2" → `11310014 — AYAM SIZE 2 FROZEN (0.8-0.9)KG(*)`). |
| `14b-editor-simpan-tapped-silent-validation.png` / `15-editor-validation-error.png` | **Real bug/behavior found live**: "Nama Driver" (driver name) was blank because the photo only shows a signature, not a printed name. Tapping **Simpan & Konfirmasi** silently does nothing the first time (Flutter's `Form.validate()` returns false and the button's `onPressed` just returns) — only a subsequent screen inspection reveals the actual inline red "Nama Driver harus diisi" error already drawn under the field. This is worth flagging to the team: there's no scroll-to-error or SnackBar telling the user *why* nothing happened. |
| `16-editor-confirmation-dialog.png` / `17-editor-confirmation-dialog-filled.png` | After all fields are valid, **Simpan & Konfirmasi** opens a second, explicit **"Konfirmasi Pengiriman"** dialog — a receiver name field + a mandatory "Saya setuju…" checkbox. The **Konfirmasi** button stays disabled/greyed until both are filled. This is the newest feature (last commit, "confirmation-gated documents") — it exists specifically so a document can't be silently marked delivered without someone at the store physically acknowledging it. |
| `18-document-saved-success.png` / `19-history-do-tab-populated.png` | Real save success: green **"Dokumen Berhasil Disimpan!"** snackbar, and the History → DO Scan list now shows the confirmed card — receiver name, PO number, date, item count, and a green **Terkonfirmasi** (confirmed) badge. |
| `20-pdf-preview.png` | Tapping the print icon on the card generates a **"TANDA TERIMA PENGIRIMAN"** (delivery receipt) PDF on-device and opens the real Android print dialog — header, item table, and both signature lines (driver + store receiver name) are populated from the same confirmed data. |

**Algorithm — upload & poll** (`lib/features/documents/pending_documents_provider.dart`):
1. `addDocument()` writes a `PendingDocument` to the Hive-backed local queue immediately (status `uploading`) — survives an app kill.
2. `POST /documents/upload` (multipart: image + GPS + `scan_mode`). On success the item moves to status `processing` and a poll loop starts.
3. Poll loop: `GET /documents/:id` every **2 seconds**, up to **130 attempts (~4.3 minutes)**. Each response's `parseStatus` is interpreted by `poll_outcome.dart`: `"done"` → success, `"failed"` → surfaced error, anything else (including missing/legacy) → keep polling.
4. On app relaunch, any item still `processing`/`uploading` in the queue automatically resumes polling/uploading — nothing is silently dropped.
5. `document_sync_merge.dart` ensures that if a local edit's `PUT` sync failed, the local corrected copy still wins over a stale server copy on the next history refresh, instead of the correction disappearing.

### 2.5 Product Scan — real DINOv2 classification + OCR expiry extraction

| Screenshot | What it shows |
|---|---|
| `22-history-product-pending-upload.png` | Product photo uploading, under History → **Product Scan** tab. |
| `23-product-parsed-ready-to-confirm.png` | Classification finished — ready to review. |
| `24-product-editor-real-result.png` | **Review Hasil Produk** screen: **AI Crop Region / Preview** (the cropped region the classifier actually scored), **AI Product Classification** — real result `12010119 — FIESTA NUGGET CHEESE 123 400 GR/PAC` at **72.5% confidence** (this is the exact SKU of the photo used) — and **AI OCR Expiry Extraction** — `Batch 1: 23/09/2026`, tagged **"Tanggal terdeteksi otomatis dari OCR"** (date auto-detected from OCR), i.e. read directly off the package's printed expiry text, not guessed. |
| `25-product-saved-success.png` / `26-history-product-tab-populated.png` | Real save success — green **"Hasil Pemindaian Produk Berhasil Disimpan!"** snackbar, and the History card shows the confirmed SKU name, `Batch 1`, and the expiry date with a **Terkonfirmasi** badge. |

**Algorithm — product classification** (backend `config/classify_ocr_server.py`, see `backend/docs/scan-product.md` for full detail):
1. **Primary: DINOv2 similarity search.** A `dinov2_vits14` vision transformer embeds the query photo (224² resize, ImageNet normalization) into a 384-dim vector; this is dot-producted (cosine similarity) against a precomputed index of 118 reference photos across 16 known SKU classes; each class's score = the **max** similarity among its own reference photos. These are *not* softmax probabilities — they're cosine similarities, typically clustered high (0.4–0.9), so they should be read relatively ("this is clearly the best match") rather than as an absolute confidence.
2. **Fallback: YOLO classifier** (`yolo26n-cls`, fine-tuned, 83.3% top-1 val accuracy) only kicks in if the DINOv2 index/model is unavailable — its scores *are* real softmax probabilities.
3. **In parallel**, PaddleOCR reads every text line off the package and regex-extracts a candidate SKU number, product name, and expiry date; the Flutter editor only shows a batch-expiry option when a real date was actually found (an empty OCR result means "no date found," not a fabricated placeholder — see the code comment in `product_editor_data_logic.dart`).
4. The gateway also independently ranks the whole `sku_master` table by Levenshtein similarity between each `nama_item` and the classifier's top-1 name, returning the top 5 candidates — this is the dropdown list the operator can override in **Pilih Produk / SKU**.
5. If the classifier is entirely unreachable, the screen shows an explicit **"Gagal Memuat Klasifikasi Produk"** retry state rather than fabricating a fake result (a deliberate fix — see G7 in `docs/api-contract-map.md`).

### 2.6 Logout (real unsynced-queue warning)

| Screenshot | What it shows |
|---|---|
| `27-logout-confirm-dialog.png` | Tapping **Logout** while a document is still sitting in the local pending queue (unsynced) triggers a real, data-driven warning: **"Ada Dokumen Belum Tersinkron"** — "Ada 1 dokumen…" (the count comes straight from `pendingDocumentsProvider`'s live length), with **Batal** (cancel) / **Ya, Logout** (yes, logout anyway). If the queue is empty, logout proceeds immediately with no dialog. |
| `28-logged-out-back-to-login.png` | After confirming, the JWT + local Hive cache are cleared and the app routes back to the login screen. |

---

## 3. Real vs. simulated — what this capture proves

Everything above was produced by actually driving the shipped app against the
live backend, with the emulator's network genuinely dropping and recovering
mid-session (captured as `11b-document-processing-failed-timeout.png`) and a
genuine client-side validation gap discovered live (`14b`/`15`). Nothing was
staged by editing local state or mocking a response — every OCR field, every
classification score, every PDF value traces back to a real POST/GET against
the Postgres-backed gateway.

## 4. Suggested presentation flow

1. Login (§2.1) → drawer/mode-toggle (§2.2) as the "here's the app" opener.
2. DO Scan end-to-end (§2.4) as the core value story: photo → real OCR → human review/correction → confirmation gate → printable receipt.
3. Product Scan (§2.5) as the second capability: photo → AI SKU ID + auto expiry read.
4. Close on the resilience story: blur gate, offline-safe queue, the real timeout screenshot, and the confirmation-gate/unsynced-logout guardrails — these are the "we thought about failure modes" slides.