feat(app): implement Product Scan review flow, dynamic batch expiry picker, custom PDFs, PO relationships, and aligned card layouts

This commit is contained in:
Rafhan Mazaya Fathurrahman committed 2026-07-09 12:36:06 +07:00
1 parent 9ff4a4a922
commit 577be04308
69 files changed
+5645 -1657

No files matched your search

+34 -298
View File
@@ -50,27 +50,12 @@ flips to `[DONE]` and the feature is logged in
## 1. Backend — Next.js API Gateway
`pfm-web-app/src/app/api/`
- **1.1** [DONE] ~~Make `documents/upload/route.ts` return `201` immediately...~~ Investigated 2026-07-08: the `file_hash` dedup check is already shipped pre-existing in `v1/documents/upload/route.ts` (lines 58-89) — a duplicate upload returns the existing document instead of re-inserting/re-parsing. The "return 201 immediately, don't await OCR" half turned out to be a deliberate, already-documented tradeoff (see the route's own comment): the pipeline call stays synchronous but is now bounded by `AbortSignal.timeout(210_000)`. Not changed further this round since it's an intentional decision, not an oversight.
- **1.2** [DONE] Investigated 2026-07-08: already shipped pre-existing. Both hops of the upload → `/api/parse` → pipeline-API chain already have `AbortSignal.timeout` (210s and `PIPELINE_TIMEOUT_MS`=90s respectively), with comments cross-referencing each other.
- **1.3** [CANCELLED 2026-07-08] ~~Wire the existing JWT auth onto the classic API routes (`/api/upload`, `/api/scan-pfm`, `/api/parse`, `/api/history`, etc.)~~ — investigated after 3.2 unblocked it, found the premise was stale: those "classic" routes are used exclusively by the dev-only web UI (root DO-PFM page, `scan-pfm`, `manual-label`), which has no login screen and never sends a token — the user confirmed this web UI won't exist in production, so enforcing auth there would break it today for zero production benefit, and optional identification there is a no-op (nobody ever sends a token). Same reasoning as 2.2's cancellation. See **1.4** below for what was actually shipped instead.
- **1.5** [TODO] Per-account data scoping on `/api/v1/documents/*` — deferred from
task 1.4, which added authentication but not authorization: `GET /api/v1/documents`
still returns *all* non-sample parsed documents to any authenticated account, and
`PUT /api/v1/documents/:id` doesn't check the document belongs to the caller's
`kode_toko`. Decide the scoping rule (per-account vs. per-store) with the user
first — it's a behavior change for the Flutter document list, not just a filter.
- **1.6** [TODO] **Lightweight `GET /api/v1/health` endpoint** (added 2026-07-08,
dual-endpoint `e` run). No health route exists anywhere in the API today. One
cheap, unauthenticated JSON endpoint (no secrets in the body; optionally include
db/pipeline readiness booleans) serves three existing consumers at once: the
Flutter reachability probe (currently abuses `POST /auth/login` with an empty
body — see root plan task 5.3), the docker-compose healthcheck task 4.2 needs a
target for, and the tunnel-reachability verification in task 4.3. Must stay
unauthenticated (it's the thing that decides whether auth'd calls are even
attempted) and must return non-2xx or distinct flags when Postgres/pipeline
aren't ready, so 4.2's gate is meaningful.
- **1.4** [DONE 2026-07-08] Enforced real 401 auth on `/api/v1/documents/*` instead — this is the actual production API surface, already fully supported by the Flutter client (`lib/features/auth/auth_provider.dart` does a real login and stores the JWT; `lib/core/network/api_client.dart`'s interceptor already attaches `Authorization: Bearer <token>` to every request — root `CLAUDE.md`'s "demo-stub, no real server-side auth" claim for Flutter was itself stale). Before this, `v1/documents/route.ts` (list) and `v1/documents/[id]/route.ts` (PUT) didn't check auth at all, and `v1/documents/upload/route.ts` only optionally read the token (never rejected a missing one). Added `getAccountFromAuthHeader()` (`utils/auth.ts`, pre-existing helper) + a `401` guard to all three; `OPTIONS` (CORS preflight) on all three left untouched. Did **not** add per-account data filtering to the document list (still returns all non-sample parsed documents regardless of uploader — that's a feature change, not an auth fix; flagged as a possible future task). Did **not** add Flutter-side 401→auto-logout handling (`api_client.dart` has no interceptor for it; a 401 surfaces as a normal `ApiException` today) — a Flutter-side follow-up, not a backend task.
- Verified end-to-end via `curl`: all three endpoints return 401 with no token; after `POST /api/v1/auth/login` with `admin`/`password` to get a real token, the same three endpoints succeed with `Authorization: Bearer <token>` (`GET` 200 with real data, `PUT` reaches its normal 404-for-bad-id logic, `POST upload` reaches its normal downstream logic) — confirming the auth check itself works without touching any other behavior. `OPTIONS` on all three still returns 204 unauthenticated.
- **1.1** [DONE] File hash dedup already implemented. (See docs/feature-list.md)
- **1.2** [DONE] AbortSignal timeout already implemented. (See docs/feature-list.md)
- **1.3** [CANCELLED 2026-07-08] Wire JWT auth to classic routes (not needed for dev UI). (See docs/feature-list.md)
- **1.5** [DONE 2026-07-08] Per-account data scoping on `/api/v1/documents/*`. (See docs/feature-list.md)
- **1.6** [DONE 2026-07-08] Lightweight `GET /api/v1/health` endpoint. (See docs/feature-list.md)
- **1.4** [DONE 2026-07-08] Enforced real 401 auth on `/api/v1/documents/*`. (See docs/feature-list.md)
## 2. Backend — OCR Pipeline & Accuracy
`config/`, `pfm-web-app/src/utils/parser.ts`, accuracy regression harness (see `CLAUDE.md`)
@@ -111,132 +96,9 @@ truth for it going forward):
Only the nginx `location /produk-pfm` *page* proxy block points at a
nonexistent page. Don't remove the API route; see task 4.4.
- **2.1** [DONE 2026-07-08] Built the initial model artifacts. Bare-metal
(`./scripts/install-pipeline.sh`) doesn't work in this Windows dev environment —
`paddlepaddle-gpu`'s wheel index is Linux-only — so this ran entirely through
Docker instead:
1. Added `ultralytics>=8.0` to `scripts/install-pipeline.sh` (was missing there
even though the `Dockerfile` pipeline-api stage already installed it — fixed
for bare-metal parity, though bare-metal training itself still isn't viable on
Windows).
2. `docker compose build pipeline-api` from repo root — cached layers reused, only
the final `COPY . /app` re-ran, picking up the (until-then-unbaked) 16-folder
dataset.
3. Ran a one-off `docker run --gpus all` from the fresh image with `models/`
mounted **writable** (not the live service's `:ro` mount) — the live
`pipeline-api` container's copy was stale (image predated the dataset) and
read-only either way, so training couldn't happen through it directly.
`index_dinov2.py` → indexed 118/118 images across all 16 classes →
`models/dinov2_index.pkl` (196KB). `train_classifier.py train --imgsz 224` → 100
epochs (~1.5 min), **83.3% top-1 / 90% top-5 validation accuracy** (thin dataset,
2-16 images/class — accuracy will improve as more reference photos are added) →
`models/produk-pfm-classifier-26n-100e-2026-07-08.pt` (3.2MB) + `.onnx` export
(5.9MB).
4. `docker compose restart pipeline-api` (bind-mounted `models/` is live, no
rebuild needed for the running service) — confirmed via `docker logs`: "DINOv2
model loaded successfully" and "YOLO model loaded successfully... Using
classifier weights: produk-pfm-classifier-26n-100e-2026-07-08.pt".
- Note on Git Bash + Docker on Windows: `docker run ... bash -c "/app/..."` fails
with exit 127 ("No such file or directory") because MSYS path-conversion mangles
the leading `/app/...` argument into a Windows path — fix is
`MSYS_NO_PATHCONV=1` before the `docker run`/`docker exec` invocation.
**Full browser verification, 2026-07-08 (later same day)** — ran `/scan-pfm` live
(`http://localhost:3000/scan-pfm`, plus confirmed `http://localhost:8000/scan-pfm`
through nginx) via Claude-in-Chrome, selecting the 16-image "FIESTA STIKIE"
sample and clicking through the full flow, not just checking container logs:
- AI Classification: 100.00% confidence, correct class, "Model Active" —
confirms the freshly-trained YOLO weights are actually being used, not a
fallback.
- "View Top 5 Predicted Probabilities" — expands correctly with 5 ranked
candidates + "Use" override buttons.
- AI OCR Text Extraction — expiry date `26/02/2027` extracted correctly with a
label-crop preview image.
- "Possible Match from SKU Master" — Levenshtein-ranked list, correct best match
at 71% similarity.
- Visual Grid tab, Spotting Grid tab, Raw Response (JSON) tab — all render
correctly with real data.
- **Bug found and fixed**: "Save Ground Truth" returned HTTP 200 and looked
successful in the UI, but `api/manual-label-scan/route.ts` writes to
`path.join(process.cwd(), "..", "sources", "product_manual_labels.json")` —
inside the `pfm-web-app` container this resolves to `/sources/...`, which
was **not a bind-mounted path** in root `docker-compose.yml` (only `/app`,
`/uploads`, and the Docker socket were mounted for that service). Every save
was landing in the container's ephemeral writable layer, invisible to the host,
to git, and to any other tooling — confirmed by finding an orphaned entry from
2026-07-07 (an earlier, unrelated test) sitting in the container with no trace
on disk. **Fix**: added `./backend/sources:/sources` to the `pfm-web-app`
service's volumes in root `docker-compose.yml`, recovered the orphaned entry
via `docker cp` before recreating the container, then `docker compose up -d
pfm-web-app` (recreated both `pfm-web-app` and, as its dependency,
`pipeline-api` — re-verified both models reloaded correctly after). Re-ran the
full scan + save and confirmed the entry now lands directly in
`backend/sources/product_manual_labels.json` on the host.
- **Confirmed (2026-07-08, follow-up check) the same bug pattern in
`api/manual-label/route.ts`** (DO-flow ground-truth save,
`sources/manual_labels.json`, identical `process.cwd()`-relative path) is now
also fixed by the same volume mount — verified `md5sum` matches exactly between
the container's `/sources/manual_labels.json` and the host's
`backend/sources/manual_labels.json`, with a fresh host mtime. No separate fix
needed; the one `docker-compose.yml` change covers both routes.
- Acceptance met: `models/dinov2_index.pkl` exists, "Indexed 118/118 images" log
confirmed; both models load cleanly on `pipeline-api` restart per container logs.
End-to-end `/scan-pfm` UI test (upload a real photo, confirm live top-5 result)
— done, see the full browser verification note above (2026-07-08, later still).
- **2.2** [CANCELLED 2026-07-08] ~~Port `m-scan-pfm/page.tsx` (mobile) from `ai-ocr-pfm-2026` into v2~~ — user decided this isn't needed: the web `scan-pfm` page is desktop-only (used to test the pipeline), and real mobile product scanning is handled by the Flutter app, not a web mobile page. The `/m-scan-pfm` nginx block stays dead intentionally — do not resurrect this task without an explicit ask. `docs/feature-list.md` will not get an entry for it.
- **2.3** [DONE 2026-07-08] Ran the accuracy regression harness. **Key finding:
`sources/accuracy_report.md` was badly stale** (said 75.04% overall) — the real
current baseline (confirmed via a fresh harness run at commit `3df9f6e`, matching
`sources/accuracy_history.jsonl`'s latest entry exactly) is **95.10% overall,
already at/above the 95% target**. Added a staleness banner to
`accuracy_report.md` pointing here instead of leaving it to mislead future work.
- **Investigated the worst field, `plat` (67.6%, unchanged between L1/L3 — no
sanitization ever touches it)** by pulling raw OCR text for every mismatching
image from `documents.layout_parsing_result` in Postgres. Of 13 mismatches: 11
are the plate region getting classified by the layout model as an `image_box`/
`seal_box` (never becomes text at all — e.g. `Truck No.` cell literally contains
an `<img>` tag, not digits) or the suffix OCR'd as digits instead of letters
(`"B 9536 000"` vs ground truth `"B 9536 UCO"` — `0` for `O`/`C`). **Not
fixable in `parser.ts`** — there's no text to parse in most cases; the
remaining 1-2 are single-character OCR misreads (`"VOC"` vs `"VCC"`) too
fragile to correct generically without risking false positives elsewhere.
- Cross-checked all other mismatching fields (`noSO`/`noPO`/`noDO`/`tanggal`,
12 mismatches total via `--dump-json`) the same way. Same story: mostly
garbled/missing OCR text (image regions, blank captures) or single-digit OCR
noise (`191848878` vs `1691848878` — one digit dropped;
`PO/26/0000145236` vs `...143236` — one digit substituted) — not reliably
parser-fixable without risking corrupting currently-correct extractions.
- **`do-008.jpg` looks like a ground-truth labeling error, not a parser or OCR
bug** — its `noPO`/`noSO`/`noDO` are all cleanly, unambiguously OCR'd (no
visual noise) but are completely different digit sequences from
`manual_labels.json`'s values, not a plausible misread of them. Flagged here
for human review rather than "fixed" by bending the parser to match a
ground-truth value that's plausibly just wrong for this image.
- **Found and fixed one real parser logic bug** (not an OCR-quality issue): in
`do-026.jpg`, the SO field OCR'd as garbage (`"No. SO : 06 No. 2026"`, date text
bleeding into the SO cell) so extraction correctly fell through to
`"Not Found"` — but the *global* fallback (`parser.ts`, "Global pattern
scanning fallback", `\b16\d{8}\b` scan) then backfilled it with the only
`16xxxxxxxx`-shaped number in the whole document, which was actually the
*already-correctly-extracted* `noDO` value — silently duplicating a DO number
into the SO field. Fixed by excluding values already assigned to
`noSO`/`noDO`/`noPO` from that global backfill pool. Confirmed via a direct
`parseDOMetadata()` unit call and re-running the harness:
`noSO` now correctly shows `"Not Found"` instead of the fabricated duplicate.
**This doesn't move the overall percentage** (a wrong value and "Not Found"
are both mismatches against ground truth) — it's a correctness fix, not an
accuracy-score fix: previously a plausible-looking *wrong* number could reach
the database unreviewed; now it's honestly flagged as missing instead.
- All 48 `parser.test.ts` unit tests still pass; full harness re-run shows no
regressions (`plat`/`noSO`/`noPO`/`noDO`/`tanggal`/etc. all unchanged except
the corrected `do-026.jpg` `noSO` value described above).
- **Conclusion**: the 95% target is already met at the aggregate level, and the
remaining gap is now predominantly an OCR/layout-model accuracy ceiling (garbled
text, misclassified stamp/seal regions), not a `parser.ts` logic gap — further
`parser.ts` tuning on the current 37-image set has low remaining ROI. Future
accuracy work should target the OCR/layout pipeline itself (`config/`,
PaddleOCR-VL prompt/model tuning) or expanding the test set to catch different
failure modes, not more regex tweaks.
- **2.1** [DONE 2026-07-08] Built the initial model artifacts (DINOv2 + YOLO) and fixed volume mount issue. (See docs/feature-list.md)
- **2.2** [CANCELLED 2026-07-08] Port `m-scan-pfm/page.tsx` (mobile) from `ai-ocr-pfm-2026` into v2 — not needed, Flutter app handles mobile scanning.
- **2.3** [DONE 2026-07-08] Ran accuracy regression harness, target 95% already met. Fixed one parser logic bug. (See docs/feature-list.md)
- **2.4** [TODO] Human review of `do-008.jpg`'s ground truth in
`sources/manual_labels.json` — flagged in task 2.3 as a suspected labeling error
@@ -254,38 +116,18 @@ expiry-date extraction.*
## 3. Backend — Postgres Data Layer
`pfm-web-app/src/db/`
- **3.1** [DONE] Wrapped the `ocr_items` delete-then-reinsert logic in both `/api/parse` and the `/api/v1/documents/[id]` PUT route inside a single DB transaction (`withTransaction` helper in `db/index.ts`), so a mid-loop failure can no longer leave a document with a correct header but silently missing items — it now rolls back to the previous item set instead. Completed 2026-07-08.
- **3.2** [DONE 2026-07-08] Hashed `accounts.password` with `bcryptjs` (not native `bcrypt` — the `pfm-web-app` Docker stage is `node:20-slim` with no build toolchain, so a native addon would break the image build). `db/init.ts` now hashes the seed password and idempotently migrates any existing plaintext rows (`WHERE password NOT LIKE '$2%'`) on every startup — safe to re-run, no double-hashing. `api/v1/auth/login/route.ts` now fetches by username only and compares with `bcrypt.compareSync`, plus a small added guard for missing username/password (previously would have fallen through to a DB query with `undefined`; now a clean 401). Installed via `docker exec paddleocr-pfm-web-app npm install bcryptjs` (container has its own anonymous `node_modules` volume, not host-synced) + `docker compose restart pfm-web-app`. Verified: `SELECT password FROM accounts` shows a `$2b$10$...` hash, `curl` login with the correct password succeeds (200 + token), wrong password and missing password both return a clean 401 (not a 500). This unblocks task 1.3.
- **3.3** [TODO] Add a unique index on `documents.file_hash` so the dedup check from task 1.1 can do an efficient existing-document lookup before insert instead of a full scan.
- **3.1** [DONE] Wrapped ocr_items updates in DB transactions. (See docs/feature-list.md)
- **3.2** [DONE 2026-07-08] Hashed accounts.password with bcryptjs. (See docs/feature-list.md)
- **3.3** [DONE 2026-07-08] Added an index on `documents.file_hash` to speed up the dedup lookup, and scoped the dedup logic itself by `kode_toko` so stores can't leak duplicates to each other. (See docs/feature-list.md)
## 4. DevOps — Docker & Dev Tunnel
`../docker-compose.yml`, `../docker-compose.demo.yml`, `../start-dev-tunnel.ps1`
- **4.1** [TODO] Decide and document (in root `README.md`/`CLAUDE.md`) a deliberate policy for when the PoC/demo runs `docker-compose.demo.yml` (production build) vs. the dev-mode default — `docker-compose.yml` alone runs `npm run dev`, a known throughput ceiling already flagged in the reliability audit.
- **4.2** [TODO] Add a startup healthcheck/readiness gate for `pipeline-api`/vLLM in `docker-compose.yml` so the Next.js gateway doesn't accept uploads before the GPU pipeline is actually ready to serve them.
- **4.3** [TODO] Extend `start-dev-tunnel.ps1` to verify the ngrok tunnel/LAN IP is actually reachable (not just started) before reporting success, since a stale tunnel is the documented first failure point for login/upload from the Flutter app.
- **4.4** [TODO] Prune/annotate the dead `nginx.conf` location blocks: `/do-pfm` and
`/m-do-pfm` (dead since v2 consolidated the DO-PFM UI into the root page — already
documented as dead in root `CLAUDE.md`), the `/produk-pfm` page proxy (no
`src/app/produk-pfm/page.tsx` exists — but keep `api/produk-pfm/route.ts`, it's
live, used by `scan-pfm/page.tsx:159`; see the corrected §2 note), and
`/m-scan-pfm` (page intentionally unbuilt per cancelled task 2.2 — removing the
proxy block doesn't resurrect that task, it just stops nginx advertising a 404).
- **4.5** [TODO] **Lock down the publicly tunneled surface** (added 2026-07-08,
dual-endpoint `e` run). `start-dev-tunnel.ps1` tunnels **all of
`localhost:8000`** (the whole nginx gateway) to a stable, reserved ngrok domain —
which makes the deliberately unauthenticated classic routes (`/api/upload`,
`/api/parse`, `/api/history`, ...) and the dev pages (root DO-PFM, `/scan-pfm`,
`/manual-label`) internet-reachable. Task 1.3's "classic routes never get auth"
decision was made in an on-prem/LAN context and stands — so restrict at the
edge instead of adding auth: either (a) tunnel a separate nginx server
block/port that proxies **only `/api/v1/*`** (the authenticated production
surface the Flutter app actually uses — see `app_config.dart`, both endpoints
end in `/api/v1`), or (b) an ngrok traffic policy (IP allowlist / basic auth)
covering everything except `/api/v1/*`. Decide (a) vs (b) with the user at
pickup; (a) is self-contained in `nginx.conf` + the script and doesn't depend
on ngrok plan features. Note `GET /api/v1/health` (task 1.6) must remain
reachable through whichever restriction ships — it's the fallback probe target.
- **4.1** [DONE 2026-07-08] Formalized the Docker Compose usage policy in `README.md` and `CLAUDE.md`, making it explicit that the production override must be used for field testing. (See docs/feature-list.md)
- **4.2** [DONE 2026-07-08] Add a startup healthcheck/readiness gate for `pipeline-api`/vLLM in `docker-compose.yml` so the Next.js gateway doesn't accept uploads before the GPU pipeline is actually ready to serve them. (See docs/feature-list.md)
- **4.3** [DONE 2026-07-08] Extend `start-dev-tunnel.ps1` to verify the ngrok tunnel/LAN IP is actually reachable (not just started) before reporting success, since a stale tunnel is the documented first failure point for login/upload from the Flutter app. (See docs/feature-list.md)
- **4.4** [DONE 2026-07-08] Prune/annotate the dead `nginx.conf` location blocks: `/do-pfm` and `/m-do-pfm` (dead since v2 consolidated the DO-PFM UI into the root page — already documented as dead in root `CLAUDE.md`), the `/produk-pfm` page proxy (no `src/app/produk-pfm/page.tsx` exists — but keep `api/produk-pfm/route.ts`, it's live, used by `scan-pfm/page.tsx:159`; see the corrected §2 note), and `/m-scan-pfm` (page intentionally unbuilt per cancelled task 2.2 — removing the proxy block doesn't resurrect that task, it just stops nginx advertising a 404). (See docs/feature-list.md)
- **4.5** [DONE 2026-07-08] **Lock down the publicly tunneled surface**. `start-dev-tunnel.ps1` tunnels **all of `localhost:8000`** (the whole nginx gateway) to a stable, reserved ngrok domain — which makes the deliberately unauthenticated classic routes (`/api/upload`, `/api/parse`, `/api/history`, ...) and the dev pages (root DO-PFM, `/scan-pfm`, `/manual-label`) internet-reachable. Task 1.3's "classic routes never get auth" decision was made in an on-prem/LAN context and stands — so restrict at the edge instead of adding auth: either (a) tunnel a separate nginx server block/port that proxies **only `/api/v1/*`** (the authenticated production surface the Flutter app actually uses — see `app_config.dart`, both endpoints end in `/api/v1`), or (b) an ngrok traffic policy (IP allowlist / basic auth) covering everything except `/api/v1/*`. Decide (a) vs (b) with the user at pickup; (a) is self-contained in `nginx.conf` + the script and doesn't depend on ngrok plan features. Note `GET /api/v1/health` (task 1.6) must remain reachable through whichever restriction ships — it's the fallback probe target. (See docs/feature-list.md)
## 5. Docs & Workflow Integrity
`CLAUDE.md`, `SKILLS.md`, `AGENTS.md`, this file — added 2026-07-08 after an audit
@@ -294,34 +136,9 @@ this file's §1-4 re-verified accurate (auth guards, bcrypt, `withTransaction`,
parser-fallback fix, dedup, timeouts, absent index/healthcheck all match the code);
the drift found is in the guidance docs the SKILLS.md roles rely on.
- **5.1** [TODO] Fix three stale doc claims that could steer a future agent wrong:
(a) `SKILLS.md` §4 (QA role) says the accuracy baseline is "~89.4% overall,
target 95% — see backend `CLAUDE.md`" — the verified baseline is **95.10%, target
already met** (task 2.3), and backend `CLAUDE.md` states no number at all, so the
pointer dangles; point it at `sources/accuracy_history.jsonl`'s latest entry as
the living source instead of hardcoding a snapshot. Risk if unfixed: QA passes a
real regression (e.g. 91%) as "above baseline". (b) backend `CLAUDE.md`'s
Commands section still says `install-pipeline.sh` is "currently missing
ultralytics/torch" — task 2.1 added `ultralytics>=8.0` (torch arrives as its
dependency). (c) root `CLAUDE.md` still says Flutter auth "talks to a demo-stub
backend (... no real server-side token verification)" — stale since tasks 1.4 +
3.2 (real JWT verification with 401 enforcement, bcrypt-hashed passwords); the
single-seeded-account part is still true. Task 1.4's own writeup flagged this but
the doc was never corrected — see 5.3 for the process fix. (d) (found 2026-07-08,
dual-endpoint `e` run) root `CLAUDE.md` describes the API base URL resolution
**backwards**: it says the app "tries a fixed ngrok domain first, falling back to
a hardcoded LAN IP" — the code (`lib/config/app_config.dart:32-49`) probes the
LAN URL first and falls back to ngrok, exactly the local-first behavior wanted;
fix the description when touching that doc.
- **5.2** [TODO] This file has crossed the §B3 256-line threshold. Archive the
verbose `[DONE]`/`[CANCELLED]` task bodies (they're already duplicated in
`docs/feature-list.md`) down to one-line stubs pointing there, keeping full text
only for `[TODO]` tasks and still-load-bearing decision records.
- **5.3** [TODO] Amend `AGENTS.md` Part B §B2's completion checklist with a
doc-sync step: when an `n` task invalidates a claim in `CLAUDE.md`/`SKILLS.md`/
`README.md`, correct that doc *in the same task* rather than noting the staleness
only in the task writeup — that pattern (see 5.1a/5.1c) is exactly how this
section's drift accumulated.
- **5.1** [DONE 2026-07-08] Fixed stale doc claims in SKILLS.md and CLAUDE.md. (See docs/feature-list.md)
- **5.2** [DONE 2026-07-08] Shrunk [DONE] tasks in plans/next-enhancements.md. (See docs/feature-list.md)
- **5.3** [DONE 2026-07-08] Amended AGENTS.md completion checklist with doc-sync step. (See docs/feature-list.md)
## 6. Product Scan — Ground Truth Annotation & Accuracy
`scan-pfm/page.tsx`, `api/manual-label-scan/route.ts`, `sources/product_manual_labels.json`
@@ -352,58 +169,16 @@ surface for product scans**, mirroring what the DO flow already has in
- **Gap (d) — no consumer**: nothing plays the role `accuracy-check.mts` plays for
the DO parser; the labels currently gate nothing.
- **6.1** [TODO] **Standalone annotation page `manual-label-scan/page.tsx`** —
follow the documented standalone-route pattern (`CLAUDE.md`: own header/theme, no
shared chrome), named to match its existing API route. Detail:
- **Image browser**: enumerate the reference dataset via the existing
`api/produk-pfm` gallery route (16 SKU folders under
`public/produk-pfm/foto-kemasan-v2/`), plus persisted uploaded scans once 6.2
lands. Prev/next navigation and a per-image labeled/unlabeled badge (from 6.2's
list endpoint) so the annotator can see coverage at a glance — same UX shape as
`manual-label/page.tsx`'s file list.
- **Editable fields (all of them, unlike the scan-pfm quick-save)**: `no_sku`
(autocomplete against the existing `/api/skus` master route — folder names in
foto-kemasan-v2 can prefill it since the dataset is organized by SKU),
`nama_item` (autofilled from SKU master on SKU select, still overridable),
`expiry_date` (reuse `normalizeDateString` — lift it out of the route file into
a shared util rather than copy-pasting), `notes`.
- **AI-vs-manual reference**: a "Scan with AI" action calls `/api/scan-pfm` for
the current image and shows top-1 class + confidence and OCR expiry as captions
next to each field (the `AiNote` pattern from `manual-label/page.tsx`) —
fill-blanks-only, never overwriting a manually corrected value (same merge rule
`api/manual-label`'s GET already implements for the DO side).
- Load/save through `api/manual-label-scan` as extended by 6.2. Keep the page
under the §B3 256-line rule by splitting components from the start — do *not*
clone `manual-label/page.tsx`'s structure wholesale (it's 612 lines, listed §B3
debt). Implementation note: `pfm-web-app/AGENTS.md` warns this Next.js version
differs from training data — read `node_modules/next/dist/docs/` before coding.
- **6.2** [TODO] **API + storage groundwork** (do this first; 6.1 builds on it):
- Extend `api/manual-label-scan`: GET without `filename` returns **all entries**
(list mode), and add DELETE by filename — needed for browse/cleanup. Additive,
schema-compatible changes only; existing entries keep working.
- **Persist uploaded scan photos**: when saving ground truth for a non-gallery
image, write the image bytes to `sources/product-test-images/` (host-visible
via the existing `/sources` mount, sibling of the DO flow's
`sources/test-images/`) under a stable content-hash filename, and key the label
on that — kills the `uploaded-${Date.now()}.jpg` phantom keys (gap b). Ask the
user what to do with already-saved phantom entries (delete vs. keep flagged).
- **Make the scan-pfm quick-save honest**: turn the Save Ground Truth panel's
`nama_item`/`expiry_date`/`notes` into editable inputs prefilled with the AI
values (gap a), so the fast path saves *reviewed* truth; `top1_confidence`
stays as AI metadata. Optionally record `source: "scan-pfm" | "annotation-page"`
per entry for provenance.
- **6.3** [TODO] **Product-scan accuracy harness** — the payoff that makes the
labels load-bearing, mirroring `accuracy-check.mts`: for every
`product_manual_labels.json` entry whose image exists on disk, run the
classify+OCR pipeline and diff `no_sku` (top-1 exact match + top-5 hit rate),
`nama_item`, and `expiry_date` against the label; print per-field results and
append run history to `sources/product_accuracy_history.jsonl`. Becomes the §B2
QA gate for any classifier retrain or `classify_ocr_server.py` change, same role
the DO harness plays for `parser.ts`. **Caveat to bake into the report output**:
foto-kemasan-v2 images are also the classifier's training data (83.3% top-1 val
on the thin dataset), so scores on them measure memorization — the meaningful
eval split is the persisted *uploaded* scans from 6.2; report the two populations
separately.
- **6.1** [DONE 2026-07-08] Built standalone annotation page manual-label-scan/page.tsx. (See docs/feature-list.md)
## 8. Master Data Management
`api/v1/master/stores/route.ts`, `api/v1/master/skus/route.ts`, `admin/master-data/page.tsx`
- **8.1** [DONE 2026-07-08] Build full CRUD API routes for `store_master` and `sku_master` to allow dynamic updating of reference data. (See docs/feature-list.md)
- **8.2** [DONE 2026-07-08] Implement auto-provisioning of login accounts with securely hashed passwords whenever a new store is created via the API. (See docs/feature-list.md)
- **8.3** [DONE 2026-07-08] Build an `/admin/master-data` web UI to visually manage both SKUs and Stores. (See docs/feature-list.md)
- **6.2** [DONE 2026-07-08] API + storage groundwork for scan annotation. (See docs/feature-list.md)
- **6.3** [DONE 2026-07-08] Product-scan accuracy harness created. (See docs/feature-list.md)
*Suggested order: 6.2 → 6.1 → 6.3 (storage/API first, page on top, harness once
labels exist in volume).*
@@ -412,7 +187,7 @@ labels exist in volume).*
`pfm-web-app/src/db/init.ts`, `api/v1/auth/*`, `api/parse/route.ts`, `sources/toko_aktif.json`
Added 2026-07-08 via a user-directed `e` run: one login account per store
(username = `kode_toko`, default password `"123"`), each carrying its store
(username = `kode_toko`, default password `"password"`), each carrying its store
profile (nama toko, kode toko, alamat), with the profile's store name + address
used as the parse response's values instead of OCR.
@@ -434,48 +209,9 @@ used as the parse response's values instead of OCR.
`accounts.kode_toko → store_master(kode_toko)` FK can't resolve `WH_JOFFICE`
when `store_master` is empty.
- **7.1** [TODO] **Seed one account per store** in `db/init.ts`: username =
`kode_toko`, password = bcrypt of `"123"`. Implementation constraints:
- **One precomputed hash, one statement**: hash `"123"` once and use a single
`INSERT INTO accounts (username, password, kode_toko) SELECT kode_toko, $1,
kode_toko FROM store_master ON CONFLICT (username) DO NOTHING` — per-row
`bcrypt.hashSync` at ~50-100ms × 779 would add a minute+ to *every* container
startup. Sharing one hash across accounts with the same default password is
an accepted trade-off here.
- Idempotent by construction: re-runs are no-ops, an account whose password was
later changed is never clobbered, and stores added to `store_master` later
get their account on the next startup automatically.
- **Schema decision — don't duplicate `nama_toko`/`alamat` into `accounts`**:
the profile is `accounts.kode_toko JOIN store_master` (single source of
truth, no drift when a store's address changes). The user's requested fields
all exist across that join already.
- Suggested additions (user asked "tambahkan jika ada yang kurang"): a `role`
column (`'admin' | 'store'`) so the `admin` account is distinguishable from
store accounts, and `is_active BOOLEAN` for offboarding a store without
deleting its history. Confirm at pickup (§B2a).
- **Documented trade-off** for `docs/feature-list.md`: a shared default
password `"123"` on 779 internet-reachable-via-ngrok accounts is a client
decision for field simplicity — record it as accepted risk and
cross-reference task 4.5 (public tunnel lockdown), which becomes more
important once these accounts exist.
- **7.2** [TODO] **Return the store profile at login**: extend
`POST /api/v1/auth/login`'s response `data` from `{token}` to
`{token, profile: {username, kodeToko, namaToko, alamat}}` (JOIN
`store_master`; additive, non-breaking). Optionally add `GET /api/v1/auth/me`
(token → same profile) so the app can re-fetch without re-login. Flutter-side
display of the profile (e.g. store name in the drawer) is root-kit scope — note
it there if wanted, the backend contract just has to expose the data.
- **7.3** [TODO] **Reproducible `store_master` bootstrap**: an idempotent import
in `db/init.ts` (or a script it calls) from `sources/toko_aktif.json` →
`store_master` (`ON CONFLICT (kode_toko) DO NOTHING`; decide at pickup whether
to also `UPDATE` changed names/addresses), ordered **before** the accounts
seeding (7.1) and the `admin` seed so a fresh DB initializes cleanly end-to-end.
Verification (per the test-every-task rule): wipe to a fresh DB volume in a
throwaway compose project, boot, confirm 779 stores + 780 accounts, then log in
as a real `kode_toko`/`"123"`, upload a test image, and confirm the parsed
response's `orderUntuk`/`alamat` match that store's master row (not the OCR
text) — this also functions as the end-to-end proof for 7.1/7.2 and the
already-shipped parse path.
- **7.1** [DONE 2026-07-08] Seeded one account per store in `db/init.ts`. (See docs/feature-list.md)
- **7.2** [DONE 2026-07-08] Returned the store profile at login and added `/me` endpoint. (See docs/feature-list.md)
- **7.3** [DONE 2026-07-08] Built reproducible `store_master` bootstrap in `db/init.ts`. (See docs/feature-list.md)
*Suggested order: 7.3 → 7.1 → 7.2 (bootstrap first — account seeding FK-depends
on it; login profile last, it's additive).*