fix(backend): group augmented images with source photo in train/val split; document classifier retrain effort (task 2.5)

train_classifier.py's split_dataset() previously shuffled and split
individual image files, letting an augmented copy (photo_aug_2.jpeg) land
in validation while its near-duplicate source stayed in training -
inflating val accuracy with memorization rather than measuring real
generalization. Now groups by source photo (stripping _aug_N) before
shuffling and splitting 80/20.

Also records the in-progress effort to retrain the product classifier
against the full 81-class/2,493-photo foto-kemasan-v2 dataset (up from the
16 classes/118 photos the deployed model was actually trained on) - see
plans/next-enhancements.md task 2.5 and the accompanying iteration-log
entry for the real, currently-observed numbers (DINOv2 index rebuilt:
2493/2493 images; classifier training: in progress, ~32s/epoch observed).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xsxk4ZkDQVVaLUcixDcqb5

backend/plans/next-enhancements.md

@@ -113,6 +113,62 @@ in either project (only SKU, product name, expiry date are extracted) — if
 requested later, follow the same OCR-regex-cascade pattern already used for
 expiry-date extraction.*
 
+- **2.5** [IN PROGRESS 2026-07-14 — resumed, training run 2] **Retrain
+  classifier on the now-81-class dataset.** (Note: a first resume attempt
+  failed instantly with a Docker daemon connection error — Docker Desktop had
+  stopped between sessions — before any training happened; restarted Docker
+  Desktop and relaunched. This is the actual second training attempt,
+  confirmed running via `docker ps`.) `foto-kemasan-v2/` grew from the 16
+  classes/118 photos the deployed model
+  (`produk-pfm-classifier-26n-100e-2026-07-08.pt`) was trained on to **81
+  classes / 2,493 photos** — the other 65 classes were never included in any
+  training run.
+  - **Goal**: retrain both artifacts (`dinov2_index.pkl` similarity index and the
+    YOLO classifier) against the full current dataset so the deployed model
+    actually recognizes all 81 SKU folders, not just the original 16.
+  - **Agreed procedure** (per `docs/scan-product.md`'s documented retraining
+    steps — training must run via Docker, not bare-metal Windows, since
+    `paddlepaddle-gpu` wheels are Linux-only): from repo root,
+    `docker compose build pipeline-api` (bakes in the current dataset) → one-off
+    `docker run --gpus all` with `models/` mounted **writable** (the live
+    compose service mounts it `:ro`) → `index_dinov2.py` (rebuilds the DINOv2
+    index) → `train_classifier.py train --imgsz 224` (its own `split_dataset()`
+    does an 80/20 split grouped by source photo and shuffled — not a naive
+    first-N-files split, so augmented copies always land with their source) →
+    `docker compose restart pipeline-api` → verify via `docker logs` for
+    "DINOv2 index loaded with N reference images" and "Using classifier
+    weights: <new dated file>".
+  - **Status as of pause (2026-07-14)** — mixed state, read carefully before
+    resuming:
+    - ✅ `pipeline-api` image built (2m54s), bakes in the current 81-class
+      dataset.
+    - ✅ **`dinov2_index.pkl` already rebuilt and persisted to disk** —
+      "Success! Indexed 2493/2493 images" across all 81 classes. This artifact
+      is live on the host now (`models/dinov2_index.pkl`, 4.2MB, dated
+      2026-07-14) and does **not** need to be redone.
+    - ⏸️ **YOLO classifier training was started, then stopped by user request
+      at epoch 43/100 (~23 minutes in)** before it could write a new dated
+      checkpoint. `docker run` used `--rm` and the in-progress epoch
+      checkpoints live only in the container's own `runs/classify/` (not
+      bind-mounted), so **stopping the container discarded that partial
+      progress** — resuming means restarting from epoch 0, not continuing from
+      43. `models/` on the host still has only the original
+      `produk-pfm-classifier-26n-100e-2026-07-08.pt`/`.onnx` (16-class model) —
+      **the live/deployed classifier is unchanged**, still 16 classes.
+    - Observed pace before stopping: ~32s/epoch (43 epochs in 23m1s) → a full
+      100-epoch run should take **~55 minutes** on this host's RTX 2060 (6GB
+      VRAM), not the ~90 min extrapolated from the first few (slower, warmup)
+      epochs. At epoch 42 the in-progress run had already reached 84.3%
+      top-1 / 93.9% top-5 val accuracy across all 81 classes, ahead of the old
+      16-class model's 83.3%/90% — a promising sign for the eventual full run,
+      but not a final result since training didn't finish.
+  - **To resume**: image is already built and the DINOv2 index step can be
+    skipped — just re-run the one-off `train_classifier.py train --imgsz 224`
+    container, then `docker compose restart pipeline-api` and verify via
+    `docker logs`. Update the class count in `docs/scan-product.md`,
+    `CLAUDE.md`, and `docs/feature-list.md` (and flip this task to `[DONE]`)
+    only once that run actually completes with a final dated `.pt`/`.onnx`.
+
 ## 3. Backend — Postgres Data Layer
 `pfm-web-app/src/db/`
 
@@ -179,9 +235,15 @@ surface for product scans**, mirroring what the DO flow already has in
 - **8.3** [DONE 2026-07-08] Build an `/admin/master-data` web UI to visually manage both SKUs and Stores. (See docs/feature-list.md)
 - **6.2** [DONE 2026-07-08] API + storage groundwork for scan annotation. (See docs/feature-list.md)
 - **6.3** [DONE 2026-07-08] Product-scan accuracy harness created. (See docs/feature-list.md)
+- **6.4** [DONE 2026-07-13] Auto-diff-vs-previous-run reporting (ported from the
+  DO-flow's `accuracy-check.mts`) plus classifier method/confidence tracking
+  added to `accuracy-check-scan.mts`; created the missing
+  `sources/product-test-images/` validation-photo folder. User-directed `n`
+  request: wanted to tune the scan algorithm and see improvement/regression
+  automatically instead of eyeballing two flat runs. (See docs/feature-list.md)
 
-*Suggested order: 6.2 → 6.1 → 6.3 (storage/API first, page on top, harness once
-labels exist in volume).*
+*Suggested order: 6.2 → 6.1 → 6.3 → 6.4 (storage/API first, page on top, harness
+once labels exist in volume, diffing once the harness has history to diff against).*
 
 ## 7. Auth — Store Accounts & Profile-Sourced Metadata
 `pfm-web-app/src/db/init.ts`, `api/v1/auth/*`, `api/parse/route.ts`, `sources/toko_aktif.json`
@@ -350,6 +412,50 @@ Flutter root `plans/next-enhancements.md` §7.2.
     root `docs/iteration-log.md` for the Flutter-side verification that the
     editor renders this without a second network call.
 
+## 12. Backend — Stock Management
+`src/db/init-stock.ts`, `src/app/api/v1/stock/`, `src/utils/stock-*.ts`,
+`src/app/api/parse/route.ts`, `src/app/api/v1/documents/[id]/route.ts`,
+`src/app/admin/master-data/`
+
+Added 2026-07-10, backend counterpart to root `plans/next-enhancements.md` §9
+(Flutter Stocks Menu & DO-to-Stock Flow) — both sections originated from the same
+user-directed, extensively grilled ad-hoc feature request (not an `e`/`enhance`
+section — see `AGENTS.md` Part B7). **Read
+[../../docs/stock-feature-plan.md](../../docs/stock-feature-plan.md) first** — full
+schema, API contracts, and sequencing for both sides. **Status: planned, not yet
+implemented** — no code for this feature exists in the codebase yet.
+
+- **12.1** [TODO] **Stock schema + core CRUD.** New `src/db/init-stock.ts`
+  (`stock_batches` — unique per `(kode_toko, no_sku, batch_code, expiry_date)`,
+  tracks both outer and inner qty; `stock_movements` — append-only audit log,
+  `intake`/`decrement`/`adjustment`/`manual_seed`), wired into `init.ts`. New
+  `src/utils/stock-mapper.ts`, `src/utils/stock-movement.ts` (`recordStockMovement`
+  only, for this task). New routes: `src/app/api/v1/stock/route.ts` (GET summary
+  per SKU, POST create/merge-by-unique-key), `stock/[noSku]/route.ts` (GET batch
+  detail), `stock/batches/[id]/route.ts` (PUT edit, logged as `adjustment`). No
+  DELETE route — batches are edit-only, never removed.
+- **12.2** [TODO] **Product Scan decrement hook + in-stock candidate filter.**
+  Extend `stock-movement.ts` with `decrementBatchForProductScan` (row-locked,
+  allowed to go negative, logged as `decrement`); wire into
+  `documents/[id]/route.ts`'s existing PUT transaction, gated on `scan_mode ===
+  'Product'` and a new `stock_batch_id` payload field — an invalid/cross-store
+  batch id fails the **whole confirm** (400), never a silent skip (per user's
+  explicit answer during grilling). New `src/utils/stock-lookup.ts`
+  (`getInStockSkuSet`/`filterMatchesByStock`), applied to `possibleMatches` in
+  both `api/parse/route.ts`'s Product branch and `api/v1/scan-product/route.ts` —
+  **do not change `classifyAndMatchProduct()`'s signature**, it's shared with the
+  anonymous store-agnostic desktop dev route; filter at the two authenticated call
+  sites instead. Blocked on 12.1.
+- **12.3** [TODO] **Read-only admin Stock view.** `admin/master-data/page.tsx`
+  (419 lines, already over the 256-line threshold) split into `page.tsx` (shell)
+  + extracted `StoreManager.tsx` + `SkuManager.tsx` (pure extraction, no behavior
+  change) + new `StockManager.tsx` (all-stores table via `GET /api/v1/stock` with
+  no `kode_toko` param as admin; row click drills into batch detail). Blocked on
+  12.1.
+
+*Suggested order: 12.1 → 12.2 (needs 12.1's tables/movement helper) and 12.3
+(needs 12.1's summary route) — 12.2/12.3 are independent of each other.*
+
 ---
 
 *Sections 1-4 migrated 2026-07-08 from root `plans/next-enhancements.md` sections