Document karung history backfill and missing-as-zero display design.
Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Alberto-AudrixandCursor committed
1 parent
5b74e0842c
commit
b3556c6ef4
1 file changed
+89
@@ -0,0 +1,89 @@
|
||||
# Karung Sync History Backfill + Missing-as-Zero Display
|
||||
|
||||
**Date:** 2026-09-04
|
||||
**Status:** Approved (awaiting written-spec confirmation)
|
||||
**Surface:** Hitung Karung sync + summary cards
|
||||
|
||||
## Problem
|
||||
|
||||
1. **Sync only writes “today.”** `request_karung` / Sync hari ini / cron use karung-web-admin `/api/v1/cycles/active/sections` with `section=today`. Older cycle days (e.g. day 0 = cycle start) never get IoT rows even when `/api/combined` already has history for those dates.
|
||||
2. **Symptom:** Rincian harian shows IOT `0` and Total `-` for dates with manual data but no `FeedSacks` row (e.g. 2 Sep), while upstream has real in/use/out.
|
||||
3. **Frontend:** Summary cards showed `N/A` when no same-date manual; product preference is to show `0` for missing values.
|
||||
|
||||
## Goals
|
||||
|
||||
- On each Sync (UI button and cron), also upsert IoT feed-sack days for the active cycle from `/api/combined` history.
|
||||
- If upstream values change after a previous sync, the next Sync updates local `FeedSacks` for those dates.
|
||||
- Zero is a valid IoT value; do not treat `0/0/0` as “needs special retry.”
|
||||
- One backfill pass per Sync invocation (no retry loops).
|
||||
- On Hitung Karung summary cards, missing Manual/IOT values display as `0` (not `N/A`).
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Changing the 17:00 (or configured) dashboard publish cutoff.
|
||||
- Re-introducing cross-day manual fallback on summary cards (already removed).
|
||||
- Syncing closed cycles unless already in scope of the existing cron filter.
|
||||
- Building a new karung-web-admin API; use existing `/api/combined` + sections.
|
||||
|
||||
## Upstream data (confirmed)
|
||||
|
||||
`GET /api/combined` returns:
|
||||
|
||||
- `masuk.history[]`: `{ date, source, counter_value }` for `K{n} - In` and `K{n} - Out`
|
||||
- `tuang.history[]`: `{ date, source, counter_value }` for `K{n} - Use`
|
||||
- Live `masuk.today` / `tuang.today` (not required for backfill of past days)
|
||||
|
||||
`GET /api/v1/cycles/active/sections` still used for **today** (and optionally yesterday_history remains available but is insufficient alone for older cycle days).
|
||||
|
||||
## Design
|
||||
|
||||
### Sync flow (single invocation)
|
||||
|
||||
For each active cycle (cron) or the requested cycle (UI):
|
||||
|
||||
1. **Today (existing):** Call `request_karung(cycle, section="today")` as today — update/create the snapshot date returned by the sections block.
|
||||
2. **History backfill (new):** Fetch `/api/combined` once. For each calendar date `D` where `cycle.start_date <= D <= min(today, cycle.end_date)` **and** combined history has at least one row for this kandang’s sources on `D`:
|
||||
- Resolve `in_today`, `out_today`, `feed_use_today` from history (`K{index} - In|Out|Use`). Missing source for that date → `0`.
|
||||
- `update_or_create` `FeedSacks(cycle, date=D)` with those daily values.
|
||||
3. **Totals recompute:** After today + backfill writes, recompute running `in_total`, `out_total`, `feed_use_total` for **all** `FeedSacks` of that cycle ordered by `date`, `pk` (same accumulation rule as current single-day prior-row logic).
|
||||
4. **Return:** API response should include the today row (compat) plus a short summary of dates upserted from history (for debugging / UI notice optional).
|
||||
|
||||
### When to write / skip a date
|
||||
|
||||
| Situation | Behavior |
|
||||
| -------------------------------------------------------------------- | ---------------------------------------------------------------------- |
|
||||
| Date in cycle range and present in combined history for this kandang | Upsert daily values from history (refresh on every Sync) |
|
||||
| Date in cycle range but **no** history rows for this kandang | Do **not** create a synthetic all-zero row; leave missing |
|
||||
| Existing local row, upstream later changes | Next Sync overwrites daily fields from history, then recomputes totals |
|
||||
| Upstream reports `0` | Store `0` (valid) |
|
||||
|
||||
Rationale: missing upstream ≠ invent zeros in DB (keeps Total `-` until data exists). Frontend still shows `0` for missing display cells.
|
||||
|
||||
### Frontend (Hitung Karung)
|
||||
|
||||
- Summary cards (`FeedStats`): when Manual or IOT daily/total value is `null`/missing for the display date, show **`0`**, not `N/A`.
|
||||
- Keep same-date manual pairing (no fallback to an older manual day).
|
||||
- Daily table already uses `0` for missing Manual/IOT; optional consistency: Total `-` may remain `-` when no IoT row (indicates no synced row) — **prefer keep `-` for Total when no row** so operators can see “not synced yet” vs synced zero. Daily Manual/IOT columns stay `0`.
|
||||
|
||||
### API / callers
|
||||
|
||||
- `POST /api/v1/karungs/request/` continues to accept `section` for today path; after today sync, always run history backfill for that cycle (section choice does not disable backfill).
|
||||
- Cron `sync_karung_from_web_admin`: same combined flow per active cycle.
|
||||
- Prefer a single service entrypoint e.g. `sync_karung_for_cycle(cycle, *, section="today")` used by view + command.
|
||||
|
||||
## Testing
|
||||
|
||||
- Unit: parse combined history → per-date in/use/out for a kandang; skip dates with no history; upsert updates existing; totals recomputed across days.
|
||||
- Unit/API: request sync creates missing cycle-start day from history while refreshing today.
|
||||
- Frontend: FeedStats / page test expects `0` instead of `N/A` when latest IoT day has no matching manual.
|
||||
|
||||
## Risks
|
||||
|
||||
- Combined payload size / timeout: one fetch per cycle sync (acceptable at current history sizes); reuse one client call per cycle.
|
||||
- Double-count totals if recompute is wrong: always full recompute from daily fields in date order, do not add “prior + today” on top of stale totals.
|
||||
- Timezone: use Asia/Jakarta “today” consistently with existing dashboard settings.
|
||||
|
||||
## Out of scope follow-ups
|
||||
|
||||
- Backfilling closed cycles.
|
||||
- Exposing a separate “Sync history only” button.
|
||||
Reference in new issue
Block a user