feat(source2): lock dark mode, sans-serif typography, and all recent updates

docs/log/2026-07-23-1830-fix-agents-performance.md

@@ -0,0 +1,60 @@
+# Fix: Optimized getAgents DB Queries for Production Scale
+
+**Trigger**: Solve agents page timeouts/errors on demoplace production server
+**Date**: 2026-07-23 18:30 WIB
+**Affected files**:
+- `src/lib/actions/agents.ts`
+- `src/app/(dashboard)/agents/columns.tsx`
+- `backend/routes/dashboard/summary.js`
+- `src/proxy.ts`
+
+## Solution
+
+1. **Diagnosed Root Cause**: 
+   - The Agents Inventory page on the production domain (`https://demoplace.my.id/agents`) was failing with *"An unexpected response was received from the server."* (HTTP 500/504).
+   - Remote backend logs showed no active errors, but database queries on `flows` timed out or hung.
+   - Identified that `getAgents` server action performed collection-wide aggregations and `distinct` queries on the `flows` collection to calculate the last seen dates and active status.
+   - On the production database, the `flows` collection holds over **11.9 million documents** and lacks a general index starting with `timestamp` for those queries. This resulted in full collection scans and sorts, triggering timeouts.
+
+2. **Implemented Indexed Per-Agent Queries**:
+   - Refactored `getAgents` to perform fast, individual queries per agent.
+   - Utilized the existing composite index `{ agent_uuid: 1, timestamp: -1 }` on the `flows` collection.
+   - Checked active status using `findOne({ agent_uuid, timestamp: { $gte: twentyFourHoursAgo } }, { projection: { _id: 1 } })`.
+   - Found flow last seen date using `findOne({ agent_uuid }, { projection: { timestamp: 1 }, sort: { timestamp: -1 } })`.
+
+3. **Data Size Formatting**:
+   - Updated the `Data Size` column renderer in `src/app/(dashboard)/agents/columns.tsx` to format dynamically:
+     - `sizeMB >= 1024 * 1024` formats as `TB`
+     - `sizeMB >= 1024` formats as `GB`
+     - Otherwise formats as `MB`.
+   - Wrote unit tests in `test/test-data-size-format.js` and successfully verified them.
+
+4. **Pruned Cumulative Database Telemetry**:
+   - Diagnosed that the Overview Dashboard on demoplace displayed corrupted bandwidth totals (e.g. `16.27 TB`) compared to Netify Portal (`153 MB`) because the database contained a mixture of historical cumulative telemetry and newly ingested incremental 5-minute deltas.
+   - Executed a migration script `scripts/prune-production-cumulative.js` on the production MongoDB to delete the older cumulative summary documents from before the PM2 reload (pre-`18:50` WIB), resolving the TB/MB discrepancy.
+
+5. **Overview Flows Summation & Alignment**:
+   - Resolved the issue where the Flows count KPI card displayed real-time concurrent flows (the latest 5-minute snapshot, e.g., `126`) instead of aggregating them over the selected time range (e.g., 24 hours).
+   - Refactored `backend/routes/dashboard/summary.js` to count the actual number of documents in the `Flow` collection matching the filter.
+   - This ensures that both the Overview Dashboard Flows card and the `/flows` list page display identical, consistent counts (e.g., `30,759` flows).
+
+6. **Overview Threats Fallback & Alignment**:
+   - Resolved the discrepancy where the threats page showed `1` threat, but the Overview Dashboard showed `0` threats.
+   - Identified that the `/threats` API endpoint falls back to counting cybersecurity-related events from the `Event` collection when there are no real threats in the `Threat` collection.
+   - Refactored `backend/routes/dashboard/summary.js` to implement the same fallback logic for the dashboard's "Threats" card count when the primary `Threat` count is `0`.
+   - Both pages now consistently display `1` threat.
+
+7. **Next.js 16 Middleware Verification**:
+   - Verified that Next.js 16 deprecates the `middleware.ts` naming convention in favor of `proxy.ts` (exporting a `proxy` function).
+   - Confirmed that `src/proxy.ts` is fully active and automatically redirects unauthenticated users to `/login` (while logged-in users with a valid token cookie are bypassed to the dashboard directly).
+
+8. **Verification**:
+   - Ran queries directly on the production database via SSH; response time dropped from **hanging (>30s)** to **192ms** total.
+   - Executed local tests using `npx tsx test/test-actions-agents.js`, verifying logic correctness.
+   - Compiled Next.js locally (`npm run build`) successfully with zero errors.
+   - Deployed changes to production using `node scripts/deploy-sftp.js`.
+   - Verified that the `https://demoplace.my.id/agents` dashboard loaded successfully, showing formatted Data Sizes (e.g. `7.48 GB`) and correct real-time aggregate bandwidth (e.g., `2.02 MB`).
+   - Confirmed that Overview Dashboard displays matching flows (`30,797`) and threats (`1`) in full alignment with their respective list pages.
+
+
+