From 41990af9af9912cfec1bb328dc11612f365398c8 Mon Sep 17 00:00:00 2001 From: dsutanto Date: Tue, 4 Aug 2026 17:18:50 +0700 Subject: [PATCH] T15: buang voter basi di scan + hapus bobot suara & UI basi di web MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit get_voters.py kini menyimpan hanya pemilih segar (last_vote tak diketahui atau > VEX_STALE_DAYS dibuang di normalize, V13). Web menampilkan kolom RANK/AKUN/STAKE/VOTE TERAKHIR, tanpa BOBOT SUARA, stat VOTE BASI, penanda/ filter ?stale=1 (V15). voters.db dipangkas 336 → 42. Oracle diperbarui dan lolos. --- AGENTS.md | 7 +++-- SPEC.md | 12 ++++---- config.py | 2 +- dashboard.py | 67 ++++++++++++-------------------------------- get_voters.py | 34 +++++++++++++++++----- static/app.js | 9 +++++- static/style.css | 26 ++++------------- templates/index.html | 26 +++-------------- test_dashboard.py | 45 +++++++---------------------- test_get_voters.py | 62 ++++++++++++++++++++++++++++++---------- 10 files changed, 131 insertions(+), 159 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 031c4fe..2726915 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -8,7 +8,7 @@ Two tools scan the Vexanium blockchain voters table for accounts whose only vote - Python tool (production): `./venv/bin/python get_voters.py`. venv is Python 3.12, deps `requests` + `flask` + `python-dotenv` (`requirements.txt`). Install with `./venv/bin/pip install -r requirements.txt`. - Web dashboard (read-only, reads `voters.db`): production `./venv/bin/gunicorn -c gunicorn.conf.py dashboard:app` → http://127.0.0.1:5000/ (run from repo dir). `gunicorn.conf.py` imports `config` → `.env` honored; `DASH_WORKERS` (default 2) controls workers, `DASH_HOST`/`DASH_PORT` the bind. Dev server (single-process) still works via `./venv/bin/python dashboard.py`. Paged 50/page, sorted `staked DESC`, live owner search (`/api/search`). Data freshness comes from the daily scan run — the dashboard never scans. - Config: all tunables load from env / `.env` via `config.py` (python-dotenv): `VEX_TARGET_BP`, `VEX_API_NODE`, `VEX_DB_PATH`, `VEX_MIN_STAKED_VEX`, `DASH_PAGE_SIZE`, `DASH_HOST`, `DASH_PORT`, `DASH_WORKERS`, `VEX_STALE_DAYS`. Copy `.env.example` → `.env` to override; `.env` is gitignored. Chain constants (`vexcore`/scope/table) stay hardcoded. -- Syntax check: `node --check get_voters.js`, `./venv/bin/python -m py_compile config.py get_voters.py dashboard.py gunicorn.conf.py`. +- Syntax check: `node --check get_voters.js`, `./venv/bin/python -m py_compile config.py get_voters.py dashboard.py gunicorn.conf.py test_get_voters.py test_dashboard.py`. - Tests (the verification oracles): `./venv/bin/python test_get_voters.py` and `./venv/bin/python test_dashboard.py` must both exit 0. They mock the network / use a temp DB — the live node is too flaky/slow for a full-scan test. Run after touching the relevant file. ## Key facts @@ -20,12 +20,13 @@ Two tools scan the Vexanium blockchain voters table for accounts whose only vote - The node sometimes returns `staked` as a string instead of a number — normalize coerces to float (spec §V8). - The voters table has a sentinel first row (`owner="...........q"`, uint64 0) that must be skipped. - Authoritative vote weight is `last_vote_weight` (high-precision decimal string) — stored verbatim as TEXT, never float-converted (spec §V6). -- Stale-vote flag (spec §V13): `last_vote` (estimated last re-vote date) is derived at scan time from the Vexanium weight formula `last_vote_weight = staked_raw × 2^(years since 2000)` → `log2(weight / (staked×10000))`. A voter is `stale` when `last_vote` is older than `VEX_STALE_DAYS` (default 28). Filter via `?stale=1`. +- Freshness filter (spec §V13): `last_vote` (estimated last re-vote date) is derived at scan time from the Vexanium weight formula `last_vote_weight = staked_raw × 2^(years since 2000)` → `log2(weight / (staked×10000))`. Voters whose `last_vote` is older than `VEX_STALE_DAYS` (default 28) — or can't be derived (zero/empty weight) — are dropped in `normalize()` and never stored; the dashboard therefore shows only fresh voters (no stale UI). +- The web list shows RANK / AKUN / STAKE (VEX) / VOTE TERAKHIR columns (3 stats cells). Vote weight stays in the DB and `/api/search` JSON but is not rendered as a column. ## Layout - `SPEC.md` — spec (goal/constraints/interfaces/invariants/tasks/bug log), in caveman encoding. Build/backprop flow through it. -- `get_voters.py` — production fetcher: scan → filter → SQLite `voters.db` (each run replaces the table = daily snapshot, with `scanned_at` + derived `last_vote`; never appends history). +- `get_voters.py` — production fetcher: scan → filter (stake + freshness, basi dibuang) → SQLite `voters.db` (each run replaces the table = daily snapshot, with `scanned_at` + derived `last_vote`; never appends history). - `get_voters.js` — reference implementation only. - `dashboard.py` + `templates/index.html` + `static/style.css` + `static/app.js` — Flask web dashboard; reads `voters.db`, styled per `DESIGN.md`; `app.js` = debounced live owner search (fetch `/api/search`), degrades to the server-side `?q=` GET form if JS is off. - `config.py` — loads env/`.env` (python-dotenv) → `TARGET_BP`, `API_NODE`, `DB_PATH`, `MIN_STAKED_VEX`, `PAGE_SIZE`, `DASH_HOST`, `DASH_PORT`, `DASH_WORKERS`, `VEX_STALE_DAYS`; shared by get_voters & dashboard. diff --git a/SPEC.md b/SPEC.md index ad9f50c..571f6e2 100644 --- a/SPEC.md +++ b/SPEC.md @@ -16,6 +16,7 @@ Re-runnable scheduled. Python 3.12, venv `./venv`. - weight = `last_vote_weight` (string, high-precision decimal) — keep verbatim - staked ≥ `MIN_STAKED_VEX` (default 1000 VEX) ! required (÷10000 dulu) - `last_vote` (perkiraan waktu revote) diturunkan saat scan dari rasio `last_vote_weight/staked_raw` (V13); `VEX_STALE_DAYS` default 28 +- voter basi (revote > `VEX_STALE_DAYS`) atau `last_vote` tak diketahui → dibuang di `normalize` (⊥ disimpan); hanya pemilih segar yang masuk DB - output/comments: Indonesian - run harian → ganti snapshot (DROP+INSERT tiap skan; ⊥ history/append) @@ -23,12 +24,11 @@ Re-runnable scheduled. Python 3.12, venv `./venv`. api: POST `https://v2.vexascan.com:2096/v1/chain/get_table_rows` → body {json:true, code:`vexcore`, scope:`vexcore`, table:`voters`, lower_bound, limit:500} → rows[owner, proxy, producers[], staked, last_vote_weight, proxied_vote_weight, is_proxy], more, next_key -db: `voters.db` → table `voters` (owner PK, weight, staked, scanned_at, last_vote) — ganti tiap skan +db: `voters.db` → table `voters` (owner PK, weight, staked, scanned_at, last_vote) — ganti tiap skan; weight disimpan (audit/asal `last_vote`) tapi ⊥ ditampilkan web cmd: `./venv/bin/python get_voters.py` → stdout summary (id-ID) web: GET `/` (Flask, disajikan gunicorn di produksi) → HTML spec-list, paged 50/halaman, `ORDER BY staked DESC, owner ASC`; ⊥ mutation (read-only) web: GET `/` + `?q=` → filter owner (server-side, no-JS fallback); pager bawa `q` -web: GET `/` + `?stale=1` → hanya voter basi (revote > VEX_STALE_DAYS); pager bawa `stale` -web: GET `/api/search?q=` → JSON `{query,count,cap,results:[{owner,staked,weight,rank,last_vote,stale}]}`, rank global, cap 500 +web: GET `/api/search?q=` → JSON `{query,count,cap,results:[{owner,staked,weight,rank,last_vote}]}`, rank global, cap 500 env: `VEX_TARGET_BP`, `VEX_API_NODE`, `VEX_DB_PATH`, `VEX_MIN_STAKED_VEX`, `DASH_PAGE_SIZE`, `DASH_HOST`, `DASH_PORT`, `DASH_WORKERS`, `VEX_STALE_DAYS` — via `config.py` (`.env`) ## §V — Invariants @@ -44,7 +44,8 @@ V9: normalize drops voters where `staked` < 1000 VEX (setelah ÷10000) V10: dashboard baca `voters.db` → rows sort `staked` DESC (terbesar dulu); ⊥ mutasi DB V11: search `q` → filter owner substring (LIKE, wildcard-escape, case-insensitive); urut+paging terjaga; `rank` global (⊥ rank-dalam-filter); `?q=` & `/api/search` read-only V12: produksi web disajikan gunicorn (`dashboard:app`, `gunicorn.conf.py`); `DASH_WORKERS` default 2; `.env` ikut termuat via import `config` -V13: `last_vote` = `2000-01-01 + log2(last_vote_weight / (staked×10000)) × 365.25 hari` (None bila bobot/stake nol); `stale` = `last_vote < now − VEX_STALE_DAYS`; `?stale=1` & `/api/search` read-only; rank tetap global +V13: `last_vote` = `2000-01-01 + log2(last_vote_weight / (staked×10000)) × 365.25 hari` (None bila bobot/stake nol); voter dgn `last_vote=None` atau `< now − VEX_STALE_DAYS` dibuang di `normalize` (⊥ disimpan) → DB hanya berisi pemilih segar +V15: web list = kolom RANK/AKUN/STAKE (VEX)/VOTE TERAKHIR — ⊥ BOBOT SUARA, ⊥ penanda/stat/filter basi; stats = 3 sel (TOTAL PEMILIH/TOTAL VEX/STAKE TERTINGGI); `/api/search` hasil tetap `{owner,staked,weight,rank,last_vote}` (weight disimpan & di-API, ⊥ dirender) ## §T — Tasks id|status|task|cites @@ -61,7 +62,8 @@ T10|x|search: ?q= server filter + /api/search JSON (substring, escape, rank glob T11|x|live JS filter (app.js) + responsive (DESIGN.md breakpoints, list→kartu mobile)|V11,I.web T12|x|dashboard oracle + search/API/config tests|V11,V10 T13|x|produksi web: gunicorn (gunicorn.conf.py, DASH_WORKERS) + WAL di persist|V12,I.web,I.env -T14|x|stale-vote: last_vote di scan (rasio bobot/stake) + flag/kolom/filter `?stale=1` di dashboard|V13,I.db,I.web +T14|x|stale-vote: last_vote di scan (rasio bobot/stake); basi/unverifiable dibuang di normalize (⊥ disimpan)|V13,I.db +T15|x|hapus bobot suara + UI basi di web (kolom/stats/penanda/filter `?stale=1`); oracle diperbarui|V15,I.web ## §B — Bug log id|date|cause|fix diff --git a/config.py b/config.py index d00373f..483d99b 100644 --- a/config.py +++ b/config.py @@ -32,5 +32,5 @@ DASH_PORT = int(os.getenv('DASH_PORT', '5000')) # Dashboard: jumlah worker gunicorn (produksi) DASH_WORKERS = int(os.getenv('DASH_WORKERS', '2')) -# Ambang batas vote "basi" (hari tanpa revote) untuk penanda staleness +# Ambang batas vote "basi" (hari tanpa revote) — pemilih lewat batas ini dibuang di scan VEX_STALE_DAYS = int(os.getenv('VEX_STALE_DAYS', '28')) diff --git a/dashboard.py b/dashboard.py index 1c41e02..c40e364 100644 --- a/dashboard.py +++ b/dashboard.py @@ -5,12 +5,11 @@ Read-only (V10/V11): tidak pernah menulis ke DB. Data disegarkan oleh `get_voter """ import sqlite3 -from datetime import datetime, timedelta, timezone +from datetime import datetime from flask import Flask, abort, jsonify, render_template, request -from config import (DASH_HOST, DASH_PORT, DB_PATH, PAGE_SIZE, TARGET_BP, - VEX_STALE_DAYS) +from config import DASH_HOST, DASH_PORT, DB_PATH, PAGE_SIZE, TARGET_BP SEARCH_CAP = 500 MAX_Q_LEN = 64 @@ -42,48 +41,27 @@ def _escape_like(q): .replace('_', '\\_')) -def _filter_sql(q, stale=False): - """V11/V13: klausa WHERE + params untuk filter owner substring dan/atau vote basi.""" +def _filter_sql(q): + """V11: klausa WHERE + params untuk filter owner substring.""" clauses = [] params = [] if q: clauses.append("owner LIKE ? ESCAPE '\\'") params.append(f'%{_escape_like(q)}%') - if stale: - clauses.append('last_vote < ?') - params.append(_stale_cutoff()) if not clauses: return '', [] return 'WHERE ' + ' AND '.join(clauses), params -def _stale_cutoff(): - """V13: waktu batas vote basi (now - VEX_STALE_DAYS), format ISO (UTC).""" - return (datetime.now(timezone.utc).replace(tzinfo=None) - - timedelta(days=VEX_STALE_DAYS)).isoformat(timespec='seconds') - - -def _is_stale(last_vote): - """V13: apakah revote terakhir sudah lewat ambang basi.""" - if not last_vote: - return False - try: - vote_at = datetime.fromisoformat(last_vote) - except ValueError: - return False - return (datetime.now(timezone.utc).replace(tzinfo=None) - vote_at) > timedelta( - days=VEX_STALE_DAYS) - - -def _db_rows(page, q, stale=False): - """V10/V11/V13: satu halaman, urut staked DESC lalu owner ASC; rank global. +def _db_rows(page, q): + """V10/V11: satu halaman, urut staked DESC lalu owner ASC; rank global. Rank via correlated-count (bukan ROW_NUMBER): SQLite mendorong WHERE masuk ke subquery sebelum window function, sehingga ROW_NUMBER jadi rank-dalam-filter. Correlated count selalu menghitung atas SELURUH tabel → rank tetap global. """ offset = (page - 1) * PAGE_SIZE - where, params = _filter_sql(q, stale) + where, params = _filter_sql(q) sql = ( 'SELECT v.owner, v.staked, v.weight, v.scanned_at, ' ' (SELECT COUNT(*) FROM voters v2 ' @@ -95,15 +73,14 @@ def _db_rows(page, q, stale=False): ) conn = sqlite3.connect(DB_PATH) try: - rows = conn.execute(sql, params + [PAGE_SIZE, offset]).fetchall() + return conn.execute(sql, params + [PAGE_SIZE, offset]).fetchall() finally: conn.close() - return [r + (_is_stale(r[5]),) for r in rows] -def _count(q, stale=False): - """V11/V13: banyak baris cocok filter (atau total bila tanpa filter).""" - where, params = _filter_sql(q, stale) +def _count(q): + """V11: banyak baris cocok filter (atau total bila tanpa filter).""" + where, params = _filter_sql(q) conn = sqlite3.connect(DB_PATH) try: return conn.execute( @@ -124,13 +101,9 @@ def _stats(): scanned = conn.execute( 'SELECT COALESCE(MAX(scanned_at), \'\') FROM voters' ).fetchone()[0] - stale_count = conn.execute( - 'SELECT COUNT(*) FROM voters WHERE last_vote < ?', - [_stale_cutoff()], - ).fetchone()[0] finally: conn.close() - return total_voters, total_staked, top_staked, scanned, stale_count + return total_voters, total_staked, top_staked, scanned @app.get('/') @@ -140,12 +113,11 @@ def index(): except ValueError: abort(400) q = request.args.get('q', '').strip()[:MAX_Q_LEN] - stale = request.args.get('stale') == '1' - total_voters, total_staked, top_staked, scanned, stale_count = _stats() - filtered_count = _count(q, stale) + total_voters, total_staked, top_staked, scanned = _stats() + filtered_count = _count(q) total_pages = max(1, (filtered_count + PAGE_SIZE - 1) // PAGE_SIZE) page = min(page, total_pages) - rows = _db_rows(page, q, stale) + rows = _db_rows(page, q) return render_template( 'index.html', @@ -156,20 +128,17 @@ def index(): total_voters=total_voters, total_staked=total_staked, top_staked=top_staked, - stale_count=stale_count, scanned=scanned, q=q, - stale=stale, filtered_count=filtered_count, ) @app.get('/api/search') def api_search(): - """V11/V13: JSON hasil pencarian owner (substring), urut staked DESC, rank global.""" + """V11: JSON hasil pencarian owner (substring), urut staked DESC, rank global.""" q = request.args.get('q', '').strip()[:MAX_Q_LEN] - stale = request.args.get('stale') == '1' - where, params = _filter_sql(q, stale) + where, params = _filter_sql(q) sql = ( 'SELECT v.owner, v.staked, v.weight, ' ' (SELECT COUNT(*) FROM voters v2 ' @@ -191,7 +160,7 @@ def api_search(): 'results': [ { 'owner': r[0], 'staked': r[1], 'weight': r[2], - 'rank': r[3], 'last_vote': r[4], 'stale': _is_stale(r[4]), + 'rank': r[3], 'last_vote': r[4], } for r in rows ], diff --git a/get_voters.py b/get_voters.py index 1441f8f..daf5e06 100644 --- a/get_voters.py +++ b/get_voters.py @@ -1,17 +1,18 @@ """Scan tabel voters Vexanium, simpan pemilih BP databisnisid + bobot suara ke SQLite. -Berjalan imparsial (repeatable) dan siap dijadwalkan (cron). Memakai requests. +Hanya pemilih segar (revote ≤ VEX_STALE_DAYS lalu) yang disimpan — vote basi +dibuang. Berjalan imparsial (repeatable) dan siap dijadwalkan (cron). """ import math import sqlite3 import sys import time -from datetime import datetime, timedelta +from datetime import datetime, timedelta, timezone import requests -from config import API_NODE, DB_PATH, MIN_STAKED_VEX, TARGET_BP +from config import API_NODE, DB_PATH, MIN_STAKED_VEX, TARGET_BP, VEX_STALE_DAYS # Kontrak & scope sistem Vexanium = vexcore (BUKAN vexio sesuai konvensi EOS) CODE = 'vexcore' @@ -104,8 +105,23 @@ def scan_voters(): return rows_all -def normalize(row): - """V4/V5/V6: saring ke baris yang relevan, bidang dinormalisasi.""" +def _is_stale(last_vote, now): + """V13: apakah revote terakhir sudah lewat ambang basi (VEX_STALE_DAYS).""" + try: + vote_at = datetime.fromisoformat(last_vote) + except (TypeError, ValueError): + return True + return (now - vote_at) > timedelta(days=VEX_STALE_DAYS) + + +def normalize(row, now=None): + """V4/V5/V6/V13: saring ke baris yang relevan, bidang dinormalisasi. + + V13: pemilih dengan last_vote tak diketahui atau basi dibuang di sini — + hanya pemilih segar yang berakhir di DB. + """ + if now is None: + now = datetime.now(timezone.utc).replace(tzinfo=None) owner = row['owner'] if owner == ZERO_OWNER: # V4: lewati sentinel nama akun 0 return None @@ -114,11 +130,14 @@ def normalize(row): staked_vex = float(row.get('staked', 0)) / 10000 # V8/V9: bisa string, ÷10000 if staked_vex < MIN_STAKED_VEX: return None + last_vote = derive_last_vote(row.get('last_vote_weight', ''), staked_vex) + if last_vote is None or _is_stale(last_vote, now): # V13: basi ⊥ disimpan + return None return { 'owner': owner, 'weight': row.get('last_vote_weight', ''), # V6: bobot verbatim (string) 'staked': staked_vex, - 'last_vote': derive_last_vote(row.get('last_vote_weight', ''), staked_vex), + 'last_vote': last_vote, } @@ -171,9 +190,10 @@ def main(): """Titik masuk CLI.""" print(f'Mulai memindai voters untuk BP: "{TARGET_BP}"...') raw = scan_voters() + now = datetime.now(timezone.utc).replace(tzinfo=None) filtered = [] for row in raw: - item = normalize(row) + item = normalize(row, now=now) # satu stempel waktu untuk seluruh skan if item is not None: filtered.append(item) scanned_at = persist(filtered) diff --git a/static/app.js b/static/app.js index c90fe9f..4b45b79 100644 --- a/static/app.js +++ b/static/app.js @@ -29,6 +29,13 @@ }); } + function shortdate(value) { + if (!value) { + return '—'; + } + return String(value).slice(0, 10); + } + function setSearchMode(on) { if (on && originalRows === null) { originalRows = rows.innerHTML; @@ -53,7 +60,7 @@ '' + r.rank + '' + '' + escapeHtml(r.owner) + '' + '' + vex(r.staked) + '' + - '' + escapeHtml(r.weight) + '' + + '' + shortdate(r.last_vote) + '' + ''; }).join(''); } diff --git a/static/style.css b/static/style.css index 8de0a29..8a4a67d 100644 --- a/static/style.css +++ b/static/style.css @@ -98,7 +98,7 @@ body { /* ————— stats band ————— */ .stats { display: grid; - grid-template-columns: repeat(auto-fit, minmax(200px, 1fr)); + grid-template-columns: repeat(3, 1fr); border-top: 1px solid var(--hairline); border-bottom: 1px solid var(--hairline); margin: 0 var(--sp-lg) var(--sp-xl); @@ -164,14 +164,13 @@ button.pill { } .search-result { margin: 0 var(--sp-lg); } .search-result:empty { display: none; } -.search-extra { margin: var(--sp-xxs) var(--sp-lg) 0; } /* ————— spec-list ————— */ .spec-list { margin: var(--sp-md) var(--sp-lg) 0; border-top: 1px solid var(--hairline-strong); } .list-header, .list-row { display: grid; - grid-template-columns: 72px 1fr 200px 170px 1fr; + grid-template-columns: 72px 1fr 200px 170px; gap: var(--sp-lg); align-items: baseline; padding: var(--sp-lg) var(--sp-sm); @@ -187,24 +186,12 @@ button.pill { } .list-row:nth-child(odd) { background: var(--surface-soft); } .rank, -.weight, .vote-date { font-family: var(--font-mono); font-size: 11px; letter-spacing: 1.5px; color: var(--muted); } -.stale-mark { - display: inline-block; - border: 1px solid var(--hairline-strong); - border-radius: 9999px; - padding: 1px 8px; - margin-right: 8px; - font-size: 10px; - letter-spacing: 2px; - color: var(--muted-soft); -} -.is-stale .owner { color: var(--muted); } .owner { font-family: var(--font-display); font-size: 20px; @@ -279,8 +266,8 @@ button.pill { /* ————— tablet (768–1024px) ————— */ @media (max-width: 1024px) { .list-header, - .list-row { grid-template-columns: 56px 1fr 160px 150px 1fr; gap: var(--sp-md); } - .spec-list, .stats, .search, .search-result, .search-extra { margin-left: var(--sp-md); margin-right: var(--sp-md); } + .list-row { grid-template-columns: 56px 1fr 160px 150px; gap: var(--sp-md); } + .spec-list, .stats, .search, .search-result { margin-left: var(--sp-md); margin-right: var(--sp-md); } } /* ————— mobile (<768px): list → kartu berlabel ————— */ @@ -300,13 +287,10 @@ button.pill { .list-row .rank { grid-row: 1 / span 3; grid-column: 1; align-self: start; padding-top: 6px; } .list-row .owner { grid-column: 2; font-size: 18px; } .list-row .stake, - .list-row .weight, .list-row .vote-date { grid-column: 2; } .list-row .stake { text-align: left; } - .list-row .stake, - .list-row .weight { overflow-wrap: anywhere; } + .list-row .stake { overflow-wrap: anywhere; } .list-row .stake::before, - .list-row .weight::before, .list-row .vote-date::before { content: attr(data-label); display: block; diff --git a/templates/index.html b/templates/index.html index 8208d65..dfb5376 100644 --- a/templates/index.html +++ b/templates/index.html @@ -36,10 +36,6 @@ {{ top_staked|vex }} STAKE TERTINGGI -
- {{ stale_count }} - VOTE BASI -