T13: produksi web dengan gunicorn + WAL (gunicorn.conf.py, DASH_WORKERS, V12)

This commit is contained in:
proitlab committed 2026-08-04 16:40:16 +07:00
1 parent 0374f9f4c2
commit 2b95be3091
8 files changed
+42 -7

No files matched your search

+3
View File
@@ -19,3 +19,6 @@ DASH_PAGE_SIZE=50
# Dashboard: host & port server
DASH_HOST=127.0.0.1
DASH_PORT=5000
# Dashboard: jumlah worker gunicorn (produksi)
DASH_WORKERS=2
+2 -2
View File
@@ -6,8 +6,8 @@ Two tools scan the Vexanium blockchain voters table for accounts whose only vote
- Node reference script: `node get_voters.js` (no deps, Node 18+).
- Python tool (production): `./venv/bin/python get_voters.py`. venv is Python 3.12, deps `requests` + `flask` + `python-dotenv` (`requirements.txt`). Install with `./venv/bin/pip install -r requirements.txt`.
- Web dashboard (read-only, reads `voters.db`): `./venv/bin/python dashboard.py` → http://127.0.0.1:5000/. Paged 50/page, sorted `staked DESC`, live owner search (`/api/search`). Data freshness comes from the daily scan run — the dashboard never scans.
- Config: all tunables load from env / `.env` via `config.py` (python-dotenv): `VEX_TARGET_BP`, `VEX_API_NODE`, `VEX_DB_PATH`, `VEX_MIN_STAKED_VEX`, `DASH_PAGE_SIZE`, `DASH_HOST`, `DASH_PORT`. Copy `.env.example` → `.env` to override; `.env` is gitignored. Chain constants (`vexcore`/scope/table) stay hardcoded.
- Web dashboard (read-only, reads `voters.db`): production `./venv/bin/gunicorn -c gunicorn.conf.py dashboard:app` → http://127.0.0.1:5000/ (run from repo dir). `gunicorn.conf.py` imports `config` → `.env` honored; `DASH_WORKERS` (default 2) controls workers, `DASH_HOST`/`DASH_PORT` the bind. Dev server (single-process) still works via `./venv/bin/python dashboard.py`. Paged 50/page, sorted `staked DESC`, live owner search (`/api/search`). Data freshness comes from the daily scan run — the dashboard never scans.
- Config: all tunables load from env / `.env` via `config.py` (python-dotenv): `VEX_TARGET_BP`, `VEX_API_NODE`, `VEX_DB_PATH`, `VEX_MIN_STAKED_VEX`, `DASH_PAGE_SIZE`, `DASH_HOST`, `DASH_PORT`, `DASH_WORKERS`. Copy `.env.example` → `.env` to override; `.env` is gitignored. Chain constants (`vexcore`/scope/table) stay hardcoded.
- Syntax check: `node --check get_voters.js`, `./venv/bin/python -m py_compile config.py get_voters.py dashboard.py`.
- Tests (the verification oracles): `./venv/bin/python test_get_voters.py` and `./venv/bin/python test_dashboard.py` must both exit 0. They mock the network / use a temp DB — the live node is too flaky/slow for a full-scan test. Run after touching the relevant file.
+6 -4
View File
@@ -6,9 +6,9 @@ Re-runnable scheduled. Python 3.12, venv `./venv`.
## §C — Constraints
- venv `./venv` exists (py3.12, only pip) — install deps there, ⊥ global
- lib: `requests`, `flask`, `python-dotenv` ! pip-installed; others stdlib
- lib: `requests`, `flask`, `python-dotenv`, `gunicorn` ! pip-installed; others stdlib
- config via env/`.env` (`config.py`, python-dotenv); default = konstanta lama
- db: SQLite stdlib `sqlite3` → `DB_PATH` (default `voters.db`)
- db: SQLite stdlib `sqlite3` → `DB_PATH` (default `voters.db`); WAL journal mode → pembaca tak terblokir saat snapshot harian
- api: `API_NODE` (default `https://v2.vexascan.com:2096`, public, flaky → retry)
- contract=scope=`vexcore` (≠ `vexio`, EOS convention ⊥) — ⊥ env-able
- filter: ∃ voter where `producers.length==1 & producers[0]==TARGET_BP`
@@ -24,10 +24,10 @@ api: POST `https://v2.vexascan.com:2096/v1/chain/get_table_rows`
→ rows[owner, proxy, producers[], staked, last_vote_weight, proxied_vote_weight, is_proxy], more, next_key
db: `voters.db` → table `voters` (owner PK, weight, staked, scanned_at) — ganti tiap skan
cmd: `./venv/bin/python get_voters.py` → stdout summary (id-ID)
web: GET `/` (Flask) → HTML spec-list, paged 50/halaman, `ORDER BY staked DESC, owner ASC`; ⊥ mutation (read-only)
web: GET `/` (Flask, disajikan gunicorn di produksi) → HTML spec-list, paged 50/halaman, `ORDER BY staked DESC, owner ASC`; ⊥ mutation (read-only)
web: GET `/` + `?q=<substring>` → filter owner (server-side, no-JS fallback); pager bawa `q`
web: GET `/api/search?q=` → JSON `{query,count,cap,results:[{owner,staked,weight,rank}]}`, rank global, cap 500
env: `VEX_TARGET_BP`, `VEX_API_NODE`, `VEX_DB_PATH`, `VEX_MIN_STAKED_VEX`, `DASH_PAGE_SIZE`, `DASH_HOST`, `DASH_PORT` — via `config.py` (`.env`)
env: `VEX_TARGET_BP`, `VEX_API_NODE`, `VEX_DB_PATH`, `VEX_MIN_STAKED_VEX`, `DASH_PAGE_SIZE`, `DASH_HOST`, `DASH_PORT`, `DASH_WORKERS` — via `config.py` (`.env`)
## §V — Invariants
V1: ∀ fetch → retry ≥3 on timeout/http err (like JS fetchWithRetry)
@@ -41,6 +41,7 @@ V8: normalize coerces `staked` → float (node kadang memberi string)
V9: normalize drops voters where `staked` < 1000 VEX (setelah ÷10000)
V10: dashboard baca `voters.db` → rows sort `staked` DESC (terbesar dulu); ⊥ mutasi DB
V11: search `q` → filter owner substring (LIKE, wildcard-escape, case-insensitive); urut+paging terjaga; `rank` global (⊥ rank-dalam-filter); `?q=` & `/api/search` read-only
V12: produksi web disajikan gunicorn (`dashboard:app`, `gunicorn.conf.py`); `DASH_WORKERS` default 2; `.env` ikut termuat via import `config`
## §T — Tasks
id|status|task|cites
@@ -56,6 +57,7 @@ T9|x|env config: config.py + .env/.env.example + dotenv; get_voters & dashboard
T10|x|search: ?q= server filter + /api/search JSON (substring, escape, rank global)|V11,I.web
T11|x|live JS filter (app.js) + responsive (DESIGN.md breakpoints, list→kartu mobile)|V11,I.web
T12|x|dashboard oracle + search/API/config tests|V11,V10
T13|x|produksi web: gunicorn (gunicorn.conf.py, DASH_WORKERS) + WAL di persist|V12,I.web,I.env
## §B — Bug log
id|date|cause|fix
+3
View File
@@ -28,3 +28,6 @@ PAGE_SIZE = int(os.getenv('DASH_PAGE_SIZE', '50'))
# Dashboard: host & port server Flask
DASH_HOST = os.getenv('DASH_HOST', '127.0.0.1')
DASH_PORT = int(os.getenv('DASH_PORT', '5000'))
# Dashboard: jumlah worker gunicorn (produksi)
DASH_WORKERS = int(os.getenv('DASH_WORKERS', '2'))
+6 -1
View File
@@ -97,10 +97,15 @@ def normalize(row):
def persist(rows):
"""V7: ganti snapshot — DROP+INSERT tiap skan, ⊥ history/append."""
"""V7: ganti snapshot — DROP+INSERT tiap skan, ⊥ history/append.
WAL: journal_mode=WAL → pembaca (dashboard multi-worker) tak terblokir
saat DROP+INSERT harian. Mode ini persisten setelah diset.
"""
scanned_at = datetime.now().isoformat(timespec='seconds')
conn = sqlite3.connect(DB_PATH)
try:
conn.execute('PRAGMA journal_mode=WAL')
conn.execute('DROP TABLE IF EXISTS voters') # snapshot bersih
conn.execute(
'CREATE TABLE voters (' # I.db
+20
View File
@@ -0,0 +1,20 @@
"""Konfigurasi gunicorn untuk dashboard (produksi).
Dipanggil lewat `./venv/bin/gunicorn -c gunicorn.conf.py dashboard:app`.
Import `config` di sini → `.env` ikut termuat (gunicorn sendiri ⊥ baca dotenv).
Setiap request membuka koneksi SQLite sendiri → worker sync aman & paling sederhana.
"""
from config import DASH_HOST, DASH_PORT, DASH_WORKERS
# Bind dari config (DASH_HOST / DASH_PORT), bukan hardcoded
bind = f'{DASH_HOST}:{DASH_PORT}'
# Worker prefork; default 2 (V12), bisa dioverride via env DASH_WORKERS
workers = DASH_WORKERS
worker_class = 'sync'
# Log ke stdout/stderr agar tampil di journal/systemd
accesslog = '-'
errorlog = '-'
+1
View File
@@ -1,3 +1,4 @@
requests>=2.31
flask>=3.0
python-dotenv>=1.0
gunicorn>=22.0
+1
View File
@@ -125,6 +125,7 @@ def main():
check('V11 config default PAGE_SIZE', config.PAGE_SIZE == 50)
check('V11 config default MIN_STAKED_VEX', config.MIN_STAKED_VEX == 1000.0)
check('V11 config default DB_PATH', config.DB_PATH == 'voters.db')
check('V12 config default DASH_WORKERS', config.DASH_WORKERS == 2)
def _extract_stakes(text):