T13: produksi web dengan gunicorn + WAL (gunicorn.conf.py, DASH_WORKERS, V12)
This commit is contained in:
1 parent
0374f9f4c2
commit
2b95be3091
8 files changed
+42
-7
No files matched your search
@@ -19,3 +19,6 @@ DASH_PAGE_SIZE=50
|
||||
# Dashboard: host & port server
|
||||
DASH_HOST=127.0.0.1
|
||||
DASH_PORT=5000
|
||||
|
||||
# Dashboard: jumlah worker gunicorn (produksi)
|
||||
DASH_WORKERS=2
|
||||
@@ -6,8 +6,8 @@ Two tools scan the Vexanium blockchain voters table for accounts whose only vote
|
||||
|
||||
- Node reference script: `node get_voters.js` (no deps, Node 18+).
|
||||
- Python tool (production): `./venv/bin/python get_voters.py`. venv is Python 3.12, deps `requests` + `flask` + `python-dotenv` (`requirements.txt`). Install with `./venv/bin/pip install -r requirements.txt`.
|
||||
- Web dashboard (read-only, reads `voters.db`): `./venv/bin/python dashboard.py` → http://127.0.0.1:5000/. Paged 50/page, sorted `staked DESC`, live owner search (`/api/search`). Data freshness comes from the daily scan run — the dashboard never scans.
|
||||
- Config: all tunables load from env / `.env` via `config.py` (python-dotenv): `VEX_TARGET_BP`, `VEX_API_NODE`, `VEX_DB_PATH`, `VEX_MIN_STAKED_VEX`, `DASH_PAGE_SIZE`, `DASH_HOST`, `DASH_PORT`. Copy `.env.example` → `.env` to override; `.env` is gitignored. Chain constants (`vexcore`/scope/table) stay hardcoded.
|
||||
- Web dashboard (read-only, reads `voters.db`): production `./venv/bin/gunicorn -c gunicorn.conf.py dashboard:app` → http://127.0.0.1:5000/ (run from repo dir). `gunicorn.conf.py` imports `config` → `.env` honored; `DASH_WORKERS` (default 2) controls workers, `DASH_HOST`/`DASH_PORT` the bind. Dev server (single-process) still works via `./venv/bin/python dashboard.py`. Paged 50/page, sorted `staked DESC`, live owner search (`/api/search`). Data freshness comes from the daily scan run — the dashboard never scans.
|
||||
- Config: all tunables load from env / `.env` via `config.py` (python-dotenv): `VEX_TARGET_BP`, `VEX_API_NODE`, `VEX_DB_PATH`, `VEX_MIN_STAKED_VEX`, `DASH_PAGE_SIZE`, `DASH_HOST`, `DASH_PORT`, `DASH_WORKERS`. Copy `.env.example` → `.env` to override; `.env` is gitignored. Chain constants (`vexcore`/scope/table) stay hardcoded.
|
||||
- Syntax check: `node --check get_voters.js`, `./venv/bin/python -m py_compile config.py get_voters.py dashboard.py`.
|
||||
- Tests (the verification oracles): `./venv/bin/python test_get_voters.py` and `./venv/bin/python test_dashboard.py` must both exit 0. They mock the network / use a temp DB — the live node is too flaky/slow for a full-scan test. Run after touching the relevant file.
|
||||
|
||||
|
||||
@@ -6,9 +6,9 @@ Re-runnable scheduled. Python 3.12, venv `./venv`.
|
||||
|
||||
## §C — Constraints
|
||||
- venv `./venv` exists (py3.12, only pip) — install deps there, ⊥ global
|
||||
- lib: `requests`, `flask`, `python-dotenv` ! pip-installed; others stdlib
|
||||
- lib: `requests`, `flask`, `python-dotenv`, `gunicorn` ! pip-installed; others stdlib
|
||||
- config via env/`.env` (`config.py`, python-dotenv); default = konstanta lama
|
||||
- db: SQLite stdlib `sqlite3` → `DB_PATH` (default `voters.db`)
|
||||
- db: SQLite stdlib `sqlite3` → `DB_PATH` (default `voters.db`); WAL journal mode → pembaca tak terblokir saat snapshot harian
|
||||
- api: `API_NODE` (default `https://v2.vexascan.com:2096`, public, flaky → retry)
|
||||
- contract=scope=`vexcore` (≠ `vexio`, EOS convention ⊥) — ⊥ env-able
|
||||
- filter: ∃ voter where `producers.length==1 & producers[0]==TARGET_BP`
|
||||
@@ -24,10 +24,10 @@ api: POST `https://v2.vexascan.com:2096/v1/chain/get_table_rows`
|
||||
→ rows[owner, proxy, producers[], staked, last_vote_weight, proxied_vote_weight, is_proxy], more, next_key
|
||||
db: `voters.db` → table `voters` (owner PK, weight, staked, scanned_at) — ganti tiap skan
|
||||
cmd: `./venv/bin/python get_voters.py` → stdout summary (id-ID)
|
||||
web: GET `/` (Flask) → HTML spec-list, paged 50/halaman, `ORDER BY staked DESC, owner ASC`; ⊥ mutation (read-only)
|
||||
web: GET `/` (Flask, disajikan gunicorn di produksi) → HTML spec-list, paged 50/halaman, `ORDER BY staked DESC, owner ASC`; ⊥ mutation (read-only)
|
||||
web: GET `/` + `?q=<substring>` → filter owner (server-side, no-JS fallback); pager bawa `q`
|
||||
web: GET `/api/search?q=` → JSON `{query,count,cap,results:[{owner,staked,weight,rank}]}`, rank global, cap 500
|
||||
env: `VEX_TARGET_BP`, `VEX_API_NODE`, `VEX_DB_PATH`, `VEX_MIN_STAKED_VEX`, `DASH_PAGE_SIZE`, `DASH_HOST`, `DASH_PORT` — via `config.py` (`.env`)
|
||||
env: `VEX_TARGET_BP`, `VEX_API_NODE`, `VEX_DB_PATH`, `VEX_MIN_STAKED_VEX`, `DASH_PAGE_SIZE`, `DASH_HOST`, `DASH_PORT`, `DASH_WORKERS` — via `config.py` (`.env`)
|
||||
|
||||
## §V — Invariants
|
||||
V1: ∀ fetch → retry ≥3 on timeout/http err (like JS fetchWithRetry)
|
||||
@@ -41,6 +41,7 @@ V8: normalize coerces `staked` → float (node kadang memberi string)
|
||||
V9: normalize drops voters where `staked` < 1000 VEX (setelah ÷10000)
|
||||
V10: dashboard baca `voters.db` → rows sort `staked` DESC (terbesar dulu); ⊥ mutasi DB
|
||||
V11: search `q` → filter owner substring (LIKE, wildcard-escape, case-insensitive); urut+paging terjaga; `rank` global (⊥ rank-dalam-filter); `?q=` & `/api/search` read-only
|
||||
V12: produksi web disajikan gunicorn (`dashboard:app`, `gunicorn.conf.py`); `DASH_WORKERS` default 2; `.env` ikut termuat via import `config`
|
||||
|
||||
## §T — Tasks
|
||||
id|status|task|cites
|
||||
@@ -56,6 +57,7 @@ T9|x|env config: config.py + .env/.env.example + dotenv; get_voters & dashboard
|
||||
T10|x|search: ?q= server filter + /api/search JSON (substring, escape, rank global)|V11,I.web
|
||||
T11|x|live JS filter (app.js) + responsive (DESIGN.md breakpoints, list→kartu mobile)|V11,I.web
|
||||
T12|x|dashboard oracle + search/API/config tests|V11,V10
|
||||
T13|x|produksi web: gunicorn (gunicorn.conf.py, DASH_WORKERS) + WAL di persist|V12,I.web,I.env
|
||||
|
||||
## §B — Bug log
|
||||
id|date|cause|fix
|
||||
|
||||
@@ -28,3 +28,6 @@ PAGE_SIZE = int(os.getenv('DASH_PAGE_SIZE', '50'))
|
||||
# Dashboard: host & port server Flask
|
||||
DASH_HOST = os.getenv('DASH_HOST', '127.0.0.1')
|
||||
DASH_PORT = int(os.getenv('DASH_PORT', '5000'))
|
||||
|
||||
# Dashboard: jumlah worker gunicorn (produksi)
|
||||
DASH_WORKERS = int(os.getenv('DASH_WORKERS', '2'))
|
||||
+6
-1
@@ -97,10 +97,15 @@ def normalize(row):
|
||||
|
||||
|
||||
def persist(rows):
|
||||
"""V7: ganti snapshot — DROP+INSERT tiap skan, ⊥ history/append."""
|
||||
"""V7: ganti snapshot — DROP+INSERT tiap skan, ⊥ history/append.
|
||||
|
||||
WAL: journal_mode=WAL → pembaca (dashboard multi-worker) tak terblokir
|
||||
saat DROP+INSERT harian. Mode ini persisten setelah diset.
|
||||
"""
|
||||
scanned_at = datetime.now().isoformat(timespec='seconds')
|
||||
conn = sqlite3.connect(DB_PATH)
|
||||
try:
|
||||
conn.execute('PRAGMA journal_mode=WAL')
|
||||
conn.execute('DROP TABLE IF EXISTS voters') # snapshot bersih
|
||||
conn.execute(
|
||||
'CREATE TABLE voters (' # I.db
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
"""Konfigurasi gunicorn untuk dashboard (produksi).
|
||||
|
||||
Dipanggil lewat `./venv/bin/gunicorn -c gunicorn.conf.py dashboard:app`.
|
||||
Import `config` di sini → `.env` ikut termuat (gunicorn sendiri ⊥ baca dotenv).
|
||||
Setiap request membuka koneksi SQLite sendiri → worker sync aman & paling sederhana.
|
||||
"""
|
||||
|
||||
from config import DASH_HOST, DASH_PORT, DASH_WORKERS
|
||||
|
||||
# Bind dari config (DASH_HOST / DASH_PORT), bukan hardcoded
|
||||
bind = f'{DASH_HOST}:{DASH_PORT}'
|
||||
|
||||
# Worker prefork; default 2 (V12), bisa dioverride via env DASH_WORKERS
|
||||
workers = DASH_WORKERS
|
||||
|
||||
worker_class = 'sync'
|
||||
|
||||
# Log ke stdout/stderr agar tampil di journal/systemd
|
||||
accesslog = '-'
|
||||
errorlog = '-'
|
||||
@@ -1,3 +1,4 @@
|
||||
requests>=2.31
|
||||
flask>=3.0
|
||||
python-dotenv>=1.0
|
||||
gunicorn>=22.0
|
||||
@@ -125,6 +125,7 @@ def main():
|
||||
check('V11 config default PAGE_SIZE', config.PAGE_SIZE == 50)
|
||||
check('V11 config default MIN_STAKED_VEX', config.MIN_STAKED_VEX == 1000.0)
|
||||
check('V11 config default DB_PATH', config.DB_PATH == 'voters.db')
|
||||
check('V12 config default DASH_WORKERS', config.DASH_WORKERS == 2)
|
||||
|
||||
|
||||
def _extract_stakes(text):
|
||||
|
||||
Reference in new issue
Block a user