databisnisid/AGENTS.md

AGENTS.md

Two tools scan the Vexanium blockchain voters table for accounts whose only vote goes to this BP (databisnisid): a Node.js reference script and the Python production tool that persists results to SQLite. A third tool (distribute.py) pays those voters daily: it signs vex.token::transfer actions with the BP active key to send the account's liquid VEX to every stored voter, proportional to stake. In the swarm stack the daily payout is driven by distribute_loop.py (service distribute), the scan equivalent of scan_loop.py.

Run

• Node reference script: node get_voters.js (no deps, Node 18+).
• Python tool (production): ./venv/bin/python get_voters.py. venv is Python 3.12, deps requests + flask + python-dotenv + gunicorn + pymysql (mysql backend only) + pyntelope (distribute signing) (requirements.txt). Install with ./venv/bin/pip install -r requirements.txt.
• Daily payout (production): ./venv/bin/python distribute.py — reads the current voters snapshot, fetches the BP account's liquid VEX, splits it pro-rata by stake (floor 4-dec, dust stays in the account), and pushes one signed vex.token::transfer per voter with memo DATABISNISID PROFIT SHARE YYYY-MM-DD. Preview the plan without signing/writing: ./venv/bin/python distribute.py --dry-run. Requires VEX_BP_PRIVATE_KEY (BP active key) in env/.env; without it --dry-run still works, a real run raises. Failed rows are recorded failed and rejoined by the next day's run — no manual cleanup needed.
• Storage backend: db.py abstracts it. Default sqlite (VEX_DB_PATH, stdlib sqlite3, WAL). Optional mysql (VEX_DB_BACKEND=mysql + VEX_DB_HOST/PORT/USER/PASS/NAME, PyMySQL). Oracle tests stay on sqlite; test_mariadb.py is opt-in (skips unless VEX_DB_BACKEND=mysql). Query SQL is written once with %s placeholders (translated to ? for sqlite); db.query always returns a list.
• Test MariaDB/MySQL via docker: docker compose -f docker-compose.dev.yml up -d (mariadb:11 container databisnisid-mariadb, localhost-only 127.0.0.1:3306, db/user/pass databisnisid/databisnis/databisnis, named volume, healthcheck). Stop/remove with docker compose -f docker-compose.dev.yml down; wipe data with docker compose -f docker-compose.dev.yml down -v. Verify with docker compose -f docker-compose.dev.yml exec mariadb mariadb -u databisnis -pdatabisnis databisnisid -e 'SELECT 1'. Smoke against the container: VEX_DB_BACKEND=mysql VEX_DB_HOST=127.0.0.1 VEX_DB_PORT=3306 VEX_DB_USER=databisnis VEX_DB_PASS=databisnis VEX_DB_NAME=databisnisid ./venv/bin/python test_mariadb.py
• Docker Swarm (production stack): images are registry-pushed git.proit.id/proitlab/databisnisid-web + databisnisid-scan + databisnisid-dist — build & push them first (docker build -t git.proit.id/proitlab/databisnisid-web . && docker push git.proit.id/proitlab/databisnisid-web, same for the scan and dist images), then from a swarm manager run docker stack deploy -c docker-compose.yml databisnisid. Stack = mariadb (internal) + web (gunicorn dashboard, published :5000) + scan (scan_loop.py, runs get_voters.py at SCAN_HOURS default 0,8,16, local timezone TZ default Asia/Jakarta, plus one scan at container start via SCAN_RUN_ON_START) + distribute (distribute_loop.py, runs distribute.py once daily at DISTRIBUTE_HOUR default 10, key VEX_BP_PRIVATE_KEY read from the bind-mounted /app/.env → host /mnt/nfs/server5.saltis.id/data/databisnisid/app/config.env via config.py load_dotenv, :ro). mariadb is pinned by placement.constraints: node.hostname == server5.saltis.id because its data lives in the host bind mount /data/db/mariadb/databisnisid/data on that node; web/scan/distribute can run on any node and reach it over the overlay network appnet. Inspect: docker stack services databisnisid, docker service logs databisnisid_scan, docker stack rm databisnisid. docker stack deploy ignores build: (images must already be in the registry) and ignores env_file (env is inlined with ${VAR} interpolation from .env). The local dev box has no swarm anymore (torn down) — if you re-init one there, the mariadb constraint leaves that task Pending since no node is named server5.saltis.id.
• Web dashboard (read-only, reads the store via db.py): production ./venv/bin/gunicorn -c gunicorn.conf.py dashboard:app → http://127.0.0.1:5000/ (run from repo dir). Easier: ./run.sh (same command, works from any cwd, $@ passed through). gunicorn.conf.py imports config → .env honored; DASH_WORKERS (default 2) controls workers, DASH_HOST/DASH_PORT the bind. Dev server (single-process) still works via ./venv/bin/python dashboard.py. Paged 50/page, sorted staked DESC, live owner search (/api/search). Data freshness comes from the daily scan run — the dashboard never scans.
• Config: all tunables load from env / .env via config.py (python-dotenv): VEX_TARGET_BP, VEX_API_NODE, VEX_DB_PATH, VEX_DB_BACKEND, VEX_DB_HOST, VEX_DB_PORT, VEX_DB_USER, VEX_DB_PASS, VEX_DB_NAME, VEX_MIN_STAKED_VEX, DASH_PAGE_SIZE, DASH_HOST, DASH_PORT, DASH_WORKERS, VEX_STALE_DAYS, DATABISNIS_API, DASH_LIQUID_TTL, VEX_BP_PRIVATE_KEY, DISTRIBUTE_HOUR, DISTRIBUTE_MAX_ATTEMPTS, TELEGRAM_BOT_TOKEN, TELEGRAM_CHAT_IDS. Copy .env.example → .env to override; .env is gitignored. Chain constants (vexcore/scope/table) stay hardcoded.
• Syntax check: node --check get_voters.js, ./venv/bin/python -m py_compile config.py get_voters.py dashboard.py db.py gunicorn.conf.py scan_loop.py distribute.py distribute_loop.py telegram.py test_get_voters.py test_dashboard.py test_mariadb.py test_distribute.py.
• Tests (the verification oracles): ./venv/bin/python test_get_voters.py, ./venv/bin/python test_dashboard.py, and ./venv/bin/python test_distribute.py must all exit 0. They mock the network / use a temp DB — the live node is too flaky/slow for a full-scan test. Run after touching the relevant file.

Key facts

• TARGET_BP (databisnisid) and API_NODE (https://v2.vexascan.com:2096) are the defaults in config.py (env-overridable); the JS reference still hardcodes them at the top of the file.
• The public Vexanium RPC node is flaky/timeout-prone; both scripts have retry logic built in — don't remove or bypass it. A full scan of the voters table takes minutes.
• The system contract on Vexanium is vexcore (not vexio), scope is also vexcore. Do not "fix" this to match EOS convention.
• VEX stake is stored scaled by 10000; the Python tool divides by 10000 before storing.
• The node sometimes returns staked as a string instead of a number — normalize coerces to float (spec §V8).
• The voters table has a sentinel first row (owner="...........q", uint64 0) that must be skipped.
• Authoritative vote weight is last_vote_weight (high-precision decimal string) — stored verbatim as TEXT, never float-converted (spec §V6).
• Freshness filter (spec §V13): last_vote (estimated last re-vote date) is derived at scan time from the Vexanium weight formula last_vote_weight = staked_raw × 2^(years since 2000) → log2(weight / (staked×10000)). Voters whose last_vote is older than VEX_STALE_DAYS (default 28) — or can't be derived (zero/empty weight) — are dropped in normalize() and never stored; the dashboard therefore shows only fresh voters (no stale UI). The heuristic over-reads when a voter unstaked without re-voting (weight ÷ smaller stake ⇒ future date), so normalize() clamps any last_vote past scan time down to now.
• The web list shows RANK / AKUN / STAKE (VEX) / TOTAL REWARD (VEX) / VOTE TERAKHIR columns (3 stats cells). TOTAL REWARD = all-time sum of distribute_payments.amount where status='sent' (0,0000 for never-paid), joined per page via a LEFT JOIN subquery. Vote weight stays in the DB and /api/search JSON but is not rendered as a column.
• Token contract is vex.token (NOT eosio.token — that name doesn't exist on Vexanium), 4-decimal VEX, chain_id f9f432b1851b5c179d2091a96f593aaed50ec7466b74f89301f957a83e56ce1f. Distribution signs vex.token::transfer with the BP active key.
• Distribution (spec §V19–V27): each run pays the whole liquid balance pro-rata by stored staked, shares floored at 4 decimals with dust left in the account, one transfer per voter. A txid that can't be confirmed (GET {DATABISNIS_API}/v2/history/get_transaction?id=<txid> returns no executed) is never re-sent the same run — that payment stays failed and rejoins the next day. No-op (exit 0, no writes) when the voters table is empty or balance < 0.0001. The dashboard's /history view renders distribute_runs + distribute_payments read-only.
• Dashboard layout (spec §V28): desktop gives AKUN/STAKE/REWARD/VOTE equal width with AKUN left and the other three centered; tablet keeps the fixed STAKE column (RANK 56 / AKUN 1fr / STAKE 160px / REWARD 1fr / VOTE 1fr); /history uses a six-column run grid and a four-column payment grid (AKUN/JUMLAH/STATUS/TXID) that shows only the latest run with its date in the title, and links each TXID to https://vexascan.com/transaction/{txid} — no TANGGAL or MEMO column (memo is on-chain only, not stored); mobile turns history rows into labeled cards. HTML responses use Cache-Control: no-store; stylesheet URL is versioned with ?v= for deploy cache-busting.
• Liquid balance (spec §V16): a 4th stat cell "SALDO LIQUID" shows the BP account's liquid VEX (account.core_liquid_balance) fetched from GET {DATABISNIS_API}/v2/state/get_account?account=<BP>. The dashboard fetches it live but caches per-worker in memory for DASH_LIQUID_TTL (default 60s); a failed fetch keeps the last value (or renders — if none ever succeeded) and the failure is also cooled-down so the API isn't hammered. The dashboard still never writes to the DB.

Layout

• SPEC.md — spec (goal/constraints/interfaces/invariants/tasks/bug log), in caveman encoding. Build/backprop flow through it.
• get_voters.py — production fetcher: scan → filter (stake + freshness, basi dibuang) → db.replace_snapshot (each run replaces the table = daily snapshot, with scanned_at + derived last_vote; never appends history).
• db.py — storage abstraction (sqlite default | mysql via PyMySQL); connect/query/queryone/replace_snapshot; %s → ? for sqlite; query returns list. Juga tabel distribusi: distribute_runs + distribute_payments (append-only) + helper ensure_distribute_schema/record_run/record_payment/update_payment_status/update_run_status/list_runs/list_payments.
• get_voters.js — reference implementation only.
• distribute.py — pembayaran harian: fetch liquid balance → compute_shares (Decimal floor 4-des, sisa di akun) → sign vex.token::transfer via pyntelope (trx.link ambil ABI+TAPOS dari node, sign dengan VEX_BP_PRIVATE_KEY, .send()) → catat sent/failed per voter; verifikasi txid via Hyperion sebelum kirim ulang; --dry-run = rencana ⊥ tanda tangan; notifikasi Telegram mulai/selesai/gagal via telegram.py (best-effort, ⊥ dry-run/no-op). test_distribute.py — oracle (mock chain+pyntelope, temp DB).
• telegram.py — kirim notifikasi status distribusi via Telegram Bot API (best-effort; ⊥ token/chat id → no-op senyap; gagal kirim → log saja). TELEGRAM_BOT_TOKEN + TELEGRAM_CHAT_IDS (CSV) dari env/.env/NFS config.env.
• dashboard.py + templates/index.html + templates/history.html + static/style.css + static/app.js — Flask web dashboard; reads the store via db.py, styled per DESIGN.md; app.js = debounced live owner search (fetch /api/search), degrades to the server-side ?q= GET form if JS is off. The voters list (voter-list section, .reward cells, V29) LEFT-joins a status='sent' payments aggregate for the TOTAL REWARD column. /history = riwayat distribusi read-only; run-list = six-column desktop/tablet grid, pay-list = four-column grid (AKUN/JUMLAH/STATUS/TXID) showing only the latest run with its date in the title, and labeled mobile cards.
• gunicorn.conf.py — gunicorn production config (bind/workers from config, sync worker). run.sh — launcher: ./run.sh = ./venv/bin/gunicorn -c gunicorn.conf.py dashboard:app from any cwd.
• config.py — loads env/.env (python-dotenv) → TARGET_BP, API_NODE, DB_PATH, DB_BACKEND, DB_HOST, DB_PORT, DB_USER, DB_PASS, DB_NAME, MIN_STAKED_VEX, PAGE_SIZE, DASH_HOST, DASH_PORT, DASH_WORKERS, VEX_STALE_DAYS, DATABISNIS_API, DASH_LIQUID_TTL, BP_PRIVATE_KEY, DISTRIBUTE_HOUR, DISTRIBUTE_MAX_ATTEMPTS, TELEGRAM_BOT_TOKEN, TELEGRAM_CHAT_IDS; shared by get_voters & dashboard.
• scan_loop.py — scheduler dalam container stack: menunggu batas SCAN_HOURS (lokal via TZ), panggil get_voters.main(); skan-awal SCAN_RUN_ON_START + tunggu DB siap; loop tak pernah keluar.
• distribute_loop.py — scheduler harian dalam container stack (service distribute): tunggu DISTRIBUTE_HOUR (lokal via TZ), panggil distribute.main(), loop tak pernah keluar; gagal dicatat dan dicoba besok; ⊥ distribusi-awal saat start (snapshot bisa basi).
• Dockerfile — image web databisnisid-web (gunicorn dashboard; DASH_HOST=0.0.0.0 di stack agar ingress menjangkaunya). Dockerfile.scan — image databisnisid-scan (scan_loop; sertakan tzdata). Dockerfile.dist — image databisnisid-dist (distribute_loop; sertakan tzdata + pyntelope via requirements). Di stack produksi ketiganya di-push ke registry git.proit.id/proitlab/databisnisid-{web,scan,dist} (⊥ build: di compose). .dockerignore — venv/.env/artifak tak masuk build context.
• docker-compose.yml — STACK SWARM PRODUKSI (mariadb internal + web :5000 + scan + distribute); image dari registry git.proit.id/proitlab/databisnisid-*; mariadb bind mount /data/db/mariadb/databisnisid/data + placement node.hostname == server5.saltis.id; network overlay appnet; distribute bawa VEX_BP_PRIVATE_KEY via bind /mnt/nfs/server5.saltis.id/data/databisnisid/app/config.env:/app/.env:ro (dibaca config.py load_dotenv; ⊥ interpolasi). docker-compose.dev.yml — mariadb uji lokal (127.0.0.1:3306, volume mariadb_data).
• DESIGN.md — Bugatti austere style guide; the dashboard's CSS maps its tokens (canvas #000000, hairline #262626, weight 400 everywhere, fonts Saira Condensed / EB Garamond / JetBrains Mono).
• voters.db — SQLite output (daily snapshot, gitignored in spirit).

Conventions

• Comments and console output are in Indonesian — keep new output/comments in Indonesian.
• Dashboard UI labels are Indonesian uppercase captions (e.g. "DAFTAR PEMILIH", "TOTAL PEMILIH").
• JS style: 2-space indent, semicolons, single quotes, trailing commas, async/await.
• Python style: 4-space indent, stdlib sqlite3, requests, flask, PEP8.
• New SQL shared across backends: %s placeholders (never ?), ESCAPE '!' for LIKE (backslash breaks MySQL string literals), no MySQL-only syntax.