59 lines
14 KiB
Markdown
59 lines
14 KiB
Markdown
# AGENTS.md
|
||
|
||
Two tools scan the Vexanium blockchain voters table for accounts whose only vote goes to this BP (`databisnisid`): a Node.js reference script and the Python production tool that persists results to SQLite. A third tool (`distribute.py`) pays those voters daily: it signs `vex.token::transfer` actions with the BP active key to send the account's liquid VEX to every stored voter, proportional to stake. In the swarm stack the daily payout is driven by `distribute_loop.py` (service `distribute`), the scan equivalent of `scan_loop.py`.
|
||
|
||
## Run
|
||
|
||
- Node reference script: `node get_voters.js` (no deps, Node 18+).
|
||
- Python tool (production): `./venv/bin/python get_voters.py`. venv is Python 3.12, deps `requests` + `flask` + `python-dotenv` + `gunicorn` + `pymysql` (mysql backend only) + `pyntelope` (distribute signing) (`requirements.txt`). Install with `./venv/bin/pip install -r requirements.txt`.
|
||
- Daily payout (production): `./venv/bin/python distribute.py` — reads the current voters snapshot, fetches the BP account's liquid VEX, splits it pro-rata by stake (floor 4-dec, dust stays in the account), and pushes one signed `vex.token::transfer` per voter with memo `DATABISNISID PROFIT SHARE YYYY-MM-DD`. Preview the plan without signing/writing: `./venv/bin/python distribute.py --dry-run`. Requires `VEX_BP_PRIVATE_KEY` (BP active key) in env/`.env`; without it `--dry-run` still works, a real run raises. Failed rows are recorded `failed` and rejoined by the next day's run — no manual cleanup needed.
|
||
- Storage backend: `db.py` abstracts it. Default `sqlite` (`VEX_DB_PATH`, stdlib `sqlite3`, WAL). Optional `mysql` (`VEX_DB_BACKEND=mysql` + `VEX_DB_HOST/PORT/USER/PASS/NAME`, PyMySQL). Oracle tests stay on sqlite; `test_mariadb.py` is opt-in (skips unless `VEX_DB_BACKEND=mysql`). Query SQL is written once with `%s` placeholders (translated to `?` for sqlite); `db.query` always returns a list.
|
||
- Test MariaDB/MySQL via docker: `docker compose -f docker-compose.dev.yml up -d` (mariadb:11 container `databisnisid-mariadb`, localhost-only `127.0.0.1:3306`, db/user/pass `databisnisid`/`databisnis`/`databisnis`, named volume, healthcheck). Stop/remove with `docker compose -f docker-compose.dev.yml down`; wipe data with `docker compose -f docker-compose.dev.yml down -v`. Verify with `docker compose -f docker-compose.dev.yml exec mariadb mariadb -u databisnis -pdatabisnis databisnisid -e 'SELECT 1'`. Smoke against the container:
|
||
`VEX_DB_BACKEND=mysql VEX_DB_HOST=127.0.0.1 VEX_DB_PORT=3306 VEX_DB_USER=databisnis VEX_DB_PASS=databisnis VEX_DB_NAME=databisnisid ./venv/bin/python test_mariadb.py`
|
||
- Docker Swarm (production stack): images are registry-pushed `git.proit.id/proitlab/databisnisid-web` + `databisnisid-scan` + `databisnisid-dist` — build & push them first (`docker build -t git.proit.id/proitlab/databisnisid-web . && docker push git.proit.id/proitlab/databisnisid-web`, same for the scan and dist images), then from a swarm manager run `docker stack deploy -c docker-compose.yml databisnisid`. Stack = mariadb (internal) + web (gunicorn dashboard, published `:5000`) + scan (`scan_loop.py`, runs `get_voters.py` at `SCAN_HOURS` default `0,8,16`, local timezone `TZ` default `Asia/Jakarta`, plus one scan at container start via `SCAN_RUN_ON_START`) + distribute (`distribute_loop.py`, runs `distribute.py` once daily at `DISTRIBUTE_HOUR` default `10`, key `VEX_BP_PRIVATE_KEY` read from the bind-mounted `/app/.env` → host `/mnt/nfs/server5.saltis.id/data/databisnisid/app/config.env` via `config.py` `load_dotenv`, `:ro`). `mariadb` is pinned by `placement.constraints: node.hostname == server5.saltis.id` because its data lives in the host bind mount `/data/db/mariadb/databisnisid/data` on that node; `web`/`scan`/`distribute` can run on any node and reach it over the overlay network `appnet`. Inspect: `docker stack services databisnisid`, `docker service logs databisnisid_scan`, `docker stack rm databisnisid`. `docker stack deploy` ignores `build:` (images must already be in the registry) and ignores `env_file` (env is inlined with `${VAR}` interpolation from `.env`). The local dev box has no swarm anymore (torn down) — if you re-init one there, the mariadb constraint leaves that task **Pending** since no node is named `server5.saltis.id`.
|
||
- Web dashboard (read-only, reads the store via `db.py`): production `./venv/bin/gunicorn -c gunicorn.conf.py dashboard:app` → http://127.0.0.1:5000/ (run from repo dir). Easier: `./run.sh` (same command, works from any cwd, `$@` passed through). `gunicorn.conf.py` imports `config` → `.env` honored; `DASH_WORKERS` (default 2) controls workers, `DASH_HOST`/`DASH_PORT` the bind. Dev server (single-process) still works via `./venv/bin/python dashboard.py`. Paged 50/page, sorted `staked DESC`, live owner search (`/api/search`). Data freshness comes from the daily scan run — the dashboard never scans.
|
||
- Config: all tunables load from env / `.env` via `config.py` (python-dotenv): `VEX_TARGET_BP`, `VEX_API_NODE`, `VEX_DB_PATH`, `VEX_DB_BACKEND`, `VEX_DB_HOST`, `VEX_DB_PORT`, `VEX_DB_USER`, `VEX_DB_PASS`, `VEX_DB_NAME`, `VEX_MIN_STAKED_VEX`, `DASH_PAGE_SIZE`, `DASH_HOST`, `DASH_PORT`, `DASH_WORKERS`, `VEX_STALE_DAYS`, `DATABISNIS_API`, `DASH_LIQUID_TTL`, `VEX_BP_PRIVATE_KEY`, `DISTRIBUTE_HOUR`, `DISTRIBUTE_MAX_ATTEMPTS`. Copy `.env.example` → `.env` to override; `.env` is gitignored. Chain constants (`vexcore`/scope/table) stay hardcoded.
|
||
- Syntax check: `node --check get_voters.js`, `./venv/bin/python -m py_compile config.py get_voters.py dashboard.py db.py gunicorn.conf.py scan_loop.py distribute.py distribute_loop.py test_get_voters.py test_dashboard.py test_mariadb.py test_distribute.py`.
|
||
- Tests (the verification oracles): `./venv/bin/python test_get_voters.py`, `./venv/bin/python test_dashboard.py`, and `./venv/bin/python test_distribute.py` must all exit 0. They mock the network / use a temp DB — the live node is too flaky/slow for a full-scan test. Run after touching the relevant file.
|
||
|
||
## Key facts
|
||
|
||
- `TARGET_BP` (`databisnisid`) and `API_NODE` (`https://v2.vexascan.com:2096`) are the defaults in `config.py` (env-overridable); the JS reference still hardcodes them at the top of the file.
|
||
- The public Vexanium RPC node is flaky/timeout-prone; both scripts have retry logic built in — don't remove or bypass it. A full scan of the voters table takes minutes.
|
||
- The system contract on Vexanium is `vexcore` (not `vexio`), scope is also `vexcore`. Do not "fix" this to match EOS convention.
|
||
- VEX stake is stored scaled by 10000; the Python tool divides by 10000 before storing.
|
||
- The node sometimes returns `staked` as a string instead of a number — normalize coerces to float (spec §V8).
|
||
- The voters table has a sentinel first row (`owner="...........q"`, uint64 0) that must be skipped.
|
||
- Authoritative vote weight is `last_vote_weight` (high-precision decimal string) — stored verbatim as TEXT, never float-converted (spec §V6).
|
||
- Freshness filter (spec §V13): `last_vote` (estimated last re-vote date) is derived at scan time from the Vexanium weight formula `last_vote_weight = staked_raw × 2^(years since 2000)` → `log2(weight / (staked×10000))`. Voters whose `last_vote` is older than `VEX_STALE_DAYS` (default 28) — or can't be derived (zero/empty weight) — are dropped in `normalize()` and never stored; the dashboard therefore shows only fresh voters (no stale UI). The heuristic over-reads when a voter unstaked without re-voting (weight ÷ smaller stake ⇒ future date), so `normalize()` clamps any `last_vote` past scan time down to `now`.
|
||
- The web list shows RANK / AKUN / STAKE (VEX) / TOTAL REWARD (VEX) / VOTE TERAKHIR columns (3 stats cells). TOTAL REWARD = all-time sum of `distribute_payments.amount` where `status='sent'` (0,0000 for never-paid), joined per page via a LEFT JOIN subquery. Vote weight stays in the DB and `/api/search` JSON but is not rendered as a column.
|
||
- Token contract is `vex.token` (NOT `eosio.token` — that name doesn't exist on Vexanium), 4-decimal VEX, chain_id `f9f432b1851b5c179d2091a96f593aaed50ec7466b74f89301f957a83e56ce1f`. Distribution signs `vex.token::transfer` with the BP `active` key.
|
||
- Distribution (spec §V19–V27): each run pays the whole liquid balance pro-rata by stored `staked`, shares floored at 4 decimals with dust left in the account, one transfer per voter. A txid that can't be confirmed (`GET {DATABISNIS_API}/v2/history/get_transaction?id=<txid>` returns no `executed`) is never re-sent the same run — that payment stays `failed` and rejoins the next day. No-op (exit 0, no writes) when the voters table is empty or balance < 0.0001. The dashboard's `/history` view renders `distribute_runs` + `distribute_payments` read-only.
|
||
- Dashboard layout (spec §V28): desktop gives AKUN/STAKE/REWARD/VOTE equal width with AKUN left and the other three centered; tablet keeps the fixed STAKE column (RANK 56 / AKUN 1fr / STAKE 160px / REWARD 1fr / VOTE 1fr); `/history` uses a six-column run grid and a four-column payment grid (AKUN/JUMLAH/STATUS/TXID) that shows only the latest run with its date in the title, and links each TXID to `https://vexascan.com/transaction/{txid}` — no TANGGAL or MEMO column (memo is on-chain only, not stored); mobile turns history rows into labeled cards. HTML responses use `Cache-Control: no-store`; stylesheet URL is versioned with `?v=` for deploy cache-busting.
|
||
- Liquid balance (spec §V16): a 4th stat cell "SALDO LIQUID" shows the BP account's liquid VEX (`account.core_liquid_balance`) fetched from `GET {DATABISNIS_API}/v2/state/get_account?account=<BP>`. The dashboard fetches it live but caches per-worker in memory for `DASH_LIQUID_TTL` (default 60s); a failed fetch keeps the last value (or renders `—` if none ever succeeded) and the failure is also cooled-down so the API isn't hammered. The dashboard still never writes to the DB.
|
||
|
||
## Layout
|
||
|
||
- `SPEC.md` — spec (goal/constraints/interfaces/invariants/tasks/bug log), in caveman encoding. Build/backprop flow through it.
|
||
- `get_voters.py` — production fetcher: scan → filter (stake + freshness, basi dibuang) → `db.replace_snapshot` (each run replaces the table = daily snapshot, with `scanned_at` + derived `last_vote`; never appends history).
|
||
- `db.py` — storage abstraction (sqlite default | mysql via PyMySQL); `connect/query/queryone/replace_snapshot`; `%s` → `?` for sqlite; `query` returns list. Juga tabel distribusi: `distribute_runs` + `distribute_payments` (append-only) + helper `ensure_distribute_schema/record_run/record_payment/update_payment_status/update_run_status/list_runs/list_payments`.
|
||
- `get_voters.js` — reference implementation only.
|
||
- `distribute.py` — pembayaran harian: fetch liquid balance → `compute_shares` (Decimal floor 4-des, sisa di akun) → sign `vex.token::transfer` via pyntelope (`trx.link` ambil ABI+TAPOS dari node, `sign` dengan `VEX_BP_PRIVATE_KEY`, `.send()`) → catat `sent/failed` per voter; verifikasi txid via Hyperion sebelum kirim ulang; `--dry-run` = rencana ⊥ tanda tangan. `test_distribute.py` — oracle (mock chain+pyntelope, temp DB).
|
||
- `dashboard.py` + `templates/index.html` + `templates/history.html` + `static/style.css` + `static/app.js` — Flask web dashboard; reads the store via `db.py`, styled per `DESIGN.md`; `app.js` = debounced live owner search (fetch `/api/search`), degrades to the server-side `?q=` GET form if JS is off. The voters list (`voter-list` section, `.reward` cells, V29) LEFT-joins a `status='sent'` payments aggregate for the TOTAL REWARD column. `/history` = riwayat distribusi read-only; `run-list` = six-column desktop/tablet grid, `pay-list` = four-column grid (AKUN/JUMLAH/STATUS/TXID) showing only the latest run with its date in the title, and labeled mobile cards.
|
||
- `gunicorn.conf.py` — gunicorn production config (bind/workers from `config`, sync worker). `run.sh` — launcher: `./run.sh` = `./venv/bin/gunicorn -c gunicorn.conf.py dashboard:app` from any cwd.
|
||
- `config.py` — loads env/`.env` (python-dotenv) → `TARGET_BP`, `API_NODE`, `DB_PATH`, `DB_BACKEND`, `DB_HOST`, `DB_PORT`, `DB_USER`, `DB_PASS`, `DB_NAME`, `MIN_STAKED_VEX`, `PAGE_SIZE`, `DASH_HOST`, `DASH_PORT`, `DASH_WORKERS`, `VEX_STALE_DAYS`, `DATABISNIS_API`, `DASH_LIQUID_TTL`, `BP_PRIVATE_KEY`, `DISTRIBUTE_HOUR`, `DISTRIBUTE_MAX_ATTEMPTS`; shared by get_voters & dashboard.
|
||
- `scan_loop.py` — scheduler dalam container stack: menunggu batas `SCAN_HOURS` (lokal via `TZ`), panggil `get_voters.main()`; skan-awal `SCAN_RUN_ON_START` + tunggu DB siap; loop tak pernah keluar.
|
||
- `distribute_loop.py` — scheduler harian dalam container stack (service `distribute`): tunggu `DISTRIBUTE_HOUR` (lokal via `TZ`), panggil `distribute.main()`, loop tak pernah keluar; gagal dicatat dan dicoba besok; ⊥ distribusi-awal saat start (snapshot bisa basi).
|
||
- `Dockerfile` — image web `databisnisid-web` (gunicorn dashboard; `DASH_HOST=0.0.0.0` di stack agar ingress menjangkaunya). `Dockerfile.scan` — image `databisnisid-scan` (scan_loop; sertakan `tzdata`). `Dockerfile.dist` — image `databisnisid-dist` (distribute_loop; sertakan `tzdata` + pyntelope via requirements). Di stack produksi ketiganya di-push ke registry `git.proit.id/proitlab/databisnisid-{web,scan,dist}` (⊥ `build:` di compose). `.dockerignore` — venv/.env/artifak tak masuk build context.
|
||
- `docker-compose.yml` — STACK SWARM PRODUKSI (mariadb internal + web `:5000` + scan + distribute); image dari registry `git.proit.id/proitlab/databisnisid-*`; mariadb bind mount `/data/db/mariadb/databisnisid/data` + placement `node.hostname == server5.saltis.id`; network overlay `appnet`; `distribute` bawa `VEX_BP_PRIVATE_KEY` via bind `/mnt/nfs/server5.saltis.id/data/databisnisid/app/config.env:/app/.env:ro` (dibaca `config.py` `load_dotenv`; ⊥ interpolasi). `docker-compose.dev.yml` — mariadb uji lokal (`127.0.0.1:3306`, volume `mariadb_data`).
|
||
- `DESIGN.md` — Bugatti austere style guide; the dashboard's CSS maps its tokens (canvas #000000, hairline #262626, weight 400 everywhere, fonts Saira Condensed / EB Garamond / JetBrains Mono).
|
||
- `voters.db` — SQLite output (daily snapshot, gitignored in spirit).
|
||
|
||
## Conventions
|
||
|
||
- Comments and console output are in Indonesian — keep new output/comments in Indonesian.
|
||
- Dashboard UI labels are Indonesian uppercase captions (e.g. "DAFTAR PEMILIH", "TOTAL PEMILIH").
|
||
- JS style: 2-space indent, semicolons, single quotes, trailing commas, async/await.
|
||
- Python style: 4-space indent, stdlib `sqlite3`, `requests`, `flask`, PEP8.
|
||
- New SQL shared across backends: `%s` placeholders (never `?`), `ESCAPE '!'` for LIKE (backslash breaks MySQL string literals), no MySQL-only syntax.
|