T1-T5: Python voter fetcher (§V1-9)

- get_voters.py: scan Vexanium voters table (vexcore), retry/paginate,
  filter databisnisid single-vote + stake ≥1000 VEX, store weight verbatim,
  daily-replace SQLite snapshot (owner PK, scanned_at)
- V8/V9 + test_get_voters.py (oracle, §V1-9)
- SPEC.md, AGENTS.md, requirements.txt
This commit is contained in:
proitlab committed 2026-08-04 15:56:19 +07:00
1 parent 539b83b330
commit c5e30a4785
7 files changed
+496 -173

No files matched your search

+5 -173
View File
@@ -1,176 +1,8 @@
# ---> Python # runtime artifacts
# Byte-compiled / optimized / DLL files voters.db
scan.log
__pycache__/ __pycache__/
*.py[cod] *.pyc
*$py.class
# C extensions # virtual env (deps installable via requirements.txt)
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
.pybuilder/
target/
# Jupyter Notebook
.ipynb_checkpoints
# IPython
profile_default/
ipython_config.py
# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# .python-version
# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock
# UV
# Similar to Pipfile.lock, it is generally recommended to include uv.lock in version control.
# This is especially recommended for binary packages to ensure reproducibility, and is more
# commonly ignored for libraries.
#uv.lock
# poetry
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
# This is especially recommended for binary packages to ensure reproducibility, and is more
# commonly ignored for libraries.
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
#poetry.lock
# pdm
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
#pdm.lock
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
# in version control.
# https://pdm.fming.dev/latest/usage/project/#working-with-version-control
.pdm.toml
.pdm-python
.pdm-build/
# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
__pypackages__/
# Celery stuff
celerybeat-schedule
celerybeat.pid
# SageMath parsed files
*.sage.py
# Environments
.env
.venv
env/
venv/ venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
dmypy.json
# Pyre type checker
.pyre/
# pytype static type analyzer
.pytype/
# Cython debug symbols
cython_debug/
# PyCharm
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/
# Ruff stuff:
.ruff_cache/
# PyPI configuration file
.pypirc
+33
View File
@@ -0,0 +1,33 @@
# AGENTS.md
Two tools scan the Vexanium blockchain voters table for accounts whose only vote goes to this BP (`databisnisid`): a Node.js reference script and the Python production tool that persists results to SQLite.
## Run
- Node reference script: `node get_voters.js` (no deps, Node 18+).
- Python tool (production): `./venv/bin/python get_voters.py`. venv is Python 3.12, dep `requests` only (`requirements.txt`). Install with `./venv/bin/pip install -r requirements.txt`.
- Syntax check: `node --check get_voters.js`, `./venv/bin/python -m py_compile get_voters.py`.
- Test (the verification oracle): `./venv/bin/python test_get_voters.py` must exit 0. It mocks the network — the live node is too flaky/slow for a full-scan test. Run it after touching `get_voters.py`.
## Key facts
- `TARGET_BP` (`databisnisid`) and `API_NODE` (`https://v2.vexascan.com:2096`) are hardcoded constants at the top of both scripts.
- The public Vexanium RPC node is flaky/timeout-prone; both scripts have retry logic built in — don't remove or bypass it. A full scan of the voters table takes minutes.
- The system contract on Vexanium is `vexcore` (not `vexio`), scope is also `vexcore`. Do not "fix" this to match EOS convention.
- VEX stake is stored scaled by 10000; the Python tool divides by 10000 before storing.
- The node sometimes returns `staked` as a string instead of a number — normalize coerces to float (spec §V8).
- The voters table has a sentinel first row (`owner="...........q"`, uint64 0) that must be skipped.
- Authoritative vote weight is `last_vote_weight` (high-precision decimal string) — stored verbatim as TEXT, never float-converted (spec §V6).
## Layout
- `SPEC.md` — spec (goal/constraints/interfaces/invariants/tasks/bug log), in caveman encoding. Build/backprop flow through it.
- `get_voters.py` — production fetcher: scan → filter → SQLite `voters.db` (each run replaces the table = daily snapshot, with `scanned_at`; never appends history).
- `get_voters.js` — reference implementation only.
- `voters.db` — SQLite output (daily snapshot, gitignored in spirit).
## Conventions
- Comments and console output are in Indonesian — keep new output/comments in Indonesian.
- JS style: 2-space indent, semicolons, single quotes, trailing commas, async/await.
- Python style: 4-space indent, stdlib `sqlite3`, `requests`, PEP8.
+48
View File
@@ -0,0 +1,48 @@
# SPEC — databisnisid voter fetcher (Python)
## §G — Goal
Scan Vexanium voters table → persist `databisnisid` voters + weight in SQLite.
Re-runnable scheduled. Python 3.12, venv `./venv`.
## §C — Constraints
- venv `./venv` exists (py3.12, only pip) — install deps there, ⊥ global
- lib: `requests` ! pip-installed; others stdlib
- db: SQLite stdlib `sqlite3` → file `voters.db`
- api: `https://v2.vexascan.com:2096` (public, flaky → retry)
- contract=scope=`vexcore` (≠ `vexio`, EOS convention ⊥)
- filter: ∃ voter where `producers.length==1 & producers[0]=='databisnisid'`
- `staked` scaled ×10000 → store ÷10000
- weight = `last_vote_weight` (string, high-precision decimal) — keep verbatim
- staked ≥ 1000 VEX ! required (÷10000 dulu)
- output/comments: Indonesian
- run harian → ganti snapshot (DROP+INSERT tiap skan; ⊥ history/append)
## §I — Interfaces
api: POST `https://v2.vexascan.com:2096/v1/chain/get_table_rows`
→ body {json:true, code:`vexcore`, scope:`vexcore`, table:`voters`, lower_bound, limit:500}
→ rows[owner, proxy, producers[], staked, last_vote_weight, proxied_vote_weight, is_proxy], more, next_key
db: `voters.db` → table `voters` (owner PK, weight, staked, scanned_at) — ganti tiap skan
cmd: `./venv/bin/python get_voters.py` → stdout summary (id-ID)
## §V — Invariants
V1: ∀ fetch → retry ≥3 on timeout/http err (like JS fetchWithRetry)
V2: page loop terminates: advance lower_bound → last owner; stop on `more=false` | empty rows
V3: ∀ page after first → shift 1st row (dup guard)
V4: skip `owner="...........q"` (uint64 0 sentinel) rows
V5: persist ∃ row where producers length 1 & [0]==TARGET_BP
V6: weight stored string verbatim, ⊥ float conversion
V7: each scan replaces table; ∀ row → latest snapshot; ⊥ append/duplicate owner
V8: normalize coerces `staked` → float (node kadang memberi string)
V9: normalize drops voters where `staked` < 1000 VEX (setelah ÷10000)
## §T — Tasks
id|status|task|cites
T1|x|setup: pip install requests, requirements.txt, project files|
T2|x|scan loop: retry + pagination|V1,V2,V3,I.api
T3|x|filter + normalize (owner, staked÷10000, weight ≥1000)|V4,V5,V6,V8,V9
T4|x|sqlite schema + replace-snapshot w/ scanned_at|V7,I.db
T5|x|CLI entry + Indonesian summary stdout|I.cmd
## §B — Bug log
id|date|cause|fix
B1|2026-08-04|node balikin `staked` sbg string utk sebagian baris → TypeError int÷int|V8
+98
View File
@@ -0,0 +1,98 @@
// Ganti dengan nama akun Block Producer Anda di Vexanium
const TARGET_BP = 'databisnisid';
// Menggunakan API dari v2.vexascan.com
const API_NODE = 'https://v2.vexascan.com:2096';
// Fungsi pembantu untuk fetch dengan mekanisme retry (karena node ini kadang timeout)
async function fetchWithRetry(url, options, maxRetries = 3) {
for (let i = 0; i < maxRetries; i++) {
try {
const response = await fetch(url, options);
if (!response.ok) throw new Error(`HTTP error! status: ${response.status}`);
return response;
} catch (error) {
if (i === maxRetries - 1) throw error;
console.log(`[Warning] Gagal menghubungi node, mencoba lagi... (${i + 1}/${maxRetries})`);
await new Promise(res => setTimeout(res, 2000)); // tunggu 2 detik sebelum retry
}
}
}
async function getVotersForMyBP() {
let myVoters = [];
let lowerBound = '';
let hasMore = true;
let totalScanned = 0;
console.log(`Mulai memindai voters untuk BP: "${TARGET_BP}"...`);
while (hasMore) {
try {
// Menggunakan fetchWithRetry yang sudah kita buat
const response = await fetchWithRetry(`${API_NODE}/v1/chain/get_table_rows`, {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify({
json: true,
// PENTING: Di Vexanium, kontrak sistem (untuk tabel voters) adalah vexcore, bukan vexio
code: 'vexcore',
scope: 'vexcore',
table: 'voters',
lower_bound: lowerBound,
limit: 500
})
});
const data = await response.json();
const rows = data.rows || [];
if (rows.length === 0) break;
// Hindari duplikasi elemen pertama saat pagination
if (lowerBound !== '') {
rows.shift();
}
totalScanned += rows.length;
// Saring hanya pemilih yang HANYA mem-vote BP ini (eksklusif 1 vote)
const filtered = rows.filter(voter =>
Array.isArray(voter.producers) &&
voter.producers.length === 1 &&
voter.producers[0] === TARGET_BP
);
myVoters = myVoters.concat(filtered);
hasMore = data.more;
if (hasMore && rows.length > 0) {
lowerBound = rows[rows.length - 1].owner;
}
} catch (error) {
console.error('\nError saat mengambil data:', error.message);
break;
}
}
console.log(`\n=== HASIL AKHIR ===`);
console.log(`Total akun dipindai di jaringan: ${totalScanned}`);
console.log(`Total voters untuk BP [${TARGET_BP}]: ${myVoters.length}`);
// Menampilkan daftar pemilih beserta VEX staked-nya
console.log('\nDaftar Pemilih:');
if (myVoters.length === 0) {
console.log('(Tidak ada pemilih ditemukan untuk BP ini atau scan terhenti di tengah jalan)');
} else {
myVoters.forEach((v, index) => {
// VEX stake disimpan dalam pecahan 1/10000 (4 desimal)
const vexStaked = (v.staked / 10000).toLocaleString('id-ID');
console.log(`${index + 1}. ${v.owner} - Stake: ${vexStaked} VEX`);
});
}
return myVoters;
}
getVotersForMyBP();
+162
View File
@@ -0,0 +1,162 @@
"""Scan tabel voters Vexanium, simpan pemilih BP databisnisid + bobot suara ke SQLite.
Berjalan imparsial (repeatable) dan siap dijadwalkan (cron). Memakai requests.
"""
import sqlite3
import sys
import time
from datetime import datetime
import requests
# Akun Block Producer (BP) yang dicari
TARGET_BP = 'databisnisid'
# Node RPC publik Vexanium
API_NODE = 'https://v2.vexascan.com:2096'
# Kontrak & scope sistem Vexanium = vexcore (BUKAN vexio sesuai konvensi EOS)
CODE = 'vexcore'
SCOPE = 'vexcore'
TABLE = 'voters'
# Ukuran halaman saat paginasi
LIMIT = 500
# Syarat minimum stake (dalam VEX) untuk disimpan
MIN_STAKED_VEX = 1000
# Lokasi basis data SQLite
DB_PATH = 'voters.db'
# Ambang-batas nama akun "0" (kodifikasi uint64 0) pada tabel voters
ZERO_OWNER = '...........q'
def fetch_with_retry(payload, max_retries=3, timeout=30):
"""POST ke RPC dengan retry (node publik sering timeout).
V1: ulang ≥3 kali pada timeout/error HTTP, jeda 2 detik antar percobaan.
"""
url = f'{API_NODE}/v1/chain/get_table_rows'
for attempt in range(max_retries):
try:
resp = requests.post(url, json=payload, timeout=timeout)
resp.raise_for_status()
return resp.json()
except requests.RequestException:
if attempt == max_retries - 1:
raise
print(f'[Peringatan] Gagal menghubungi node, mencoba lagi... '
f'({attempt + 1}/{max_retries})')
time.sleep(2)
def scan_voters():
"""Iterasi seluruh tabel voters (paginasi), kembalikan daftar mentah.
V2: majukan lower_bound ke owner terakhir; berhenti saat more=false | baris kosong.
V3: pada halaman kelanjutan, buang baris pertama (induk pamungkas duplikat).
"""
rows_all = []
lower_bound = ''
has_more = True
while has_more:
payload = {
'json': True,
'code': CODE,
'scope': SCOPE,
'table': TABLE,
'lower_bound': lower_bound,
'limit': LIMIT,
}
data = fetch_with_retry(payload)
rows = data.get('rows') or []
if not rows:
break
if lower_bound != '':
rows = rows[1:]
rows_all.extend(rows)
has_more = data.get('more', False)
if has_more:
lower_bound = rows_all[-1]['owner']
return rows_all
def normalize(row):
"""V4/V5/V6: saring ke baris yang relevan, bidang dinormalisasi."""
owner = row['owner']
if owner == ZERO_OWNER: # V4: lewati sentinel nama akun 0
return None
if not (row.get('producers') == [TARGET_BP]): # V5
return None
staked_vex = float(row.get('staked', 0)) / 10000 # V8/V9: bisa string, ÷10000
if staked_vex < MIN_STAKED_VEX:
return None
return {
'owner': owner,
'weight': row.get('last_vote_weight', ''), # V6: bobot verbatim (string)
'staked': staked_vex,
}
def persist(rows):
"""V7: ganti snapshot — DROP+INSERT tiap skan, ⊥ history/append."""
scanned_at = datetime.now().isoformat(timespec='seconds')
conn = sqlite3.connect(DB_PATH)
try:
conn.execute('DROP TABLE IF EXISTS voters') # snapshot bersih
conn.execute(
'CREATE TABLE voters (' # I.db
'owner TEXT PRIMARY KEY, '
'weight TEXT NOT NULL, '
'staked REAL NOT NULL, '
'scanned_at TEXT NOT NULL)'
)
conn.executemany(
'INSERT INTO voters (owner, weight, staked, scanned_at) '
'VALUES (?, ?, ?, ?)',
[(r['owner'], r['weight'], r['staked'], scanned_at) for r in rows],
)
conn.commit()
finally:
conn.close()
return scanned_at
def summarize(rows, scanned_at):
"""I.cmd: ringkasan stdout dalam bahasa Indonesia, format id-ID."""
print('\n=== HASIL AKHIR ===')
print(f'Total pemilih untuk BP [{TARGET_BP}]: {len(rows)}')
print(f'Waktu pemindaian: {scanned_at}')
print('\nDaftar Pemilih:')
if not rows:
print('(Tidak ada pemilih ditemukan untuk BP ini)')
return
for index, row in enumerate(rows, start=1):
print(f"{index}. {row['owner']} - Stake: "
f"{row['staked']:,.4f} VEX - Bobot: {row['weight']}")
def main():
"""Titik masuk CLI."""
print(f'Mulai memindai voters untuk BP: "{TARGET_BP}"...')
raw = scan_voters()
filtered = []
for row in raw:
item = normalize(row)
if item is not None:
filtered.append(item)
scanned_at = persist(filtered)
summarize(filtered, scanned_at)
return 0
if __name__ == '__main__':
sys.exit(main())
+1
View File
@@ -0,0 +1 @@
requests>=2.31
+149
View File
@@ -0,0 +1,149 @@
"""Uji verifikasi invarian §V tanpa bergantung node publik (flaky).
Oracle: `./venv/bin/python test_get_voters.py` harus exit 0.
Dalam setiap contoh, 'last_vote_weight' berisi 17 desimal (bobot teliti).
"""
import os
import sqlite3
import tempfile
import requests
import get_voters as gv
class FakePayload:
passes = 0
@staticmethod
def call(payload):
lb = payload['lower_bound']
pages = {
'': {'rows': [
{'owner': gv.ZERO_OWNER, 'producers': [],
'staked': 10000, 'last_vote_weight': '0.00'},
{'owner': 'aaa1', 'producers': [gv.TARGET_BP],
'staked': 12000000,
'last_vote_weight': '12345.678901234567890'},
{'owner': 'bbb2', 'producers': ['wind'],
'staked': 9000, 'last_vote_weight': '999.0'},
], 'more': True},
'bbb2': {'rows': [
{'owner': 'aaa1', 'producers': [gv.TARGET_BP],
'staked': 12000000,
'last_vote_weight': '12345.678901234567890'},
{'owner': 'ccc3', 'producers': [gv.TARGET_BP],
'staked': 10000000, 'last_vote_weight': '0.50000000000000000'},
], 'more': True},
'ccc3': {'rows': [
{'owner': 'ccc3', 'producers': [gv.TARGET_BP],
'staked': 10000000, 'last_vote_weight': '0.50000000000000000'},
], 'more': False},
}
FakePayload.passes += 1
if FakePayload.passes > 20:
raise AssertionError('V2 gagal: loop tidak berterminasi')
return pages[lb]
def check(tag, cond):
if not cond:
raise AssertionError(f'GAGAL: {tag}')
print(f'ok: {tag}')
def main():
tmp = tempfile.mkdtemp()
gv.DB_PATH = os.path.join(tmp, 'test.db')
# V1: ulang ≥3x saat node timeout → hits, lalu sukses
calls = {'n': 0}
def fake_post(url, json=None, timeout=30):
calls['n'] += 1
if calls['n'] < 3:
raise requests.exceptions.Timeout('node timeout')
resp = type('R', (), {'ok': True, 'raise_for_status': lambda self: None, 'json': lambda self: {'rows': [], 'more': False}})()
return resp
gv.requests.post = fake_post
out = gv.fetch_with_retry({})
check(f'V1 retry ≥3 lalu sukses ({calls["n"]} panggilan)', calls['n'] == 3 and out == {'rows': [], 'more': False})
# bila gagal terus → mengangkat setelah retries habis
def always_fail(url, json=None, timeout=30):
raise requests.exceptions.Timeout('node timeout')
gv.requests.post = always_fail
try:
gv.fetch_with_retry({}, max_retries=3)
raise AssertionError('V1 gagal: seharusnya mengangkat setelah retries habis')
except requests.exceptions.Timeout:
check('V1 gagal total → error dinaikkan', True)
# V2 + V3: paginasi berakhir & baris duplikat antar-halaman dibuang
gv.fetch_with_retry = FakePayload.call
raw = gv.scan_voters()
owners = [r['owner'] for r in raw]
check('V2/V3 pembuangan duplikat',
owners == [gv.ZERO_OWNER, 'aaa1', 'bbb2', 'ccc3'])
# V4 + V5 + V6: saringan sentinel & BP target terjadi di normalize
norm = [gv.normalize(r) for r in raw]
norm = [n for n in norm if n is not None]
norm_owners = [n['owner'] for n in norm]
check('V4 sentinel "0" dilewati', gv.ZERO_OWNER not in norm_owners)
check('V5 hanya BP target', all(n['owner'] in ('aaa1', 'ccc3') for n in norm))
by = {n['owner']: n for n in norm}
check('V6 bobot verbatim TEXT', by['aaa1']['weight'] == '12345.678901234567890')
check('V6 bobot bukan float-rounded',
by['aaa1']['weight'] != str(float(by['aaa1']['weight'])))
check('staked ÷10000', by['aaa1']['staked'] == 1200.0)
check('staked batas ≥1000 ÷10000', by['ccc3']['staked'] == 1000.0)
# V9: staked < 1000 VEX → dibuang; tepat 1000 → disimpan
check('V9 di bawah 1000 dibuang',
gv.normalize({'owner': 'low9', 'producers': [gv.TARGET_BP],
'staked': 9999999, 'last_vote_weight': '1.0'}) is None)
check('V9 tepat 1000 disimpan',
gv.normalize({'owner': 'eq10', 'producers': [gv.TARGET_BP],
'staked': 10000000, 'last_vote_weight': '1.0'})['staked'] == 1000.0)
# V8: node kadang memberi staked sebagai string → tetap dinormalisasi
srow = {'owner': 'eee5', 'producers': [gv.TARGET_BP],
'staked': '10000010000', 'last_vote_weight': '42.0'}
snorm = gv.normalize(srow)
check('V8 staked string → float ÷10000', snorm['staked'] == 1000001.0)
# V7: replace-snapshot — tiap persist mengganti tabel, ⊥ append/owner ganda
set_a = [
{'owner': 'aaa1', 'weight': '111.0', 'staked': 1200.0},
{'owner': 'ccc3', 'weight': '222.0', 'staked': 1000.0},
]
gv.persist(set_a)
conn = sqlite3.connect(gv.DB_PATH)
cnt = conn.execute('SELECT COUNT(*) FROM voters').fetchone()[0]
conn.close()
check('V7 snapshot pertama = 2', cnt == 2)
set_b = [ # aaa1 weight berubah; ccc3 dihapus; ddd4 baru
{'owner': 'aaa1', 'weight': '999.0', 'staked': 5000.0},
{'owner': 'ddd4', 'weight': '333.0', 'staked': 2000.0},
]
gv.persist(set_b)
conn = sqlite3.connect(gv.DB_PATH)
rows = conn.execute('SELECT owner, weight, staked, scanned_at FROM voters').fetchall()
by = {r[0]: r for r in rows}
cols = [r[1] for r in conn.execute('PRAGMA table_info(voters)').fetchall()]
pk = [r[1] for r in conn.execute('PRAGMA table_info(voters)').fetchall() if r[5]]
conn.close()
check('V7 count = |set_b|', len(rows) == 2)
check('V7 ccc3 (dihapus) tak ada', 'ccc3' not in by)
check('V7 ddd4 (baru) ada', 'ddd4' in by)
check('V7 bobot aaa1 terbaru', by['aaa1'][1] == '999.0')
stamps = set(r[3] for r in rows)
check('V7 semua baris cap waktu scan ini', len(stamps) == 1)
check('V7 owner PRIMARY KEY', 'owner' in cols and pk == ['owner'])
print('\nSEMUA UJI PASS')
if __name__ == '__main__':
main()