T18: stack swarm docker — image web+scan, scan_loop.py, mariadb bind-mount server5.saltis.id
- Dockerfile (web) + Dockerfile.scan (scan; sertakan tzdata) + .dockerignore
- scan_loop.py: jadwal skan SCAN_HOURS (0,8,16 lokal via TZ) + skan-awal SCAN_RUN_ON_START; loop tak keluar; tunggu DB siap
- docker-compose.yml → STACK SWARM produksi (mariadb bind mount /data/db/mariadb/databisnisid/data + placement node.hostname == server5.saltis.id; web :5000; scan); image dari registry git.proit.id/proitlab/databisnisid-{web,scan}
- docker-compose.dev.yml: mariadb uji lokal (127.0.0.1:3306) — pengganti docker-compose.yml lama
- SPEC.md §V18/T18; AGENTS.md runbook swarm + dev compose
- .env.example: TZ/SCAN_HOURS/SCAN_RUN_ON_START/MARIADB_*
This commit is contained in:
1 parent
fdb5d0de15
commit
656f26085f
9 files changed
+244
-11
No files matched your search
@@ -0,0 +1,9 @@
|
||||
# Artifak runtime & env lokal — jangan masuk build context
|
||||
venv/
|
||||
.git/
|
||||
.env
|
||||
voters.db
|
||||
.oc/
|
||||
__pycache__/
|
||||
*.pyc
|
||||
*.log
|
||||
@@ -42,3 +42,17 @@ DATABISNIS_API=https://api.databisnis.id
|
||||
|
||||
# Dashboard: TTL cache (detik) untuk saldo liquid — gagal fetch juga di-cooldown
|
||||
DASH_LIQUID_TTL=60
|
||||
|
||||
# — Stack swarm (docker-compose.yml) —
|
||||
# Interpolasi compose; override default di bawah lewat .env bila perlu.
|
||||
# TZ dipakai juga sebagai zona waktu jadwal skan (`datetime.now()` lokal).
|
||||
TZ=Asia/Jakarta
|
||||
SCAN_HOURS=0,8,16
|
||||
# Skan sekali saat container start (tabel voters langsung ada; dashboard ⊥ 500)
|
||||
SCAN_RUN_ON_START=1
|
||||
|
||||
# Kredensial mariadb stack — wajib konsisten antar-service (mariadb/web/scan)
|
||||
MARIADB_ROOT_PASSWORD=root
|
||||
MARIADB_DATABASE=databisnisid
|
||||
MARIADB_USER=databisnis
|
||||
MARIADB_PASSWORD=databisnis
|
||||
@@ -7,11 +7,12 @@ Two tools scan the Vexanium blockchain voters table for accounts whose only vote
|
||||
- Node reference script: `node get_voters.js` (no deps, Node 18+).
|
||||
- Python tool (production): `./venv/bin/python get_voters.py`. venv is Python 3.12, deps `requests` + `flask` + `python-dotenv` + `gunicorn` + `pymysql` (mysql backend only) (`requirements.txt`). Install with `./venv/bin/pip install -r requirements.txt`.
|
||||
- Storage backend: `db.py` abstracts it. Default `sqlite` (`VEX_DB_PATH`, stdlib `sqlite3`, WAL). Optional `mysql` (`VEX_DB_BACKEND=mysql` + `VEX_DB_HOST/PORT/USER/PASS/NAME`, PyMySQL). Oracle tests stay on sqlite; `test_mariadb.py` is opt-in (skips unless `VEX_DB_BACKEND=mysql`). Query SQL is written once with `%s` placeholders (translated to `?` for sqlite); `db.query` always returns a list.
|
||||
- Test MariaDB/MySQL via docker: `docker compose up -d` (mariadb:11 container `databisnisid-mariadb`, localhost-only `127.0.0.1:3306`, db/user/pass `databisnisid`/`databisnis`/`databisnis`, named volume, healthcheck). Stop/remove with `docker compose down`; wipe data with `docker compose down -v`. Verify with `docker compose exec mariadb mariadb -u databisnis -pdatabisnis databisnisid -e 'SELECT 1'`. Smoke against the container:
|
||||
- Test MariaDB/MySQL via docker: `docker compose -f docker-compose.dev.yml up -d` (mariadb:11 container `databisnisid-mariadb`, localhost-only `127.0.0.1:3306`, db/user/pass `databisnisid`/`databisnis`/`databisnis`, named volume, healthcheck). Stop/remove with `docker compose -f docker-compose.dev.yml down`; wipe data with `docker compose -f docker-compose.dev.yml down -v`. Verify with `docker compose -f docker-compose.dev.yml exec mariadb mariadb -u databisnis -pdatabisnis databisnisid -e 'SELECT 1'`. Smoke against the container:
|
||||
`VEX_DB_BACKEND=mysql VEX_DB_HOST=127.0.0.1 VEX_DB_PORT=3306 VEX_DB_USER=databisnis VEX_DB_PASS=databisnis VEX_DB_NAME=databisnisid ./venv/bin/python test_mariadb.py`
|
||||
- Docker Swarm (production stack): images are registry-pushed `git.proit.id/proitlab/databisnisid-web` + `databisnisid-scan` — build & push them first (`docker build -t git.proit.id/proitlab/databisnisid-web . && docker push git.proit.id/proitlab/databisnisid-web`, same for the scan image), then from a swarm manager run `docker stack deploy -c docker-compose.yml databisnisid`. Stack = mariadb (internal) + web (gunicorn dashboard, published `:5000`) + scan (`scan_loop.py`, runs `get_voters.py` at `SCAN_HOURS` default `0,8,16`, local timezone `TZ` default `Asia/Jakarta`, plus one scan at container start via `SCAN_RUN_ON_START`). `mariadb` is pinned by `placement.constraints: node.hostname == server5.saltis.id` because its data lives in the host bind mount `/data/db/mariadb/databisnisid/data` on that node; `web`/`scan` can run on any node and reach it over the overlay network `appnet`. Inspect: `docker stack services databisnisid`, `docker service logs databisnisid_scan`, `docker stack rm databisnisid`. `docker stack deploy` ignores `build:` (images must already be in the registry) and ignores `env_file` (env is inlined with `${VAR}` interpolation from `.env`). The local dev box has no swarm anymore (torn down) — if you re-init one there, the mariadb constraint leaves that task **Pending** since no node is named `server5.saltis.id`.
|
||||
- Web dashboard (read-only, reads the store via `db.py`): production `./venv/bin/gunicorn -c gunicorn.conf.py dashboard:app` → http://127.0.0.1:5000/ (run from repo dir). Easier: `./run.sh` (same command, works from any cwd, `$@` passed through). `gunicorn.conf.py` imports `config` → `.env` honored; `DASH_WORKERS` (default 2) controls workers, `DASH_HOST`/`DASH_PORT` the bind. Dev server (single-process) still works via `./venv/bin/python dashboard.py`. Paged 50/page, sorted `staked DESC`, live owner search (`/api/search`). Data freshness comes from the daily scan run — the dashboard never scans.
|
||||
- Config: all tunables load from env / `.env` via `config.py` (python-dotenv): `VEX_TARGET_BP`, `VEX_API_NODE`, `VEX_DB_PATH`, `VEX_DB_BACKEND`, `VEX_DB_HOST`, `VEX_DB_PORT`, `VEX_DB_USER`, `VEX_DB_PASS`, `VEX_DB_NAME`, `VEX_MIN_STAKED_VEX`, `DASH_PAGE_SIZE`, `DASH_HOST`, `DASH_PORT`, `DASH_WORKERS`, `VEX_STALE_DAYS`, `DATABISNIS_API`, `DASH_LIQUID_TTL`. Copy `.env.example` → `.env` to override; `.env` is gitignored. Chain constants (`vexcore`/scope/table) stay hardcoded.
|
||||
- Syntax check: `node --check get_voters.js`, `./venv/bin/python -m py_compile config.py get_voters.py dashboard.py db.py gunicorn.conf.py test_get_voters.py test_dashboard.py test_mariadb.py`.
|
||||
- Syntax check: `node --check get_voters.js`, `./venv/bin/python -m py_compile config.py get_voters.py dashboard.py db.py gunicorn.conf.py scan_loop.py test_get_voters.py test_dashboard.py test_mariadb.py`.
|
||||
- Tests (the verification oracles): `./venv/bin/python test_get_voters.py` and `./venv/bin/python test_dashboard.py` must both exit 0. They mock the network / use a temp DB — the live node is too flaky/slow for a full-scan test. Run after touching the relevant file.
|
||||
|
||||
## Key facts
|
||||
@@ -36,7 +37,9 @@ Two tools scan the Vexanium blockchain voters table for accounts whose only vote
|
||||
- `dashboard.py` + `templates/index.html` + `static/style.css` + `static/app.js` — Flask web dashboard; reads the store via `db.py`, styled per `DESIGN.md`; `app.js` = debounced live owner search (fetch `/api/search`), degrades to the server-side `?q=` GET form if JS is off.
|
||||
- `gunicorn.conf.py` — gunicorn production config (bind/workers from `config`, sync worker). `run.sh` — launcher: `./run.sh` = `./venv/bin/gunicorn -c gunicorn.conf.py dashboard:app` from any cwd.
|
||||
- `config.py` — loads env/`.env` (python-dotenv) → `TARGET_BP`, `API_NODE`, `DB_PATH`, `DB_BACKEND`, `DB_HOST`, `DB_PORT`, `DB_USER`, `DB_PASS`, `DB_NAME`, `MIN_STAKED_VEX`, `PAGE_SIZE`, `DASH_HOST`, `DASH_PORT`, `DASH_WORKERS`, `VEX_STALE_DAYS`, `DATABISNIS_API`, `DASH_LIQUID_TTL`; shared by get_voters & dashboard.
|
||||
- `docker-compose.yml` — mariadb:11 test container (`databisnisid-mariadb`, localhost-only 3306) for the mysql backend.
|
||||
- `scan_loop.py` — scheduler dalam container stack: menunggu batas `SCAN_HOURS` (lokal via `TZ`), panggil `get_voters.main()`; skan-awal `SCAN_RUN_ON_START` + tunggu DB siap; loop tak pernah keluar.
|
||||
- `Dockerfile` — image web `databisnisid-web` (gunicorn dashboard; `DASH_HOST=0.0.0.0` di stack agar ingress menjangkaunya). `Dockerfile.scan` — image `databisnisid-scan` (scan_loop; sertakan `tzdata`). Di stack produksi keduanya di-push ke registry `git.proit.id/proitlab/databisnisid-{web,scan}` (⊥ `build:` di compose). `.dockerignore` — venv/.env/artifak tak masuk build context.
|
||||
- `docker-compose.yml` — STACK SWARM PRODUKSI (mariadb internal + web `:5000` + scan); image dari registry `git.proit.id/proitlab/databisnisid-*`; mariadb bind mount `/data/db/mariadb/databisnisid/data` + placement `node.hostname == server5.saltis.id`; network overlay `appnet`. `docker-compose.dev.yml` — mariadb uji lokal (`127.0.0.1:3306`, volume `mariadb_data`).
|
||||
- `DESIGN.md` — Bugatti austere style guide; the dashboard's CSS maps its tokens (canvas #000000, hairline #262626, weight 400 everywhere, fonts Saira Condensed / EB Garamond / JetBrains Mono).
|
||||
- `voters.db` — SQLite output (daily snapshot, gitignored in spirit).
|
||||
|
||||
|
||||
+13
@@ -0,0 +1,13 @@
|
||||
FROM python:3.12-slim
|
||||
ENV PYTHONUNBUFFERED=1 PIP_NO_CACHE_DIR=1
|
||||
WORKDIR /app
|
||||
|
||||
COPY requirements.txt .
|
||||
RUN pip install --no-cache-dir -r requirements.txt
|
||||
|
||||
COPY config.py db.py dashboard.py gunicorn.conf.py ./
|
||||
COPY templates ./templates
|
||||
COPY static ./static
|
||||
|
||||
EXPOSE 5000
|
||||
CMD ["gunicorn", "-c", "gunicorn.conf.py", "dashboard:app"]
|
||||
@@ -0,0 +1,15 @@
|
||||
FROM python:3.12-slim
|
||||
ENV PYTHONUNBUFFERED=1 PIP_NO_CACHE_DIR=1
|
||||
|
||||
# tzdata → TZ env (jadwal skan lokal, default Asia/Jakarta) bekerja di container
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends tzdata \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY requirements.txt .
|
||||
RUN pip install --no-cache-dir -r requirements.txt
|
||||
|
||||
COPY config.py db.py get_voters.py scan_loop.py ./
|
||||
|
||||
CMD ["python", "scan_loop.py"]
|
||||
@@ -8,7 +8,7 @@ Re-runnable scheduled. Python 3.12, venv `./venv`.
|
||||
- venv `./venv` exists (py3.12, only pip) — install deps there, ⊥ global
|
||||
- lib: `requests`, `flask`, `python-dotenv`, `gunicorn` ! pip-installed; `pymysql` utk backend mysql (opt-in); others stdlib
|
||||
- config via env/`.env` (`config.py`, python-dotenv); default = konstanta lama
|
||||
- db: `db.py` abstraksi backend — `sqlite` (default, stdlib `sqlite3` → `DB_PATH`, WAL → pembaca tak terblokir) atau `mysql` (MariaDB/MySQL via PyMySQL, `VEX_DB_*`); container uji via `docker-compose.yml`
|
||||
- db: `db.py` abstraksi backend — `sqlite` (default, stdlib `sqlite3` → `DB_PATH`, WAL → pembaca tak terblokir) atau `mysql` (MariaDB/MySQL via PyMySQL, `VEX_DB_*`); container uji via `docker-compose.dev.yml`
|
||||
- api: `API_NODE` (default `https://v2.vexascan.com:2096`, public, flaky → retry)
|
||||
- saldo liquid akun BP: API databisnis (`DATABISNIS_API`, default `https://api.databisnis.id`) — diambil dashboard, cache TTL `DASH_LIQUID_TTL` (default 60s), cooldown kegagalan
|
||||
- contract=scope=`vexcore` (≠ `vexio`, EOS convention ⊥) — ⊥ env-able
|
||||
@@ -20,6 +20,7 @@ Re-runnable scheduled. Python 3.12, venv `./venv`.
|
||||
- voter basi (revote > `VEX_STALE_DAYS`) atau `last_vote` tak diketahui → dibuang di `normalize` (⊥ disimpan); hanya pemilih segar yang masuk DB
|
||||
- output/comments: Indonesian
|
||||
- run harian → ganti snapshot (DROP+INSERT tiap skan; ⊥ history/append)
|
||||
- produksi stack: Docker Swarm multi-node (node DB `server5.saltis.id`) — image dari registry `git.proit.id/proitlab/databisnisid-web` & `-scan` (⊥ `build:` di stack), deploy `docker stack deploy -c docker-compose.yml databisnisid`; mariadb bind mount `/data/db/mariadb/databisnisid/data` pd node `server5.saltis.id` (placement constraint `node.hostname == server5.saltis.id`), web/scan di node mana pun via overlay `appnet`; scan berjalan pd jam `SCAN_HOURS` (default 0,8,16, waktu lokal `TZ` default Asia/Jakarta) + skan-awal `SCAN_RUN_ON_START`
|
||||
|
||||
## §I — Interfaces
|
||||
api: POST `https://v2.vexascan.com:2096/v1/chain/get_table_rows`
|
||||
@@ -30,7 +31,7 @@ cmd: `./venv/bin/python get_voters.py` → stdout summary (id-ID)
|
||||
web: GET `/` (Flask, disajikan gunicorn di produksi) → HTML spec-list, paged 50/halaman, `ORDER BY staked DESC, owner ASC`; ⊥ mutation (read-only)
|
||||
web: GET `/` + `?q=<substring>` → filter owner (server-side, no-JS fallback); pager bawa `q`
|
||||
web: GET `/api/search?q=` → JSON `{query,count,cap,results:[{owner,staked,weight,rank,last_vote}]}`, rank global, cap 500
|
||||
env: `VEX_TARGET_BP`, `VEX_API_NODE`, `VEX_DB_PATH`, `VEX_DB_BACKEND`, `VEX_DB_HOST`, `VEX_DB_PORT`, `VEX_DB_USER`, `VEX_DB_PASS`, `VEX_DB_NAME`, `VEX_MIN_STAKED_VEX`, `DASH_PAGE_SIZE`, `DASH_HOST`, `DASH_PORT`, `DASH_WORKERS`, `VEX_STALE_DAYS`, `DATABISNIS_API`, `DASH_LIQUID_TTL` — via `config.py` (`.env`)
|
||||
env: `VEX_TARGET_BP`, `VEX_API_NODE`, `VEX_DB_PATH`, `VEX_DB_BACKEND`, `VEX_DB_HOST`, `VEX_DB_PORT`, `VEX_DB_USER`, `VEX_DB_PASS`, `VEX_DB_NAME`, `VEX_MIN_STAKED_VEX`, `DASH_PAGE_SIZE`, `DASH_HOST`, `DASH_PORT`, `DASH_WORKERS`, `VEX_STALE_DAYS`, `DATABISNIS_API`, `DASH_LIQUID_TTL`, `SCAN_HOURS`, `SCAN_RUN_ON_START`, `TZ` — via `config.py` (`.env`)
|
||||
|
||||
## §V — Invariants
|
||||
V1: ∀ fetch → retry ≥3 on timeout/http err (like JS fetchWithRetry)
|
||||
@@ -49,6 +50,7 @@ V13: `last_vote` = `2000-01-01 + log2(last_vote_weight / (staked×10000)) × 365
|
||||
V15: web list = kolom RANK/AKUN/STAKE (VEX)/VOTE TERAKHIR — ⊥ BOBOT SUARA, ⊥ penanda/stat/filter basi; stats = 3 sel (TOTAL PEMILIH/TOTAL VEX/STAKE TERTINGGI); `/api/search` hasil tetap `{owner,staked,weight,rank,last_vote}` (weight disimpan & di-API, ⊥ dirender)
|
||||
V16: saldo liquid akun `TARGET_BP` dari `GET {DATABISNIS_API}/v2/state/get_account` (parse `account.core_liquid_balance`) → sel stat SALDO LIQUID; cache in-memory per worker TTL `DASH_LIQUID_TTL`; gagal fetch → nilai lama (atau `—` bila belum pernah sukses) + `at` ikut diset (cooldown, ⊥ pukulan berulang); read-only, ⊥ tulis DB
|
||||
V17: penyimpanan lewat `db.py` (backend `VEX_DB_BACKEND` = `sqlite` default | `mysql`); query berbagi sintaks pakai placeholder `%s` (diterjemahkan `?` utk sqlite); LIKE escape pakai `ESCAPE '!'` — backslash memutus literal string MySQL (B2); `db.query` → list (fetchall sqlite=list / PyMySQL=tuple diseragamkan, B3); replace snapshot mysql = CREATE IF NOT EXISTS + DELETE + INSERT satu transaksi (MVCC → pembaca dapat snapshot konsisten saat ganti harian, tak kena torn read); oracle utama (test_get_voters/test_dashboard) tetap sqlite hermetik; `test_mariadb.py` opt-in (skip exit 0 bila `VEX_DB_BACKEND != mysql`)
|
||||
V18: stack produksi = Docker Swarm (`docker-compose.yml`, `docker stack deploy -c docker-compose.yml databisnisid`): mariadb bind mount `/data/db/mariadb/databisnisid/data` pd node `server5.saltis.id` + placement constraint `node.hostname == server5.saltis.id` (data node-lokal → mariadb harus selalu di node itu; di swarm yg ⊥ punya node itu task-nya Pending, bukan bug); web (gunicorn, publik `:5000`) + scan (`scan_loop.py`) di node mana pun, terhubung via overlay `appnet`; image dari registry `git.proit.id/proitlab/databisnisid-web` & `-scan` (⊥ `build:`); scan berjalan pd jam `SCAN_HOURS` (default 0,8,16, lokal `TZ` default Asia/Jakarta via `datetime.now()`) + skan-awal `SCAN_RUN_ON_START` (setelah DB siap) → tabel voters langsung ada (dashboard ⊥ 500); loop tak pernah keluar; `stack deploy` ⊥ `env_file` (env inline via `${VAR}` interpolasi `.env`); image web set `DASH_HOST=0.0.0.0` agar ingress menjangkau; healthcheck web = cek socket TCP (⊥ ketergantungan isi DB)
|
||||
|
||||
## §T — Tasks
|
||||
id|status|task|cites
|
||||
@@ -69,6 +71,7 @@ T14|x|stale-vote: last_vote di scan (rasio bobot/stake); basi/unverifiable dibua
|
||||
T15|x|hapus bobot suara + UI basi di web (kolom/stats/penanda/filter `?stale=1`); oracle diperbarui|V15,I.web
|
||||
T16|x|saldo liquid akun di dashboard: API databisnis + cache TTL + cooldown; sel SALDO LIQUID|V16,I.web,I.env
|
||||
T17|x|backend shared db.py (sqlite|mysql, `%s`→`?`, ESCAPE '!'), config VEX_DB_*, refactor fetcher+dashboard, test_mariadb.py opt-in, runbook docker-compose|V17,I.db,I.env
|
||||
T18|x|stack swarm: Dockerfile web + Dockerfile.scan, scan_loop.py (SCAN_HOURS 0,8,16 lokal, skan-awal), docker-compose.yml stack (mariadb+web+scan), docker-compose.dev.yml utk uji lokal|V18,I.web,I.env
|
||||
|
||||
## §B — Bug log
|
||||
id|date|cause|fix
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
services:
|
||||
mariadb:
|
||||
image: mariadb:11
|
||||
container_name: databisnisid-mariadb
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
MARIADB_ROOT_PASSWORD: ${MARIADB_ROOT_PASSWORD:-root}
|
||||
MARIADB_DATABASE: ${MARIADB_DATABASE:-databisnisid}
|
||||
MARIADB_USER: ${MARIADB_USER:-databisnis}
|
||||
MARIADB_PASSWORD: ${MARIADB_PASSWORD:-databisnis}
|
||||
ports:
|
||||
- "127.0.0.1:3306:3306"
|
||||
volumes:
|
||||
- mariadb_data:/var/lib/mysql
|
||||
healthcheck:
|
||||
test: ["CMD", "healthcheck.sh", "--connect", "--innodb_initialized"]
|
||||
interval: 5s
|
||||
timeout: 5s
|
||||
retries: 12
|
||||
|
||||
volumes:
|
||||
mariadb_data:
|
||||
+77
-6
@@ -1,22 +1,93 @@
|
||||
# Stack Docker Swarm — deploy via `docker stack deploy -c docker-compose.yml databisnisid`
|
||||
#
|
||||
# 3 service: mariadb (storage, internal), web (dashboard gunicorn, publik :5000),
|
||||
# scan (skan get_voters.py pada SCAN_HOURS, waktu lokal TZ).
|
||||
#
|
||||
# Catatan swarm:
|
||||
# - `build:` hanya untuk `docker compose build`; `docker stack deploy` memakai `image:`
|
||||
# (harus dibangun lebih dulu; multi-node perlu registry).
|
||||
# - `env_file` ⊥ didukung stack → environment inline dgn interpolasi `.env`.
|
||||
# - Tanpa `container_name`/`restart:` — pakai `deploy.restart_policy`.
|
||||
|
||||
services:
|
||||
mariadb:
|
||||
image: mariadb:11
|
||||
container_name: databisnisid-mariadb
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
MARIADB_ROOT_PASSWORD: ${MARIADB_ROOT_PASSWORD:-root}
|
||||
MARIADB_DATABASE: ${MARIADB_DATABASE:-databisnisid}
|
||||
MARIADB_USER: ${MARIADB_USER:-databisnis}
|
||||
MARIADB_PASSWORD: ${MARIADB_PASSWORD:-databisnis}
|
||||
ports:
|
||||
- "127.0.0.1:3306:3306"
|
||||
volumes:
|
||||
- mariadb_data:/var/lib/mysql
|
||||
- /data/db/mariadb/databisnisid/data:/var/lib/mysql
|
||||
networks:
|
||||
- appnet
|
||||
healthcheck:
|
||||
test: ["CMD", "healthcheck.sh", "--connect", "--innodb_initialized"]
|
||||
interval: 5s
|
||||
timeout: 5s
|
||||
retries: 12
|
||||
deploy:
|
||||
replicas: 1
|
||||
placement:
|
||||
constraints:
|
||||
- node.hostname == server5.saltis.id
|
||||
restart_policy:
|
||||
condition: any
|
||||
|
||||
web:
|
||||
image: git.proit.id/proitlab/databisnisid-web:latest
|
||||
environment:
|
||||
VEX_DB_BACKEND: mysql
|
||||
VEX_DB_HOST: mariadb
|
||||
VEX_DB_PORT: "3306"
|
||||
VEX_DB_USER: ${MARIADB_USER:-databisnis}
|
||||
VEX_DB_PASS: ${MARIADB_PASSWORD:-databisnis}
|
||||
VEX_DB_NAME: ${MARIADB_DATABASE:-databisnisid}
|
||||
DASH_HOST: "0.0.0.0" # wajib di container → gunicorn bind terbuka utk ingress
|
||||
DASH_PORT: "5000"
|
||||
DASH_WORKERS: "2"
|
||||
DATABISNIS_API: ${DATABISNIS_API:-https://api.databisnis.id}
|
||||
DASH_LIQUID_TTL: "60"
|
||||
ports:
|
||||
- published: 5000
|
||||
target: 5000
|
||||
networks:
|
||||
- appnet
|
||||
healthcheck:
|
||||
test: ["CMD", "python", "-c", "import socket,sys;sys.exit(0 if socket.create_connection(('127.0.0.1',5000),5) else 1)"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
start_period: 20s
|
||||
deploy:
|
||||
replicas: 1
|
||||
restart_policy:
|
||||
condition: any
|
||||
|
||||
scan:
|
||||
image: git.proit.id/proitlab/databisnisid-scan:latest
|
||||
command: ["python", "scan_loop.py"]
|
||||
environment:
|
||||
VEX_DB_BACKEND: mysql
|
||||
VEX_DB_HOST: mariadb
|
||||
VEX_DB_PORT: "3306"
|
||||
VEX_DB_USER: ${MARIADB_USER:-databisnis}
|
||||
VEX_DB_PASS: ${MARIADB_PASSWORD:-databisnis}
|
||||
VEX_DB_NAME: ${MARIADB_DATABASE:-databisnisid}
|
||||
TZ: ${TZ:-Asia/Jakarta}
|
||||
SCAN_HOURS: ${SCAN_HOURS:-0,8,16}
|
||||
SCAN_RUN_ON_START: ${SCAN_RUN_ON_START:-1}
|
||||
networks:
|
||||
- appnet
|
||||
deploy:
|
||||
replicas: 1
|
||||
restart_policy:
|
||||
condition: any
|
||||
|
||||
networks:
|
||||
appnet:
|
||||
driver: overlay
|
||||
|
||||
# volume khusus stack — jangan pakai mariadb_data (milik docker-compose.dev.yml)
|
||||
volumes:
|
||||
mariadb_data:
|
||||
db_data:
|
||||
@@ -0,0 +1,83 @@
|
||||
"""Jadwal skan berkala: jalankan get_voters.py pada jam SCAN_HOURS (waktu lokal).
|
||||
|
||||
Menunggu hingga batas jam berikutnya dalam daftar `SCAN_HOURS` (default 0,8,16)
|
||||
lalu memanggil `get_voters.main()`. Loop tak pernah keluar — service dijaga
|
||||
hidup oleh restart_policy swarm. Gagal skan dicatat lalu lanjut ke batas
|
||||
berikutnya (tidak membunuh loop). `TZ` env menentukan jam lokal via
|
||||
`datetime.now()` (libc); image Dockerfile.scan sudah menyertakan tzdata.
|
||||
|
||||
`SCAN_RUN_ON_START` (default 1): skan sekali saat container start setelah
|
||||
basis data siap — penting agar tabel voters langsung ada (dashboard ⊥ 500
|
||||
sebelum batas jam pertama).
|
||||
"""
|
||||
|
||||
import os
|
||||
import time
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
import db
|
||||
import get_voters
|
||||
|
||||
|
||||
def parse_hours(raw):
|
||||
"""Daftar jam integer dari string CSV, diurutkan naik."""
|
||||
hours = sorted({int(h) for h in raw.split(',') if h.strip()})
|
||||
if not hours or not all(0 <= h <= 23 for h in hours):
|
||||
raise ValueError(f'SCAN_HOURS tak valid: {raw!r}')
|
||||
return hours
|
||||
|
||||
|
||||
def next_boundary(now, hours):
|
||||
"""datetime berikutnya pada jam-jam `hours` yang lebih besar dari `now`."""
|
||||
for h in hours:
|
||||
cand = now.replace(hour=h, minute=0, second=0, microsecond=0)
|
||||
if cand > now:
|
||||
return cand
|
||||
# semua jam hari ini sudah lewat → besok jam pertama
|
||||
nxt = now + timedelta(days=1)
|
||||
return nxt.replace(hour=hours[0], minute=0, second=0, microsecond=0)
|
||||
|
||||
|
||||
def _wait_db(timeout=120):
|
||||
"""Tunggu basis data siap (mariadb baru boot) sebelum skan awal."""
|
||||
start = time.monotonic()
|
||||
while time.monotonic() - start < timeout:
|
||||
try:
|
||||
db.queryone('SELECT 1')
|
||||
return
|
||||
except Exception:
|
||||
time.sleep(3)
|
||||
raise RuntimeError('basis data tak siap setelah menunggu')
|
||||
|
||||
|
||||
def _run_scan(label):
|
||||
print(f'[{label}] memulai skan...', flush=True)
|
||||
try:
|
||||
rc = get_voters.main()
|
||||
except Exception as exc:
|
||||
print(f'[Error] Skan gagal: {exc}', flush=True)
|
||||
return
|
||||
if rc != 0:
|
||||
print(f'[Peringatan] get_voters keluar dengan kode {rc}', flush=True)
|
||||
|
||||
|
||||
def main():
|
||||
hours = parse_hours(os.getenv('SCAN_HOURS', '0,8,16'))
|
||||
run_on_start = os.getenv('SCAN_RUN_ON_START', '1').lower() not in ('0', 'false', 'no')
|
||||
print(f'Jadwal skan: jam {", ".join(str(h) for h in hours)} (waktu lokal); '
|
||||
f'skan-awal: {"ya" if run_on_start else "tidak"}')
|
||||
if run_on_start:
|
||||
_wait_db()
|
||||
_run_scan('skan awal')
|
||||
while True:
|
||||
now = datetime.now()
|
||||
target = next_boundary(now, hours)
|
||||
wait = (target - now).total_seconds()
|
||||
print(f'{now.isoformat(timespec="seconds")} tidur {wait / 3600:.2f} jam '
|
||||
f'sampai {target.isoformat(timespec="seconds")}', flush=True)
|
||||
time.sleep(max(wait, 0))
|
||||
_run_scan('skan terjadwal')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
Reference in new issue
Block a user