feat: GET /api/getWhitelist (walletList) + WalletAdmin; verifier grace no longer resurrects false rows

This commit is contained in:
proitlab committed 2026-08-26 18:41:57 +07:00
1 parent 7ae6b8f4f8
commit b00bd4f740
7 files changed
+83 -8

No files matched your search

+3 -3
View File
@@ -4,9 +4,9 @@ Project conventions and operational gotchas for agents working in this repo.
## Layout ## Layout
- `app/main.py` — FastAPI app: public endpoints `GET /api/getTokenInfo`, `GET /api/version/android`, `GET /api/getDapps`, `POST /api/increaseDappCounter` (header `X-Api-Key` = env `IDRS_SECRET`, fail-closed 401 if unset/wrong), `GET /img/token/<filename>`, `GET /img/dapps/<filename>` + `/public/img/...` aliases (seed iconUrl path); mounts `/static`; startup calls `db.init_db()`. - `app/main.py` — FastAPI app: public endpoints `GET /api/getTokenInfo`, `GET /api/version/android`, `GET /api/getDapps`, `POST /api/increaseDappCounter` (header `X-Api-Key` = env `IDRS_SECRET`, fail-closed 401 if unset/wrong), `GET /img/token/<filename>`, `GET /img/dapps/<filename>` + `/public/img/...` aliases (seed iconUrl path); mounts `/static`; startup calls `db.init_db()`.
- `app/admin.py` — sqladmin panel: `TokenAdmin` (custom WTForms form, `on_model_change`), `VersionAdmin`, `DappAdmin` (URL field + `imageFile` upload, auto `created_at`/`updated_at`, `id` not writable), `create_admin(app)`. Upload icons saved under `app/static/icons/token/` with `iconUrl = {IDRS_PUBLIC_URL}/img/token/<filename>`; dapp uploads under `icons/dapps/`. - `app/admin.py` — sqladmin panel: `TokenAdmin` (custom WTForms form, `on_model_change`), `VersionAdmin`, `DappAdmin` (URL field + `imageFile` upload, auto `created_at`/`updated_at`, `id` not writable), `WalletAdmin` (unique account, dup rejected, auto `created_at`, `id` not writable), `create_admin(app)`. Upload icons saved under `app/static/icons/token/` with `iconUrl = {IDRS_PUBLIC_URL}/img/token/<filename>`; dapp uploads under `icons/dapps/`.
- `app/models.py` — SQLAlchemy `Token`/`Version`/`Dapp`, engine, `SessionLocal`, `ICON_DIR`, `DB_PATH` (env `IDRS_DB_PATH`). - `app/models.py` — SQLAlchemy `Token`/`Version`/`Dapp`/`Wallet`, engine, `SessionLocal`, `ICON_DIR`, `DB_PATH` (env `IDRS_DB_PATH`).
- `app/db.py` — seed from `tokenList.json`/`versionAndroid.json`/`dapps.json` (once, empty DB only). Dapp seed mirrors each `image` into `icons/dapps/` and rewrites to `{IDRS_PUBLIC_URL}/img/dapps/<file>` (gate: `image/*` only, sniff `octet-stream`, `text/html`→skip; on failure keep original URL; collision → `-N` suffix). - `app/db.py` — seed from `tokenList.json`/`versionAndroid.json`/`dapps.json` (once, empty DB only). Dapp seed mirrors each `image` into `icons/dapps/` and rewrites to `{IDRS_PUBLIC_URL}/img/dapps/<file>` (gate: `image/*` only, sniff `octet-stream`, `text/html`→skip; on failure keep original URL; collision → `-N` suffix). Also seeds first two whitelisted wallets (`susukudaliar`, `vexessential`) when the wallets table is empty.
- `app/verify.py` — icon/link-reachability verifiers: daemon threads started on app startup, gated by env `IDRS_VERIFY_TOKENS` (token `iconUrl`, absent/0/false/off → disabled, default) and `IDRS_VERIFY_DAPPS` (dapp `link`, **enabled by default**; `0/false/off` disables). When enabled, `is_live` = reachability (local file stat under `icons/token/`, or external HTTP HEAD/GET 2xx; dapp link strict 2xx, whitespace-stripped); empty → false; flips after 2 consecutive failed passes, auto-recovers; interval env `IDRS_VERIFY_INTERVAL` (default 600s). When disabled, `is_live` is admin-owned (editable form field). `IDRS_VERIFY_TOKENS` only. Dapp admin-set `is_live=false` is sticky via `is_live_override` (verifier skips overridden dapps). External hosts must be reachable from the server or those tokens/dapps flip false while enabled. - `app/verify.py` — icon/link-reachability verifiers: daemon threads started on app startup, gated by env `IDRS_VERIFY_TOKENS` (token `iconUrl`, absent/0/false/off → disabled, default) and `IDRS_VERIFY_DAPPS` (dapp `link`, **enabled by default**; `0/false/off` disables). When enabled, `is_live` = reachability (local file stat under `icons/token/`, or external HTTP HEAD/GET 2xx; dapp link strict 2xx, whitespace-stripped); empty → false; flips after 2 consecutive failed passes, auto-recovers; interval env `IDRS_VERIFY_INTERVAL` (default 600s). When disabled, `is_live` is admin-owned (editable form field). `IDRS_VERIFY_TOKENS` only. Dapp admin-set `is_live=false` is sticky via `is_live_override` (verifier skips overridden dapps). External hosts must be reachable from the server or those tokens/dapps flip false while enabled.
- `app/auth.py` — `AdminAuthBackend`, fail-closed creds. - `app/auth.py` — `AdminAuthBackend`, fail-closed creds.
- `tokenList.json` / `versionAndroid.json` / `dapps.json` — seed source (read-only after seed). - `tokenList.json` / `versionAndroid.json` / `dapps.json` — seed source (read-only after seed).
+5 -1
View File
@@ -17,6 +17,7 @@ FastAPI + SQLite server mirroring idrs-api live endpoints — `GET /api/getToken
- dapps: `Dapp` table seeded once from repo-root `dapps.json` (empty DB only); at seed, mirror each dapp `image` to `icons/dapps/` + rewrite `image` = `{IDRS_PUBLIC_URL}/img/dapps/<file>`; download gate: accept `image/*` only (sniff magic for `octet-stream`; `text/html`/other → skip), filename = sanitized URL basename stem + content-type ext, collision → `-N` suffix; download/validation failure → keep original URL, continue (seed never hard-fails); admin panel can upload a new image (any `image/*`, ≤2MB) → rewrites URL, or edit URL directly - dapps: `Dapp` table seeded once from repo-root `dapps.json` (empty DB only); at seed, mirror each dapp `image` to `icons/dapps/` + rewrite `image` = `{IDRS_PUBLIC_URL}/img/dapps/<file>`; download gate: accept `image/*` only (sniff magic for `octet-stream`; `text/html`/other → skip), filename = sanitized URL basename stem + content-type ext, collision → `-N` suffix; download/validation failure → keep original URL, continue (seed never hard-fails); admin panel can upload a new image (any `image/*`, ≤2MB) → rewrites URL, or edit URL directly
- dapp `is_new_version`: non-nullable Boolean, default false; admin-editable (form + details); surfaced in `/api/getDapps` as `is_new_version:bool`; existing DBs migrated additively via startup `ALTER TABLE dapps ADD COLUMN is_new_version BOOLEAN NOT NULL DEFAULT 0` guarded by column-existence check (⊥ data loss; all existing rows → false); ⊥ reseed - dapp `is_new_version`: non-nullable Boolean, default false; admin-editable (form + details); surfaced in `/api/getDapps` as `is_new_version:bool`; existing DBs migrated additively via startup `ALTER TABLE dapps ADD COLUMN is_new_version BOOLEAN NOT NULL DEFAULT 0` guarded by column-existence check (⊥ data loss; all existing rows → false); ⊥ reseed
- dapp counter: public POST endpoint increments `counter` by 1; **auth-gated** by shared secret env `IDRS_SECRET` sent as `X-Api-Key` header (constant-time compare; unset → fail-closed 401); `id` query param; atomic `UPDATE ... SET counter=counter+1`; unknown id → 404 - dapp counter: public POST endpoint increments `counter` by 1; **auth-gated** by shared secret env `IDRS_SECRET` sent as `X-Api-Key` header (constant-time compare; unset → fail-closed 401); `id` query param; atomic `UPDATE ... SET counter=counter+1`; unknown id → 404
- whitelist: `Wallet` table (account unique), admin-managed via panel (`WalletAdmin`); seeded once on empty DB with first entries `susukudaliar`, `vexessential`; public endpoint returns account strings
- out of scope: multi-user table, roles, 2FA, password reset, on-chain calls, audit trail, change history - out of scope: multi-user table, roles, 2FA, password reset, on-chain calls, audit trail, change history
- data survival: existing `idrs.db` file + schema kept; startup runs idempotent additive `ALTER TABLE dapps ADD COLUMN is_live_override BOOLEAN NOT NULL DEFAULT 0` (guarded by `PRAGMA table_info`) — columns added, no data rewritten; ⊥ reseed if DB non-empty - data survival: existing `idrs.db` file + schema kept; startup runs idempotent additive `ALTER TABLE dapps ADD COLUMN is_live_override BOOLEAN NOT NULL DEFAULT 0` (guarded by `PRAGMA table_info`) — columns added, no data rewritten; ⊥ reseed if DB non-empty
- run: `./venv/bin/uvicorn app.main:app --host 127.0.0.1 --port 8000` + `run.sh` wrapper - run: `./venv/bin/uvicorn app.main:app --host 127.0.0.1 --port 8000` + `run.sh` wrapper
@@ -27,6 +28,7 @@ api: GET `/api/getTokenInfo` → 200 `{tokenList:[{idfToken:int, position:int, t
api: GET `/api/version/android` → 200 `{idVersion:int, appName:string, acceptableVersion:int, latestVersion:int}` api: GET `/api/version/android` → 200 `{idVersion:int, appName:string, acceptableVersion:int, latestVersion:int}`
api: GET `/api/getDapps` → 200 `{dappList:[{id:int, name, description, link, image, is_live:bool, counter:int, category_id:int|null, is_available_indonesia:bool|null, is_new_version:bool, created_at, updated_at}]}` (id ASC) api: GET `/api/getDapps` → 200 `{dappList:[{id:int, name, description, link, image, is_live:bool, counter:int, category_id:int|null, is_available_indonesia:bool|null, is_new_version:bool, created_at, updated_at}]}` (id ASC)
api: POST `/api/increaseDappCounter?id=<int>` header `X-Api-Key: <IDRS_SECRET>` → 200 `{id:int, counter:int}` | 401 bad/missing secret (or secret unset) | 404 unknown id api: POST `/api/increaseDappCounter?id=<int>` header `X-Api-Key: <IDRS_SECRET>` → 200 `{id:int, counter:int}` | 401 bad/missing secret (or secret unset) | 404 unknown id
api: GET `/api/getWhitelist` → 200 `{walletList:[string, ...]}` (account names, insertion order)
api: GET `/img/dapps/<filename>` + `/public/img/dapps/<filename>` → 200 image (mime from file) | 404 unknown (traversal-guarded) api: GET `/img/dapps/<filename>` + `/public/img/dapps/<filename>` → 200 image (mime from file) | 404 unknown (traversal-guarded)
api: GET `/img/<filename>` → 200 image/png | 404 unknown api: GET `/img/<filename>` → 200 image/png | 404 unknown
web: GET `/admin` → sqladmin panel (login-gated; login page when unauthenticated) web: GET `/admin` → sqladmin panel (login-gated; login page when unauthenticated)
@@ -55,8 +57,9 @@ V15: token edit/create form upload → `on_model_change` sets `filename` + `icon
V16: `idfToken` auto-increment only — not in form, never written from form data (⊥ injectable); `typeBlockchain` fixed `"Vexanium"` — not in form, always forced on write (⊥ injectable); `position` editable, default = next idfToken value V16: `idfToken` auto-increment only — not in form, never written from form data (⊥ injectable); `typeBlockchain` fixed `"Vexanium"` — not in form, always forced on write (⊥ injectable); `position` editable, default = next idfToken value
V17: `is_live` admin-owned by default (editable in form, writes honored). Optional verifier — runs only when `IDRS_VERIFY_TOKENS` truthy (absent/0/false/off → off): when enabled, verifier overwrites token `is_live` each pass (live iff iconUrl reachable; empty → false; flips after 2 consecutive failed passes, recovers next success) V17: `is_live` admin-owned by default (editable in form, writes honored). Optional verifier — runs only when `IDRS_VERIFY_TOKENS` truthy (absent/0/false/off → off): when enabled, verifier overwrites token `is_live` each pass (live iff iconUrl reachable; empty → false; flips after 2 consecutive failed passes, recovers next success)
V18: `Dapp` seeded from `dapps.json` (empty DB only); seed-mirrored image URLs = `{IDRS_PUBLIC_URL}/img/dapps/<file>`; download gate accepts `image/*` only (sniff `octet-stream`; `text/html` → keep original URL); admin-uploaded image = any `image/*` ≤2MB (magic-sniffed), rewrites URL; `id` never writable from form (⊥ injectable); `created_at` set on create, `updated_at` on every change V18: `Dapp` seeded from `dapps.json` (empty DB only); seed-mirrored image URLs = `{IDRS_PUBLIC_URL}/img/dapps/<file>`; download gate accepts `image/*` only (sniff `octet-stream`; `text/html` → keep original URL); admin-uploaded image = any `image/*` ≤2MB (magic-sniffed), rewrites URL; `id` never writable from form (⊥ injectable); `created_at` set on create, `updated_at` on every change
V19: dapp-link verifier — **enabled by default**; disabled when `IDRS_VERIFY_DAPPS` is `0/false/off`: dapp live iff `link` reachable (whitespace-stripped; empty → false; HTTP HEAD/GET **strict 2xx**, `Accept: text/html`, 3s timeout, follow redirects); flips after 2 consecutive failed passes, recovers next success; **admin-set `is_live=false` is sticky: dapp sets `is_live_override=True`, and the verifier never flips an overridden dapp (in either direction)** V19: dapp-link verifier — **enabled by default**; disabled when `IDRS_VERIFY_DAPPS` is `0/false/off`: dapp live iff `link` reachable (whitespace-stripped; empty → false; HTTP HEAD/GET **strict 2xx**, `Accept: text/html`, 3s timeout, follow redirects); grace (2 consecutive failed passes) only delays a **live→false** flip — a row already `false` stays false until its link verifies reachable (⊥ grace-resurrection after restart); recovers next success; **admin-set `is_live=false` is sticky: dapp sets `is_live_override=True`, and the verifier never flips an overridden dapp (in either direction)**
V20: dapp counter increment — POST only, `id` required, secret-gated (`IDRS_SECRET`, `X-Api-Key` header, constant-time compare; unset → always 401); `+1` atomic update; returns new counter; unknown id → 404; ⊥ decrement/reset via this endpoint V20: dapp counter increment — POST only, `id` required, secret-gated (`IDRS_SECRET`, `X-Api-Key` header, constant-time compare; unset → always 401); `+1` atomic update; returns new counter; unknown id → 404; ⊥ decrement/reset via this endpoint
V21: whitelist — `Wallet.account` unique (⊥ duplicate add via panel or seed); endpoint returns plain account strings only (⊥ leaks ids/timestamps); public ⊥ auth
V21: `Dapp.is_new_version` — non-nullable Boolean, default false; surfaced as `is_new_version:bool` in `/api/getDapps`; admin-editable (form + `column_details_list`); existing DBs auto-migrated via additive startup `ALTER TABLE dapps ADD COLUMN is_new_version BOOLEAN NOT NULL DEFAULT 0` guarded by column-existence check (mirrors `is_live_override`; all pre-existing rows → false); ⊥ reseed V21: `Dapp.is_new_version` — non-nullable Boolean, default false; surfaced as `is_new_version:bool` in `/api/getDapps`; admin-editable (form + `column_details_list`); existing DBs auto-migrated via additive startup `ALTER TABLE dapps ADD COLUMN is_new_version BOOLEAN NOT NULL DEFAULT 0` guarded by column-existence check (mirrors `is_live_override`; all pre-existing rows → false); ⊥ reseed
## §T — Tasks ## §T — Tasks
@@ -90,6 +93,7 @@ T26|x|`Dapp` table + seed from `dapps.json` (mirror images to `icons/dapps/`, re
T27|x|dapp-link verifier (`DappVerifier` in `app/verify.py`): strict 2xx on `link`, whitespace-stripped, 2-pass grace + recover, **enabled by default** (`IDRS_VERIFY_DAPPS=0/false/off` disables; admin-owned then), skips dapps with `is_live_override=True` (sticky admin-off); tokens opt-in via `IDRS_VERIFY_TOKENS`; verifier refactored to base class + `TokenVerifier`/`DappVerifier`; env `IDRS_VERIFY_ENABLED` renamed → `IDRS_VERIFY_TOKENS`|V17,V19 T27|x|dapp-link verifier (`DappVerifier` in `app/verify.py`): strict 2xx on `link`, whitespace-stripped, 2-pass grace + recover, **enabled by default** (`IDRS_VERIFY_DAPPS=0/false/off` disables; admin-owned then), skips dapps with `is_live_override=True` (sticky admin-off); tokens opt-in via `IDRS_VERIFY_TOKENS`; verifier refactored to base class + `TokenVerifier`/`DappVerifier`; env `IDRS_VERIFY_ENABLED` renamed → `IDRS_VERIFY_TOKENS`|V17,V19
T28|x|`is_live_override` column on `Dapp` (additive startup ALTER TABLE); `DappAdmin.on_model_change` sets it `True` on admin-set `false` (create, or edit only when `is_live` changes), never injected; verifier skips overridden dapps|V19 T28|x|`is_live_override` column on `Dapp` (additive startup ALTER TABLE); `DappAdmin.on_model_change` sets it `True` on admin-set `false` (create, or edit only when `is_live` changes), never injected; verifier skips overridden dapps|V19
T29|x|`POST /api/increaseDappCounter?id=<int>` (header `X-Api-Key` = `IDRS_SECRET`, fail-closed, constant-time; atomic `counter+1`; 200/401/404); `db.increment_dapp_counter` with SQLite `RETURNING`|V20 T29|x|`POST /api/increaseDappCounter?id=<int>` (header `X-Api-Key` = `IDRS_SECRET`, fail-closed, constant-time; atomic `counter+1`; 200/401/404); `db.increment_dapp_counter` with SQLite `RETURNING`|V20
T30|x|`Wallet` table + `WalletAdmin` (unique account, dup rejected, auto created_at, id ⊥ injectable) + seed first entries (`susukudaliar`, `vexessential`, empty-DB-only); `GET /api/getWhitelist` → `{walletList:[...]}`|V21
T30|x|add `is_new_version` Boolean column to `Dapp` model (default False, nullable=False) + additive startup `ALTER TABLE dapps ADD COLUMN is_new_version BOOLEAN NOT NULL DEFAULT 0` guarded by column-existence check in `db._ensure_migrations` (mirrors `is_live_override` handling)|V21,§C T30|x|add `is_new_version` Boolean column to `Dapp` model (default False, nullable=False) + additive startup `ALTER TABLE dapps ADD COLUMN is_new_version BOOLEAN NOT NULL DEFAULT 0` guarded by column-existence check in `db._ensure_migrations` (mirrors `is_live_override` handling)|V21,§C
T31|x|expose `is_new_version:bool` in `/api/getDapps` (`main.py` `get_dapps` returns `d.is_new_version`) + `DappForm` `BooleanField` + `DappAdmin` `column_list`/`column_details_list`; seeded rows default false via DB default (dapps.json has no such key)|V21,§I T31|x|expose `is_new_version:bool` in `/api/getDapps` (`main.py` `get_dapps` returns `d.is_new_version`) + `DappForm` `BooleanField` + `DappAdmin` `column_list`/`column_details_list`; seeded rows default false via DB default (dapps.json has no such key)|V21,§I
+33 -1
View File
@@ -12,7 +12,7 @@ from wtforms.validators import DataRequired
from sqlalchemy import text from sqlalchemy import text
from app.auth import AdminAuthBackend, get_session_secret from app.auth import AdminAuthBackend, get_session_secret
from app.models import ICON_DIR, SessionLocal, Dapp, Token, Version from app.models import ICON_DIR, SessionLocal, Dapp, Token, Version, Wallet
BASE_DIR = Path(__file__).resolve().parent.parent BASE_DIR = Path(__file__).resolve().parent.parent
TEMPLATES_DIR = str(BASE_DIR / "app" / "templates") TEMPLATES_DIR = str(BASE_DIR / "app" / "templates")
@@ -284,6 +284,37 @@ class VersionAdmin(ModelView, model=Version):
raise Exception("acceptableVersion must be <= latestVersion") raise Exception("acceptableVersion must be <= latestVersion")
class WalletForm(Form):
account = StringField("Account", validators=[DataRequired()])
class WalletAdmin(ModelView, model=Wallet):
name = "Whitelisted Wallets"
icon = "fa-solid fa-wallet"
form = WalletForm
column_list = ["account", "created_at"]
column_details_list = ["id", "account", "created_at"]
column_searchable_list = ["account"]
column_default_sort = [("id", True)]
page_size = 25
def is_accessible(self, request: Request) -> bool:
return _is_authenticated(request)
async def on_model_change(self, data, model, is_created, request):
data.pop("id", None)
account = (data.get("account") or "").strip()
if not account:
raise Exception("Account is required")
with SessionLocal() as session:
exists = session.query(Wallet).filter(Wallet.account == account).first()
if exists and (is_created or exists.id != model.id):
raise Exception(f"'{account}' is already whitelisted")
data["account"] = account
if is_created:
data["created_at"] = _now_iso()
def create_admin(app): def create_admin(app):
auth_backend = AdminAuthBackend(get_session_secret()) auth_backend = AdminAuthBackend(get_session_secret())
admin = Admin( admin = Admin(
@@ -297,4 +328,5 @@ def create_admin(app):
admin.add_view(TokenAdmin) admin.add_view(TokenAdmin)
admin.add_view(VersionAdmin) admin.add_view(VersionAdmin)
admin.add_view(DappAdmin) admin.add_view(DappAdmin)
admin.add_view(WalletAdmin)
return admin return admin
+27 -1
View File
@@ -13,6 +13,7 @@ from app.models import (
Dapp, Dapp,
Token, Token,
Version, Version,
Wallet,
Base, Base,
engine, engine,
) )
@@ -105,11 +106,28 @@ def seed():
) )
) )
seed_dapps(session) seed_dapps(session)
seed_wallets(session)
session.commit() session.commit()
finally: finally:
session.close() session.close()
SEED_WALLETS = ["susukudaliar", "vexessential"]
def _utc_now_iso() -> str:
from datetime import datetime, timezone
return datetime.now(timezone.utc).isoformat(timespec="seconds")
def seed_wallets(session):
if session.query(Wallet).count() != 0:
return
for account in SEED_WALLETS:
session.add(Wallet(account=account, created_at=_utc_now_iso()))
def _image_extension(content_type: str, url: str) -> str: def _image_extension(content_type: str, url: str) -> str:
ext = IMAGE_EXT.get((content_type or "").split(";")[0].strip().lower()) ext = IMAGE_EXT.get((content_type or "").split(";")[0].strip().lower())
if ext: if ext:
@@ -224,4 +242,12 @@ def increment_dapp_counter(dapp_id: int):
{"dapp_id": dapp_id}, {"dapp_id": dapp_id},
).fetchone() ).fetchone()
session.commit() session.commit()
return row[0] if row else None return row[0] if row else None
def list_wallets():
session = SessionLocal()
try:
return [w.account for w in session.query(Wallet).order_by(Wallet.id.asc()).all()]
finally:
session.close()
+5
View File
@@ -126,6 +126,11 @@ def increase_dapp_counter(id: int = Query(...), x_api_key: str = Header(default=
return {"id": id, "counter": new_counter} return {"id": id, "counter": new_counter}
@app.get("/api/getWhitelist")
def get_whitelist():
return {"walletList": db.list_wallets()}
@app.on_event("startup") @app.on_event("startup")
def startup(): def startup():
db.init_db() db.init_db()
+9 -1
View File
@@ -68,4 +68,12 @@ class Dapp(Base):
created_at = Column("created_at", String, nullable=True) created_at = Column("created_at", String, nullable=True)
updated_at = Column("updated_at", String, nullable=True) updated_at = Column("updated_at", String, nullable=True)
is_live_override = Column("is_live_override", Boolean, nullable=False, default=False) is_live_override = Column("is_live_override", Boolean, nullable=False, default=False)
imageFile = None imageFile = None
class Wallet(Base):
__tablename__ = "wallets"
id = Column(Integer, primary_key=True)
account = Column("account", String, unique=True, nullable=False)
created_at = Column("created_at", String, nullable=True)
+1 -1
View File
@@ -78,7 +78,7 @@ class Verifier(threading.Thread):
else: else:
fails = self._fail_counts.get(key, 0) + 1 fails = self._fail_counts.get(key, 0) + 1
self._fail_counts[key] = fails self._fail_counts[key] = fails
live = fails < GRACE_PASSES live = fails < GRACE_PASSES and bool(item.is_live)
if bool(item.is_live) != live: if bool(item.is_live) != live:
item.is_live = live item.is_live = live
logger.info("%s is_live -> %s (%s)", key, live, self._label(item)) logger.info("%s is_live -> %s (%s)", key, live, self._label(item))