dedysutanto
c550fe0684
- CI: build/validate use backone selftest + binary names, add lint.yml - make-*.mk: emit/install backone binaries, drop zerotier-one paths - doc/debian/ext.installfiles: backone manpages, service, init, te - rule-compiler: npm test runs test.js; attic world planet files - AGENTS.md repository guidelines, tools/lint.sh changed-lines gate - gitignore: /backone-selftest, *.gcno, *.gcda
118 lines
11 KiB
Markdown
118 lines
11 KiB
Markdown
# Repository Guidelines
|
||
|
||
## Project Overview
|
||
|
||
BackOne — fork of ZeroTier 1.14.1: peer-to-peer SDN daemon. Builds virtual Ethernet (TUN/TAP), tunnels L2 frames over encrypted UDP mesh, issues membership certs from an embedded controller. One fat binary `backone` doubles as CLI (`backone-cli`) and identity tool (`backone-idtool`) via `argv[0]` dispatch (`one.cpp:2140-2142`). Local JSON API on `127.0.0.1:9993`.
|
||
|
||
## Architecture & Data Flow
|
||
|
||
Four layers, top → bottom:
|
||
|
||
1. **Entry** — `one.cpp`: `main()` → `cli()` / `idtool()` / daemon → `OneService::newInstance()` + `run()`.
|
||
2. **Service** — `service/OneService.cpp` (`OneServiceImpl`, line 776): owns sockets (`Phy<>`), httplib control plane, TAP devices, `Node`, `EmbeddedNetworkController`, main loop.
|
||
3. **Core** — `node/`: OS-independent switch. Reached only through the C API (`ZT_Node_*` in `include/ZeroTierOne.h`, implemented at bottom of `node/Node.cpp`).
|
||
4. **OS glue** — `osdep/`: `Phy` (select()-based reactor, the only event loop), `EthernetTap` (per-OS TUN/TAP), `OSUtils`, `ManagedRoute`, `Thread`, `BlockingQueue`.
|
||
|
||
**Core pattern = inversion of control.** `OneServiceImpl` fills `struct ZT_Node_Callbacks` (`service/OneService.cpp:1045-1054`) with `Snode*` static functions; `node/` never touches OS code, it calls back. Shared context is `RuntimeEnvironment` passed as `const RuntimeEnvironment *RR` to every `node/` function (`node/RuntimeEnvironment.hpp`).
|
||
|
||
Flows:
|
||
|
||
- **Wire → app:** `Phy::poll` → `phyOnDatagram` (`service/OneService.cpp:2940`) → `Node::processWirePacket` (`node/Node.cpp:206`) → `Switch::onRemotePacket` → verb dispatch `switch` (`node/IncomingPacket.cpp:94-151`) → peer/network/crypto. Heavy work offloaded to `PacketMultiplexer` worker threads.
|
||
- **App → wire:** TAP handler → `tapFrameHandler` (`service/OneService.cpp:3776`) → `Node::processVirtualNetworkFrame` → `Switch::onLocalEthernet` → `Peer`/`Path` → `_phy.udpSend`.
|
||
- **Controller:** `IncomingPacket::_doNETWORK_CONFIG_REQUEST` → `EmbeddedNetworkController::request` posts `_RQEntry*` to `BlockingQueue` → `hardware_concurrency()` workers → `DBMirrorSet`/`DB` → signed cert back via `NetworkController::Sender`.
|
||
- **Main loop** (`OneServiceImpl::run`, `service/OneService.cpp:~1150-1310`): single-threaded, time-sliced; refresh binds → `processBackgroundTasks` when due → `_phy.poll(delay)`. `Phy::whack()` (self-pipe) is the only cross-thread-safe Phy call.
|
||
|
||
## Key Directories
|
||
|
||
| Path | Purpose |
|
||
|---|---|
|
||
| `one.cpp` | Daemon/CLI/idtool entry; privilege drop, daemonize, signals |
|
||
| `node/` | Overlay engine: `Switch`, `Topology`, `Peer`, `IncomingPacket`, `Identity`, `Bond`, `Metrics` — OS-independent by rule |
|
||
| `service/` | `OneService` (whole runtime), `SoftwareUpdater` |
|
||
| `controller/` | `EmbeddedNetworkController` + `DB` backends (FileDB default, LFDB, PostgreSQL, Redis) |
|
||
| `osdep/` | All OS-dependent code: `Phy`, `EthernetTap*`, `Binder`, `ManagedRoute`, netlink/DNS helpers |
|
||
| `include/` | Public C API (`ZeroTierOne.h`, 57KB) — changes ripple to Java/libzt/SDK |
|
||
| `ext/` | Vendored: nlohmann/json, cpp-httplib, prometheus-cpp-lite, libpqxx, redis++, miniupnpc, ASM crypto |
|
||
| `rustybits/` | Cargo workspace: `zeroidc` (SSO FFI), `smeeclient` (PostgreSQL workflow client) |
|
||
| `java/` | JNI wrapper (`java/jni/…Node.cpp`, ant build) |
|
||
| `tcp-proxy/` | Standalone TCP fallback relay; own Makefile (C++11) |
|
||
| `rule-compiler/` | JS network-rules compiler (`node cli.js <rules>`), npm |
|
||
| `pkg/`, `debian/`, `windows/` | Packaging (snap/QNAP/Synology/ASUSTOR/WD, RPM `backone.spec`, MSVC sln) |
|
||
|
||
## Development Commands
|
||
|
||
```sh
|
||
make # = make one → ./backone + backone-cli/backone-idtool symlinks
|
||
make -j$(nproc) one
|
||
make core # libbackonecore.a
|
||
make selftest && ./backone-selftest
|
||
make debug # ZT_DEBUG=1 (adds -g, forces ZT_TRACE=1)
|
||
make ZT_SANITIZE=1 one # ASan
|
||
make central-controller # ZT_CONTROLLER=1 (needs ext/ libpqxx + hiredis + redis++)
|
||
make install DESTDIR=/tmp/root
|
||
make debian | make redhat # debuild / rpmbuild
|
||
make manpages # cd doc && ./build.sh (needs ronn or node marked-man)
|
||
cd rustybits && cargo build # zeroidc/smeeclient (also auto via `make zeroidc`)
|
||
cd java && ant build_java | build_android | build_jar
|
||
cd tcp-proxy && make
|
||
```
|
||
|
||
`Makefile` is a uname dispatcher → `make-linux.mk` / `make-mac.mk` / `make-bsd.mk` / `make-netbsd.mk`. Windows: `MSBuild windows/ZeroTierOne.sln /property:Configuration=Release`. Top-level `CMakeLists.txt` is a 323B stub — never `cmake .` as product build.
|
||
|
||
Env knobs (make vars): `ZT_DEBUG`, `ZT_TRACE`, `ZT_SANITIZE`, `ZT_STATIC`, `ZT_OFFICIAL`, `ZT_CONTROLLER`, `ZT_SSO_SUPPORTED` (0 on Linux default → no cargo needed), `ZT_VAULT_SUPPORT`, `ZT_IA32`.
|
||
|
||
## Code Conventions & Common Patterns
|
||
|
||
- **Standard: C++17** (`-std=c++17` in all makefiles; `tcp-proxy` is C++11). Ignore stale comments claiming otherwise (`node/README.md` "No C++11", `osdep/BlockingQueue.hpp:30`).
|
||
- **Formatting**: `.clang-format` — LLVM base, **tabs always**, indent 4, Stroustrup braces, `BinPackArguments: false`, `PointerAlignment: Left`. Gate: `make lint` → `tools/lint.sh` (clang-format on lines changed vs base commit; `--all` checks whole files — whole repo has format debt, don't reformat untouched lines). No clang-tidy, no `-Werror`.
|
||
- **Namespace**: everything in `namespace ZeroTier { … }`, close with `} // namespace ZeroTier`. File-local helpers in anonymous namespace or `static`.
|
||
- **Naming**: classes `PascalCase`; methods `camelCase`; private members `_camelCase`; static C callbacks `S` + class (`SnodeWirePacketSendFunction`); macros/enums `ZT_UPPER_SNAKE`; include guards `ZT_<NAME>_HPP` (no `#pragma once`).
|
||
- **Includes**: quoted relative (`"../node/Constants.hpp"` from subdirs, `"node/Constants.hpp"` from root). Third-party via angle brackets resolved by `-isystem ext`. `node/Constants.hpp` FIRST in `node/` files — canonicalizes `__LINUX__`/`__APPLE__`/`__UNIX_LIKE__`/`__WINDOWS__`; never test raw `__linux__`/`_WIN32`.
|
||
- **Error handling**: three regimes — (1) C API boundary returns `ZT_ResultCode`, fatal = 100–999, checked via `ZT_ResultCode_isFatal`; (2) `node/` throws bare ints (`ZT_EXCEPTION_OUT_OF_BOUNDS`, `node/Constants.hpp:757+`), caught as `catch (int e)` in `OneServiceImpl::run`; (3) service/controller use `std::exception`/`catch (...)` at thread boundaries. **No exceptions across the C API.**
|
||
- **Threading**: no global lock; per-object `ZeroTier::Mutex` scoped guard (`Mutex::Lock _l(_m);`) or `std::mutex` + `lock_guard`/`shared_lock` in controller. Cross-thread wake only via `Phy::whack()`; producer/consumer via `BlockingQueue<T>`. Threads: `std::thread`.
|
||
- **Async**: single-threaded reactor (`Phy::poll` callbacks), not futures; no `std::async`/`future` in C++ paths (tokio only inside `rustybits/smeeclient`).
|
||
- **Memory**: raw `new`/`delete` in service/controller; intrusive `SharedPtr<T>` (`node/SharedPtr.hpp` — class needs `friend class SharedPtr<X>` + `AtomicCounter __refCount`) for `Peer`/`Network`/`Path`; `std::shared_ptr` for `DB` backends. Zero secrets with `Utils::burn`.
|
||
- **DI/state**: no framework. Two seams only: `ZT_Node_Callbacks` function-pointer struct (core ↔ OS) and `const RuntimeEnvironment *RR` as first param. Controller injected via `Node::setNetconfMaster`.
|
||
- **JSON**: `nlohmann::json` + typed accessors `OSUtils::jsonString/jsonInt/jsonBool` (defaults); API is type-sensitive.
|
||
- **Metrics**: `node/Metrics.hpp` namespace, increment inline (`Metrics::udp_recv += len;`).
|
||
- **License header**: every new `.cpp/.hpp/.h` gets the 11-line BSL block verbatim (copy `node/Mutex.hpp:1-12`).
|
||
- **Placement rule**: sockets/files/interfaces → `osdep/`; orchestration → `service/`; persistence → `controller/`; `node/` stays OS-independent. New compiled file must be added to `objects.mk` (`CORE_OBJS` or `ONE_OBJS`) or it won't link.
|
||
|
||
## Important Files
|
||
|
||
Read before editing:
|
||
|
||
1. `include/ZeroTierOne.h` — public contract (`ZT_Node_Callbacks:1733`, `ZT_ResultCode:375`); ripples to Java/SDK.
|
||
2. `node/Constants.hpp` — platform macros, exception ints, constants; include first.
|
||
3. `node/RuntimeEnvironment.hpp` — object graph every `node/` fn navigates.
|
||
4. `service/OneService.cpp` lines 690–1100 (callback table) and 1150–1400 (`run()` loop) — the integration seam.
|
||
5. `osdep/Phy.hpp` lines 55–160 — handler contract for any socket behavior.
|
||
6. `objects.mk` + `make-linux.mk:12,62-73,370-457` — what compiles with which flags.
|
||
7. Per-dir READMEs: `service/README.md` (local.conf schema + JSON API — primary ops doc), `controller/README.md`, `node/README.md`.
|
||
|
||
Module-specific: wire protocol → `node/IncomingPacket.cpp:94-151` + `node/Packet.hpp`; routing → `node/Switch.cpp`; DB backend → `controller/DB.hpp` + `EmbeddedNetworkController.cpp:465-580`; TAP → `osdep/EthernetTap.cpp`.
|
||
|
||
## Runtime/Tooling Preferences
|
||
|
||
- **Build**: GNU make (g++ or clang auto-detected, `make-linux.mk:3-10`); MSVC on Windows; ant + NDK for Java; cargo for `rustybits/`; npm for `rule-compiler/`.
|
||
- **No package manager for C++** — deps vendored in `ext/` (do not add new ones casually).
|
||
- **Daemon paths (Linux)**: home `/var/lib/backone` (macOS: `/Library/Application Support/BackOne`); port 9993; key files `identity.secret`, `authtoken.secret`, `networks.d/`, `local.conf`, `controller.db`. Unprivileged CLI: `cp authtoken.secret ~/.backOneOneAuthToken` (source: `one.cpp:283`).
|
||
- **CLI**: `backone-cli info|listpeers|listnetworks|join <nwid>|leave <nwid>` (`-j` JSON, `-p<port>`, `-D<dir>`); `backone-idtool generate|validate|getpublic|sign|verify|mkcom`.
|
||
- **Dangerous scripts**: `update_controllers.sh`, `cycle_controllers.sh` are live ZeroTier-Central kubectl ops — never run locally.
|
||
- **Rebrand is partial**: build outputs, CI (`build.yml`, `validate-linux.sh`), `Dockerfile.ci`, `make-bsd.mk`/`make-netbsd.mk`, and debian units now use `backone*`. Still upstream-named: `pkg/snap/snapcraft.yaml` (builds `github.com/zerotier/zerotierone.git` → broken) and cosmetic strings (`/etc/zerotier-version` in `Dockerfile.ci`, `windows/zerotier-cli.bat`, `pkg/` vendor paths). Expect occasional `zerotier-*` names in docs/scripts.
|
||
|
||
## Testing & QA
|
||
|
||
No C++ unit framework. Practical loop:
|
||
|
||
```sh
|
||
make selftest && ./backone-selftest # crypto KATs, identity, certs, packet codec, Phy loopback; non-zero exit on failure
|
||
make debug # ZT_DEBUG=1 build of one + selftest
|
||
```
|
||
|
||
- **Sections** in `selftest.cpp`: `testCrypto`, `testIdentity`, `testCertificate`, `testPacket`, `testOther`, `testPhy` (binds 127.0.0.1 UDP/TCP loopback; no root needed).
|
||
- **CI** (`.github/workflows/`): `build.yml` = build smoke (`make`, `make selftest`); `validate.yml` = `make one ZT_COVERAGE=1 ZT_TRACE=1` then `.github/workflows/validate-linux.sh` (two-node netns + ping + iperf3 + valgrind) and `validate-report.sh` (fails on any definite leak or non-zero test exit — the only hard gate). `ZT_COVERAGE=1` adds `--coverage` flags (`make-linux.mk:322-326`); coverage recorded via gcovr, never thresholded.
|
||
- **Lint/format**: `make lint` → `tools/lint.sh` (clang-format on changed lines vs base; CI: `.github/workflows/lint.yml`, clang-format pinned 23.1.1). No clang-tidy, no `-Werror`. Do not reformat untouched lines.
|
||
- **Gaps**: no Rust tests, no Java tests (orphaned `java/test/` deleted), no `make test`/`check` target. Prefer extending `selftest.cpp` for regression coverage; `rule-compiler` has `npm test` (real, `rule-compiler/test.js`).
|
||
- **Windows smoke** (`windows/README.md`): run exe `-p9994 -C <tmpdir>` then `zerotier-cli.bat -p9994 -D<tmpdir> info|join`.
|