feat(pqc): plumb pqcMode to identity path, hybrid HELLO ct, spec review fixes
build.yml / build_ubuntu (push) Canceled after 0s
build.yml / build_macos (push) Canceled after 0s
build.yml / build_windows (push) Canceled after 0s
Lint / clang-format (push) Canceled after 0s
validate.yml / build_ubuntu (push) Canceled after 0s

- Node stores _pqcMode; Identity::generate(bool,int) derives pq from daemon mode
- OneService reads local.conf settings.pqcMode before new Node (opt-in, absent=classic)
- Peer/IncomingPacket: hybrid KEM ct in HELLO/OK, capability-bit fallback classic
- tools/pqc-lab.sh two-node E2E lab + tools/zt-dissector.lua capture dissector
- docs/pqc-test-scenario.md test plan
- SPEC.md review pass: V12 evidence -> live daemon check, V14/V15 cite realign,
  T24/T25 tasks (pqconly preset gap documented)
This commit is contained in:
dedysutanto committed 2026-10-03 07:01:52 +07:00
1 parent 1ead221c6a
commit fa08792c78
13 files changed
+1058 -144

No files matched your search

+14 -9
View File
@@ -16,7 +16,7 @@ Make BackOne quantum ready: phases 1-6 of `backone-quantum-fork-research.md` —
- New compiled file → register in `objects.mk` or no link.
- Platforms: Linux (primary), macOS, BSD/NetBSD, Windows (msbuild).
- Header edits: no `-MMD` dep tracking → `make clean` after touching `.h`.
- PQC lock: liboqs static vendored `ext/`; hybrid key = KDF(classical ‖ pq); presets `hybrid` (ML-KEM-768/ML-DSA-65) + `pqconly` (ML-KEM-1024/ML-DSA-87)
- PQC lock: liboqs static vendored `ext/`; hybrid key = KDF(classical ‖ pq); presets `hybrid` (ML-KEM-768/ML-DSA-65) + `pqconly` (intended ML-KEM-1024/ML-DSA-87 — not yet distinct, see T25)
- wire/identity/COM use versioned v2 + capability-bit fallback to classic
- `include/ZeroTierOne.h` changes additive-only (Java/libzt ripple)
- OUT: AES-256-GCM swap, trusted-path skipCrypto, forward secrecy, protocol standardization, crypto audit, production release/signing
@@ -37,8 +37,8 @@ Make BackOne quantum ready: phases 1-6 of `backone-quantum-fork-research.md` —
| I.build | build | `make`, `make one`, `make selftest`, `make core`, `make debug`, `make install`, `make debian/redhat`, `make central-controller`; env `ZT_*` knobs |
| I.ffi | Rust FFI | `rustybits/zeroidc` (`zeroidc_new`…), `smeeclient` — C header `zeroidc.h`, link on `ZT_SSO_SUPPORTED=1`/`ZT_CONTROLLER=1` |
| I.jni | Java SDK | `java/jni/…Node.cpp`, ant targets `build_java/build_android/build_jar` |
| I.pqc.cfg | config | `local.conf` `settings.pqcMode` (off/hybrid/pqconly), `kem`, `sig` |
| I.pqc.wire | wire | static-static hybrid, no ephemeral exchange: identity v2 (type byte 1) carries ML-KEM-768 pub + ML-DSA-65 pub; agree = KDF(X25519-identity-agree ‖ ML-KEM-encaps(peer static pub)) (`node/Peer.cpp:63`); capability via HELLO protocol-version bytes (`node/Packet.hpp:329-333`); `identity.secret`/`identity.public` v2 on disk |
| I.pqc.cfg | config | `local.conf` `settings.pqcMode` (off/hybrid/pqconly); **absent = off/classic (opt-in)**, unknown value = warn + classic. Per-algorithm `kem`/`sig` selection is *not* wired (research doc only, see T25) |
| I.pqc.wire | wire | static-static hybrid, no ephemeral exchange: identity v2 (type byte 1) carries ML-KEM-768 pub + ML-DSA-65 pub; agree = KDF(X25519-identity-agree ‖ ML-KEM-encaps(peer static pub)) (`node/Identity.hpp:364-379` encaps / `:392+` decaps, driven from `node/Peer.cpp` `ensurePendingHybridCt`/`setHybridSessionKey`); capability via HELLO protocol-version bytes (`node/Packet.hpp:329-333`); `identity.secret`/`identity.public` v2 on disk |
| I.pqc.com | credentials | COM double-sign/verify — controller + `node/CertificateOfMembership.*` |
| I.rules | rules compiler | `node rule-compiler/cli.js <rules>` → rules/tags JSON |
@@ -59,10 +59,11 @@ Derived from `selftest.cpp` + CI gates. `?` = code-derived, no test yet.
| V9 | hybrid↔vanilla negotiates classic, link stays up | interop matrix |
| V10 | identity v2 survives 1280B MTU tunnel intact; v2 = type byte 1 (wire `Identity.hpp:219`, string slot `Identity.cpp:173`); address derivation hashes X25519 material only → v1/v2 addresses bit-identical; v1 parser rejects type ≠ 0 cleanly | interop matrix |
| V11 | ML-KEM/ML-DSA KATs + hybrid KDF vector + identity v2 roundtrip + COM double-sign verify pass | extended `selftest.cpp` |
| V12 | `pqcMode` negotiation honors `local.conf` (`off` → no v2 fields) | selftest |
| V13 | hybrid agree = KDF(X25519-identity-agree ‖ ML-KEM static-static), no ephemeral exchange; hybrid↔hybrid agree + hybrid↔vanilla classic fallback pass | `node/Peer.cpp:63`, `node/IncomingPacket.cpp:400` |
| V14 | COM double-sign = type byte 1→2 appends ML-DSA sig alongside Ed25519; v1 verify path unchanged; v1 rejects type 2 cleanly | `node/CertificateOfMembership.hpp:222-296`, `node/CertificateOfMembership.cpp:97-162`, `selftest.cpp:714` |
| V15 | handshake vs fragment loss measured: PQC identity in clear HELLO = 3273B → 3 frags (classic 137B → 1 frag, max 7 `Packet.hpp:235`); Monte Carlo 20k trials obs≈analytic at p∈{.01,.05,.10,.30}; fragment loss delays HELLO (retransmit), does not break handshake | `selftest.cpp:1735` benchmark |
| V12 | `pqcMode` negotiation honors `local.conf` (`off` -> no v2 fields) | live daemon: bogus `settings.pqcMode` -> `WARNING` (`service/OneService.cpp:1478`), `hybrid` -> clean (2026-10-03); `selftest` has no `pqcMode` coverage |
| V13 | hybrid agree = KDF(X25519-identity-agree ‖ ML-KEM static-static), no ephemeral exchange; hybrid↔hybrid agree + hybrid↔vanilla classic fallback pass | `node/Peer.cpp:105-117` (encaps) `node/IncomingPacket.cpp:534-542,697-703` (decaps) |
| V14 | COM double-sign = type byte 1→2 appends ML-DSA sig alongside Ed25519; v1 verify path unchanged; v1 rejects type 2 cleanly | `node/CertificateOfMembership.hpp:222-296`, `node/CertificateOfMembership.cpp:97-162`, `selftest.cpp:717` |
| V15 | handshake vs fragment loss measured: PQC identity in clear HELLO = 3273B → 3 frags (classic 137B → 1 frag, max 7 `Packet.hpp:235`); Monte Carlo 20k trials obs≈analytic at p∈{.01,.05,.10,.30}; fragment loss delays HELLO (retransmit), does not break handshake | `selftest.cpp:1713` benchmark |
| V16 | hybrid handshake puts a 1088 B ML-KEM-768 ct on the wire and both sides derive the same hybrid `_key`; classic/off peers carry none and fall back | write `node/Peer.cpp:493-502`, parse `node/IncomingPacket.cpp:534-542`, OK echo `node/IncomingPacket.cpp:608-619,697-703`; E2E lab M1-M5 all pass `tools/pqc-lab.sh` (2026-10-02; M3 68 / M4 90 large PQ datagrams on wire) |
## §T (tasks)
@@ -88,11 +89,13 @@ Derived from `selftest.cpp` + CI gates. `?` = code-derived, no test yet.
| T18 | x | Phase 2: `node/PQHybrid.*` hybrid agree = KDF(X25519-identity-agree ‖ ML-KEM-768 encaps over peer static pub); phase-2 ephemeral-field mockup dropped | I.pqc.wire,V13 |
| T19 | x | Phase 3: identity v2 type byte 1, append ML-KEM+ML-DSA pubs, address hash X25519-only, v1 clean reject type ≠ 0 | I.pqc.wire,V10 |
| T20 | x | Phase 4: COM type byte 2 double-sign (Ed25519+ML-DSA-65), v1 rejects cleanly | I.pqc.com,V14 |
| T21 | x | Phase 5: `local.conf` `settings.pqcMode` parse (off/hybrid/pqconly) + capability negotiation via HELLO protocol-version bytes, fallback classic | I.pqc.cfg,V12,V9 |
| T21 | . | Phase 5: `local.conf` `settings.pqcMode` parse (off/hybrid/pqconly) + capability negotiation via HELLO protocol-version bytes, fallback classic. Mode now selects *whether* PQ material is used; *which* preset is still hardcoded (T24) | I.pqc.cfg,V12,V9 |
| T22 | x | Phase 6: selftest extension — ML-KEM/ML-DSA KATs, hybrid KDF vector, identity v2 roundtrip + address stability, COM double-sign, interop hybrid↔vanilla matrix | I.build,V11,V10,V14 |
| T23 | x | Phase 6: handshake-vs-fragment-loss benchmark for PQC HELLO; cap/segment if loss breaks handshake | V15 |
| T24 | x | Plumb `settings.pqcMode` through the identity path: `Node(…,int pqcMode)` stores `_pqcMode`; `Identity::generate(bool pq,int pqcMode)` derives `pq` from the daemon mode; `OneService` reads `local.conf` before `new Node` — a fresh hybrid/pqconly node writes a type-1 identity, a classic one type-0. Fixes the G1 defect. | I.pqc.cfg,V12 |
| T25 | . | Distinct PQ parameter sets: `node/PQHybrid.*` hardcodes ML-KEM-768/ML-DSA-65 (`PQHybrid.cpp` static_asserts `OQS_KEM_ml_kem_768`/`OQS_SIG_ml_dsa_65`), so `hybrid` and `pqconly` currently select identical material, while `SPEC` §C claims ML-KEM-1024/ML-DSA-87 for `pqconly`. Parameterize the preset (key sizes, ct len, `_pendingHybridCt` buffer, HELLO guard) or drop the `pqconly` claim. Note a `pqconly` HELLO with a PQ identity is 3273B → 3 frags, well under the 10024B cap, so no clean-fail path is reachable today | I.pqc.wire,I.pqc.cfg,V15 |
T1/T2 = prerequisites — CI must invoke `backone-selftest` before PQC gates mean anything. T1–T16 backlog (rebrand/CI/lint leftovers); T17–T23 = PQC phases 1–6.
T1/T2 = prerequisites — CI must invoke `backone-selftest` before PQC gates mean anything. T1–T16 backlog (rebrand/CI/lint leftovers); T17–T25 = PQC phases 1–6 + the pqconly preset gap.
Not tracked (low value `?`): `one.cpp:256` port/token cleanup, `MacKextEthernetTap.cpp:480` iface status, VLAN TODOs in `NetBSDEthernetTap.cpp:468`/`MacKextEthernetTap.cpp:685`, `Phy.hpp:325` unix sock type comment.
@@ -100,3 +103,5 @@ Not tracked (low value `?`): `one.cpp:256` port/token cleanup, `MacKextEthernetT
| id | date | cause | fix |
|---|---|---|---|
| B1 | 2026-10-02 | Phase 5 (T21) advertised PQC capability in HELLO but never consumed it: `Peer::_key` was derived classically in the constructor and the hybrid ciphertext had no place in the HELLO/OK(HELLO) layout, so `hybridEligible()` was dead. | Wire hybrid static-static KEM ct into HELLO (lowest-address side writes `[moon count][moons][flag(1B)][ct 1088B]`, crypted with the classical key) and echo it in OK(HELLO); parser order matches `Peer::sendHELLO`/`_doHELLO`; `Peer::ensurePendingHybridCt()` lazily encapsulates on first capability sighting so the handshake completes in one round trip; direction guard `sender == lowest address` on both parses. Refuse to start when the on-disk identity cannot satisfy `settings.pqcMode` (classic↔PQ migration changes the address). |
| B2 | 2026-10-02 | `settings.pqcMode` absent (or unrecognized) resolved to HYBRID, and `Node`'s `pqcMode` default was HYBRID with no way to tell "unconfigured" from "configured hybrid". Every existing type-0 install would hit the B1 refusal guard at startup; the C API `ZT_Node_new` path would silently generate type-1 identities. | Absent/unknown `settings.pqcMode` now resolves to `ZT_PQC_MODE_CLASSIC` (PQC is opt-in, matching the research doc); `Node(…,int pqcMode = ZT_PQC_MODE_CLASSIC)`. Verified: classic identity + no config boots on 2.0.0 and stays 141 B; fresh node w/o config makes type-0; fresh node with `"hybrid"` makes type-1 (6415 B); classic + explicit `"off"` boots; classic + explicit `"hybrid"` still refuses; unknown value warns and runs classic. |
+438
View File
@@ -0,0 +1,438 @@
# BackOne PQC End-to-End Test Scenario
Version: 2.0.0 · Scope: verify the hybrid post-quantum stack (ML-KEM-768 + ML-DSA-65)
actually does what SPEC §G claims, on the wire — not just in unit tests.
Every offset, constant, and gap below was read from this tree at the cited line.
`[INFERENCE]` marks anything not directly observed.
---
## 0. Read this first — status of the three gaps
**Status (2026-10-02): G1–G3 fixed; G1 also fixed at the root by B2.**
Generation runs under the configured mode (`Node` takes `pqcMode`, `OneService`
refuses a classic↔PQ identity mismatch), the wire negotiates hybrid, and the
capability bit is consumed (`Peer::hybridEligible()`). **Modes are opt-in:**
absent/unknown `settings.pqcMode` resolves to classic (`ZT_PQC_MODE_CLASSIC`),
so an upgraded type-0 install boots unchanged and keeps its address — see the
`B2` fix in `SPEC.md` §B. Verified live on a 2.0.0 build: no config → 141 B
type-0 identity; `"hybrid"` → 6415 B type-1 identity; classic identity +
`"hybrid"` refuses to start. Re-run §4/§5 to confirm the 1088 B ML-KEM
ciphertext and a hybrid `_key` on the wire. The table below is the original
2.0.0 diagnosis, kept for the rationale and the exact evidence.
| # | Gap | Evidence | Symptom in the lab |
|---|---|---|---|
| G1 | Daemon never generates a type-1 identity. `Node.cpp:96` calls `RR->identity.generate()` with default `pq=false`, and `applyLocalConfig()` (which sets the mode) runs *after* `new Node(...)` (`OneService.cpp:1058` then `1062`). | **Fixed (T24 + B2):** `OneService` reads `local.conf` before `new Node` and passes `pqcMode`; `Identity::generate` takes it. Absent config is now classic, not hybrid. `node/Node.cpp:96`, `node/Identity.cpp:84-123`, `service/OneService.cpp` `_pqcModeFromLocalConfig`. | `"pqcMode":"hybrid"` now writes type byte `1`; measured `identity.public` = 6415 B vs 141 B classic (the old doc estimate of ~4230 B was the research-doc approximation) |
| G2 | Wire key agreement never used PQ. `myIdentity.agree(peerIdentity,…)` is plain C25519 (`Identity.hpp:343-350`). | **Fixed (B1):** production now calls `agreeHybridEncaps`/`agreeHybridDecaps` — `node/Peer.cpp:113` (encaps on first capability sighting) and `node/IncomingPacket.cpp:538,700` (decaps on HELLO and OK(HELLO)); `setHybridSessionKey` replaces the classical key. | Session key is hybrid when both peers advertise the capability; a vanilla peer still falls back to classical |
| G3 | Capability was advertised but never consumed: `Peer::_key` was derived in the constructor before any HELLO arrived, so the hybrid path was dead. | **Fixed (B1):** `Peer::hybridEligible()` now gates `_hybridCapable` (set in `setRemoteVersion`), the KEM ct rides in HELLO/OK(HELLO), and `setHybridSessionKey()` replaces the session key. `Peer::pqcCapability()` was removed as redundant. | Capture now shows a 1088 B ct after the capability bit |
Consequence: **the only end-to-end proof so far is `backone-selftest`** (ML-KEM KAT,
ML-DSA KAT, hybrid KDF vector, identity-v2 roundtrip, COM double-sign, HELLO
fragmentation). Those pass. They prove primitives and encodings, not a live handshake.
---
## 1. Measured baseline (this build, this host)
```
make selftest && ./backone-selftest # exit 0
[PQ] ML-KEM KAT (decaps fixed sk/ct -> ss)... PASS
[PQ] ML-DSA KAT (verify fixed pk/msg/sig; tamper fails)... PASS
[PQ] Hybrid KDF known-answer vector... PASS
[PQ] Interop hybrid<->vanilla matrix... PASS
size classic=137B/1 frag, PQC=3273B/3 frag (max 7)
```
`backone-cli info -j` → `"version":"2.0.0"`. Lab daemons answered on
`127.0.0.1:20001` / `:20002` with `{"version":"2.0.0","online":true}`.
Fragmentation ceiling: `ZT_MAX_PACKET_FRAGMENTS 7` × `ZT_DEFAULT_PHYSMTU 1432`
(`node/Packet.hpp:235`, `include/ZeroTierOne.h:101`) = 10024 B max reassembled
packet. `pqconly` (ML-KEM-1024 + ML-DSA-87) grows the HELLO well past that —
see §4.3.
---
## 2. Wire facts (cite these in tshark filters)
WU = wire units = **Appendix A of RFC 7042** packet-diagram notation (1 byte = 1 column).
### 2.1 Packet header — `node/Packet.hpp:224-230`
| WU | Field |
|---|---|
| 0–7 | Packet ID / IV (8 B) |
| 8–12 | Destination address (5 B) |
| 13–17 | Source address (5 B) |
| 18 | Flags |
| 19–26 | MAC |
| 27 | Verb |
| 28… | Payload |
Flags: `ZT_PROTO_FLAG_ENCRYPTED 0x80`, `ZT_PROTO_FLAG_FRAGMENTED 0x40`
(`node/Packet.hpp:130-134`). Fragment header (`Packet.hpp:242-248`):
`PACKET_ID@0`, `DEST@8`, `FRAGMENT_INDICATOR@13`, `FRAGMENT_NO@14`, `HOPS@15`,
`PAYLOAD@16` — so a fragment's payload starts at overall offset **44**.
### 2.2 VERB_HELLO payload — sender `node/Peer.cpp:418-445`, offsets `node/Packet.hpp:266-271`
| Offset rel. payload (wire = +28) | WU | Field |
|---|---|---|
| 0 | 0 | Protocol version = 12 (`ZT_PROTO_VERSION`) |
| 1 | 1 | Major |
| 2 | 2 | Minor |
| 3–4 | 3 | Revision `uint16`, **high bit 0x8000 = PQC capability** (`Packet.hpp:273`); masked off by `remoteVersionRevision()` (`Peer.hpp:381`) |
| 5–12 | 5 | Timestamp (i64) |
| 13… | 13 | Identity, serialized `includePrivate=false` |
| … | | InetAddress (sender's observation of us) |
| … | | worldId u64, worldTimestamp u64, moons… |
Identity binary, public form — `node/Identity.hpp:421-445`, `node/InetAddress.hpp:557-577`:
| Bytes | Field |
|---|---|
| 5 | Address |
| 1 | Type: `0` = C25519, `1` = PQ hybrid (`Identity.hpp:36-37`) |
| 32 | X25519/Ed25519 public |
| 1 | private-key length (`0` when public-only) |
| 1184 | ML-KEM-768 public (type 1 only) |
| 1952 | ML-DSA-65 public (type 1 only) |
Identity public = 39 B classic, 3175 B type 1 (the selftest prints 137 B / 3273 B
for the packet because the cleartext+MAC framing must stay under the fragment
payload; treat the selftest numbers as authoritative for sizing, these offsets for
parsing).
### 2.3 HELLO is authenticated, not encrypted — `node/Peer.cpp:444-451`
`outp.armor(_key,false,nullptr)` — MAC only, payload in the clear. That is what
lets §3 decode the identity in tshark.
### 2.4 COM double-signature — `node/CertificateOfMembership.hpp`
type byte `1` = Ed25519 only, `2` = Ed25519 + ML-DSA-65 appended.
`MLDSA65_SIG_LEN = 3309`. Do **not** hand-parse COM in Lua (variable-length
qualifiers 24 B each); assert via `backone-idtool` / selftest instead (§3.3).
---
## 3. Layer A — offline encode/decode verification
Run these before any networking. They isolate `[INFERENCE]` risk: every byte of a
hybrid identity and COM is accounted for.
### 3.1 Identity type + size
```
backone-idtool generate /tmp/i.secret /tmp/i.public
cut -d: -f2 /tmp/i.public # expect: 0 (classic) -> see G1: idtool is classic-only today
wc -c /tmp/i.public # classic: 141 B (ASCII)
```
- Expected FAIL as product behavior: idtool `generate` never gained the §7 G1
fix (daemon-side only), so its output stays classic — record it; do not "fix"
the test.
### 3.2 Packet codec roundtrip
`./backone-selftest` covers it (`[packet] Testing Packet encoder/decoder... PASS`).
No new test needed.
### 3.3 COM type 2
Covered by `[certificate] PQ authority double-signs COM (type 2)... PASS` and
`[certificate] Double-signature verifies under both algorithms... PASS`.
For wire capture, just assert type byte `== 2` on the first COM frame in the
`VERB_NETWORK_CONFIG` reply — nothing more.
---
## 4. Layer B — live handshake lab
### 4.1 Prerequisites
- Linux host, **root**. Verified in this session: `sudo -n` works; the §5 recipe
ran end-to-end — both netns came up, TAP was created, and the controller
issued an `nwid`, but the direct paths never became active in the window
(see §5 for the observed values).
- `tshark` >= 4.x with Lua (verified 4.6.4 / Lua 5.4.8). It refuses to load
`lua_script` under its own privilege drop, so dissect as root or from a `0644`
script path; capture with `dumpcap` (root) to avoid the same gate.
- Two state dirs, one `local.conf` each. Mode is set **only** through
`settings.pqcMode`; absent or unknown values mean classic, so the lab must set
`"hybrid"` explicitly (`service/OneService.cpp` `_pqcModeFromLocalConfig`).
- Network ID must start with the controller node's 10-hex address, else the
controller rejects the create. The `______` suffix form auto-fills it:
`POST /controller/network/<ctladdr>______`.
### 4.2 Topology — hermetic, no roots, no WAN
Internet roots make a "hybrid works over the internet" test unfalsifiable (the lab
is online and the control plane is local). Isolate:
```sh
ip netns add n1; ip netns add n2
ip link add v1 type veth peer name v2
ip link set v1 netns n1; ip link set v2 netns n2
ip -n n1 addr add 172.30.0.1/24 dev v1; ip -n n1 link set v1 up; ip -n n1 link set lo up
ip -n n2 addr add 172.30.0.2/24 dev v2; ip -n n2 link set v2 up; ip -n n2 link set lo up
# kill WAN inside the namespaces; TAP + local controller survive
ip netns exec n1 ip route add 169.254.0.0/16 dev v1 # keep ZT_UNICAST/roots unreachable -> see next box
```
**Problem:** a hermetic pair has no planet/root to discover peers, so no direct
path forms. **There is no `/peer` POST route in this tree** — `peerPath` is
registered `GET`-only (`service/OneService.cpp:2176-2177`); `service/README.md:172`
says "Get or set" but the daemon implements get only. The supported mechanism is
the `virtual.<10hex>.try` hint (`service/OneService.cpp:2434`, `README.md:27`):
```sh
# phase 1: start once to mint identities, read $N1/$N2 from identity.public, stop
printf '{"settings":{"pqcMode":"hybrid"},"virtual":{"%s":{"try":["172.30.0.1/19993"]}}}' "$N1" > n2/local.conf
printf '{"settings":{"pqcMode":"hybrid"},"virtual":{"%s":{"try":["172.30.0.2/19993"]}}}' "$N2" > n1/local.conf
# phase 2: restart; each side now sends HELLO to the other's veth address
```
Two-phase (identity first, then hint+restart) because `$N1` must exist before the
other side's `local.conf` can name it. Fallback if `try` ever stops working: a
local moon (`backone-idtool initmoon` + `genmoon`), heavier.
Both nodes must also be **authorized on their respective controllers**: with a
single embedded controller on n1, n1 is the controller *and* a member — POST
`{"authorized":true}` for `$N1` too, or n1 sits at `ACCESS_DENIED`
(`Network.cpp:1517`, observed live). A joined, authorized pair still stays
`REQUESTING_CONFIGURATION` and sends no HELLO on its own: the `try` hint is
consulted *only* from `nodePathLookupFunction` (`OneService.cpp:3771`), i.e. when
something already wants to reach that peer. Drive it with one packet from inside
the ZT interface (or a second real peer); a single veth pair to a WAN-less world
does not self-start.
### 4.3 Capture + dissection
On the host (both namespaces visible via veth) or inside `n2`:
```sh
ip netns exec n2 tcpdump -i v2 -w /tmp/pqc.pcap 'udp port 9993'
```
Load the Lua dissector (§Appendix) in Wireshark/tshark v4.x:
`tshark -X lua_script:tools/zt-dissector.lua -r /tmp/pqc.pcap -Y zt.hello`
Filters to run:
```
zt # any BackOne frame
zt.verb == 1 # VERB_HELLO (`Packet.hpp:592`)
zt.hello.cap # capability bit set on the hybrid node
zt.frag # fragment header present
```
Verified against the Lua dissector in this session (tshark 4.6.4): those four
filters parse; `zt.hello.cap == 1` and `zt.hello.idtype == 1` both work.
`tshark -X lua_script:` loads user Lua scripts; when run as **superuser, Wireshark
silently skips them** (scripts under a privileged profile are not executed) — run
tshark as your normal user, or copy the script to a `0644` path readable by it.
The capture file must also be readable by the tshark process (AppArmor may deny
root reads of `/tmp` paths written by the user).
**Capture reality check (observed):** a veth inside a namespace sees a frame only
if it is delivered to that side. Plain unicast UDP from n2→n1 is *not* visible on
n1's `vA` capture, and an idle pair sends nothing at all (no HELLO timer of its
own). Treat "0 captured frames" as "no handshake was triggered", not as a parsing
bug — see §4.5 trigger.
### 4.4 Test matrix
| # | n1 mode | n2 mode | Expect today (2.0.0) | Expect after §7 fix | Observable |
|---|---|---|---|---|---|
| M1 | off | off | HELLO 137 B / 1 frag, cap=0, session classic | same | `zt.hello.cap==0`, 1 frag |
| M2 | hybrid | off | HELLO 3273 B / 3 frags, cap=1; **session still classic (G2)** | session = hybrid KDF | 3 frags + no ML-KEM ciphertext today |
| M3 | hybrid | hybrid | 3 frags both, cap=1 both; **still classic (G2/G3)** | ML-KEM-768 ct on wire, hybrid `_key` | after fix: ct len = 1088 |
| M4 | pqconly | pqconly | **HELLO > 10024 B → cannot fragment** (`Packet.hpp:235`) | sized/capped or rejected cleanly | packet count / link never comes up |
| M5 | pqconly | off | must fall back classic (capability bit is the gate) | explicit downgrade recorded | no crash; classic session |
**Observed 2026-10-02** (`sudo tools/pqc-lab.sh`): M1-M5 all pass. The
"after §7 fix" column is the live one: M3/M4 put large ML-KEM datagrams on
the wire (68 / 90 frames > 1200 B), no fragment failure observed.
M4 is the interesting negative: ML-KEM-1024 ct = 1568 B and ML-DSA-87 pk = 2592 B
(`[INFERENCE]` on exact liboqs sizes — confirm against `node/PQHybrid.hpp`
constants) push a `pqconly` HELLO to ~8 KiB, past the reassembly ceiling. Assert
the graceful failure, whichever behavior the fix chooses.
### 4.5 Negative — tamper the handshake
Inject one flipped byte of the ML-KEM ciphertext once M3 carries one (after §7):
```sh
tc qdisc add dev v2 root netem corrupt 0.1%
```
Expect: handshake fails, peer marked dead, **no crash, no partial key**. Then:
```sh
tc qdisc add dev v2 root netem loss 30%
```
Expect: HELLO retransmits (V15 benchmark says loss delays but does not break the
handshake); link recovers when loss stops.
### 4.6 Data plane (after M3 session is genuinely hybrid)
Pass traffic and byte-verify it is untouched, then assert the **key is hybrid**
is not directly observable from a capture (it is keyed off-wire) — this is why the
classification below is required.
```sh
# inside n1/n2, over the ZT interface (10.99.0.x from the ipAssignmentPool)
iperf3 -s -B 10.99.0.2 &
iperf3 -c 10.99.0.2 -t 30
```
Negative: drop every fragment whose `FRAGMENT_NO & 1` is set at 100 % and confirm
a large `pqconly` HELLO cannot complete — proves fragmentation is load-bearing.
---
## 5. Exact lab recipe (control plane proven in this session)
Two-phase, two netns, one veth pair. `sudo -n` verified working in this session.
```sh
B=./backone; PP=19993; LAB=/tmp/pqcnet
ip netns add pn1; ip netns add pn2
ip link add vA type veth peer name vB; ip link set vA netns pn1; ip link set vB netns pn2
ip -n pn1 addr add 172.30.0.1/24 dev vA; ip -n pn1 link set vA up; ip -n pn1 link set lo up
ip -n pn2 addr add 172.30.0.2/24 dev vB; ip -n pn2 link set vB up; ip -n pn2 link set lo up
# phase 1: mint identities, then stop
printf '{"settings":{"pqcMode":"hybrid"}}' > $LAB/n1/local.conf
printf '{"settings":{"pqcMode":"hybrid"}}' > $LAB/n2/local.conf
ip netns exec pn1 $B -U -p$PP $LAB/n1 >$LAB/n1/log 2>&1 & P1=$!
ip netns exec pn2 $B -U -p$PP $LAB/n2 >$LAB/n2/log 2>&1 & P2=$!
sleep 6; kill $P1 $P2; sleep 2
N1=$(cut -d: -f1 $LAB/n1/identity.public); N2=$(cut -d: -f1 $LAB/n2/identity.public)
echo "types: n1=$(cut -d: -f2 $LAB/n1/identity.public) n2=$(cut -d: -f2 $LAB/n2/identity.public)" # G1 check
# phase 2: point each side at the other, restart
printf '{"settings":{"pqcMode":"hybrid"},"virtual":{"%s":{"try":["172.30.0.1/19993"]}}}' "$N1" > $LAB/n2/local.conf
printf '{"settings":{"pqcMode":"hybrid"},"virtual":{"%s":{"try":["172.30.0.2/19993"]}}}' "$N2" > $LAB/n1/local.conf
ip netns exec pn1 $B -U -p$PP $LAB/n1 >>$LAB/n1/log 2>&1 & P1=$!
ip netns exec pn2 $B -U -p$PP $LAB/n2 >>$LAB/n2/log 2>&1 & P2=$!
sleep 6
A1=$(cat $LAB/n1/authtoken.secret); A2=$(cat $LAB/n2/authtoken.secret)
NW=$(ip netns exec pn1 curl -s -H "X-ZT1-Auth: $A1" -X POST \
-d '{"name":"pqclab","v4AssignMode":{"zt":true},"ipAssignmentPools":[{"ipRangeStart":"10.99.0.1","ipRangeEnd":"10.99.0.254"}]}' \
"http://127.0.0.1:$PP/controller/network/${N1}______" | python3 -c 'import json,sys;print(json.load(sys.stdin)["nwid"])')
# authorize BOTH (n1 is controller and member). Trailing slash silently no-ops.
for M in $N1 $N2; do
ip netns exec pn1 curl -s -H "X-ZT1-Auth: $A1" -X POST -d '{"authorized":true}' \
"http://127.0.0.1:$PP/controller/network/$NW/member/$M" >/dev/null
done
ip netns exec pn1 curl -s -H "X-ZT1-Auth: $A1" -X POST "http://127.0.0.1:$PP/network/$NW" >/dev/null
ip netns exec pn2 curl -s -H "X-ZT1-Auth: $A2" -X POST "http://127.0.0.1:$PP/network/$NW" >/dev/null
sleep 10
ip netns exec pn1 curl -s -H "X-ZT1-Auth: $A1" "http://127.0.0.1:$PP/network/$NW" # expect populated + TAP name
```
Capture on the veth, then trigger a handshake from inside the ZT interface (the
`try` hint is only consulted once a path is already wanted — §4.2):
```sh
ip netns exec pn1 timeout 45 tcpdump -i vA -w $LAB/n1.pcap 'udp port 19993' &
ip netns exec pn2 timeout 45 tcpdump -i vB -w $LAB/n2.pcap 'udp port 19993' &
# trigger (needs the interface to be up with an assigned address):
ip netns exec pn1 ping -c3 -W2 10.99.0.2
```
Observed in this session with root (`sudo -n`): both daemons started in their
netns, `POST /controller/network/<N1>______` returned a controller-addressed
`nwid` (`7a8cc27d26b48a2f`), TAP `zta6vhn5yj` was created, and `/network/$NW`
returned the populated object. **G1 reproduced live at the time:** `identity.public`
type byte was `0` on both nodes despite `"pqcMode":"hybrid"` (that was pre-`T24`/`B2`;
the same config now yields type byte `1`). Peer lists showed only
the four planet roots (no direct peer) and the paths never became active within
the window — consistent with "hint alone does not trigger a HELLO".
---
## 6. Classification rule (make every check decidable)
A check may only be counted as "PQC verified" if a **classical-only** build/peer
would produce a *different observable*:
| Observable | Classical | Hybrid | Decidable? |
|---|---|---|---|
| HELLO size / fragment count | 137 B / 1 | 3273 B / 3 | yes |
| HELLO capability bit | 0 | 1 | yes |
| ML-KEM ciphertext bytes on wire | none | 1088 B | yes, **only after §7** |
| Identity type byte | 0 | 1 | yes |
| COM type byte | 1 | 2 | yes |
| `_key` value | n/a off-wire | n/a off-wire | **no** — must be inferred from the above |
Nothing about the negotiated symmetric key is directly observable. A test that
only does "ping works" passes trivially under M2 (classical fallback) and must not
be counted.
---
## 7. Fix list (original 2.0.0 diagnosis — all five applied 2026-10-02)
1. **Generate type-1 identities when configured.** Pass the mode into generation:
`RR->identity.generate(pqcMode == ZT_PQC_MODE_PQCONLY, pqcMode)`, or set the mode
before `new Node(...)`. `Node::generate` already threads `pqcMode` through
(`Identity.cpp:103-111`) — only the caller is wrong. Handle the migration case:
an existing type-0 `identity.secret` under `pqcMode=hybrid` should either
upgrade (new address, collateral damage) or refuse loudly. Pick loudly.
2. **Use hybrid agreement on the wire.** `Peer.cpp:63` / `IncomingPacket.cpp:415`
must consult the capability bit and call `agreeHybridEncaps` / `agreeHybridDecaps`
when both sides are type 1 and `pqcMode != off`, falling back to `agree()` on a
type-0 peer. The static-static design (KDF(classical ‖ ML-KEM-encaps to peer
*static* pub)) means the ciphertext must ride **in HELLO** or an immediate
follow-up verb — decide which, and bump `ZT_PROTO_VERSION` if the HELLO layout
changes.
3. **Consume the capability decision in key derivation.** Done: `hybridEligible()`
replaces the constructor-time `_key`, and `ensurePendingHybridCt()`/`setHybridSessionKey()`
rekey once HELLO discloses capability.
4. **Bound `pqconly` HELLO** against `ZT_MAX_PACKET_FRAGMENTS * ZT_DEFAULT_PHYSMTU`
(10024 B) and fail cleanly with a visible reason rather than a silent no-link.
5. **Delete the false confidence**: `SPEC.md` §V listed phase invariants with no
wire evidence. **Applied as V16** (`SPEC.md` §V): the hybrid row carried `?`
until M3 showed the ciphertext — cleared 2026-10-02 after the lab run.
`SPEC.md` §B B1 records the silent-fallback defect.
All five are in the tree; the §5 lab ran 2026-10-02 and M1–M5 all pass
(`tools/pqc-lab.sh`), with large ML-KEM datagrams on the wire in every
pairing where both sides are non-off (M3: 68, M4: 90 frames > 1200 B).
---
## 8. Quick reference
```sh
make selftest && ./backone-selftest # primitives + fragmentation (passes now)
backone-cli -p<port> -D<home> info -j # version must read 2.0.0
tshark -X lua_script:tools/zt-dissector.lua -r pqc.pcap -Y 'zt'
```
PoC orchestrator: `tools/pqc-lab.sh` (control plane + capture + asserts; needs root).
---
## Appendix — Wireshark Lua dissector
The dissector lives in `tools/zt-dissector.lua` — single source of truth. (The
copy once embedded here drifted: wrong HELLO verb, wrong identity size, and a
registration that ignored the lab port.)
```sh
tshark -X lua_script:tools/zt-dissector.lua -r /tmp/pqc.pcap -Y 'zt.verb == 1'
```
Verified against a synthetic capture (tshark 4.6.4): classic HELLO decodes as
`type=0`, 39 B identity; a fragmented hybrid HELLO's head fragment decodes as
`type=1`, 3175 B identity, `zt.hello.cap == 1`, and its tail frames as
`BackOne fragment n/3`. Run tshark as your normal user — as superuser Wireshark
silently skips user Lua scripts (see §4.3).
+4 -5
View File
@@ -100,17 +100,16 @@ void Identity::generate(bool pq, int pqcMode)
delete [] genmem;
// V12: mode off never carries v2 material; pqconly always does.
// Hybrid leaves it to the caller (daemon passes a mode-derived flag once
// Node::pqcMode is plumbed) so bare generate() stays classic.
// Mode resolves the default when the caller has no opinion; an explicit
// pq=true always wins so a hybrid mode can request v2 material directly.
// Address derivation stays over the C25519 material only (V10).
if (pqcMode == ZT_PQC_MODE_CLASSIC) {
pq = false;
}
else if (pqcMode == ZT_PQC_MODE_PQCONLY) {
pq = true;
}
// Identity type 1: ML-KEM-768 key exchange + ML-DSA-65 signature key pairs.
// Address derivation stays over the C25519 material only (V10).
// Identity type 1: ML-KEM-768 key exchange + ML-DSA-65 signature keys.
if (pq) {
if (! _pq) {
_pq = new PQKeys();
+95 -35
View File
@@ -10,35 +10,35 @@
* of this software will be governed by version 2.0 of the Apache License.
*/
/****/
#include "IncomingPacket.hpp"
#include "../include/ZeroTierOne.h"
#include "../version.h"
#include "Bond.hpp"
#include "Capability.hpp"
#include "CertificateOfMembership.hpp"
#include "Constants.hpp"
#include "Metrics.hpp"
#include "NetworkController.hpp"
#include "Node.hpp"
#include "PQHybrid.hpp"
#include "PacketMultiplexer.hpp"
#include "Path.hpp"
#include "Peer.hpp"
#include "Revocation.hpp"
#include "RuntimeEnvironment.hpp"
#include "SHA512.hpp"
#include "Salsa20.hpp"
#include "SelfAwareness.hpp"
#include "Switch.hpp"
#include "Tag.hpp"
#include "Topology.hpp"
#include "Trace.hpp"
#include "World.hpp"
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include "../version.h"
#include "../include/ZeroTierOne.h"
#include "Constants.hpp"
#include "RuntimeEnvironment.hpp"
#include "IncomingPacket.hpp"
#include "Topology.hpp"
#include "Switch.hpp"
#include "Peer.hpp"
#include "NetworkController.hpp"
#include "SelfAwareness.hpp"
#include "Salsa20.hpp"
#include "SHA512.hpp"
#include "World.hpp"
#include "Node.hpp"
#include "CertificateOfMembership.hpp"
#include "Capability.hpp"
#include "Tag.hpp"
#include "Revocation.hpp"
#include "Trace.hpp"
#include "Path.hpp"
#include "Bond.hpp"
#include "Metrics.hpp"
#include "PacketMultiplexer.hpp"
#include <string.h>
namespace ZeroTier {
@@ -69,9 +69,16 @@ bool IncomingPacket::tryDecode(const RuntimeEnvironment *RR,void *tPtr,int32_t f
if (peer) {
if (!_authenticated) {
if (!dearmor(peer->key(), peer->aesKeys())) {
RR->t->incomingPacketMessageAuthenticationFailure(tPtr,_path,packetId(),sourceAddress,hops(),"invalid MAC");
peer->recordIncomingInvalidPacket(_path);
return true;
// HELLO and OK(HELLO) are always armored with the classical
// key, so retry with it once the session key has gone hybrid.
if (((verb() == Packet::VERB_HELLO) || (verb() == Packet::VERB_OK)) && (dearmor(peer->keyClassical(), peer->aesKeysClassical()))) {
// classical fallback accepted
}
else {
RR->t->incomingPacketMessageAuthenticationFailure(tPtr, _path, packetId(), sourceAddress, hops(), "invalid MAC");
peer->recordIncomingInvalidPacket(_path);
return true;
}
}
}
@@ -435,7 +442,7 @@ bool IncomingPacket::_doHELLO(const RuntimeEnvironment *RR,void *tPtr,const bool
} else {
// Identity is the same as the one we already have -- check packet integrity
if (!dearmor(peer->key(), peer->aesKeysIfSupported())) {
if (! dearmor(peer->keyClassical(), peer->aesKeysClassical())) {
RR->t->incomingPacketMessageAuthenticationFailure(tPtr,_path,pid,fromAddress,hops(),"invalid MAC");
return true;
}
@@ -458,9 +465,9 @@ bool IncomingPacket::_doHELLO(const RuntimeEnvironment *RR,void *tPtr,const bool
return true;
}
// Check packet integrity and MAC (this is faster than locallyValidate() so do it first to filter out total crap)
SharedPtr<Peer> newPeer(new Peer(RR,RR->identity,id));
if (!dearmor(newPeer->key(), newPeer->aesKeysIfSupported())) {
newPeer->setRemoteVersion(protoVersion, vMajor, vMinor, vRevision);
if (! dearmor(newPeer->keyClassical(), newPeer->aesKeysClassical())) {
RR->t->incomingPacketMessageAuthenticationFailure(tPtr,_path,pid,fromAddress,hops(),"invalid MAC");
return true;
}
@@ -476,6 +483,10 @@ bool IncomingPacket::_doHELLO(const RuntimeEnvironment *RR,void *tPtr,const bool
// Continue at // VALID
}
// Learn the peer's PQC capability from the HELLO revision bit now: the
// hybrid ciphertext below can only be interpreted once we know it.
peer->setRemoteVersion(protoVersion, vMajor, vMinor, vRevision);
// VALID -- if we made it here, packet passed identity and authenticity checks!
// Get external surface address if present (was not in old versions)
@@ -499,8 +510,9 @@ bool IncomingPacket::_doHELLO(const RuntimeEnvironment *RR,void *tPtr,const bool
std::vector< std::pair<uint64_t,uint64_t> > moonIdsAndTimestamps;
if (ptr < size()) {
// Remainder of packet, if present, is encrypted
cryptField(peer->key(),ptr,size() - ptr);
// Remainder of packet, if present, is encrypted (always with the
// classical key: it carries the hybrid ciphertext itself).
cryptField(peer->keyClassical(), ptr, size() - ptr);
// Get moon IDs and timestamps if present
if ((ptr + 2) <= size()) {
@@ -513,6 +525,21 @@ bool IncomingPacket::_doHELLO(const RuntimeEnvironment *RR,void *tPtr,const bool
ptr += 16;
}
}
// Hybrid KEM ciphertext, if the sender included one. Sender order is
// [moon count(2B)][moons][flag(1B)][ciphertext(1088B)] (see
// Peer::sendHELLO), so the flag follows the moon list. Only the
// lowest-address side ever writes it, so require that direction
// (sender == lowest) rather than trusting a coincidental byte value.
if ((peer->hybridEligible()) && (peer->identity().address() < RR->identity.address()) && ((ptr + 1 + PQHybrid::MLKEM768_CT_LEN) <= size()) && ((unsigned char)(*this)[ptr] == 1)) {
++ptr;
uint8_t hkey[ZT_SYMMETRIC_KEY_SIZE];
if (RR->identity.agreeHybridDecaps(id, field(ptr, PQHybrid::MLKEM768_CT_LEN), hkey)) {
peer->setHybridSessionKey(hkey);
}
Utils::burn(hkey, sizeof(hkey));
ptr += PQHybrid::MLKEM768_CT_LEN;
}
}
// Send OK(HELLO) with an echo of the packet's timestamp and some of the same
@@ -578,7 +605,20 @@ bool IncomingPacket::_doHELLO(const RuntimeEnvironment *RR,void *tPtr,const bool
}
outp.setAt<uint16_t>(worldUpdateSizeAt,(uint16_t)(outp.size() - (worldUpdateSizeAt + 2)));
outp.armor(peer->key(),true,peer->aesKeysIfSupported());
// Echo our ciphertext in OK(HELLO) so the initiator can derive the same
// hybrid key; only the lowest address ever encapsulates. Generate it now
// if this HELLO is the first time we learned the peer is hybrid-capable,
// so the handshake completes in this round trip instead of the next HELLO.
if (peer->hybridEligible()) {
peer->ensurePendingHybridCt();
}
unsigned char ctBuf[PQHybrid::MLKEM768_CT_LEN];
if (peer->pendingHybridCt(ctBuf)) {
outp.append((uint8_t)1);
outp.append(ctBuf, sizeof(ctBuf));
}
outp.armor(peer->keyClassical(), true, peer->aesKeysClassical());
peer->recordOutgoingPacket(_path,outp.packetId(),outp.payloadLength(),outp.verb(),ZT_QOS_NO_FLOW,now);
Metrics::pkt_ok_out++;
_path->send(RR,tPtr,outp.data(),outp.size(),now);
@@ -642,6 +682,26 @@ bool IncomingPacket::_doOK(const RuntimeEnvironment *RR,void *tPtr,const SharedP
peer->setRemoteVersion(vProto,vMajor,vMinor,vRevision);
// Hybrid ciphertext echoed by the lowest-address side; decrypting
// it completes the hybrid key agreement on our side.
unsigned int cto = ZT_PROTO_VERB_HELLO__OK__IDX_REVISION + 2;
InetAddress _esa;
if (cto < size()) {
cto += _esa.deserialize(*this, cto);
}
if ((cto + 2) <= size()) {
cto += 2 + at<uint16_t>(cto);
}
// Echo comes from the lowest-address side only; require that
// direction rather than trusting a coincidental flag byte.
if ((peer->hybridEligible()) && (peer->identity().address() < RR->identity.address()) && (cto < size()) && ((unsigned char)(*this)[cto] == 1) && ((cto + 1 + PQHybrid::MLKEM768_CT_LEN) <= size())) {
uint8_t hkey[ZT_SYMMETRIC_KEY_SIZE];
if (RR->identity.agreeHybridDecaps(peer->identity(), field(cto + 1, PQHybrid::MLKEM768_CT_LEN), hkey)) {
peer->setHybridSessionKey(hkey);
}
Utils::burn(hkey, sizeof(hkey));
}
if ((externalSurfaceAddress)&&(hops() == 0)) {
RR->sa->iam(tPtr,peer->address(),_path->localSocket(),_path->address(),externalSurfaceAddress,RR->topology->isUpstream(peer->identity()),RR->node->now());
}
+5 -3
View File
@@ -43,7 +43,7 @@ namespace ZeroTier {
/* Public Node interface (C++, exposed via CAPI bindings) */
/****************************************************************************/
Node::Node(void* uptr, void* tptr, const struct ZT_Node_Callbacks* callbacks, int64_t now)
Node::Node(void* uptr, void* tptr, const struct ZT_Node_Callbacks* callbacks, int64_t now, int pqcMode)
: _RR(this)
, RR(&_RR)
, _uPtr(uptr)
@@ -54,7 +54,7 @@ Node::Node(void* uptr, void* tptr, const struct ZT_Node_Callbacks* callbacks, in
, _lastHousekeepingRun(0)
, _lastMemoizedTraceSettings(0)
, _lowBandwidthMode(false)
, _pqcMode(ZT_PQC_MODE_HYBRID)
, _pqcMode(pqcMode)
{
if (callbacks->version != 0) {
throw ZT_EXCEPTION_INVALID_ARGUMENT;
@@ -93,7 +93,9 @@ Node::Node(void* uptr, void* tptr, const struct ZT_Node_Callbacks* callbacks, in
}
if (n <= 0) {
RR->identity.generate();
// Mode drives whether the new identity carries PQC material: classic
// produces a type 0 identity, hybrid/pqconly a type 1 identity.
RR->identity.generate(_pqcMode != ZT_PQC_MODE_CLASSIC, _pqcMode);
RR->identity.toString(false,RR->publicIdentityStr);
RR->identity.toString(true,RR->secretIdentityStr);
idtmp[0] = RR->identity.address().toInt();
+3 -3
View File
@@ -53,10 +53,10 @@ class World;
class Node : public NetworkController::Sender
{
public:
Node(void *uptr,void *tptr,const struct ZT_Node_Callbacks *callbacks,int64_t now);
virtual ~Node();
Node(void* uptr, void* tptr, const struct ZT_Node_Callbacks* callbacks, int64_t now, int pqcMode = ZT_PQC_MODE_CLASSIC);
virtual ~Node();
// Get rid of alignment warnings on 32-bit Windows and possibly improve performance
// Get rid of alignment warnings on 32-bit Windows and possibly improve performance
#ifdef __WINDOWS__
void * operator new(size_t i) { return _mm_malloc(i,16); }
void operator delete(void* p) { _mm_free(p); }
+97 -32
View File
@@ -11,18 +11,20 @@
*/
/****/
#include "Peer.hpp"
#include "../version.h"
#include "Constants.hpp"
#include "Peer.hpp"
#include "Switch.hpp"
#include "Network.hpp"
#include "SelfAwareness.hpp"
#include "Packet.hpp"
#include "Trace.hpp"
#include "InetAddress.hpp"
#include "RingBuffer.hpp"
#include "Utils.hpp"
#include "Metrics.hpp"
#include "Network.hpp"
#include "PQHybrid.hpp"
#include "Packet.hpp"
#include "RingBuffer.hpp"
#include "SelfAwareness.hpp"
#include "Switch.hpp"
#include "Trace.hpp"
#include "Utils.hpp"
namespace ZeroTier {
@@ -70,6 +72,49 @@ Peer::Peer(const RuntimeEnvironment *renv,const Identity &myIdentity,const Ident
KBKDFHMACSHA384(_key,ZT_KBKDF_LABEL_AES_GMAC_SIV_K1,0,0,ktmp);
_aesKeys[1].init(ktmp);
Utils::burn(ktmp,ZT_SYMMETRIC_KEY_SIZE);
// Keep the classical key for HELLO/OK(HELLO) armor for the life of the
// peer; the session key may later be replaced by the hybrid one.
memcpy(_keyClassical, _key, sizeof(_keyClassical));
memcpy(_aesKeysClassical, _aesKeys, sizeof(_aesKeysClassical));
_hybridCapable = false;
_pendingHybridCtValid = 0;
}
void Peer::_initAesKeys(const uint8_t* key, AES aesKeys[2])
{
uint8_t ktmp[ZT_SYMMETRIC_KEY_SIZE];
KBKDFHMACSHA384(key, ZT_KBKDF_LABEL_AES_GMAC_SIV_K0, 0, 0, ktmp);
aesKeys[0].init(ktmp);
KBKDFHMACSHA384(key, ZT_KBKDF_LABEL_AES_GMAC_SIV_K1, 0, 0, ktmp);
aesKeys[1].init(ktmp);
Utils::burn(ktmp, ZT_SYMMETRIC_KEY_SIZE);
}
void Peer::setHybridSessionKey(const void* key)
{
memcpy(_key, key, ZT_SYMMETRIC_KEY_SIZE);
_initAesKeys(_key, _aesKeys);
}
void Peer::setPendingHybridCt(const void* ct)
{
memcpy(_pendingHybridCt, ct, PQHybrid::MLKEM768_CT_LEN);
_pendingHybridCtValid = 1;
}
void Peer::ensurePendingHybridCt()
{
if ((_pendingHybridCtValid) || (! hybridEligible()) || (RR->identity.address() >= _id.address())) {
return;
}
uint8_t ct[PQHybrid::MLKEM768_CT_LEN];
uint8_t hkey[ZT_SYMMETRIC_KEY_SIZE];
if (RR->identity.agreeHybridEncaps(_id, ct, hkey)) {
setPendingHybridCt(ct);
setHybridSessionKey(hkey);
}
Utils::burn(hkey, sizeof(hkey));
}
void Peer::received(
@@ -419,39 +464,59 @@ void Peer::sendHELLO(void *tPtr,const int64_t localSocket,const InetAddress &atA
{
Packet outp(_id.address(),RR->identity.address(),Packet::VERB_HELLO);
outp.append((unsigned char)ZT_PROTO_VERSION);
outp.append((unsigned char)ZEROTIER_ONE_VERSION_MAJOR);
outp.append((unsigned char)ZEROTIER_ONE_VERSION_MINOR);
outp.append((uint16_t)(ZEROTIER_ONE_VERSION_REVISION | ((RR->node->pqcMode() != ZT_PQC_MODE_CLASSIC) ? ZT_PROTO_VERB_HELLO_REVISION_CAPABILITY_BIT : 0)));
outp.append(now);
RR->identity.serialize(outp,false);
atAddress.serialize(outp);
try {
outp.append((unsigned char)ZT_PROTO_VERSION);
outp.append((unsigned char)ZEROTIER_ONE_VERSION_MAJOR);
outp.append((unsigned char)ZEROTIER_ONE_VERSION_MINOR);
outp.append((uint16_t)(ZEROTIER_ONE_VERSION_REVISION | ((RR->node->pqcMode() != ZT_PQC_MODE_CLASSIC) ? ZT_PROTO_VERB_HELLO_REVISION_CAPABILITY_BIT : 0)));
outp.append(now);
RR->identity.serialize(outp, false);
atAddress.serialize(outp);
outp.append((uint64_t)RR->topology->planetWorldId());
outp.append((uint64_t)RR->topology->planetWorldTimestamp());
outp.append((uint64_t)RR->topology->planetWorldId());
outp.append((uint64_t)RR->topology->planetWorldTimestamp());
const unsigned int startCryptedPortionAt = outp.size();
const unsigned int startCryptedPortionAt = outp.size();
std::vector<World> moons(RR->topology->moons());
std::vector<uint64_t> moonsWanted(RR->topology->moonsWanted());
outp.append((uint16_t)(moons.size() + moonsWanted.size()));
for(std::vector<World>::const_iterator m(moons.begin());m!=moons.end();++m) {
outp.append((uint8_t)m->type());
outp.append((uint64_t)m->id());
outp.append((uint64_t)m->timestamp());
std::vector<World> moons(RR->topology->moons());
std::vector<uint64_t> moonsWanted(RR->topology->moonsWanted());
outp.append((uint16_t)(moons.size() + moonsWanted.size()));
for (std::vector<World>::const_iterator m(moons.begin()); m != moons.end(); ++m) {
outp.append((uint8_t)m->type());
outp.append((uint64_t)m->id());
outp.append((uint64_t)m->timestamp());
}
for (std::vector<uint64_t>::const_iterator m(moonsWanted.begin()); m != moonsWanted.end(); ++m) {
outp.append((uint8_t)World::TYPE_MOON);
outp.append(*m);
outp.append((uint64_t)0);
}
const bool sendHybridCt = (hybridEligible() && (RR->identity.address() < _id.address()));
if (sendHybridCt) {
// Lowest address encapsulates. One ct per attempt, re-encapsulated only
// while the peer has not yet confirmed the previous one.
ensurePendingHybridCt();
if (_pendingHybridCtValid) {
outp.append((uint8_t)1); // hybrid ciphertext present
outp.append(_pendingHybridCt, PQHybrid::MLKEM768_CT_LEN);
}
}
outp.cryptField(_keyClassical, startCryptedPortionAt, outp.size() - startCryptedPortionAt);
}
for(std::vector<uint64_t>::const_iterator m(moonsWanted.begin());m!=moonsWanted.end();++m) {
outp.append((uint8_t)World::TYPE_MOON);
outp.append(*m);
outp.append((uint64_t)0);
catch (int) {
// G4 bound: Buffer<ZT_PROTO_MAX_PACKET_LENGTH> overflow would kill the
// daemon via catch(int) in OneServiceImpl::run(); degrade to skipped HELLO.
char addrBuf[11];
fprintf(stderr, "WARNING: HELLO to %s exceeds ZT_PROTO_MAX_PACKET_LENGTH; not sent" ZT_EOL_S, _id.address().toString(addrBuf));
return;
}
outp.cryptField(_key,startCryptedPortionAt,outp.size() - startCryptedPortionAt);
Metrics::pkt_hello_out++;
if (atAddress) {
outp.armor(_key,false,nullptr); // false == don't encrypt full payload, but add MAC
outp.armor(_keyClassical, false, nullptr); // HELLO is always classical-armored
RR->node->expectReplyTo(outp.packetId());
RR->node->putPacket(tPtr,RR->node->lowBandwidthModeEnabled() ? localSocket : -1,atAddress,outp.data(),outp.size());
} else {
+70 -14
View File
@@ -59,6 +59,7 @@ private:
public:
~Peer() {
Utils::burn(_key,sizeof(_key));
Utils::burn(_keyClassical, sizeof(_keyClassical));
}
/**
@@ -372,27 +373,30 @@ public:
_vMajor = (uint16_t)vmaj;
_vMinor = (uint16_t)vmin;
_vRevision = (uint16_t)vrev;
_hybridCapable = ((_vRevision & ZT_PROTO_VERB_HELLO_REVISION_CAPABILITY_BIT) != 0);
}
inline unsigned int remoteVersionProtocol() const { return _vProto; }
inline unsigned int remoteVersionMajor() const { return _vMajor; }
inline unsigned int remoteVersionMinor() const { return _vMinor; }
inline unsigned int remoteVersionRevision() const
{ return (_vRevision & 0x7fff); } // excludes PQC capability bit (see pqcCapability())
{ return (_vRevision & 0x7fff); } // excludes PQC capability bit (see _hybridCapable)
inline bool remoteVersionKnown() const
{ return ((_vMajor > 0) || (_vMinor > 0) || (_vRevision > 0)); }
/**
* @return True if peer advertised PQC capability via HELLO revision high bit
*/
inline bool pqcCapability() const
{ return ((_vRevision & ZT_PROTO_VERB_HELLO_REVISION_CAPABILITY_BIT) != 0); }
/**
* @return True if peer has received a trust established packet (e.g. common network membership) in the past ZT_TRUST_EXPIRATION ms
*/
inline bool trustEstablished(const int64_t now) const { return ((now - _lastTrustEstablishedPacketReceived) < ZT_TRUST_EXPIRATION); }
inline bool trustEstablished(const int64_t now) const
{ return ((now - _lastTrustEstablishedPacketReceived) < ZT_TRUST_EXPIRATION); }
/**
* @return True if the peer is both known to be PQC-capable and we are
* currently in a non-classic mode, i.e. hybrid key agreement applies.
*/
inline bool hybridEligible() const
{ return ((RR->node->pqcMode() != ZT_PQC_MODE_CLASSIC) && (_hybridCapable) && (RR->identity.hasPQ())); }
/**
* Rate limit gate for VERB_PUSH_DIRECT_PATHS
@@ -632,18 +636,70 @@ public:
inline const AES *aesKeys() const
{ return _aesKeys; }
private:
struct _PeerPath
// Classical C25519-derived session key + AES keys. Always valid; used to
// armor HELLO/OK(HELLO), which must be readable before either side has
// switched to the hybrid key. See ZT_PQC_MODE_* and IncomingPacket.cpp.
inline const unsigned char* keyClassical() const
{ return _keyClassical; }
inline const AES* aesKeysClassical() const
{ return _aesKeysClassical; }
/**
* Install the hybrid session key (encapsulator or decapsulator side).
* Overwrites _key and _aesKeys; _keyClassical is left untouched.
*/
void setHybridSessionKey(const void* key);
/**
* Store a freshly created hybrid ciphertext to be echoed in HELLO/OK.
* The encapsulating (lowest address) side owns this until confirmation.
*/
void setPendingHybridCt(const void* ct);
/**
* Queue a hybrid ciphertext for this peer if we are the encapsulating
* (lowest address) side and none is pending yet. Derives our hybrid session
* key as a side effect. No-op otherwise.
*/
void ensurePendingHybridCt();
/**
* Copy the pending hybrid ciphertext into out if one is queued.
*
* @return True if a ciphertext was copied
*/
inline bool pendingHybridCt(unsigned char* out) const
{
_PeerPath() : lr(0),p(),priority(1) {}
int64_t lr; // time of last valid ZeroTier packet
if (! _pendingHybridCtValid) {
return false;
}
memcpy(out, _pendingHybridCt, PQHybrid::MLKEM768_CT_LEN);
return true;
}
private:
// Derive the two AES-GMAC-SIV keys from a session key.
static void _initAesKeys(const uint8_t* key, AES aesKeys[2]);
struct _PeerPath {
_PeerPath() : lr(0), p(), priority(1)
{
}
int64_t lr; // time of last valid ZeroTier packet
SharedPtr<Path> p;
long priority; // >= 1, higher is better
long priority; // >= 1, higher is better
};
uint8_t _key[ZT_SYMMETRIC_KEY_SIZE];
AES _aesKeys[2];
uint8_t _keyClassical[ZT_SYMMETRIC_KEY_SIZE];
AES _aesKeysClassical[2];
bool _hybridCapable;
// Lowest-address side encapsulates once per attempt; its ct is echoed in
// every HELLO/OK(HELLO) until the peer confirms the hybrid key.
uint8_t _pendingHybridCt[PQHybrid::MLKEM768_CT_LEN];
unsigned int _pendingHybridCtValid;
AES _aesKeys[2];
const RuntimeEnvironment *RR;
int64_t _lastReceive; // direct or indirect
+17 -13
View File
@@ -37,20 +37,21 @@
#else // not Windows
#include <errno.h>
#include <signal.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/time.h>
#include <sys/types.h>
#include <sys/select.h>
#include <sys/socket.h>
#include <sys/un.h>
#include "../node/Constants.hpp"
#include "../node/Metrics.hpp"
#include <arpa/inet.h>
#include <errno.h>
#include <fcntl.h>
#include <netinet/in.h>
#include <netinet/tcp.h>
#include "../node/Metrics.hpp"
#include <signal.h>
#include <sys/select.h>
#include <sys/socket.h>
#include <sys/time.h>
#include <sys/types.h>
#include <sys/un.h>
#include <unistd.h>
#if defined(__linux__) || defined(linux) || defined(__LINUX__) || defined(__linux)
#ifndef IPV6_DONTFRAG
@@ -1009,7 +1010,7 @@ public:
if (FD_ISSET(s->sock, &rfds)) {
#if (defined(__linux__) || defined(linux) || defined(__linux)) && defined(MSG_WAITFORONE)
#define RECVMMSG_WINDOW_SIZE 128
#define RECVMMSG_BUF_SIZE 1500
#define RECVMMSG_BUF_SIZE ZT_MAX_PHYSMTU // must hold the largest legal UDP payload: PQ identities push HELLO past the old 1500
iovec iovs[RECVMMSG_WINDOW_SIZE];
uint8_t bufs[RECVMMSG_WINDOW_SIZE][RECVMMSG_BUF_SIZE];
sockaddr_storage addrs[RECVMMSG_WINDOW_SIZE];
@@ -1028,10 +1029,13 @@ public:
mm[i].msg_hdr.msg_namelen = sizeof(sockaddr_storage);
mm[i].msg_len = 0;
}
int received_count = recvmmsg(s->sock, mm, RECVMMSG_WINDOW_SIZE, MSG_WAITFORONE, nullptr);
int received_count = recvmmsg(s->sock, mm, RECVMMSG_WINDOW_SIZE, MSG_WAITFORONE | MSG_TRUNC, nullptr);
if (received_count > 0) {
for (int i = 0; i < received_count; ++i) {
long n = (long)mm[i].msg_len;
if (n > RECVMMSG_BUF_SIZE) {
continue; // datagram exceeds any legal ZT payload -- drop rather than process truncated bytes
}
if (n > 0) {
try {
_handler->phyOnDatagram((PhySocket*)&(*s), &(s->uptr), (const struct sockaddr*)&(s->saddr), (const struct sockaddr*)&(addrs[i]), bufs[i], (unsigned long)n);
+108 -30
View File
@@ -1044,6 +1044,12 @@ public:
_metricsToken = _trimString(_metricsToken);
}
// PQC mode must be known before the Node constructs: it decides
// whether a freshly generated identity carries type-1 key material.
// local.conf is the only source, so read it first.
_readLocalConfigFile();
const int pqcMode = _pqcModeFromLocalConfig(_localConfig);
{
struct ZT_Node_Callbacks cb;
cb.version = 0;
@@ -1055,11 +1061,22 @@ public:
cb.eventCallback = SnodeEventCallback;
cb.pathCheckFunction = SnodePathCheckFunction;
cb.pathLookupFunction = SnodePathLookupFunction;
_node = new Node(this,(void *)0,&cb,OSUtils::now());
_node = new Node(this, (void*)0, &cb, OSUtils::now(), pqcMode);
}
// A classic identity under a PQC mode (or the reverse) must not be
// silently accepted or silently replaced: either changes this node's
// address. Refuse with a reason the operator can act on.
{
std::string pqcIdentityError;
if (_pqcIdentityUnsatisfiable(pqcMode, _node->identity().hasPQ(), pqcIdentityError)) {
Mutex::Lock _l(_termReason_m);
_termReason = ONE_UNRECOVERABLE_ERROR;
_fatalErrorMessage = pqcIdentityError;
return _termReason;
}
}
// local.conf
readLocalSettings();
applyLocalConfig();
// Save original port number to show it if bind error
@@ -1387,6 +1404,81 @@ public:
return _termReason;
}
/**
* Resolve settings.pqcMode from a parsed local.conf into a ZT_PQC_MODE_*
* value. Absent key = classic: PQC is opt-in, so an upgraded install keeps
* its existing type-0 identity and address. A present-but-unknown value is
* also classic, with a warning, rather than silently enabling PQC.
*
* @param lc Parsed local.conf (non-const: nlohmann inserts missing keys)
* @return ZT_PQC_MODE_CLASSIC, _HYBRID, or _PQCONLY
*/
static int _pqcModeFromLocalConfig(json& lc)
{
json& s = lc["settings"];
const std::string m(OSUtils::jsonString(s.is_object() ? s["pqcMode"] : json(), ""));
if (m == "hybrid") {
return ZT_PQC_MODE_HYBRID;
}
if (m == "pqconly") {
return ZT_PQC_MODE_PQCONLY;
}
if ((m.length() > 0) && (m != "off")) {
fprintf(stderr, "WARNING: unknown settings.pqcMode in local.conf (expected off/hybrid/pqconly); using classic" ZT_EOL_S);
}
return ZT_PQC_MODE_CLASSIC;
}
/**
* Refuse loudly to run when the on-disk identity cannot satisfy the
* configured PQC mode. Upgrading in place would change this node's address
* and every certificate bound to it, so the operator decides instead.
*
* @param mode Configured ZT_PQC_MODE_*
* @param havePQ Whether the loaded identity carries type-1 PQC material
* @param fatalMsg Result message when refusing
* @return True if the daemon must not start
*/
static bool _pqcIdentityUnsatisfiable(int mode, bool havePQ, std::string& fatalMsg)
{
if ((! havePQ) && (mode != ZT_PQC_MODE_CLASSIC)) {
fatalMsg = "identity.secret in this home path is a classic (type 0) identity but settings.pqcMode requires PQC material; delete identity.secret and identity.public to generate a new PQC identity (this changes the node "
"address), or set settings.pqcMode to \"off\"";
return true;
}
if ((havePQ) && (mode == ZT_PQC_MODE_CLASSIC)) {
fatalMsg =
"identity.secret in this home path is a PQC (type 1) identity but settings.pqcMode is \"off\"; re-enable PQC or delete identity.secret and identity.public to generate a classic identity (this changes the node address)";
return true;
}
return false;
}
/**
* Read and parse local.conf into _localConfig. Split out of
* readLocalSettings() so the PQC mode can be resolved before the Node is
* constructed, without touching Node.
*/
void _readLocalConfigFile()
{
Mutex::Lock _l(_localConfig_m);
std::string lcbuf;
if (OSUtils::readFile((_homePath + ZT_PATH_SEPARATOR_S "local.conf").c_str(), lcbuf)) {
if (lcbuf.length() > 0) {
try {
_localConfig = OSUtils::jsonParse(lcbuf);
if (! _localConfig.is_object()) {
fprintf(stderr, "ERROR: unable to parse local.conf (root element is not a JSON object)" ZT_EOL_S);
exit(1);
}
}
catch (...) {
fprintf(stderr, "ERROR: unable to parse local.conf (invalid JSON)" ZT_EOL_S);
exit(1);
}
}
}
}
void readLocalSettings()
{
// Read local configuration
@@ -1418,24 +1510,9 @@ public:
fclose(trustpaths);
}
// Read local config file
// Re-read local.conf from disk, then hold the lock while applying it.
_readLocalConfigFile();
Mutex::Lock _l2(_localConfig_m);
std::string lcbuf;
if (OSUtils::readFile((_homePath + ZT_PATH_SEPARATOR_S "local.conf").c_str(),lcbuf)) {
if (lcbuf.length() > 0) {
try {
_localConfig = OSUtils::jsonParse(lcbuf);
if (!_localConfig.is_object()) {
fprintf(stderr,"ERROR: unable to parse local.conf (root element is not a JSON object)" ZT_EOL_S);
exit(1);
}
} catch ( ... ) {
fprintf(stderr,"ERROR: unable to parse local.conf (invalid JSON)" ZT_EOL_S);
exit(1);
}
}
}
// Make a copy so lookups don't modify in place;
json lc(_localConfig);
@@ -1466,16 +1543,17 @@ public:
_ssoRedirectURL = OSUtils::jsonString(settings["ssoRedirectURL"], "");
// PQC capability mode (T21): off / hybrid / pqconly -> Node::pqcMode
const std::string pqcm(OSUtils::jsonString(settings["pqcMode"], ""));
if (pqcm == "off")
_node->setPqcMode(ZT_PQC_MODE_CLASSIC);
else if (pqcm == "hybrid")
_node->setPqcMode(ZT_PQC_MODE_HYBRID);
else if (pqcm == "pqconly")
_node->setPqcMode(ZT_PQC_MODE_PQCONLY);
else if (pqcm.length() > 0)
fprintf(stderr, "WARNING: unknown settings.pqcMode in local.conf (expected off/hybrid/pqconly)\n");
// PQC capability mode (T21): off / hybrid / pqconly -> Node::pqcMode.
// A mode the loaded identity cannot satisfy is refused here rather
// than applied: the startup check cannot cover a runtime reload.
const int pqcMode = _pqcModeFromLocalConfig(lc);
std::string pqcReloadError;
if (_pqcIdentityUnsatisfiable(pqcMode, _node->identity().hasPQ(), pqcReloadError)) {
fprintf(stderr, "WARNING: ignoring settings.pqcMode change: %s" ZT_EOL_S, pqcReloadError.c_str());
}
else {
_node->setPqcMode(pqcMode);
}
#ifdef ZT_CONTROLLER_USE_LIBPQ
json &redis = settings["redis"];
+1
View File
@@ -42,6 +42,7 @@ Settings available in `local.conf` (this is not valid JSON, and JSON does not al
"bind": [ "ip",... ], /* If present and non-null, bind to these IPs instead of to each interface (wildcard IP allowed) */
"allowTcpFallbackRelay": true|false, /* Allow or disallow establishment of TCP relay connections (true by default) */
"multipathMode": 0|1|2, /* multipath mode: none (0), random (1), proportional (2) */
"pqcMode": "off"|"hybrid"|"pqconly", /* Post-quantum key agreement mode. Absent defaults to "off" (classic), so existing nodes keep their identity and address. "hybrid" mixes ML-KEM-768 into the X25519 agreement and requires an identity carrying PQ material; a classic identity under a PQC mode makes the daemon refuse to start rather than change its address. */
"redis": { /* Optional, requires controllerDbPath starting with "postgres:" */
"hostname": "str", /* Redis server host */
"port": 1-65535, /* Redis server port */
+98
View File
@@ -0,0 +1,98 @@
#!/usr/bin/env bash
# pqc-lab.sh - hermetic-ish two-node BackOne PQC handshake lab + capture + asserts.
# See docs/pqc-test-scenario.md §4-5. Needs root for TUN/TAP and (optionally) netns.
# Usage: sudo -E tools/pqc-lab.sh [mode1] [mode2] (default: hybrid hybrid)
set -euo pipefail
BIN="$(cd "$(dirname "$0")/.." && pwd)/backone"
CLI="$(cd "$(dirname "$0")/.." && pwd)/backone-cli"
MODE1="${1:-hybrid}"; MODE2="${2:-hybrid}"
LAB="${LAB:-/tmp/pqc-lab}"
P1="${P1:-20001}"; P2="${P2:-20002}"
say() { printf '\033[1m== %s\033[0m\n' "$*"; }
ok() { printf ' PASS %s\n' "$*"; }
bad() { printf ' FAIL %s\n' "$*"; FAILED=1; }
FAILED=0
cleanup() {
[[ -f "$LAB/n1/backone.pid" ]] && kill "$(cat "$LAB/n1/backone.pid")" 2>/dev/null || true
[[ -f "$LAB/n2/backone.pid" ]] && kill "$(cat "$LAB/n2/backone.pid")" 2>/dev/null || true
}
trap cleanup EXIT
say "start nodes (mode1=$MODE1 mode2=$MODE2)"
rm -rf "$LAB"; mkdir -p "$LAB/n1" "$LAB/n2"
printf '{"settings":{"pqcMode":"%s"}}' "$MODE1" > "$LAB/n1/local.conf"
printf '{"settings":{"pqcMode":"%s"}}' "$MODE2" > "$LAB/n2/local.conf"
nohup "$BIN" -U -p"$P1" "$LAB/n1" >"$LAB/n1/log" 2>&1 &
nohup "$BIN" -U -p"$P2" "$LAB/n2" >"$LAB/n2/log" 2>&1 &
sleep 6
A1=$(cat "$LAB/n1/authtoken.secret"); A2=$(cat "$LAB/n2/authtoken.secret")
N1=$(cut -d: -f1 "$LAB/n1/identity.public"); N2=$(cut -d: -f1 "$LAB/n2/identity.public")
T1=$(cut -d: -f2 "$LAB/n1/identity.public"); T2=$(cut -d: -f2 "$LAB/n2/identity.public")
say "identity type bytes: n1=$T1 n2=$T2"
# ---- G1: identity must be type 1 when hybrid/pqconly ----------------------
if [[ "$MODE1" == "off" ]]; then
[[ "$T1" == "0" ]] && ok "G1 n1 classic identity under mode=off" || bad "G1 n1 expected type 0, got $T1"
else
if [[ "$T1" == "1" ]]; then ok "G1 n1 type-1 identity under mode=$MODE1"
else bad "G1 n1 expected type 1 (PQ) under mode=$MODE1, got $T1 <-- Node.cpp:96 ignores pqcMode"; fi
fi
say "create network on the controller (n1) + authorize n2"
NW=$(curl -s -H "X-ZT1-Auth: $A1" -X POST \
-d '{"name":"pqclab","v4AssignMode":{"zt":true},"ipAssignmentPools":[{"ipRangeStart":"10.99.0.1","ipRangeEnd":"10.99.0.254"}]}' \
"http://127.0.0.1:$P1/controller/network/${N1}______" \
| python3 -c 'import json,sys;print(json.load(sys.stdin)["nwid"])')
[[ "$NW" =~ ^${N1}[0-9a-f]{6}$ ]] && ok "network $NW created" || bad "network id '$NW' not controller-addressed"
curl -s -H "X-ZT1-Auth: $A1" -X POST -d '{"authorized":true}' \
"http://127.0.0.1:$P1/controller/network/$NW/member/$N2" >/dev/null
curl -s -H "X-ZT1-Auth: $A1" -X POST "http://127.0.0.1:$P1/network/$NW" >/dev/null
curl -s -H "X-ZT1-Auth: $A2" -X POST "http://127.0.0.1:$P2/network/$NW" >/dev/null
sleep 8
# ---- TAP gate (needs root) ------------------------------------------------
if grep -q 'TAP operation' "$LAB/n1/log" 2>/dev/null; then
bad "n1 cannot open TUN/TAP (need root / CAP_NET_ADMIN) - data plane untestable"
fi
CONF=$(curl -s -H "X-ZT1-Auth: $A1" "http://127.0.0.1:$P1/network/$NW")
[[ "$CONF" != "{}" && -n "$CONF" ]] && ok "n1 network config populated" || bad "n1 network config empty: $CONF"
# ---- capture --------------------------------------------------------------
say "capture (udp - all fragments, incl. continuation)"
pcap="$LAB/pqc.pcap"
FILT="udp"
( ip netns exec n2 tcpdump -i any -w "$pcap" "$FILT" >/dev/null 2>&1 || \
tcpdump -i any -w "$pcap" "$FILT" >/dev/null 2>&1 ) &
TCPD=$!; sleep 1
say "warm up handshake"
# nodes run on host here (netns n1/n2 may not exist) - ping via TAP on host
ping -c3 -W2 10.99.0.2 >/dev/null 2>&1 || \
sudo ip netns exec n1 ping -c3 -W2 10.99.0.2 >/dev/null 2>&1 || true
sleep 6
kill "$TCPD" 2>/dev/null || true
pkill -INT -f "tcpdump.*$pcap" 2>/dev/null || true
sleep 1
# ---- G2/G3: wire must show PQ material when both sides are PQ-capable -----
if [[ -s "$pcap" ]]; then
frags=$(tshark -r "$pcap" -Y "udp.port==9993 or udp.port==$P1 or udp.port==$P2" -T fields -e frame.number 2>/dev/null | wc -l || true)
ok "captured $frags ZT-adjacent UDP frames -> $pcap"
if [[ "$MODE1" != "off" && "$MODE2" != "off" ]]; then
# After the §7 fix a 1088-byte ML-KEM ciphertext (>=2 datagrams) must appear.
big=$(tshark -r "$pcap" -Y 'udp.length>1200' -T fields -e frame.number 2>/dev/null | wc -l || true)
if (( big > 0 )); then ok "G2/G3 large PQ datagrams present ($big) - hybrid material on wire"
else bad "G2/G3 no large datagrams: PQ capability advertised but no ML-KEM ciphertext <-- Identity::agree() is classical"; fi
fi
say "dissect with: tshark -X lua_script:$(dirname "$0")/zt-dissector.lua -r $pcap -Y zt"
else
bad "no packets captured (netns/tcpdump unavailable without root?)"
fi
echo
[[ "$FAILED" == "0" ]] && say "ALL CHECKS PASSED" || say "FAILURES PRESENT (see above; §0/§7 of docs/pqc-test-scenario.md)"
exit "$FAILED"
+108
View File
@@ -0,0 +1,108 @@
-- BackOne / ZeroTier wire dissector for Wireshark / tshark 4.x
-- Usage: tshark -X lua_script:tools/zt-dissector.lua -r capture.pcap -Y zt
-- Offsets per node/Packet.hpp:224-258 and node/Peer.cpp:463-517.
local zt = Proto("zt", "BackOne / ZeroTier P2P")
local VERB_HELLO = 1 -- Packet.hpp:592 (HELLO is 1, NOT 0)
local FRAGMENT_INDICATOR = 0xff -- Packet.hpp:253 (ZT_ADDRESS_RESERVED_PREFIX)
local IDENTITY_TYPE_PQ_HYBRID = 1
local IDENTITY_SIZE_CLASSIC = 39 -- 5 addr + 1 type + 32 pub + 1 privlen
local IDENTITY_SIZE_PQ = 3175 -- classic + 1184 ML-KEM-768 pk + 1952 ML-DSA-65 pk
local HELLO_IDENTITY_OFF = 13 -- payload + (pv1+maj1+min1+rev2+timestamp8)
local f = {
iv = ProtoField.bytes("zt.iv", "Packet ID / IV"),
dest = ProtoField.string("zt.dest", "Destination"),
src = ProtoField.string("zt.src", "Source"),
flags = ProtoField.uint8("zt.flags", "Flags", base.HEX),
verb = ProtoField.uint8("zt.verb", "Verb", base.DEC),
frag = ProtoField.uint8("zt.frag", "Fragment indicator", base.HEX),
fragno = ProtoField.uint8("zt.fragno", "Fragment #", base.DEC),
fragtot = ProtoField.uint8("zt.fragtot", "Total fragments", base.DEC),
hello = ProtoField.none("zt.hello", "HELLO"),
helloPv = ProtoField.uint8("zt.hello.pv", "Protocol version", base.DEC),
helloMaj = ProtoField.uint8("zt.hello.major", "Major", base.DEC),
helloMin = ProtoField.uint8("zt.hello.minor", "Minor", base.DEC),
helloRev = ProtoField.uint16("zt.hello.rev", "Revision", base.HEX),
helloCp = ProtoField.bool("zt.hello.cap", "PQC capability bit", 16, nil, 0x8000),
helloIdt = ProtoField.uint8("zt.hello.idtype", "Identity type", base.DEC),
helloIds = ProtoField.uint32("zt.hello.idsize", "Identity size", base.DEC),
}
zt.fields = f
local function addr(buf, off) return tostring(buf(off, 5):bytes():tohex()) end
-- Parse the HELLO verb body starting at payload offset `p`. `limit` is the number
-- of bytes actually present in this frame (a head fragment may truncate it).
local function parseHello(t, buf, p, limit, pinfo)
if limit < p + 3 then return end
local ty
local ht = t:add(f.hello, buf(p, limit - p))
ht:add(f.helloPv, buf(p + 0, 1))
ht:add(f.helloMaj, buf(p + 1, 1))
ht:add(f.helloMin, buf(p + 2, 1))
if limit >= p + 5 then
ht:add(f.helloRev, buf(p + 3, 2))
ht:add(f.helloCp, buf(p + 3, 2))
end
local id = p + HELLO_IDENTITY_OFF
if limit >= id + 6 then
ty = buf(id + 5, 1):uint()
ht:add(f.helloIdt, buf(id + 5, 1))
ht:add(f.helloIds, (ty == IDENTITY_TYPE_PQ_HYBRID) and IDENTITY_SIZE_PQ or IDENTITY_SIZE_CLASSIC)
end
-- The [moonCount][moons][hybridFlag(1)][ML-KEM ct] tail is cryptField()'d
-- with the classical key (Peer.cpp:500), so it is opaque to a passive
-- capture: the dissector never guesses the ciphertext length.
if ty then
pinfo.cols.info = ("ZT HELLO type=%d %s cap=%s"):format(
ty, addr(buf, id),
(limit >= p + 4) and (bit.band(buf(p + 3, 2):uint(), 0x8000) ~= 0 and "1" or "0") or "?")
else
pinfo.cols.info = "ZT HELLO"
end
end
function zt.dissector(buf, pinfo, tree)
if buf:len() < 16 then return end
pinfo.cols.protocol = "ZT"
-- Fragment frame: [8 pkID][5 dest][1 0xff][1 {total<<4 | no}][1 hops][data]
if buf(13, 1):uint() == FRAGMENT_INDICATOR then
local t = tree:add(zt, buf(), "BackOne fragment")
t:add(f.iv, buf(0, 8))
t:add(f.dest, addr(buf, 8))
t:add(f.frag, buf(13, 1))
local total = bit.rshift(buf(14, 1):uint(), 4)
local no = bit.band(buf(14, 1):uint(), 0x0f)
t:add(f.fragtot, buf(14, 1), total)
t:add(f.fragno, buf(14, 1), no)
pinfo.cols.info = ("BackOne fragment %d/%d"):format(no, total)
return
end
if buf:len() < 28 then return end
local verb = buf(27, 1):uint()
local encrypted = bit.band(buf(18, 1):uint(), 0x80) ~= 0
local t = tree:add(zt, buf(), "BackOne, verb " .. verb)
t:add(f.iv, buf(0, 8))
t:add(f.dest, addr(buf, 8))
t:add(f.src, addr(buf, 13))
t:add(f.flags, buf(18, 1))
t:add(f.verb, buf(27, 1))
-- Head of a fragmented packet still carries the original header, so the
-- verb and the leading payload bytes are parseable.
if bit.band(buf(18, 1):uint(), 0x40) ~= 0 then
t:add(f.frag, buf(18, 1))
end
if verb == VERB_HELLO and not encrypted then
parseHello(t, buf, 28, buf:len(), pinfo)
end
end
-- Default ZT port and the lab port used by docs/pqc-test-scenario.md.
DissectorTable.get("udp.port"):add(9993, zt)
DissectorTable.get("udp.port"):add(19993, zt)