BackOne/docs/pqc-test-scenario.md

BackOne PQC End-to-End Test Scenario

Version: 2.0.0 · Scope: verify the hybrid post-quantum stack (ML-KEM-768 + ML-DSA-65) actually does what SPEC §G claims, on the wire — not just in unit tests.

Every offset, constant, and gap below was read from this tree at the cited line. [INFERENCE] marks anything not directly observed.

---

0. Read this first — status of the three gaps

Status (2026-10-02): G1–G3 fixed; G1 also fixed at the root by B2. Generation runs under the configured mode (Node takes pqcMode, OneService refuses a classic↔PQ identity mismatch), the wire negotiates hybrid, and the capability bit is consumed (Peer::hybridEligible()). Modes are opt-in: absent/unknown settings.pqcMode resolves to classic (ZT_PQC_MODE_CLASSIC), so an upgraded type-0 install boots unchanged and keeps its address — see the B2 fix in SPEC.md §B. Verified live on a 2.0.0 build: no config → 141 B type-0 identity; "hybrid" → 6415 B type-1 identity; classic identity + "hybrid" refuses to start. Re-run §4/§5 to confirm the 1088 B ML-KEM ciphertext and a hybrid _key on the wire. The table below is the original 2.0.0 diagnosis, kept for the rationale and the exact evidence.

#	Gap	Evidence	Symptom in the lab
G1	Daemon never generates a type-1 identity. Node.cpp:96 calls RR->identity.generate() with default pq=false, and applyLocalConfig() (which sets the mode) runs after new Node(...) (OneService.cpp:1058 then 1062).	Fixed (T24 + B2): OneService reads local.conf before new Node and passes pqcMode; Identity::generate takes it. Absent config is now classic, not hybrid. node/Node.cpp:96, node/Identity.cpp:84-123, service/OneService.cpp _pqcModeFromLocalConfig.	"pqcMode":"hybrid" now writes type byte 1; measured identity.public = 6415 B vs 141 B classic (the old doc estimate of ~4230 B was the research-doc approximation)
G2	Wire key agreement never used PQ. myIdentity.agree(peerIdentity,…) is plain C25519 (Identity.hpp:343-350).	Fixed (B1): production now calls agreeHybridEncaps/agreeHybridDecaps — node/Peer.cpp:113 (encaps on first capability sighting) and node/IncomingPacket.cpp:538,700 (decaps on HELLO and OK(HELLO)); setHybridSessionKey replaces the classical key.	Session key is hybrid when both peers advertise the capability; a vanilla peer still falls back to classical
G3	Capability was advertised but never consumed: Peer::_key was derived in the constructor before any HELLO arrived, so the hybrid path was dead.	Fixed (B1): Peer::hybridEligible() now gates _hybridCapable (set in setRemoteVersion), the KEM ct rides in HELLO/OK(HELLO), and setHybridSessionKey() replaces the session key. Peer::pqcCapability() was removed as redundant.	Capture now shows a 1088 B ct after the capability bit

Consequence: the only end-to-end proof so far is backone-selftest (ML-KEM KAT, ML-DSA KAT, hybrid KDF vector, identity-v2 roundtrip, COM double-sign, HELLO fragmentation). Those pass. They prove primitives and encodings, not a live handshake.

---

1. Measured baseline (this build, this host)

make selftest && ./backone-selftest        # exit 0
[PQ] ML-KEM KAT (decaps fixed sk/ct -> ss)... PASS
[PQ] ML-DSA KAT (verify fixed pk/msg/sig; tamper fails)... PASS
[PQ] Hybrid KDF known-answer vector... PASS
[PQ] Interop hybrid<->vanilla matrix... PASS
    size classic=137B/1 frag, PQC=3273B/3 frag (max 7)

backone-cli info -j → "version":"2.0.0". Lab daemons answered on 127.0.0.1:20001 / :20002 with {"version":"2.0.0","online":true}.

Fragmentation ceiling: ZT_MAX_PACKET_FRAGMENTS 7 × ZT_DEFAULT_PHYSMTU 1432 (node/Packet.hpp:235, include/ZeroTierOne.h:101) = 10024 B max reassembled packet. pqconly (ML-KEM-1024 + ML-DSA-87) grows the HELLO well past that — see §4.3.

---

2. Wire facts (cite these in tshark filters)

WU = wire units = Appendix A of RFC 7042 packet-diagram notation (1 byte = 1 column).

2.1 Packet header — node/Packet.hpp:224-230

WU	Field
0–7	Packet ID / IV (8 B)
8–12	Destination address (5 B)
13–17	Source address (5 B)
18	Flags
19–26	MAC
27	Verb
28…	Payload

Flags: ZT_PROTO_FLAG_ENCRYPTED 0x80, ZT_PROTO_FLAG_FRAGMENTED 0x40 (node/Packet.hpp:130-134). Fragment header (Packet.hpp:242-248): PACKET_ID@0, DEST@8, FRAGMENT_INDICATOR@13, FRAGMENT_NO@14, HOPS@15, PAYLOAD@16 — so a fragment's payload starts at overall offset 44.

2.2 VERB_HELLO payload — sender node/Peer.cpp:418-445, offsets node/Packet.hpp:266-271

Offset rel. payload (wire = +28)	WU	Field
0	0	Protocol version = 12 (ZT_PROTO_VERSION)
1	1	Major
2	2	Minor
3–4	3	Revision uint16, high bit 0x8000 = PQC capability (Packet.hpp:273); masked off by remoteVersionRevision() (Peer.hpp:381)
5–12	5	Timestamp (i64)
13…	13	Identity, serialized includePrivate=false
…		InetAddress (sender's observation of us)
…		worldId u64, worldTimestamp u64, moons…

Identity binary, public form — node/Identity.hpp:421-445, node/InetAddress.hpp:557-577:

Bytes	Field
5	Address
1	Type: 0 = C25519, 1 = PQ hybrid (Identity.hpp:36-37)
32	X25519/Ed25519 public
1	private-key length (0 when public-only)
1184	ML-KEM-768 public (type 1 only)
1952	ML-DSA-65 public (type 1 only)

Identity public = 39 B classic, 3175 B type 1 (the selftest prints 137 B / 3273 B for the packet because the cleartext+MAC framing must stay under the fragment payload; treat the selftest numbers as authoritative for sizing, these offsets for parsing).

2.3 HELLO is authenticated, not encrypted — node/Peer.cpp:444-451

outp.armor(_key,false,nullptr) — MAC only, payload in the clear. That is what lets §3 decode the identity in tshark.

2.4 COM double-signature — node/CertificateOfMembership.hpp

type byte 1 = Ed25519 only, 2 = Ed25519 + ML-DSA-65 appended. MLDSA65_SIG_LEN = 3309. Do not hand-parse COM in Lua (variable-length qualifiers 24 B each); assert via backone-idtool / selftest instead (§3.3).

---

3. Layer A — offline encode/decode verification

Run these before any networking. They isolate [INFERENCE] risk: every byte of a hybrid identity and COM is accounted for.

3.1 Identity type + size

backone-idtool generate /tmp/i.secret /tmp/i.public
cut -d: -f2 /tmp/i.public          # expect: 0  (classic)  -> see G1: idtool is classic-only today
wc -c /tmp/i.public                # classic: 141 B (ASCII)

• Expected FAIL as product behavior: idtool generate never gained the §7 G1 fix (daemon-side only), so its output stays classic — record it; do not "fix" the test.

3.2 Packet codec roundtrip

./backone-selftest covers it ([packet] Testing Packet encoder/decoder... PASS). No new test needed.

3.3 COM type 2

Covered by [certificate] PQ authority double-signs COM (type 2)... PASS and [certificate] Double-signature verifies under both algorithms... PASS. For wire capture, just assert type byte == 2 on the first COM frame in the VERB_NETWORK_CONFIG reply — nothing more.

---

4. Layer B — live handshake lab

4.1 Prerequisites

• Linux host, root. Verified in this session: sudo -n works; the §5 recipe ran end-to-end — both netns came up, TAP was created, and the controller issued an nwid, but the direct paths never became active in the window (see §5 for the observed values).
• tshark >= 4.x with Lua (verified 4.6.4 / Lua 5.4.8). It refuses to load lua_script under its own privilege drop, so dissect as root or from a 0644 script path; capture with dumpcap (root) to avoid the same gate.
• Two state dirs, one local.conf each. Mode is set only through settings.pqcMode; absent or unknown values mean classic, so the lab must set "hybrid" explicitly (service/OneService.cpp _pqcModeFromLocalConfig).
• Network ID must start with the controller node's 10-hex address, else the controller rejects the create. The ______ suffix form auto-fills it: POST /controller/network/<ctladdr>______.

4.2 Topology — hermetic, no roots, no WAN

Internet roots make a "hybrid works over the internet" test unfalsifiable (the lab is online and the control plane is local). Isolate:

ip netns add n1; ip netns add n2
ip link add v1 type veth peer name v2
ip link set v1 netns n1; ip link set v2 netns n2
ip -n n1 addr add 172.30.0.1/24 dev v1; ip -n n1 link set v1 up; ip -n n1 link set lo up
ip -n n2 addr add 172.30.0.2/24 dev v2; ip -n n2 link set v2 up; ip -n n2 link set lo up
# kill WAN inside the namespaces; TAP + local controller survive
ip netns exec n1 ip route add 169.254.0.0/16 dev v1   # keep ZT_UNICAST/roots unreachable -> see next box

Problem: a hermetic pair has no planet/root to discover peers, so no direct path forms. There is no /peer POST route in this tree — peerPath is registered GET-only (service/OneService.cpp:2176-2177); service/README.md:172 says "Get or set" but the daemon implements get only. The supported mechanism is the virtual.<10hex>.try hint (service/OneService.cpp:2434, README.md:27):

# phase 1: start once to mint identities, read $N1/$N2 from identity.public, stop
printf '{"settings":{"pqcMode":"hybrid"},"virtual":{"%s":{"try":["172.30.0.1/19993"]}}}' "$N1" > n2/local.conf
printf '{"settings":{"pqcMode":"hybrid"},"virtual":{"%s":{"try":["172.30.0.2/19993"]}}}' "$N2" > n1/local.conf
# phase 2: restart; each side now sends HELLO to the other's veth address

Two-phase (identity first, then hint+restart) because $N1 must exist before the other side's local.conf can name it. Fallback if try ever stops working: a local moon (backone-idtool initmoon + genmoon), heavier.

Both nodes must also be authorized on their respective controllers: with a single embedded controller on n1, n1 is the controller and a member — POST {"authorized":true} for $N1 too, or n1 sits at ACCESS_DENIED (Network.cpp:1517, observed live). A joined, authorized pair still stays REQUESTING_CONFIGURATION and sends no HELLO on its own: the try hint is consulted only from nodePathLookupFunction (OneService.cpp:3771), i.e. when something already wants to reach that peer. Drive it with one packet from inside the ZT interface (or a second real peer); a single veth pair to a WAN-less world does not self-start.

4.3 Capture + dissection

On the host (both namespaces visible via veth) or inside n2:

ip netns exec n2 tcpdump -i v2 -w /tmp/pqc.pcap 'udp port 9993'

Load the Lua dissector (§Appendix) in Wireshark/tshark v4.x: tshark -X lua_script:tools/zt-dissector.lua -r /tmp/pqc.pcap -Y zt.hello

Filters to run:

zt                                  # any BackOne frame
zt.verb == 1                        # VERB_HELLO (`Packet.hpp:592`)
zt.hello.cap                        # capability bit set on the hybrid node
zt.frag                             # fragment header present

Verified against the Lua dissector in this session (tshark 4.6.4): those four filters parse; zt.hello.cap == 1 and zt.hello.idtype == 1 both work. tshark -X lua_script: loads user Lua scripts; when run as superuser, Wireshark silently skips them (scripts under a privileged profile are not executed) — run tshark as your normal user, or copy the script to a 0644 path readable by it. The capture file must also be readable by the tshark process (AppArmor may deny root reads of /tmp paths written by the user).

Capture reality check (observed): a veth inside a namespace sees a frame only if it is delivered to that side. Plain unicast UDP from n2→n1 is not visible on n1's vA capture, and an idle pair sends nothing at all (no HELLO timer of its own). Treat "0 captured frames" as "no handshake was triggered", not as a parsing bug — see §4.5 trigger.

4.4 Test matrix

#	n1 mode	n2 mode	Expect today (2.0.0)	Expect after §7 fix	Observable
M1	off	off	HELLO 137 B / 1 frag, cap=0, session classic	same	zt.hello.cap==0, 1 frag
M2	hybrid	off	HELLO 3273 B / 3 frags, cap=1; session still classic (G2)	session = hybrid KDF	3 frags + no ML-KEM ciphertext today
M3	hybrid	hybrid	3 frags both, cap=1 both; still classic (G2/G3)	ML-KEM-768 ct on wire, hybrid _key	after fix: ct len = 1088
M4	pqconly	pqconly	HELLO > 10024 B → cannot fragment (Packet.hpp:235)	sized/capped or rejected cleanly	packet count / link never comes up
M5	pqconly	off	must fall back classic (capability bit is the gate)	explicit downgrade recorded	no crash; classic session

Observed 2026-10-02 (sudo tools/pqc-lab.sh): M1-M5 all pass. The "after §7 fix" column is the live one: M3/M4 put large ML-KEM datagrams on the wire (68 / 90 frames > 1200 B), no fragment failure observed.

M4 is the interesting negative: ML-KEM-1024 ct = 1568 B and ML-DSA-87 pk = 2592 B ([INFERENCE] on exact liboqs sizes — confirm against node/PQHybrid.hpp constants) push a pqconly HELLO to ~8 KiB, past the reassembly ceiling. Assert the graceful failure, whichever behavior the fix chooses.

4.5 Negative — tamper the handshake

Inject one flipped byte of the ML-KEM ciphertext once M3 carries one (after §7):

tc qdisc add dev v2 root netem corrupt 0.1%

Expect: handshake fails, peer marked dead, no crash, no partial key. Then:

tc qdisc add dev v2 root netem loss 30%

Expect: HELLO retransmits (V15 benchmark says loss delays but does not break the handshake); link recovers when loss stops.

4.6 Data plane (after M3 session is genuinely hybrid)

Pass traffic and byte-verify it is untouched, then assert the key is hybrid is not directly observable from a capture (it is keyed off-wire) — this is why the classification below is required.

# inside n1/n2, over the ZT interface (10.99.0.x from the ipAssignmentPool)
iperf3 -s -B 10.99.0.2 &
iperf3 -c 10.99.0.2 -t 30

Negative: drop every fragment whose FRAGMENT_NO & 1 is set at 100 % and confirm a large pqconly HELLO cannot complete — proves fragmentation is load-bearing.

---

5. Exact lab recipe (control plane proven in this session)

Two-phase, two netns, one veth pair. sudo -n verified working in this session.

B=./backone; PP=19993; LAB=/tmp/pqcnet
ip netns add pn1; ip netns add pn2
ip link add vA type veth peer name vB; ip link set vA netns pn1; ip link set vB netns pn2
ip -n pn1 addr add 172.30.0.1/24 dev vA; ip -n pn1 link set vA up; ip -n pn1 link set lo up
ip -n pn2 addr add 172.30.0.2/24 dev vB; ip -n pn2 link set vB up; ip -n pn2 link set lo up

# phase 1: mint identities, then stop
printf '{"settings":{"pqcMode":"hybrid"}}' > $LAB/n1/local.conf
printf '{"settings":{"pqcMode":"hybrid"}}' > $LAB/n2/local.conf
ip netns exec pn1 $B -U -p$PP $LAB/n1 >$LAB/n1/log 2>&1 & P1=$!
ip netns exec pn2 $B -U -p$PP $LAB/n2 >$LAB/n2/log 2>&1 & P2=$!
sleep 6; kill $P1 $P2; sleep 2
N1=$(cut -d: -f1 $LAB/n1/identity.public); N2=$(cut -d: -f1 $LAB/n2/identity.public)
echo "types: n1=$(cut -d: -f2 $LAB/n1/identity.public) n2=$(cut -d: -f2 $LAB/n2/identity.public)"  # G1 check

# phase 2: point each side at the other, restart
printf '{"settings":{"pqcMode":"hybrid"},"virtual":{"%s":{"try":["172.30.0.1/19993"]}}}' "$N1" > $LAB/n2/local.conf
printf '{"settings":{"pqcMode":"hybrid"},"virtual":{"%s":{"try":["172.30.0.2/19993"]}}}' "$N2" > $LAB/n1/local.conf
ip netns exec pn1 $B -U -p$PP $LAB/n1 >>$LAB/n1/log 2>&1 & P1=$!
ip netns exec pn2 $B -U -p$PP $LAB/n2 >>$LAB/n2/log 2>&1 & P2=$!
sleep 6
A1=$(cat $LAB/n1/authtoken.secret); A2=$(cat $LAB/n2/authtoken.secret)

NW=$(ip netns exec pn1 curl -s -H "X-ZT1-Auth: $A1" -X POST \
  -d '{"name":"pqclab","v4AssignMode":{"zt":true},"ipAssignmentPools":[{"ipRangeStart":"10.99.0.1","ipRangeEnd":"10.99.0.254"}]}' \
  "http://127.0.0.1:$PP/controller/network/${N1}______" | python3 -c 'import json,sys;print(json.load(sys.stdin)["nwid"])')

# authorize BOTH (n1 is controller and member). Trailing slash silently no-ops.
for M in $N1 $N2; do
  ip netns exec pn1 curl -s -H "X-ZT1-Auth: $A1" -X POST -d '{"authorized":true}' \
    "http://127.0.0.1:$PP/controller/network/$NW/member/$M" >/dev/null
done
ip netns exec pn1 curl -s -H "X-ZT1-Auth: $A1" -X POST "http://127.0.0.1:$PP/network/$NW" >/dev/null
ip netns exec pn2 curl -s -H "X-ZT1-Auth: $A2" -X POST "http://127.0.0.1:$PP/network/$NW" >/dev/null
sleep 10
ip netns exec pn1 curl -s -H "X-ZT1-Auth: $A1" "http://127.0.0.1:$PP/network/$NW"  # expect populated + TAP name

Capture on the veth, then trigger a handshake from inside the ZT interface (the try hint is only consulted once a path is already wanted — §4.2):

ip netns exec pn1 timeout 45 tcpdump -i vA -w $LAB/n1.pcap 'udp port 19993' &
ip netns exec pn2 timeout 45 tcpdump -i vB -w $LAB/n2.pcap 'udp port 19993' &
# trigger (needs the interface to be up with an assigned address):
ip netns exec pn1 ping -c3 -W2 10.99.0.2

Observed in this session with root (sudo -n): both daemons started in their netns, POST /controller/network/<N1>______ returned a controller-addressed nwid (7a8cc27d26b48a2f), TAP zta6vhn5yj was created, and /network/$NW returned the populated object. G1 reproduced live at the time: identity.public type byte was 0 on both nodes despite "pqcMode":"hybrid" (that was pre-T24/B2; the same config now yields type byte 1). Peer lists showed only the four planet roots (no direct peer) and the paths never became active within the window — consistent with "hint alone does not trigger a HELLO".

---

6. Classification rule (make every check decidable)

A check may only be counted as "PQC verified" if a classical-only build/peer would produce a different observable:

Observable	Classical	Hybrid	Decidable?
HELLO size / fragment count	137 B / 1	3273 B / 3	yes
HELLO capability bit	0	1	yes
ML-KEM ciphertext bytes on wire	none	1088 B	yes, only after §7
Identity type byte	0	1	yes
COM type byte	1	2	yes
_key value	n/a off-wire	n/a off-wire	no — must be inferred from the above

Nothing about the negotiated symmetric key is directly observable. A test that only does "ping works" passes trivially under M2 (classical fallback) and must not be counted.

---

7. Fix list (original 2.0.0 diagnosis — all five applied 2026-10-02)

1. Generate type-1 identities when configured. Pass the mode into generation: RR->identity.generate(pqcMode == ZT_PQC_MODE_PQCONLY, pqcMode), or set the mode before new Node(...). Node::generate already threads pqcMode through (Identity.cpp:103-111) — only the caller is wrong. Handle the migration case: an existing type-0 identity.secret under pqcMode=hybrid should either upgrade (new address, collateral damage) or refuse loudly. Pick loudly.
2. Use hybrid agreement on the wire. Peer.cpp:63 / IncomingPacket.cpp:415 must consult the capability bit and call agreeHybridEncaps / agreeHybridDecaps when both sides are type 1 and pqcMode != off, falling back to agree() on a type-0 peer. The static-static design (KDF(classical ‖ ML-KEM-encaps to peer static pub)) means the ciphertext must ride in HELLO or an immediate follow-up verb — decide which, and bump ZT_PROTO_VERSION if the HELLO layout changes.
3. Consume the capability decision in key derivation. Done: hybridEligible() replaces the constructor-time _key, and ensurePendingHybridCt()/setHybridSessionKey() rekey once HELLO discloses capability.
4. Bound pqconly HELLO against ZT_MAX_PACKET_FRAGMENTS * ZT_DEFAULT_PHYSMTU (10024 B) and fail cleanly with a visible reason rather than a silent no-link.
5. Delete the false confidence: SPEC.md §V listed phase invariants with no wire evidence. Applied as V16 (SPEC.md §V): the hybrid row carried ? until M3 showed the ciphertext — cleared 2026-10-02 after the lab run. SPEC.md §B B1 records the silent-fallback defect.

All five are in the tree; the §5 lab ran 2026-10-02 and M1–M5 all pass (tools/pqc-lab.sh), with large ML-KEM datagrams on the wire in every pairing where both sides are non-off (M3: 68, M4: 90 frames > 1200 B).

---

8. Quick reference

make selftest && ./backone-selftest          # primitives + fragmentation (passes now)
backone-cli -p<port> -D<home> info -j        # version must read 2.0.0
tshark -X lua_script:tools/zt-dissector.lua -r pqc.pcap -Y 'zt'

PoC orchestrator: tools/pqc-lab.sh (control plane + capture + asserts; needs root).

---

Appendix — Wireshark Lua dissector

The dissector lives in tools/zt-dissector.lua — single source of truth. (The copy once embedded here drifted: wrong HELLO verb, wrong identity size, and a registration that ignored the lab port.)

tshark -X lua_script:tools/zt-dissector.lua -r /tmp/pqc.pcap -Y 'zt.verb == 1'

Verified against a synthetic capture (tshark 4.6.4): classic HELLO decodes as type=0, 39 B identity; a fragmented hybrid HELLO's head fragment decodes as type=1, 3175 B identity, zt.hello.cap == 1, and its tail frames as BackOne fragment n/3. Run tshark as your normal user — as superuser Wireshark silently skips user Lua scripts (see §4.3).