tether/ARCHITECTURE.md

tether — architecture & north star

What this is. tether began as a private Universal Clipboard. The clipboard is the proof, not the point. The point is the thing underneath it: a resilient, transport-agnostic P2P substrate that any private cross-platform app can be built on. The clipboard is app #1.

This document is the durable record of where tether is and where it's going, so the vision survives outside any one conversation. It is a map, not a spec — where it describes things that don't exist yet, they are marked (target).

---

1. The thesis

As long as nodes can come up, find each other, prove who they are, and pass data — securely, over whatever medium is available — everything else is an app-layer concern.

That one sentence is the whole bet. It implies a clean separation into three tiers. Each tier only ever talks to the one below it through a narrow contract, so any tier can be swapped without disturbing the others.

┌─────────────────────────────────────────────────────────────┐
│  APP            clipboard · (next app) · (your app)          │  ← per-app
│                 reads/writes named state; sends/receives msgs │     code
├═══════════════════════════════════════════════════════════════┤
│  SYNC           CRDT state · message/data passing · presence  │  ┐
│                 turns "bytes between peers" into shared state  │  │
├─────────────────────────────────────────────────────────────┤  │ tethercore
│  MESH           identity · transports · discovery ·           │  │ (one crate,
│  (substrate)    secure channel · reachability/relay           │  │  UniFFI'd to
│                 promises: "named, authenticated peers you can  │  │  every client)
│                 reach and send bytes to, over any medium"      │  ┘
└─────────────────────────────────────────────────────────────┘

Tiers vs. crates. The three tiers are a contract boundary, not a packaging one. Mesh and Sync both live in tethercore — sync is app-agnostic (every app reuses dedup, ordering, CRDT merge, delivery, presence), so it belongs in the shared core precisely because it's shared, not pushed up into each app. The only thing outside the crate is the App tier. The double line above (═) is the crate boundary; the single line (─) inside it is the internal Mesh/Sync seam.

The seam is now real (as of the MeshEvent refactor). Transports emit MeshEvents up to a single engine event loop that is the sole owner/writer of all derived state; transports no longer touch engine state. The three clean boundaries today: the Transport trait (Mesh's downward edge), MeshEvent (the internal Mesh→Sync seam), and MessageHandler (the App edge). See §5a.

The contracts between tiers:

• Mesh → Sync: "Here is a set of authenticated, named peers. You can send bytes to any of them and receive bytes from them. I don't care what the bytes mean. I'll tell you who's present and reachable; I hide whether that's LAN, WebRTC, QUIC, Bluetooth, or LoRa."
• Sync → App: "Here is replicated state and a message stream, scoped to an identity. Read it, write it, subscribe to it. I handle conflict resolution, ordering, dedup, and delivery; I hide which peer a change came from."

Everything tether-specific and clever lives in Mesh. Everything reusable across future apps lives in Sync. The clipboard is a thin App.

The one seam that's easy to get wrong

"Find each other locally" is the easy half — mDNS / link-local broadcast on a shared L2 is basically solved. The irreducible hard part of the Mesh — the part that must not leak up into the app — is everything that happens when two nodes are not on the same network:

• Non-local discovery — peers on different networks finding each other → rendezvous, then DHT.
• NAT traversal — hole-punching with a relay fallback → supernodes.
• Identity & secure channel — knowing who a peer is and that the link is private → durable keypairs + an authenticated handshake.

These five — identity, transport, discovery, secure channel, reachability — are the Mesh. Get them right once and the transport medium becomes a detail.

---

2. Transport-agnostic by construction

The Mesh sees every medium through one Transport trait. The upper tiers never know which one carried a byte. Today's ladder, with room to grow downward:

Decided: adopt iroh as the Mesh (spike proven, §8 M1–M3). The hand-rolled ladder below collapses into iroh's one key-addressed substrate — and crucially, the media we'd have hand-rolled (QUIC, BLE) are iroh's, not ours.

Transport	Status	Reach	Notes
LAN (mDNS + length-prefixed TCP)	shipped, to retire	same L2	hand-rolled; iroh does same-LAN direct natively
RTC (WebRTC data channel)	shipped, to retire	cross-network	hand-rolled STUN/TURN; iroh's holepunch+relay replaces it
SSE relay	shipped, to retire	cross-network	signaling + fallback; iroh relay replaces it
iroh QUIC	adopted	LAN + cross-network	one substrate: key-dial, holepunch, relay, 0-RTT
iroh BLE	feature flag	proximity, no IP	iroh-ble-transport via add_custom_transport() — same connection over BLE. Experimental (unstable-custom-transports), ~100kbps, AGPL
LoRa / other	someday	long-range, low-bw	iroh custom-transport API (same hook as BLE) if ever needed

The payoff of adopting iroh: BT is no longer a Transport we write — it's a flag, and the same EndpointId peer is reachable over BLE or IP with iroh choosing the path. The convergence is the validation — re-deriving QUIC + dial-keys-not-IPs + key-exchange + holepunch independently, then finding iroh already implements exactly that, means it's the correct substrate, adopted knowingly rather than cargo-culted.

---

3. Identity & crypto (the two-tier model)

Identity is part of the Mesh, not the app. The model is two layers:

1. Room key — shared symmetric secret. Everyone in a room/group can decrypt group content. Today: key = sha256(account), ChaCha20-Poly1305, payload sealed before any transport touches it (see core/src/crypto.rs). The server and any LAN sniffer see only ciphertext.

   • Known weakness (documented, honest): the account is an email — low entropy. This stops a passive eavesdropper, not someone who knows the account. (target) Replace the secret with a high-entropy source (Sign-in-with-Apple sub, or a passphrase) fed into the same from_secret. No protocol change — just a better secret.

2. Device keypairs — durable per-device identity. (target) Each device has a long-lived X25519 keypair. Peer-to-peer links run an authenticated handshake (Noise via snow, or adopt libsignal/MLS rather than hand-roll) so each pair has forward-secret session keys. The room key is then distributed by sealing it to each device's public key (sealed-box) instead of being derived from a guessable string.

Rotation & unlinkability (target). New shared key → new room. New device identity → new public key, re-handshake. This buys forward secrecy and unlinkability, but creates a discovery tension (if your identity keeps changing, how does someone find you?). Resolved by a stable handle + directory indirection: a durable, user-facing handle maps — privately — to whatever the current key/room is.

Private discovery (target). Looking someone up by email/phone/handle without revealing the lookup table:

• OPRF / PSI for the lookup ("do we both know this contact?" without either side learning the other's full set).
• ZKP for ownership proofs ("I control the key behind this handle"), not for the lookup itself.

These are v2+ research arcs. Don't hand-roll the primitives — stand on vetted libraries.

---

4. The supernode model (there is no "server")

There is no server/client split. There are only nodes. Some nodes — by virtue of public reachability + uptime + bandwidth — volunteer extra roles. Those are supernodes (Skype/ultrapeer/libp2p-relay model). The current Go "server" is a supernode; the architecture just generalizes it from the one to one of many, peer-selected.

A supernode offers any subset of these — none of which touch plaintext:

• Signaling / rendezvous — peer-relayed SDP/ICE exchange (today: the SSE bus).
• TURN relay — forwards E2E ciphertext when hole-punching fails; blind to payload.
• DHT bootstrap + capacity — stable entry points; hold more of the keyspace.
• Discovery hints — same-ASN / same-egress-IP co-location hints (the node that can see public IPs). Strong CGNAT false-positive caveat; mostly useful to bias toward the LAN transport and to avoid TURN.
• Store-and-forward mailbox (target) — hold sealed messages for offline peers.

Promotion = reachable + up + has bandwidth. Auto-detected (a node that finds it's publicly reachable volunteers) or designated (flag the homelab). Patrick's homelab is supernode #1; the hosted relay is #2.

Why "many, peer-selected" matters: a single coordinator is a server with extra steps. Several supernodes, cross-checked, are what resist the real failure modes — eclipse attacks (one node controlling your view of the network) and traffic analysis (metadata is visible even though content is E2E).

Two honest constraints:

• Bootstrap is unavoidable. You must know one supernode to start — a hardcoded list or a DNS name. Even BitTorrent's DHT ships bootstrap nodes. "Decentralized" never meant "zero fixed entry point."
• Incentives at public scale. For Patrick's own devices, he runs the supernodes. For a public backbone, who runs them is a real economics question (Skype conscripted user machines — controversial; BitTorrent/Veilid lean on altruism). Decide deliberately, don't default into it.

---

5. Current state (what actually exists)

At commit 33d0fb7. Verified on real hardware: bidirectional cross-network clipboard sync (iPhone-on-cellular ⇄ Mac), file transfer (RTC + LAN), prefer-direct (relay saw only presence), and E2E (relay carried only ciphertext).

• core/ — tethercore Rust crate, the Mesh + a thin Sync. UniFFI-exported to Swift/Kotlin. Engine multiplexes LAN + RTC + SSE, dedups across them, prefer-direct (suppress relay once all present peers are directly reachable), delivery receipts (hash, not content), presence, nearby (mDNS), file transfer, E2E (crypto.rs). Account-derived room: room = sha256(account)[..4].
• agent/ — headless desktop node; installs as a launchd/systemd service; clipboard loop; saves received files.
• ios/ — SwiftUI iPhone + Mac apps over the same core (TetherKit).
• android/ — scaffold; needs NDK build wired up.
• server/ — Go supernode: pion signaling + integrated TURN + /api/turn-cred (HMAC ephemeral creds) + web client. ⚠ The deployed binary is ahead of this source (integrated TURN, advanced web client, built from an uncommitted tree ~2026-05-21). Do not redeploy server/ as-is — it would regress TURN. Decided (post-iroh): the Go server is retired entirely. iroh subsumes every networking role it has — STUN/TURN → holepunch + relay, RTC signaling → endpoint.connect() (gone), SSE relay → iroh relay, presence → iroh-gossip. Its replacement is off-the-shelf: a self-hosted iroh-relay on the homelab (supernode #1) + a tiny account → {EndpointId} rendezvous for gossip bootstrap. Web is not a reason to keep it (§6.3: web = WASM build of the core). Migration is graceful, not big-bang — IrohTransport runs alongside SSE/RTC/LAN as another MeshEvent source until it wins, then they retire.

What's Mesh today: identity (weak), LAN/RTC/SSE transports, mDNS + relay discovery, E2E channel, presence/reachability. What's Sync today: message passing, dedup, receipts, file frames — but no replicated state (no CRDT). What's App: the clipboard UIs.

5a. Separation-of-concerns: honest grade

Graded against the code (core/src/).

Clean and real (the three boundaries):

• Transport trait — the downward edge. SSE/RTC/LAN each implement it; adding QUIC/BT is "implement the trait." Capability is now typed (direct() / is_relay()), so the engine never matches on name().
• MeshEvent enum — the Mesh→Sync seam. Transports emit events (Message/Status/File/Present/DirectUp/DirectDown/Discovered/ Transfer) over an EventTx; a single engine event loop is the sole owner/writer of all derived state (presence, reachability, discovery, transfers, dedup, receipts). Transports hold no engine state.
• MessageHandler trait — the App edge. The clipboard reaches core only through on_message/on_status/on_file; apps never touch engine internals.
• crypto.rs — isolated; one seal point at the engine boundary before any transport.

The MeshEvent refactor (done) fixed the three gaps this section used to list:

1. No Mesh/Sync seam → MeshEvent is the seam; a new transport (e.g. Iroh) slots in as just another event source without touching the engine loop.
2. Transports mutate Sync state → the dependency arrow is inverted: transports emit, the engine reacts. No more Arc<Mutex> state handed into transports.
3. Stringly-typed capabilities → Transport::direct() / is_relay() replace the name() == "rtc" matches.

Remaining watch-item:

• Engine is still a large struct — it owns the state maps and the event loop. The loop centralizes writes (good), but the struct still has many fields. Fine for now; revisit if a second app lands on the core.

Next, building on the seam: spike Iroh as an alternative Mesh — it becomes a MeshEvent source behind the same Transport/event contract, so the Sync/App tiers above don't change.

---

6. The big gaps (clipboard → backbone)

The hard networking kernel exists. To become a general substrate:

1. State sync, not just messages — adopt Automerge. Apps need replicated state (a doc, list, board), conflict-free across peers. **automerge (Rust)
   • its transport-agnostic automerge::sync protocol over an iroh bi-stream.** The clipboard becomes an Automerge document (history as a CRDT list) synced per-peer; app #2 is another document. This is the heart of the Sync tier and the biggest single addition. Bonus reuse — iroh-blobs subsumes the hand-rolled M/C/E file-frame protocol entirely: content-addressed, resumable, deduped blob transfer for files/images, for free.
2. An app API / SDK. A clean surface: declare data, subscribe, send. The thing a developer building app #2 actually touches.
3. Web target = WASM (resolved by iroh). The future web client is just tethercore compiled to wasm32 as a relay-only iroh peer — same core, no bespoke server bridge ever. A browser iroh endpoint can't holepunch (sandbox: no UDP), so it always reaches peers through the relay (E2E, relay blind); iroh-gossip compiles to WASM too, so discovery (§ M3) works in-browser unchanged. The MeshEvent seam makes this a build target, not a project: native transports (LAN/mdns, webrtc-rs, reqwest) don't compile to WASM and don't need to — feature-gate the transports and the web build is "engine + IrohTransport only, default-features=false, wasm32." Cost: browser peers lean on the relay (no P2P offload) — trivial for clipboard, relay-carried for big files.
4. Capabilities / permissions (target). Multi-user needs object-capabilities (UCAN), not ACLs.
5. Persistence / offline-first (target). Local store + sync-on-reconnect (append-only log or CRDT doc store).

---

7. Don't reinvent the substrate

This space has serious, aligned prior art. The moat is product/DX/UX (the "AirDrop-grade just works" + the app model) — not the transport/DHT plumbing, which is being commoditized. Stand on vetted infra:

• Iroh (n0, Rust) — QUIC P2P + relay hole-punching + discovery (DNS/pkarr-DHT) + iroh-docs/iroh-blobs for sync. Possibly ~70% of the Mesh already, in Rust. Its relay nodes = our supernodes; its bootstrap/DHT = our rendezvous. Evaluate first.
• libp2p — the components (relay-v2 + DCUtR + Kademlia) if you want them à la carte.
• Veilid (cDc, Rust) — privacy-first P2P app framework, DHT + E2E. Aligned with the privacy/DHT angle specifically.
• Automerge / Yjs — the CRDT Sync layer. UCAN — capability auth. Willow / Earthstar — capability-scoped P2P data.

Strategic recommendation: adopt Iroh (Mesh) + Automerge (Sync), build identity/E2E/discovery-UX + the app API on top, keep the clipboard as the flagship app that proves the DX. Run the homelab as a self-hosted Iroh relay + bootstrap node — that's supernode #1 — and the substrate hands you peer-relay + DHT + multi-supernode selection for free, instead of a year of hand-rolling NAT and crypto and getting it subtly wrong.

---

8. Roadmap

Near-term (ship the product):

• Coordinated rebuild + reinstall of all clients so E2E actually activates (running apps still carry the old plaintext core).
• Swap the room secret to a high-entropy source (SiwA sub / passphrase).
• Recover the uncommitted deployed server/ source (don't redeploy until then).
• iOS multicast entitlement (Nearby/LAN are dormant on iOS).
• Android NDK build; clipboard-image auto-send; file-size cap.

Next chapter (start the backbone, clean):

• Extract the Mesh/Sync seam via MeshEvent (§5a) — transports emit events, engine owns state. Fixed the transport→sync coupling and the stringly-typed capability check; verified behavior-preserving over LAN + crypto round-trip. Prerequisite for the Iroh spike — done.
• [~] Spike Iroh as the Mesh — homelab as self-hosted relay/bootstrap (supernode #1). Compare against the current hand-rolled RTC/LAN/SSE. (Drops in as a MeshEvent source once the seam exists.) - [x] Milestone 1 — it builds & connects. examples/iroh_spike.rs: iroh 1.0, two key-addressed endpoints, QUIC bi-stream round-trip in-process. ProtocolHandler::accept ≈ DirectUp + inbound events; connect() ≈ the dial path; connection.remote_id() is a durable per-device public key — identity is the transport (no separate keypair/Noise layer needed, cf. §3). - [x] Milestone 2 — n0's hosted relay carries us. examples/iroh_relay.rs: connect side with address lookup OFF + a relay-only EndpointAddr → the connection had to bootstrap through n0's relay (use1-1.relay.n0.iroh.link). remote_info then showed a DCUtR upgrade to a direct path (relay + direct both Active) — relay bootstrap and holepunch upgrade, on one machine. - [ ] Milestone 2b — true cross-NAT between two machines on different networks; then stand up the homelab as a self-hosted iroh relay (supernode #1) and repeat against it instead of n0's. - [x] Milestone 3 — discovery via iroh-gossip. examples/iroh_gossip.rs: two endpoints derive the same topic from a shared account (TopicId = sha256(account) — its first 4 bytes are today's room_for_account, so the models line up), join, and learn each other's EndpointId by broadcasting presence — zero hardcoded addresses. Bootstrap = knowing one peer's EndpointId; iroh's discovery (n0 DNS here, a rendezvous supernode in prod) maps id→addr. A GossipPresence source would emit MeshEvent::Present/Discovered, retiring the SSE presence bus. Confirmed: gossip compiles to WASM, so the web client uses the same discovery unchanged (§6.3). - [ ] Milestone 4 — wrap as IrohTransport behind the Transport/ MeshEvent contract; A/B against RTC+LAN on latency & connect rate.
• Spike Automerge as the Sync tier; re-express the clipboard as a tiny app over replicated state to validate the App API.
• Durable device keypairs + authenticated handshake (Noise/snow, or libsignal/MLS); seal the room key to device pubkeys.

Horizon (the real vision — stateless P2P app backbone):

• DHT + multi-supernode peer-relay (mostly adopted via Iroh/libp2p, not built).
• Rotatable rooms/keys/identities + stable-handle directory indirection.
• Private discovery (OPRF/PSI lookup; ZKP ownership proofs).
• Capability model (UCAN); offline mailbox.
• Web client = tethercore on wasm32 — relay-only iroh peer, same core, no server bridge. Gated on feature-gating the transports (iroh-only build) + the IrohTransport (M4). A build target, not a project.
• Retire the Go server once IrohTransport wins — replace with a self-hosted iroh-relay (supernode #1) + a tiny gossip-bootstrap rendezvous.

---

9. One-line summary

Peers with durable keypairs · E2E content · pluggable transports (LAN/RTC/QUIC/BT/LoRa) · supernodes volunteering signaling/relay/DHT/discovery · CRDT state sync · clipboard as app #1. Built on Iroh + Automerge, differentiated on the "just works" DX. The network is the product; the clipboard is the proof.