tether/ARCHITECTURE.md

tether — architecture & north star

What this is. tether began as a private Universal Clipboard. The clipboard is the proof, not the point. The point is the thing underneath it: a resilient, transport-agnostic P2P substrate that any private cross-platform app can be built on. The clipboard is app #1.

This document is the durable record of where tether is and where it's going, so the vision survives outside any one conversation. It is a map, not a spec — where it describes things that don't exist yet, they are marked (target).

---

1. The thesis

As long as nodes can come up, find each other, prove who they are, and pass data — securely, over whatever medium is available — everything else is an app-layer concern.

That one sentence is the whole bet. It implies a clean separation into three tiers. Each tier only ever talks to the one below it through a narrow contract, so any tier can be swapped without disturbing the others.

┌─────────────────────────────────────────────────────────────┐
│  APP            clipboard · (next app) · (your app)          │  ← per-app
│                 reads/writes named state; sends/receives msgs │     code
├═══════════════════════════════════════════════════════════════┤
│  SYNC           CRDT state · message/data passing · presence  │  ┐
│                 turns "bytes between peers" into shared state  │  │
├─────────────────────────────────────────────────────────────┤  │ tethercore
│  MESH           identity · transports · discovery ·           │  │ (one crate,
│  (substrate)    secure channel · reachability/relay           │  │  UniFFI'd to
│                 promises: "named, authenticated peers you can  │  │  every client)
│                 reach and send bytes to, over any medium"      │  ┘
└─────────────────────────────────────────────────────────────┘

Tiers vs. crates. The three tiers are a contract boundary, not a packaging one. Mesh and Sync both live in tethercore — sync is app-agnostic (every app reuses dedup, ordering, CRDT merge, delivery, presence), so it belongs in the shared core precisely because it's shared, not pushed up into each app. The only thing outside the crate is the App tier. The double line above (═) is the crate boundary; the single line (─) inside it is the internal Mesh/Sync seam.

The seam is now real (as of the MeshEvent refactor). Transports emit MeshEvents up to a single engine event loop that is the sole owner/writer of all derived state; transports no longer touch engine state. The three clean boundaries today: the Transport trait (Mesh's downward edge), MeshEvent (the internal Mesh→Sync seam), and MessageHandler (the App edge). See §5a.

The contracts between tiers:

• Mesh → Sync: "Here is a set of authenticated, named peers. You can send bytes to any of them and receive bytes from them. I don't care what the bytes mean. I'll tell you who's present and reachable; I hide whether that's LAN, WebRTC, QUIC, Bluetooth, or LoRa."
• Sync → App: "Here is replicated state and a message stream, scoped to an identity. Read it, write it, subscribe to it. I handle conflict resolution, ordering, dedup, and delivery; I hide which peer a change came from."

Everything tether-specific and clever lives in Mesh. Everything reusable across future apps lives in Sync. The clipboard is a thin App.

The one seam that's easy to get wrong

"Find each other locally" is the easy half — mDNS / link-local broadcast on a shared L2 is basically solved. The irreducible hard part of the Mesh — the part that must not leak up into the app — is everything that happens when two nodes are not on the same network:

• Non-local discovery — peers on different networks finding each other → rendezvous, then DHT.
• NAT traversal — hole-punching with a relay fallback → supernodes.
• Identity & secure channel — knowing who a peer is and that the link is private → durable keypairs + an authenticated handshake.

These five — identity, transport, discovery, secure channel, reachability — are the Mesh. Get them right once and the transport medium becomes a detail.

---

2. Transport-agnostic by construction

The Mesh sees every medium through one Transport trait. The upper tiers never know which one carried a byte. Today's ladder, with room to grow downward:

Transport	Status	Reach	Notes
LAN (mDNS + length-prefixed TCP)	shipped	same L2	zero-infra, fastest, works server-down
RTC (WebRTC data channel)	shipped	cross-network	STUN/TURN via supernode; the web-reachable transport
SSE relay	shipped	cross-network	signaling + last-resort carry of (E2E) ciphertext
QUIC	target	cross-network	likely via Iroh; better than hand-rolled RTC
Bluetooth LE	someday	proximity, no IP	"bitchat-style" — works with no network at all
LoRa / other	someday	long-range, low-bw	same trait, tiny MTU; sync layer must tolerate it

The reason BT/LoRa are plausible later rather than fantasy is precisely the trait boundary: a transport only has to (a) discover peers on its medium and (b) move framed bytes between them. The sync layer already tolerates multi-transport delivery (it dedups across transports today), which is the same discipline a high-latency / low-MTU medium demands.

---

3. Identity & crypto (the two-tier model)

Identity is part of the Mesh, not the app. The model is two layers:

1. Room key — shared symmetric secret. Everyone in a room/group can decrypt group content. Today: key = sha256(account), ChaCha20-Poly1305, payload sealed before any transport touches it (see core/src/crypto.rs). The server and any LAN sniffer see only ciphertext.

   • Known weakness (documented, honest): the account is an email — low entropy. This stops a passive eavesdropper, not someone who knows the account. (target) Replace the secret with a high-entropy source (Sign-in-with-Apple sub, or a passphrase) fed into the same from_secret. No protocol change — just a better secret.

2. Device keypairs — durable per-device identity. (target) Each device has a long-lived X25519 keypair. Peer-to-peer links run an authenticated handshake (Noise via snow, or adopt libsignal/MLS rather than hand-roll) so each pair has forward-secret session keys. The room key is then distributed by sealing it to each device's public key (sealed-box) instead of being derived from a guessable string.

Rotation & unlinkability (target). New shared key → new room. New device identity → new public key, re-handshake. This buys forward secrecy and unlinkability, but creates a discovery tension (if your identity keeps changing, how does someone find you?). Resolved by a stable handle + directory indirection: a durable, user-facing handle maps — privately — to whatever the current key/room is.

Private discovery (target). Looking someone up by email/phone/handle without revealing the lookup table:

• OPRF / PSI for the lookup ("do we both know this contact?" without either side learning the other's full set).
• ZKP for ownership proofs ("I control the key behind this handle"), not for the lookup itself.

These are v2+ research arcs. Don't hand-roll the primitives — stand on vetted libraries.

---

4. The supernode model (there is no "server")

There is no server/client split. There are only nodes. Some nodes — by virtue of public reachability + uptime + bandwidth — volunteer extra roles. Those are supernodes (Skype/ultrapeer/libp2p-relay model). The current Go "server" is a supernode; the architecture just generalizes it from the one to one of many, peer-selected.

A supernode offers any subset of these — none of which touch plaintext:

• Signaling / rendezvous — peer-relayed SDP/ICE exchange (today: the SSE bus).
• TURN relay — forwards E2E ciphertext when hole-punching fails; blind to payload.
• DHT bootstrap + capacity — stable entry points; hold more of the keyspace.
• Discovery hints — same-ASN / same-egress-IP co-location hints (the node that can see public IPs). Strong CGNAT false-positive caveat; mostly useful to bias toward the LAN transport and to avoid TURN.
• Store-and-forward mailbox (target) — hold sealed messages for offline peers.

Promotion = reachable + up + has bandwidth. Auto-detected (a node that finds it's publicly reachable volunteers) or designated (flag the homelab). Patrick's homelab is supernode #1; the hosted relay is #2.

Why "many, peer-selected" matters: a single coordinator is a server with extra steps. Several supernodes, cross-checked, are what resist the real failure modes — eclipse attacks (one node controlling your view of the network) and traffic analysis (metadata is visible even though content is E2E).

Two honest constraints:

• Bootstrap is unavoidable. You must know one supernode to start — a hardcoded list or a DNS name. Even BitTorrent's DHT ships bootstrap nodes. "Decentralized" never meant "zero fixed entry point."
• Incentives at public scale. For Patrick's own devices, he runs the supernodes. For a public backbone, who runs them is a real economics question (Skype conscripted user machines — controversial; BitTorrent/Veilid lean on altruism). Decide deliberately, don't default into it.

---

5. Current state (what actually exists)

At commit 33d0fb7. Verified on real hardware: bidirectional cross-network clipboard sync (iPhone-on-cellular ⇄ Mac), file transfer (RTC + LAN), prefer-direct (relay saw only presence), and E2E (relay carried only ciphertext).

• core/ — tethercore Rust crate, the Mesh + a thin Sync. UniFFI-exported to Swift/Kotlin. Engine multiplexes LAN + RTC + SSE, dedups across them, prefer-direct (suppress relay once all present peers are directly reachable), delivery receipts (hash, not content), presence, nearby (mDNS), file transfer, E2E (crypto.rs). Account-derived room: room = sha256(account)[..4].
• agent/ — headless desktop node; installs as a launchd/systemd service; clipboard loop; saves received files.
• ios/ — SwiftUI iPhone + Mac apps over the same core (TetherKit).
• android/ — scaffold; needs NDK build wired up.
• server/ — Go supernode: pion signaling + integrated TURN + /api/turn-cred (HMAC ephemeral creds) + web client. ⚠ The deployed binary is ahead of this source (integrated TURN, advanced web client, built from an uncommitted tree ~2026-05-21). Do not redeploy server/ as-is — it would regress TURN. Recovering that source is an open task.

What's Mesh today: identity (weak), LAN/RTC/SSE transports, mDNS + relay discovery, E2E channel, presence/reachability. What's Sync today: message passing, dedup, receipts, file frames — but no replicated state (no CRDT). What's App: the clipboard UIs.

5a. Separation-of-concerns: honest grade

Graded against the code (core/src/).

Clean and real (the three boundaries):

• Transport trait — the downward edge. SSE/RTC/LAN each implement it; adding QUIC/BT is "implement the trait." Capability is now typed (direct() / is_relay()), so the engine never matches on name().
• MeshEvent enum — the Mesh→Sync seam. Transports emit events (Message/Status/File/Present/DirectUp/DirectDown/Discovered/ Transfer) over an EventTx; a single engine event loop is the sole owner/writer of all derived state (presence, reachability, discovery, transfers, dedup, receipts). Transports hold no engine state.
• MessageHandler trait — the App edge. The clipboard reaches core only through on_message/on_status/on_file; apps never touch engine internals.
• crypto.rs — isolated; one seal point at the engine boundary before any transport.

The MeshEvent refactor (done) fixed the three gaps this section used to list:

1. No Mesh/Sync seam → MeshEvent is the seam; a new transport (e.g. Iroh) slots in as just another event source without touching the engine loop.
2. Transports mutate Sync state → the dependency arrow is inverted: transports emit, the engine reacts. No more Arc<Mutex> state handed into transports.
3. Stringly-typed capabilities → Transport::direct() / is_relay() replace the name() == "rtc" matches.

Remaining watch-item:

• Engine is still a large struct — it owns the state maps and the event loop. The loop centralizes writes (good), but the struct still has many fields. Fine for now; revisit if a second app lands on the core.

Next, building on the seam: spike Iroh as an alternative Mesh — it becomes a MeshEvent source behind the same Transport/event contract, so the Sync/App tiers above don't change.

---

6. The big gaps (clipboard → backbone)

The hard networking kernel exists. To become a general substrate:

1. State sync, not just messages. Apps need replicated state (a doc, list, board), conflict-free across peers → CRDTs (Automerge / Yjs). This is the biggest single addition and the heart of the Sync tier. The clipboard is stateless message-passing; a backbone is replicated state.
2. An app API / SDK. A clean surface: declare data, subscribe, send. The thing a developer building app #2 actually touches.
3. Web target = WASM. The Rust core compiles to WASM, but the browser sandbox limits web peers to WebRTC / WebSocket / WebTransport (no raw UDP/TCP/mDNS — same wall as iOS's multicast entitlement). Web is a first-class but transport-limited peer.
4. Capabilities / permissions (target). Multi-user needs object-capabilities (UCAN), not ACLs.
5. Persistence / offline-first (target). Local store + sync-on-reconnect (append-only log or CRDT doc store).

---

7. Don't reinvent the substrate

This space has serious, aligned prior art. The moat is product/DX/UX (the "AirDrop-grade just works" + the app model) — not the transport/DHT plumbing, which is being commoditized. Stand on vetted infra:

• Iroh (n0, Rust) — QUIC P2P + relay hole-punching + discovery (DNS/pkarr-DHT) + iroh-docs/iroh-blobs for sync. Possibly ~70% of the Mesh already, in Rust. Its relay nodes = our supernodes; its bootstrap/DHT = our rendezvous. Evaluate first.
• libp2p — the components (relay-v2 + DCUtR + Kademlia) if you want them à la carte.
• Veilid (cDc, Rust) — privacy-first P2P app framework, DHT + E2E. Aligned with the privacy/DHT angle specifically.
• Automerge / Yjs — the CRDT Sync layer. UCAN — capability auth. Willow / Earthstar — capability-scoped P2P data.

Strategic recommendation: adopt Iroh (Mesh) + Automerge (Sync), build identity/E2E/discovery-UX + the app API on top, keep the clipboard as the flagship app that proves the DX. Run the homelab as a self-hosted Iroh relay + bootstrap node — that's supernode #1 — and the substrate hands you peer-relay + DHT + multi-supernode selection for free, instead of a year of hand-rolling NAT and crypto and getting it subtly wrong.

---

8. Roadmap

Near-term (ship the product):

• Coordinated rebuild + reinstall of all clients so E2E actually activates (running apps still carry the old plaintext core).
• Swap the room secret to a high-entropy source (SiwA sub / passphrase).
• Recover the uncommitted deployed server/ source (don't redeploy until then).
• iOS multicast entitlement (Nearby/LAN are dormant on iOS).
• Android NDK build; clipboard-image auto-send; file-size cap.

Next chapter (start the backbone, clean):

• Extract the Mesh/Sync seam via MeshEvent (§5a) — transports emit events, engine owns state. Fixed the transport→sync coupling and the stringly-typed capability check; verified behavior-preserving over LAN + crypto round-trip. Prerequisite for the Iroh spike — done.
• Spike Iroh as the Mesh — homelab as self-hosted relay/bootstrap (supernode #1). Compare against the current hand-rolled RTC/LAN/SSE. (Drops in as a MeshEvent source once the seam exists.)
• Spike Automerge as the Sync tier; re-express the clipboard as a tiny app over replicated state to validate the App API.
• Durable device keypairs + authenticated handshake (Noise/snow, or libsignal/MLS); seal the room key to device pubkeys.

Horizon (the real vision — stateless P2P app backbone):

• DHT + multi-supernode peer-relay (mostly adopted via Iroh/libp2p, not built).
• Rotatable rooms/keys/identities + stable-handle directory indirection.
• Private discovery (OPRF/PSI lookup; ZKP ownership proofs).
• Capability model (UCAN); offline mailbox; web (WASM) peer.

---

9. One-line summary

Peers with durable keypairs · E2E content · pluggable transports (LAN/RTC/QUIC/BT/LoRa) · supernodes volunteering signaling/relay/DHT/discovery · CRDT state sync · clipboard as app #1. Built on Iroh + Automerge, differentiated on the "just works" DX. The network is the product; the clipboard is the proof.