Guest User

Untitled

a guest
Nov 19th, 2026
63
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 23.60 KB | None | 0 0
  1. # Distributed P2P Learning System — Design Spec
  2.  
  3. Nov 19, 2026 · @Jesse
  4.  
  5. ## Overview
  6.  
  7. This spec defines a general model that trains and runs across volunteer desktops with no central server. "AGI" is not an acceptance criterion, since no known architecture produces general intelligence; the spec targets the substrate and measurable results.
  8.  
  9. **Goals**
  10.  
  11. - Run the model locally on Mac, Windows and Linux, GPU first with CPU fallback.
  12. - Train it collaboratively over home internet links, tolerating nodes that join, leave and crash.
  13. - Merge only updates that measurably improve held-out evaluations.
  14. - Give every user a chat app, and network resources in proportion to their node's contribution.
  15. - Stay safe to install: opt-in, sandboxed, resource-capped, stoppable by the owner at any time.
  16.  
  17. **Non-goals (v1):** frontier-scale training, code that updates itself, Android.
  18.  
  19. **Success metric:** at a fixed parameter budget, the swarm-trained model matches a centrally trained baseline on a public eval suite within a stated margin, at a stated cost in node-hours.
  20.  
  21. ## Constraints
  22.  
  23. Bandwidth is the binding constraint: a home uplink moves about 1–5 MB/s, roughly 1/100,000th of a datacenter interconnect.
  24.  
  25. | Constraint | Typical value (approx.) | Design consequence |
  26. | --- | --- | --- |
  27. | Home uplink | 10–50 Mbps | A full sync of a 7B model takes 40 min to 4 h, so syncs must be rare and compressed |
  28. | Peer latency | 30–300 ms | Inference stays local |
  29. | Device memory | 8–24 GB VRAM; 8–64 GB RAM | Nodes hold a small quantized model or a shard |
  30. | Compute spread | \~100x across devices | Work is sized per node; no one waits on stragglers |
  31. | Availability | Hours a day, unpredictable | Fully asynchronous protocol |
  32. | Trust | Any peer may be malicious | Every contribution is verified before it is merged |
  33.  
  34. So capacity grows by adding **experts**, not by enlarging one dense model, and nodes take hundreds of local steps between syncs.
  35.  
  36. ## Architecture
  37.  
  38. Every node runs the same six-layer stack and reaches the network only through its P2P layer. Validators are ordinary high-reputation nodes picked at random each round; releases come from the public code repository.
  39.  
  40. [embedded content: node architecture · 6 layers, network, 3 outside roles\]
  41.  
  42. The update and governance layer is the gate: nothing from the network changes the model without passing signature and eval checks, and nothing from the network changes the code at all.
  43.  
  44. ## Runtime and compute
  45.  
  46. The node is written in Rust, with hand-tuned code only in about 20 compute kernels (matmul, attention, norms, optimizer). Elsewhere the bottleneck is memory and network, where assembly buys nothing. Rust gives native speed with no garbage collector, and memory safety in the code that faces untrusted peers.
  47.  
  48. | Backend | Hardware | OS | Kernels written in |
  49. | --- | --- | --- | --- |
  50. | Metal | Apple silicon GPUs | macOS | Metal Shading Language |
  51. | CUDA | NVIDIA GPUs | Windows, Linux | CUDA C compiled to PTX |
  52. | Vulkan compute | AMD, Intel, NVIDIA; Android later | Windows, Linux, Android | GLSL or Slang compiled to SPIR-V |
  53. | CPU | Any x86-64 or ARM64 | All | Rust with SIMD: AVX2, AVX-512, NEON |
  54.  
  55. - **Backend choice:** probed at startup. Vulkan covers every non-Apple GPU; CUDA is kept for NVIDIA tensor-core speed.
  56. - **Kernels:** forward and backward passes for every op, plus the optimizer. Backward kernels are the main engineering cost; existing runtimes such as llama.cpp cover inference well but training only thinly.
  57. - **Precision:** 4- or 8-bit weights for inference; bf16/fp16 training with fp32 master copies. CPU-only nodes train small experts or validate.
  58. - **Execution:** a static graph per model block, memory planned up front, no allocation in the hot loop.
  59.  
  60. ## Model and training
  61.  
  62. A small shared backbone runs on every node, and a large pool of experts is spread across the network. Nodes train locally for hundreds of steps, then send compressed weight changes; validators merge them only if evaluations improve (asynchronous, DiLoCo-style training).
  63.  
  64. **Model**
  65.  
  66. - **Backbone:** a dense transformer of about 1–3B parameters, 4-bit quantized (\~1–2 GB). It runs locally, so chat works offline.
  67. - **Experts:** 50–300M parameters each, chosen by a learned top-k router, each hosted on at least 3 nodes. A missing expert is fetched and cached, never called over the network per token.
  68. - **Ownership:** each node trains the backbone lightly and its hosted experts heavily. New experts fork from a parent; replay of earlier data and the eval gate guard against forgetting.
  69.  
  70. **Built from scratch.** No outside weights, tokenizer or post-training data with incompatible terms. Everything the model is made of can be released under the AGPL.
  71.  
  72. - **Tokenizer:** a byte-level BPE vocabulary trained on the project's own corpus.
  73. - **Pre-training:** starts from random initialisation. A rough compute guide: training compute ≈ 6 × parameters × tokens. A 1B backbone on 20B tokens (the compute-optimal minimum) is \~1.2×10²⁰ FLOPs, about 70 days of one consumer GPU at realistic efficiency. Useful small chat models today are trained on far more data: 1B parameters on 1T tokens is \~6×10²¹ FLOPs, about 3,500 GPU-days, or a few weeks across a few hundred nodes before low-communication overhead.
  74. - **Post-training:** a chat model needs instruction and preference data. Only data with AGPL-compatible terms is used: openly licensed human-written sets and data the network generates itself. Many popular open instruction sets were generated by commercial models whose terms restrict this use, so each source is checked.
  75. - **Cold start:** the first public model will be small and noticeably weaker than current chat assistants. The chat app should say so, and quality rises as the network trains.
  76.  
  77. **One training round**
  78.  
  79. 1. Pull the current signed checkpoint and an assigned data shard.
  80. 2. Run H local steps (100–1,000, sized to a 30–90 minute round on that node).
  81. 3. Compute the pseudo-gradient (local weights minus the starting checkpoint), compress it 100–1,000x (4-bit quantisation, top-k sparsity, error feedback) and publish it.
  82. 4. Validators drop outliers (trimmed mean), down-weight stale updates, and apply the result with an outer Nesterov optimizer.
  83. 5. Randomly chosen validators run held-out quality and behaviour evals on the candidate.
  84. 6. If it does not regress, a validator quorum signs and publishes it; otherwise it is dropped and its contributors flagged.
  85.  
  86. Training never runs while the owner is using the machine. **Unproven:** convergence on home links with heavy churn. Prior work (DiLoCo, Branch-Train-Merge, DiPaCo, Petals) used far better-connected hardware, so Phase 2 measures this before anything is built on it.
  87.  
  88. ## Training data
  89.  
  90. Nodes gather text from the open web but act as one polite crawler: each page is fetched once by one node and shared with peers, so no site sees more load than from a single well-behaved crawler.
  91.  
  92. **Sources, in order of preference:** public dumps (Common Crawl, Wikipedia, Project Gutenberg, arXiv, openly licensed code), mirrored between peers by hash; official APIs within their limits; direct crawling for the rest.
  93.  
  94. **Crawl rules,** built into the binary rather than left to settings:
  95.  
  96. - Obey robots.txt (including rules for AI crawlers), noai opt-out tags, and the opt-out list kept in the public repository.
  97. - One crawler per domain at a time, through a DHT lease; at most one request every few seconds, or the site's crawl-delay; back off on 429 or 503.
  98. - Fetch only when no fresh shared copy exists, using conditional requests.
  99. - Never log in or bypass paywalls or CAPTCHAs; skip pages about private individuals.
  100.  
  101. **Identity:** robots.txt token `DoraBot`, fixed across versions. User-Agent: `Mozilla/5.0 (compatible; DoraBot/1.0; +https://<project-site>/bot)`. Rules under `User-agent: DoraBot` override `User-agent: *`, and the bot page explains the crawler, its limits and how to opt out. A site that blocks crawlers generally but wants Dora adds:
  102.  
  103. ```
  104. User-agent: DoraBot
  105. Allow: /
  106. ```
  107.  
  108. Requests are signed (HTTP Message Signatures, RFC 9421) with network-certified keys so sites can verify them. A signature proves the request came from a network node, not that the node behaves well.
  109.  
  110. **Pipeline:** extract text, remove duplicates (MinHash), filter for quality, scrub personal data, check against known-illegal-content hash lists, pack into shards, assign shards to trainers by hash.
  111.  
  112. **Storage:** shards hold text only and are encrypted per shard (XChaCha20-Poly1305), addressed by the hash of the ciphertext. Seeding nodes never hold keys; trainers hold them in memory only. Removal works through the takedown log below.
  113.  
  114. Encryption protects integrity and privacy; it does not provide legal deniability. Trainers decrypt, the code is open, and in MGM v. Grokster (2005) the Supreme Court treated a P2P design's aims as evidence of intent to induce infringement.
  115.  
  116. **Takedowns.** A signed, append-only takedown log spreads through the network like any other content, and every node checks it before storing, serving or training on a shard.
  117.  
  118. - **Authority:** one named takedown steward, chosen in governance, with a published policy that favours speech. It acts only on (1) matches against known-illegal-content hash lists and (2) legally valid notices or court orders. It rejects notices that fail on their face, and publishes every notice it acts on, redacted where the law requires, to the Lumen database.
  119. - **Entries:** each entry names a shard ID (the hash of the encrypted shard), a source-URL hash so the crawler never re-fetches it, a reason code, and a date. Entries never contain the illegal-content hashes themselves, since those lists are confidential to their providers.
  120. - **Distribution:** entries are signed with the steward's key, which is pinned in each release, and published on a gossipsub topic. Their DHT records mark the shard as revoked. Nodes keep the full log locally, as a filter that can be checked quickly.
  121. - **Enforcement:** before fetching, seeding, serving or assigning a shard, a node checks the log. On a match it deletes the shard, refuses to send it, and drops any training job that uses it. Peers that keep serving revoked shards lose reputation.
  122. - **Accountability:** the log is append-only and public (like Certificate Transparency), so every removal is visible and none can be made silently. The steward can remove data; it can never add data or change code. The public history is never erased, but the active list nodes enforce changes (see Removing entries).
  123.  
  124. **Removing entries.** Every entry can come off the active list. Nodes enforce only the active set: all entries, minus those lifted or expired.
  125.  
  126. - **Lift:** the steward publishes a signed lift entry naming the original entry. This is used after a successful counter-notice, a withdrawn notice, a court reversal, or a mistake. Nodes drop the block, and the crawler may fetch the page again.
  127. - **Expire:** copyright entries carry the date the work enters the US public domain, when known. On that date every node drops the entry automatically, with no action from the steward.
  128. - **Never expire:** illegal-content entries. They come off the list only by an explicit lift, if one turns out to be a false match.
  129. - **Compaction:** nodes keep only active entries in their local filter. The full history stays available from the log for audit.
  130.  
  131. **Public domain.** The steward rejects copyright notices for public-domain works on their face. In the US that includes works published before 1931 (as of 2026; the cut-off moves forward one year each 1 January), US federal government works, and works whose owners have dedicated them to the public domain (for example CC0). Public-domain sources such as Project Gutenberg's US texts are preferred data and are never subject to copyright takedowns, though illegal-content checks still apply to everything.
  132.  
  133. **Limits:** a takedown removes data going forward, but cannot remove what the model has already learned from it. The steward's key is a single point of control; if it is coerced or stolen it can remove content network-wide, though only visibly. No existing free-speech organisation offers this role as a service as far as I know, so the steward is a partner or body the project has to set up.
  134.  
  135. **Risks:** deliberate poisoning by crafted web pages, which reputation, de-duplication, filters and the eval gate reduce but do not remove. Shard traffic counts against each node's bandwidth caps.
  136.  
  137. ## Legal position (US law)
  138.  
  139. The project relies on fair use to train on web content: a defensible position, not a settled one. This records the design's assumptions, not legal advice.
  140.  
  141. - **Training:** Bartz v. Anthropic and Kadrey v. Meta (2025) found training on lawfully acquired works to be fair use; Bartz rejected keeping pirated copies. In October 2026 the Third Circuit held in Thomson Reuters v. Ross that training a non-generative tool was not fair use. The Supreme Court has not ruled. So: publicly reachable pages only, no circumvention of access controls, pirated sources blocklisted.
  142. - **Sharing:** peers passing crawled copies to each other is distribution, which is less clearly covered than training. Encryption does not change that. See Open questions.
  143. - **Outputs:** purely AI-generated output cannot be copyrighted (Thaler v. Perlmutter; Supreme Court review denied March 2026), and the network claims none. A person who meaningfully edits output may own their contribution. Verbatim reproduction of training text can still infringe, so nodes block long overlaps with the training data.
  144.  
  145. ## Networking and credits
  146.  
  147. The network runs on libp2p's Rust implementation, which already handles transport, discovery and NAT traversal. Remote resources are paid for in credits earned by contributing; the local model is always free.
  148.  
  149. | Concern | Mechanism |
  150. | --- | --- |
  151. | Transport | QUIC, with TCP + Noise encryption as fallback |
  152. | Identity | Ed25519 keypair per node; peer ID = hash of the public key |
  153. | Discovery | Kademlia DHT seeded from bootstrap nodes; mDNS on the local network |
  154. | NAT traversal | AutoNAT, hole punching (DCUtR), relays as a last resort |
  155. | Messaging | Gossipsub topic per shard for checkpoints and updates; one network-wide topic for the takedown log |
  156. | Bulk data | Content-addressed chunks fetched from any holder |
  157. | Directory | DHT records map each expert to its hosts and latest checkpoint |
  158.  
  159. **Credits**
  160.  
  161. - **Earned** per hour of verified uptime, scaled by benchmarked compute, proven storage and bandwidth, plus accepted work: training, validation, serving, crawling. Random peer challenges verify availability; a missed challenge pauses accrual.
  162. - **Expire** 30 days after they are earned, oldest spent first. They cannot be transferred and are not a currency.
  163. - **Spent** on remote experts and offloaded jobs. No contribution means no remote resources. Jobs larger than spare capacity run in chunks, and every serving node rate-limits each peer.
  164.  
  165. **Risks:** people renting cloud machines to farm uptime (scaling credit to verified resources, not hours, keeps this no more profitable than honest contribution). The largest affordable job is about one month of contribution. A peer that computes part of a request reads it in plain text, so the chat app flags remote requests and offers a local-only mode.
  166.  
  167. ## Updates
  168.  
  169. Weights update automatically through training; code changes only when the owner installs a new release by hand. A network that can push code to its members is structurally a botnet's command channel, so it gets no such path.
  170.  
  171. - **Weights:** a fixed tensor format with no executable content, accepted only with validator-quorum signatures and passing evals. Nodes keep recent checkpoints and can roll back.
  172. - **Code:** downloaded from the public repository's releases on GitHub. There are no maintainer keys: public CI builds each release, Sigstore keyless signing and GitHub provenance attestations tie each binary to its commit, and builds are reproducible. The tray can report a new version; it never installs one.
  173. - **Compatibility:** the network accepts the current and previous protocol versions, so owners upgrade on their own schedule.
  174. - **The model cannot propose, write or publish code.**
  175.  
  176. **Risks:** whoever controls the repository controls releases, so a hijacked account can ship malware to anyone who does not verify downloads. There is no network-wide kill switch: a harmful checkpoint that passes evals can only be rolled back node by node.
  177.  
  178. ## Security and sandbox
  179.  
  180. Assume malicious peers from day one. Defences are layered because no single check is reliable, and cheaply verifying arbitrary training work is an unsolved problem.
  181.  
  182. | Threat | Defence | Residual risk |
  183. | --- | --- | --- |
  184. | Poisoned updates | Trimmed-mean aggregation, eval gate, per-peer reputation | Subtle backdoors can pass general evals |
  185. | Fake work | Validators re-run random slices of a peer's steps from its declared seed and data | Probabilistic; cheating must be rare to pay |
  186. | Many identities (Sybil) | New identities start with zero credit and near-zero update weight; credit scales with verified resources | Patient attackers can build reputation |
  187. | Validator capture | Validators drawn at random each round; quorum signatures | A large colluding bloc |
  188. | Malicious checkpoint files | Fixed tensor format, strict parser in safe Rust, fuzz-tested | Parser bugs |
  189. | Compromised node | OS sandbox, never runs as root or Administrator | Sandbox escapes |
  190.  
  191. **OS sandbox:** macOS App Sandbox; Windows AppContainer; Linux seccomp, Landlock and namespaces under a dedicated user. The node never trains on the owner's own files.
  192.  
  193. **Owner settings:** the node cannot write its own settings file. The service manager starts the node with a sandbox profile built from the file, so a compromised node cannot raise its own permissions.
  194.  
  195. | Area | Settings (default) | Enforced by |
  196. | --- | --- | --- |
  197. | Network | Peers off / LAN / internet (internet); crawling (on); remote inference (on, local-only switch in chat); caps (1 MB/s up, 100 GB/month); metered connections treated as offline | OS on Linux; node elsewhere |
  198. | Files | Data folder (50 GB cap); chat attachments granted per folder, read-only; nothing else | OS sandbox |
  199. | Hardware | GPU choice; VRAM 70%, RAM 25%, half the CPU cores; thermal ceiling; pause on battery | Node, plus OS limits where available |
  200. | Devices | Camera, microphone, USB: never | OS sandbox |
  201. | Schedule | Start at login (on); train only when idle (on) | Service manager, node |
  202.  
  203. Presets set these at once: **Strict** (local chat only), **Standard** (the defaults), **Contributor** (higher caps, training whenever idle). Settings enforced by the node guard against accidents, not against a tampered build; reproducible builds cover that gap.
  204.  
  205. ## Apps, platforms and distribution
  206.  
  207. Owners see two apps in front of the background node: a chat app and a tray icon. Both are built with Tauri (a Rust core with a web interface in the OS's own webview) and talk to the node over a local socket authenticated with a per-install token.
  208.  
  209. - **Chat app:** streaming replies with Markdown and code rendering, light and dark themes. History is stored only on the device, encrypted. It shows when a request leaves the machine and which experts answered, plus the credit balance.
  210. - **Tray icon:** status (training, idle, paused, offline, update available), pause and resume, settings and presets, live usage and credit, logs, and a link to new releases.
  211.  
  212. | Platform | GPU backend | Runs as | Package |
  213. | --- | --- | --- | --- |
  214. | macOS (Apple silicon; Intel at low priority) | Metal | launchd user agent | Signed, notarized .pkg |
  215. | Windows 10/11 x64 | CUDA, Vulkan | Low-privilege service | Signed MSI |
  216. | Linux x64 / ARM64 | CUDA, Vulkan | systemd user service | .deb, .rpm, tarball |
  217. | Android (later) | Vulkan | Foreground service while charging on Wi-Fi | APK / Play Store |
  218.  
  219. **Footprint:** under 40 MB per installer (estimate, to confirm in Phase 1): node binary 15–25 MB, apps 5–10 MB, GPU kernels 1–5 MB. The model is not in the installer: the 1–2 GB backbone downloads from peers on first run, and experts (50–300 MB each) only when needed. It stays small because CUDA kernels load through NVIDIA's driver API (no cuBLAS or cuDNN), each OS gets its own installer, no runtimes are bundled, and every role lives in one binary.
  220.  
  221. ## Open source and documentation
  222.  
  223. Everything is AGPL-3.0: all code (node, kernels, protocol library, apps, crawler, build tooling) and the model weights. The protocol specification and docs are CC BY 4.0, so independent nodes can be built from them.
  224.  
  225. - **Dependencies** must be AGPL-compatible (Apache-2.0, MIT, BSD, MPL-2.0, GPL-3.0; not GPL-2.0-only). CI checks licenses on every build.
  226. - **Contributions** come with a Developer Certificate of Origin sign-off, not a copyright assignment, so no one can later close the code. Relicensing would need every contributor's agreement.
  227. - **Source link:** each node shows its exact source (release tag and build hash) on the bot page and in the chat app's About screen, as the AGPL's network clause requires.
  228. - **Weights:** the license notice treats the weights as their own source and serving them as network use. How AGPL applies to weights is untested, so this records intent more than it guarantees enforcement.
  229. - **Closed layers:** CUDA and Metal are proprietary platform frameworks. A Vulkan-plus-CPU build is fully open and ships alongside.
  230.  
  231. **Docs,** versioned in the repository: user guide, architecture guide (this spec plus a decision log), protocol specification, API reference (rustdoc, with examples tested in CI), kernel guide, threat model and security policy, the DoraBot page, governance, contributor guide. No change to behaviour, protocol or settings merges without its docs.
  232.  
  233. ## Roadmap and open questions
  234.  
  235. Each phase can fail cheaply, and the riskiest assumption, that training converges on home links, is tested in Phase 2 before networking and security work is spent on it.
  236.  
  237. 1. **Local runtime.** Inference kernels on CPU, Metal, CUDA and Vulkan, tested on randomly initialised weights against a reference implementation, with the chat app and tray on all three OSes. *Gate:* within 20% of llama.cpp speed on a model of the same shape and hardware.
  238. 2. **Local training, then simulated swarm.** Backward kernels; train the tokenizer and a 100–300M model from scratch on one machine; simulate 50–500 nodes on one box with injected bandwidth limits, latency and churn. *Gate:* within a set margin of centralised training loss.
  239. 3. **Trusted network.** 5–20 real machines run by trusted people, over libp2p, training the first 1B backbone that ships to users. *Gate:* the Phase 2 result holds on real links.
  240. 4. **Open network.** Validators, reputation, spot checks, storage proofs, credits, attested releases, external security review. *Gate:* red-team poisoning and Sybil attacks fail.
  241. 5. **Growth and Android.** Experts added on demand; the Vulkan path ported to phones.
  242.  
  243. **Open questions**
  244.  
  245. - **Data sharing:** may peers share copies of crawled pages, or only fetch lists, and for how long? Needs a US copyright lawyer before the open network launches.
  246. - **Governance:** who controls the repository and releases, and what happens if that account is compromised or abandoned?
  247. - **Verification:** can spot checks catch enough cheating at acceptable cost? An open research question.
  248. - **Scale ceiling:** the largest model this design can usefully train on home hardware; Phase 2 gives the first estimate.
  249. - **Project site:** the URL for the DoraBot page and source links.
  250. - **Takedown steward:** who holds the role, how the steward's key is replaced, and whether the project registers a DMCA agent. The DMCA safe harbor was written for service providers; how it applies to a P2P network with no central host is uncertain.
  251.  
Advertisement
Add Comment
Please, Sign In to add comment