The Next Chapter in the Story of Cloud Computing is P2P

— Written by heapwolf

Today we're announcing the general public preview our new Peer To Peer network protocol.

Why

There is more hardware in the world than ever before. As hardware proliferates, it becomes more powerful and produces more data at a velocity that is only increasing. With more than 16 billion mobile devices, 3 billion laptops, and countless IoT devices, the world is entering an era of ubiquitous computing; a thesis held by many experienced technologists for over a decade.

Cloud services and client-server based architectures have been a successful solution for high availability and on-demand computing and storage. But they have also demonstrated that, as scale up, they become more expensive, more complex, and less reliable.

P2P can reduce cost, complexity, and improve reliability by moving workloads out of the data-center and onto the end-points of the network. This also provides opportunities for improved privacy and security.

This is why we see P2P as the next chapter in the story of cloud computing.

Scope of the Protocol

P2P is a broad term that can include a lot of things. To create a framework for having a conversation, we like to break the problem space into 5 distinct areas, Networking, Compute, Replication, Consensus, and Storage. Today, we solve the Networking problem. We help you discover, connect, and communicate with peers in the most reliable way. This is the solid foundation that Web developers need to build interesting things in the other areas.

High Level Summary

  • This is a network protocol is called Stream Relay Protocol (aka SRP).

  • The entire protocol is implemented in JavaScript for the purposes of being easy to audit, document, and reproduce.

  • It has zero dependencies. And at the time of this writing has 2171 lines of code.

  • It's a gossip protocol that demonstrates self-organizing and small-world properties.

  • It's based on highly peer reviewed computer science research; SRP is the only protocol that runs in a web-like environment that was designed to satisfy the full criteria for DDIL and DTN.

  • Comprehensive NAT traversal.

  • There is no case where an end user or developer using the protocol is required to subsidize any server infrastructure.

A Global Network

Most P2P protocols produce isolated networks of participants — a problem when Alice wants to send a message to Bob when he is offline. SRP takes the opposite approach, creating a shared global network where each peer, regardless of who they know, help each other connect and relay packets without needing server infrastructure.

Since packets are encrypted and causally linked, they can be acquired by any peer in any order, and read only by the intended recipients.

Every peer in the network has exactly the same capabilities, and this symmetry makes it highly tolerant to failures and partitions. This also ensures that every peer continues to operate normally regardless of the condition of any other peers in the network.

Hello, World

The API uses an event emitter pattern so it should be immediately familiar to any JavaScript developer.

import { network, Encryption } from 'socket:network'

After importing the module, create (or read from storage) a peer ID and an ed25519 key-pair for signing packets that this peer creates.

const peerId = await Encryption.createId()
const signingKeys = await Encryption.createKeyPair()

Note To make this example more readable, we don't save and read the peer's ID. In practice, you should save it after creating it, and then read it from where ever you saved it because you usually want a peer IDs to be deterministic across application restarts.

Next, create (or read from storage) a clusterID. The purpose of this is to help peers in the network introduce you to peers who you know.

Don't actually use 'TEST', you most likely want to pick a unique seed or use a random values for these. If you don't, you may end up receiving unexpected messages with unexpected encodings from strangers.

const clusterId = await Encryption.createClusterId('TEST')

Next, create the instance of the socket and pass the values you've created.

const socket = await network({ peerId, clusterId, signingKeys })

Before we create data for other peers in our network to receive, we should create a subcluster — this is a way to "bucket" the data in your cluster. In this case we will create a partition called greetings. Again, see the earlier note about persisting keys for determinism.

const sharedKey = await Encryption.createSharedKey('greetings')
const greetings = await socket.subcluster({ sharedKey })

Now that we have that out of the way we can create a small program.

//
// A published message on this subcluster has arrived!
//
greetings.on('hello, world', value => console.log(value))

//
// A message will be published into this subcluster
//
socket.on('#ready', () => {
  greetings.emit('hello, world', { language: 'english' })
})

//
// Another peer from this subcluster has directly connected to you.
//
greetings.on('#join', peer => {
  peer.on('hello, world', value => {
    console.log(value)
  })
})

Opportunity

The way most people are using SRP is to augment their existing Cloud services and offload work to end-user hardware.

But any front-end developer familiar with HTML, CSS, and JavaScript should be able to create a fully functional chat app like Telegram, a social media app like Twitter, a coordination app like Uber, or a collaborative content creation app like Figma or Notion without needing any servers or Cloud services at all.

Network topology

Client-Server

Contemporary client-server designs tend to keep the majority of intelligence off the client and in the data-center. In the many-to-few model, millions of clients connect to a small number of servers, a natural concurrency bottle-neck.

Peer-To-Peer

P2P is a many-to-many model, so concurrency is less of a concern. Intelligence is distributed equally in the network. Clients retain data and do compute that is relevant for them. This has better scaling properties and lower costs, and at the same time it doesn't preclude business intelligence or intellectual property.

NAT Traversal (aka Hole Punching)

SRP makes sure every peer is able to communicate with other peers directly or indirectly, regardless of the NAT type in front of it.

A Deeper Dive

This is a high-level introduction to our early preview. But you can take a much deeper dive with our p2p guides. Our work derives from highly peer-reviewed computer science research and we cite our sources frequently.
Prev — Next