Node hosting on Google Cloud: a pillar of Web3 infrastructure

Blockchain nodes are the physical machines that power the virtual computer that comprises a blockchain network and store the distributed ledger. There are several types of blockchain nodes, such as:

• RPC nodes, which DApps, wallets, and other blockchain “clients” use as their blockchain “gateway” to read or submit transactions

• Validator nodes, which secures the network by participating in consensus and producing blocks

• Archive nodes, which indexers use to archive nodes to get the full history of on-chain transactions 

Deploying and managing nodes can be costly, time consuming, and complex. Cloud providers can help abstract away the complexities of node hosting so that Web3 developers do not need to think about infrastructure. In this article, we’ll explore both how organizations can avoid challenges by running their own nodes on Google Cloud, and how in many scenarios, our fully managed offering, Blockchain Node Engine, can make node hosting even easier.

Why running nodes is often difficult and costly 

Developers often choose a mix of deploying their own nodes or using shared nodes provided by third parties. Free RPC nodes are sufficient to start exploring but may not offer the required latency or performance. Web3 infrastructure providers’ APIs or dedicated nodes are another option, letting developers focus on their app without worrying about the underlying blockchain node infrastructure. There are situations, however, in which it is beneficial to run your own nodes in the cloud. For example:

• Privacy is too critical for RPC calls to go over the public internet.

• Certain regulated industries require organizations to operate in a specific jurisdiction and control their nodes

• Node hardware needs to be configured for optimal performance.

• A DApp requires low latency to the node.

• An organization is a validator with a significant stake and needs to be in control of the uptime of its validator node and security.

• An organization needs predictable and consistent high performance that will not be impacted by others using your node.

In Ethereum, the fee recipient is an address nominated by a validator to receive tips from user transactions. The node controls the fee recipient, not the validator client, so to guarantee control of the fee recipient, the organization must run its own nodes.

Organizations can face challenges running their own nodes. At a macro level, node infrastructure challenges fall into one of these buckets:

• Sustainability (impact on the environment)

• Security (DDoS attacks, private key management)

• Performance (can the hardware keep up with the blockchain software)

• Scalability (how a network starts and grows)

In addition, there is a learning curve related to how each protocol works (e.g., Ethereum, Solana, Arbitrum, Aptos, etc.), what hardware specifications the protocol requires (compute, memory, disk, network), and how to optimize (e.g., sync modes).

Hyperscalers have been perceived as not performant enough and too expensive. As a result, a lot of the Web3 infrastructure today runs in bare-metal server providers or in one hyperscaler. For example, as of September 20, 2022, more than 40% of Solana validators ran in Hetzner. But then, Hetzner blocked Solana activity on its servers, causing disruption to the protocol. Similarly, as of October 2022, 5 out of the top 10 Solana validators by SOL staked (representing 8.3% of all staked SOL) ran in AWS, per validators.app. 

Simply put, this concentration of validators creates a dependency on only a select few hosting providers. As a result, an outage–or a ban–from a single provider can lead to a material failure of the underlying protocol. Moreover, this centralization goes against the Web3 ethos of decentralization and diversification. Healthy protocols require a diversity of participants, clients, and geographic distribution. In fact, the Solana Foundation, via its delegation program, incentivizes infrastructure diversity with the data center criteria.

Running nodes on Google Cloud for security, resiliency, and speed

To avoid the aforementioned challenges and improve decentralization on major protocols, organizations have been using Google Cloud to host nodes for several years. For example, we are a validator for protocols like Aptos, Arbitrum, Solana, and Hedera, and Web3 customers use Google Cloud to power nodes include Blockdaemon, Bullish, Coinbase and Dapper Labs. 

We support a diverse set of ecosystems and use cases, for example:

• The nodes can run in Google Cloud, regardless of the protocol (we run nodes for Ethereum, layer 2’s, and alternative layer 1’s, etc.). Please note that Proof of Work mining is restricted.

• We have nodes running in both live and test networks. This is important for the learnings required for each protocol.

• While these examples are public (permissionless) networks, we also support the private networks favored by some of our regulated customers.

Streamlining and accelerating node hosting with Blockchain Node Engine

Blockchain Node Engine provides streamlined provisioning, and a secure environment, as a fully managed service. A developer using Blockchain Node Engine doesn't need to worry about configuring or running nodes. Blockchain Node Engine does all this so that the developer can focus on building a superb DApp. We've simplified this process and collapsed all the required node hosting steps into one.

For protocols not supported by Blockchain Node Engine, or if an organization wants to manage their own nodes themselves,  services in Google Cloud are built to cover an organization’s full Web3 journey:

•  An organization might start with a simple Compute Engine VM instance using the machine family that works for the protocol. (We support the most demanding protocols, including Solana.)

• Then, they’ll make their architecture more resilient with managed instance group fronted by Cloud Load Balancer

• Next, the organization might secure the user-facing nodes by fronting them with Cloud Armor as a Web Application Firewall and DDoS protection

• This node hosting infrastructure is fully automated and integrated with the organization’s DevOps pipelines, helping them to seamlessly accelerate development.

• As the organization grows and its apps attract more traffic, Kubernetes becomes a natural choice for health monitoring and management. Blockchain nodes can be migrated to GKE node pools (pun intended). (Note: Organizations can also start directly in GKE, rather than Compute Engine.)

• As the organization continues to grow, it can benefit from access to the cloud-native services close to the nodes. For example, customers use various caching solutions like Cloud CDN, Memorystore and/or Spanner (like blockchain.com) so that most requests do not even have to hit your nodes.

• On the data side, the organization can implement pipelines that extract data from the node and ingest into BigQuery to make it available for analysis and ML.

• It can also leverage Confidential Computing for data encrypted while in use (e.g., Multi-Party Computation, Bullish).

Next steps

As we’ve shown with the formation of both customer-facing and product teams dedicated to Web3, Google Cloud is inspired by the Web3 community and grateful to work with so many innovators within it. We’ve been excited to see our work in open-source projects, security, reliability, and sustainability address core needs we see in Web3 communities, and we look forward to seeing more creative decentralized apps and services as Web3 businesses continue to accelerate. 

To get started with Blockchain Node Engine or explore hosting your own nodes in Google Cloud, contact sales or visit our Google Cloud for Web3 page. 

^{Acknowledgements: I’d like to thank customer engineers David Mehi and Sam Padilla and staff software engineer Ross Nicoll, who helped me to better understand node hosting, and Richard Widmann, digital assets head of strategy for his review of this post.}