Could you please elaborate:
a) How a bare metal node compares to a cloud node in terms of performance, security and decentralization?
b) How the change from 3 cloud hosted nodes to 2 bare metal nodes would impact the performance for users and dApps, also taking into account a growing demand for chain state queries, RPC calls, etc.?


Ben from Dwellir here

a) Great question:

**Performance** - https://www.comparenodes.com/global-node-comparison/fbd2a857-dd50-4633-b9a0-8f0408fb992f/

Dwellir performs marginally better OnFinality globally. The one region we do not is Asia and this may improve once we set up more bare metal in that region. 

**Security** - Dwellir controls all of our own bare-metal and have had zero instances of interruptions. Our CTO ran the entire DevOps stack for a large Swedish company called Skania managing a 50 mil USD yearly IT budget. Security is the most important aspect. Without that, there is not trust. Without trust, well ... then you have nothing.

**Decentralisation** - Dwellir owns bare metal, OnFinality uses Public clouds. 

b) Dwellir can run 3 nodes if required for the same price. From what I can see OnFinality is only running a single node in Singapore currently. https://onfinality.io/networks/interlay

James from OnFinality here

**There is some detail missing here from this proposal that may change the outcome. We've recently reduced our service price for Interlay and Kintsugi to be able to provide a fixed price service for Interlay and Kintsugi for $4,000 per quarter (total across both).**

OnFinality for a long time has been the sole commercial operator of Interlay and Kintsugi nodes, as a result, we've received huge traffic to our Interlay and Kintsugi nodes. From inception of the Kintsugi API Service on 2023-10-12 until 2023-05-31 our Kintsugi and Interlay API service has:

- Served a combined total of over 2.5 billion (2,531,891,335) requests
- Accumulated almost 9 TB of data egress 
- The highest daily responses in a single 24 hour period for Interlay occurred on the 21st of December 2022 with 92,949,041  (almost 93 million responses)
- The highest daily responses in a single 24 hour period for Kintsugi occurred on the 20th of December 2022 with 66,337,717  (over 66 million responses)
- At the end of this period the 90 day combined uptime for Interlay and Kintsugi was 99.99%
- [Raw Data can be found here](https://docs.google.com/spreadsheets/d/18fhVtJ7E-7tPrtsjqb4Buit8K_AWq-J79PkKd4NjTRs/edit#gid=1222701687).

As you can see, OnFinality is designed to massively scale to handle bursty traffic with almost zero downtime. Our previous advice from the Interlay governance body was to manage a public service where pure uptime and performance were the main considerations, our 99.99% uptime means that OnFinality was down for less than 14 minutes in the entirety of the last quarter shows that we have clearly achieved this, despite bursty traffic. That advice has since changed and we're pleased to be able to show the results of these cost controls with this fixed price service of $4,000 per calendar quarter.

Additionally, we now have better analytics on where our customers are making requests from (you can see some [aggregated data here](https://docs.google.com/document/d/1pPMvUxpWc0tX70t12yW9QskbrF1VY8vKCo3sFw2E_TY/edit), so we will also move our nodes to better serve these users in the United States, Germany, and the Netherlands.

Ben from Dwellir.

When we take a closer look at the data egress figures, we find ourselves in a position to provide a more refined price evaluation. The breakdown for this quarter is as follows:

**Interlay: $663 per quarter
Kintsugi: $1621  per quarter
Combined Total : $2284 per quarter**

These numbers provide us with a clearer picture of the costs involved and allow for a more accurate budget projection.

Secondly, I'd like to highlight Dwellir's impressive performance over the past year. We've expanded our utility from **1 billion RPC requests per month** to an astounding **30 billion RPC requests per month** this year alone, with virtually zero downtime. This represents a significant increase in the scale and reliability of our operations.

Looking ahead, Dwellir has ambitious plans to launch an API developer service before the end of 2023. This is part of our ongoing commitment to innovation and supporting our user community.

Finally, I want to assure everyone that Dwellir remains steadfast in our commitment to the decentralized vision of owning and operating bare metal machines. This approach significantly reduces the risk of downtime from third-party providers such as AWS. It's a critical part of our strategy to ensure consistent and reliable service to our users.

We appreciate your ongoing support and look forward to continuing to deliver high-quality services that meet your needs.

Will (Paradox) from LuckyFriday:

Hi All,

I got wind of this forum post from a general discussion on the topic of RPC provision for Parachain teams. As an organization, we can appreciate the demands required for deploying enterprise-level RPC nodes. We also appreciate that there are certain advantages when operating said services at scale.

The Lucky Friday team is deploying RPC services on self-owned, bare-metal servers hosted in our own rack space.  Our services are live on Polkadot & Kusama and you'll soon see us on many other chains. We are currently operating our RPC services out of data centres in the US, with expansion to Europe imminent and Asia following. We aim to provide high-quality, geo-diverse, censorship-resistant RPC services for the Dotsama ecosystem.

Lucky Friday has had a long-lasting relationship with Interlay from its beginnings on Kusama. **We value this relationship** and look forward to the success of the respective project(s). In light of this, we would like to offer RPC services to the project(s) **at no cost for twelve months**.  Interlay can decide if they would like this service to replace Interlay Labs or as an independent option.

We do not intend to affect the present vote – we respect Interlay's history of service provision and identify with Dwellir's vision of a bare-metal future. However, it would have been remiss of us not to aid Interlay and demonstrate to the ecosystem that relationships matter.

We hope the community, the Interlay team, and fellow providers receive this offer well.

Regards,
Will

Hi All from RadiumBlock,

We would like to make the Interlay team and community aware of our RPC offering. 

- RadiumBlock is a high performance infrastructure provider for the Polkadot/Substrate ecosystem. 
- We operate the highest performance RPC service in polkadot ecosystem https://www.comparenodes.com/global-node-comparison/407c5245-0c6f-4abe-be49-1ff0b25a2938/
- We propose to run an Interlay and Kintsugi RPC for a <ins>combined total of $699 per month</ins>
- As a startup offering, we can run these nodes <ins>free for 6 months for a $1000 signup fee </ins>
- RadiumBlock would offer an RPC package to:
- - Operate archive node based RPC endpoints for Interlay and Kintsugi 
- - Operate 3 location PoP (US,EU,Asia or 3 of customers choice) for Interlay and Kintsugi 
- - Using our on-prem high availability hardware at collocated data centers for better cost and decentralization
- - Offer low latency access across the world

Here is a more [detailed proposal](https://docs.google.com/document/d/1QqcrRK08534aPhcjSHrs0HKdDu6OMusJ/edit?usp=sharing&ouid=114172945417100820622&rtpof=true&sd=true) 

We too respect Interlay's history of service provision with some of the providers and identify with Dwellir's vision of a bare-metal first design to strengthen decentralization and better cost control.

regards,
RadiumBlock Team

RPC nodes are used to connect to the Interlay and Kintsugi networks. Dwellir and OnFinality are both providers in the Dotsama space to provide such services (think Infura for Dotsama).

It's generally a good idea to have two providers for redundancy in case one provider might go offline. However, the treasury should also not overspend on RPC node provision.

*Current status*

Currently, there are the following options for RPC nodes:
- Kintsugi/Interlay Labs. These are not invoiced against the treasury but the Interlay team operates these.
- OnFinality. OnFinality operates nodes for Kintsugi and Interlay networks. The currently charge the Kintsugi treasury only for the RPC node provision.
- Dwellir. Dwllir has recently set up Kintsugi and Interlay nodes as well. They are not charging the treasury but opened a discussion post on the Interlay forum to operate nodes in the future.

**Update 20 June 2023**

*Current costing*

- Please see the proposals and discussion by both OnFinality and Dwellir in their respective posts

*Next steps*

- OnFinality shared their proposal here: https://interlay.subsquare.io/post/23
- Dwellir shared their proposal here: https://interlay.subsquare.io/post/21

*What you can do*
Please share your opinion in the poll below. If you have comments, please leave them below.


Future of RPC node providers