The decentralization of Cardano puts responsibility for running the blockchain in the hands of stake pools. An essential element in this is reliable and effective connections between all the distributed nodes, and ensuring that the network is resilient to failure.

With the simpler Byron version of the blockchain, federated (OBFT) nodes controlled by the Cardano Foundation, Emurgo, and IOHK were wholly responsible for managing block production and network connections. This maintained the network, while building up a system of thousands of distributed nodes, operated by stake pools. To achieve decentralization, Cardano has now ended the prevalence of the federated nodes that have supported the system since its creation in 2017.

On December 6, 2020, we set the k parameter to 500 to expand the number of ‘viable’ pools and further promote decentralization. We have also gradually reduced d to put the power of block production entirely into the hands of the community. 100% of blocks are now being produced by the stake pool operator (SPO) community, which means that block production in Cardano is completely decentralized. These parameter changes support long-term chain sustainability and encourage the spreading of stake and potential rewards more evenly among stake pools.

In just over six months, we have evolved from a system reliant upon a handful of federated nodes, into a proof-of-stake system run by the community, with thousands of blocks produced every epoch by over 2,000 stake pools.

The network

Cardano’s networking layer is a physical infrastructure that combines nodes and their interaction into a unified system. The network distributes information about transactions and block creation among all active nodes. In this way, the system validates and adds blocks to the chain and verifies transactions. Thus, a distributed network of nodes must keep communication delays to a minimum, and be resilient enough to cope with failures, capacity constraints, or hackers.

Under the old federated system, nodes were connected by a static configuration defined in a topology file. Since the introduction of Shelley, the system has been functioning in a hybrid mode, where nodes connect to federated relays and to other SPO’s relays. This connectivity is partially manually constructed, however, SPOs can exchange block and transaction information without relying on federated nodes.

In his article 'Cardano’s path to decentralization', Marcin Szamotulski discussed the network’s design and explained Cardano’s approach to network decentralization with the advent of Shelley. Now that we have reached full decentralization in terms of block production, it is also essential that the network connectivity is decentralized too. Cardano will achieve this through a shift to peer-to-peer (P2P) connectivity.

P2P networking

At this point, we should talk about the network ‘stack’, a set of software tools recently enhanced by our engineering team to cope with a larger, more dynamic, and complex network.

P2P communication will enhance the flow of information between nodes, thus reducing (and ultimately removing) the network’s reliance on the federated nodes, and enabling the decentralization of Cardano. To achieve the desired resilience, IOHK’s networking team has been busy improving the network stack with advanced P2P capabilities. These improvements do not require a protocol change, but rather enable automated peer selection and communication.

The P2P networking is enabled due to the use of the following components:

Figure 1. P2P architecture

Let's take a closer look at the process of how node connections are established and see how the latest developments streamline data exchange between nodes.

Mini protocols

A set of mini protocols enables communication between nodes. Each protocol implements a basic information exchange requirement, such as informing peers of the latest block, sharing blocks, or processing transactions. Chain-sync, block-fetch, and tx-submission protocols have been used to distribute chains of blocks and transactions for node-to-node communication in the network:

• block-fetch draws information from the chain database.
• chain-sync synchronizes fetched data across the network.
• tx-submission2 consumes transactions from peer mempools and adds them to the local mempool, which enables peers to submit their transactions to the node. This is a modification of the current tx-submission protocol.

These mini protocols support the Ouroboros consensus protocol. To ensure optimal networking service, the team has implemented additional protocols:

• keep-alive: this ensures continuous connection between nodes and minimizes performance faults.
• tip-sample: this provides information about which peers offer better connectivity in terms of performance.

You can find out more about the network architecture and mini protocol examples on the Cardano documentation website.

Connection management

The networking service supports Linux, Windows, and macOS, but the number of connections supported by each operating system varies.

To avoid system overloading, a multiplexer combines several channels into a single Transmission Control Protocol (TCP) connection channel. This offers two advantages: One, bidirectional communication between peers (so any peer can initiate communication with no restrictions as both parties have read and write permissions within the same channel), and enhanced node-to-node communication without affecting performance.

The networking team has implemented a bidirectional-aware ‘connection manager’ that integrates with the P2P governor, which is currently undergoing final testing before deployment. Additionally, the multiplexer’s API has been upgraded to monitor new connections and protocols. This enhancement introduces more efficient connection management and improved issue tracking.

P2P governor functionality

The Cardano network involves multiple peer nodes. Some are more active than others, some have established connections, and some should be promoted to ensure the best system performance. As discussed in 'Cardano’s path to decentralization', peers are mapped into three categories:

• cold peers
• warm peers
• hot peers

To establish bidirectional connections between them, it is crucial that we know which connections are active.

Figure 2. Peer discovery on Cardano

The P2P governor manages connections and provides information on which peers are active and performing well. This feature promotes peer connections for enhanced system performance and also provides excellent visibility by building and maintaining a connectivity map of the entire network. The governor will simplify the process of connection definitions by handling these automatically so a few central stake pools no longer have to configure them manually. The governor promotes or demotes peers between cold, warm, and hot states, and also interacts with the connection manager to open new connections or reuse existing ones.

P2P deployment roadmap

The IOHK networking team is in the final stages of quality testing the P2P governor integration with the node. After this, the team will extend the network stack with more protocols – gossip, in particular, which will provide seamless data exchange between peers and help construct a decentralized communication map.

These technical upgrades allow us to simplify Cardano node interfaces and improve the system’s configuration. When testing is finalized, all SPOs will be able to update and simplify their configuration preferences for enhanced connectivity.

This involves the following stages before full P2P deployment:

Figure 3. P2P deployment roadmap

For a walkthrough of the plan from chief architect Duncan Coutts, check out this video from the March Cardano360 show.

While governance played an important role in the network establishment, maintenance and support, only with decentralization we can achieve true network sustainability to ensure equal opportunities for all stake pools. Thus, the goal of stack improvements is to allow all stake pools to run the same configurations, establishing equal capabilities within a decentralized environment.

We’ll keep providing more development updates in this blog, and you can also follow Cardano status updates to learn about recent improvements and developments.

As we celebrated in yesterday’s blog about D=0 day, block production is now 100% in the hands of the Cardano stake pool operator (SPO) community. Community pools are the lifeblood of the network. And earlier this year we confirmed our commitment to a vibrant proof-of-stake (PoS) ecosystem when we delegated over 300M ada to our first cohort of community stake pools.

The number of pools now stands at over 2,000 – responsible for nearly three-quarters of the ada in circulation. Their efforts sustain the network and greatly contribute to the success and recognition of the Ouroboros protocol, setting a clear standard for any competing PoS protocol.

At the start of the year, we committed to making fresh delegations quarterly to help bootstrapping community pools. Running a successful small pool is hard, and needs to be entered into like any other business endeavour, with fixed and variable costs planned, hours needed to operate, reach break even point, plan marketing approach, etc. With each delegation round, our goal is to delegate enough ada to every SPO that they can create blocks, giving them a certain level of financial security while they look to grow their delegation base.

We’re again asking every operator out there to seek delegation from IOHK’s ada funds. We’ve already done a survey of the existing pools to guide our progress, and now we want every pool to fill in our survey to apply. Besides supporting your own application, your answers will help us shape our community delegation approach based on factors such as potential impact and each stake pool’s goals.

We favour operators running single pools, and we ask for details in support of each application. We are also looking for pools with a clear sense of mission or purpose – pool operators need to be in it for more than just the ada. We also look for an ability to ‘market’ – whether it’s through content creation, clear articulation of mission etc. Evidence of technical commitment to supporting the Cardano system is also likely to influence our decisions, so a clear statement of your pool’s goals and motivation is vital. You may have created educational content; have strong environmental credentials; or run the pool to raise funds for charitable works – tell us about it and provide evidence of your successes. For ease of reference, here’s an outline of what we’ll ask you in the application form.

• Your name
• The region and country your pool is located in
• Your pool ticker
• When you established your pool
• Details of any SPO group or guild affiliation (e.g., SPOCRA, Guild, SPA etc.)
• Details of your web presence, Twitter handle, Telegram, YouTube, etc. so we can better understand how you market your pool
• Number of pools (we favor operators running single pool)
• Your mission statement and the goals of your pool
• Whether you or your Stake Pool have created educational content that is publicly available and the language it is in
• Links to evidence of that content and any successes
• If you host a meetup, what the results have been
• Your pool’s environmental credentials (carbon offset/renewables etc.)
• Whether your stake pool is a charity or runs on behalf of a charity, non-profit, or non-governmental organization, with details
• If your stake pool has ever been bootstrapped or compensated by another organization, including - but not limited to - the Cardano Foundation or EMURGO
• Details of any contribution that you have made to the community or ecosystem not mentioned above that support your application for the delegation (e.g., running a pool on the ITN, technical contribution, GitHub PRs, CIPs, content/marketing contribution etc...)

To apply, you can access the typeform here.

While we ask all SPOs applying to warrant that the information they have supplied is truthful and accurate, we do not ‘police’ pool activity or forensically examine claims for accuracy. We do however expect transparency from every member of the community.

Stake pools can apply for delegation from today; applications will close for this cohort at midnight UTC on Thursday 15th April. We will then review all the applications and announce which SPOs will receive delegation, which will be made in early May. We look forward to your applications.

Next week sees the start of the first in a series of our Plutus pioneer training programs where participants can learn the fundamentals of Plutus and help to test the code before the official release of our new smart contract language. This new innovative program aims to recruit and train developers within the ecosystem so that they are fully prepared when Plutus is deployed to the Cardano mainnet later this year.

Since we announced this new course on March’s Cardano360 show we have had a huge expression of interest from our developer community, both from developers who want to create decentralized applications (DApps), and smart contract programmers who want to work with Cardano’s principal development language. This week we have contacted everyone interested (over 2,000) to get a better idea of their experience and expertise. Those of you who have been selected for the first round of this program will hear from us very soon. If you don’t receive an invitation at this stage, don’t worry as we will be running several rounds of this program, so please do stay in touch!

Course structure

The first iteration of the new program starts next week. The cohort joining this program will be true pioneers. As well as being part of a group that will have early access to a set of learning modules, your feedback will help us develop and iterate the overall learning experience.

The course will teach you the core principles of how to code in both Haskell and Plutus. The course modules will cover the building blocks of Haskell and Plutus, including functions and data types, type classes, monads, template Haskell, using the Plutus Playground, the Extended UTXO model, working with Plutus on and off the chain, minting policies, state machines, the Plutus application framework, as well as some case studies and practical exercises.

The course will follow a modular approach and will be highly interactive – each week we will be releasing new teaching videos from our director of education, Lars Brünjes, along with a set of practical exercises to complete during the week as part of each module. We will also be holding regular Q&A sessions and you will have access to a dedicated community channel on Discord, created especially to help you to connect with other course participants as you learn.

Outside the exercises and videos, students will be encouraged to learn at a pace that is right for them and to collaborate with fellow students. As with all learning experiences, the more you put in the more you will get out! We encourage all participants to engage with each other and work collaboratively to answer questions and solve problems.

We will have a small team of moderators who will check in from time to time to help facilitate and assist. They can also help triage issues or questions that may come up in your learning during the week. These moderators are all graduates of the Mongolia class of 2020 that completed the Haskell MOOC and are well placed to assist with questions and challenges. At the close of each week, Lars will engage directly with the group to resolve more complex technical questions and provide feedback on the subject matter covered that week.

Prior experience

As Plutus is based largely on Haskell, having some prior experience with Haskell (or another functional programming language), will be very helpful. At a minimum, you should have some programming experience and a mathematical and technical mindset. You should be as keen to learn as to help us; while we have run a number of successful courses already, this is the first time we have challenged ourselves to teach at this scale. So be prepared for road bumps along the way as we learn and improve, too!

Bear in mind that this course is not for coding beginners. While you do not need to be an expert in formal methods, programming experience and a general aptitude for logical and mathematical thinking are highly desirable. The course will include advanced features like Template Haskell, type-level programming, and effect systems. If you need a refresher or get an introduction to Haskell, we recommend that you read the Learn You a Haskell guide before you participate in the course. We’ll open several new cohorts this year, so you won’t miss out.

When does the course start?

The course starts next week and will run concurrently for ten weeks through to mid June. It will involve approximately ten hours a week of your time and effort each week.

Will there be other pioneer programs?

Yes, we are keen to be as inclusive as we can on our path to rolling out smart contracts. We are also developing education programs for both Marlowe and Glow, so don't feel like you have missed out if this Plutus course is not for you. We’re still keen to have you onboard, so do watch this space for details of these other courses!

Certification

We will reward participants for their efforts in participating in this course and certify those pioneers that complete the entire program and are successful. These Plutus pioneer certificates will be represented as non fungible tokens (on the testnet) and locked by a Plutus contract. Pioneers can demonstrate their knowledge and qualification by constructing an appropriate transaction to unlock their individual token.

Ready to start

We are excited to have so many developers from our ecosystem on board and ready to get started and learn Plutus. You’ll not only be learning yourselves, but acting as a pioneer to help us determine the best way to teach and deliver this course – truly helping pave the way for future cohorts who enroll! We look forward to bringing you on this learning journey before we launch Plutus to the world!