In our previous blog post, we discussed the deterministic nature of transaction and script validation on the Alonzo ledger, which provides assurance that the outcome of on-chain transaction application and script validation can be accurately predicted locally, before the transaction is ever submitted.

Relying on the guarantees provided by the deterministic design of the Alonzo ledger, we have implemented a specific two-phase validation scheme. It is designed to minimize the resources the nodes use to validate network transactions while also eliminating unexpected costs for the user. In this blog post, we dive deeper into how the two-phase validation works.

In the Shelley, Allegra, and Mary eras, transaction validation was a one-step process. The effect of a valid transaction on the ledger was fully predictable before it was applied. If a transaction was valid, it got included into a block and added to the ledger. If not, a node would reject it after a failed validation attempt and the transaction would not be included in a block. However, nodes that validated incoming transactions used time and resources, regardless of whether or not the transaction ended up in a block.

Alonzo introduces Plutus scripts, which might require significantly more resources to validate them compared to simple scripts in previous eras. To address the issue of nodes expending resources on validating scripts of transactions that get rejected, Alonzo introduces a two-phase validation approach. This strategy maintains a predictable outcome of applying transactions to the ledger, and also ensures fair compensation to the nodes for their work and resource usage.

Two-phase transaction validation

Transaction validation on Cardano is divided into two phases. The main reason for introducing two-phase validation is to limit the amount of uncompensated validation work by nodes. Each phase serves a purpose in achieving this goal. Roughly speaking, the first phase checks whether the transaction is constructed correctly and can pay its processing fee. The second phase runs the scripts included in the transaction. If the transaction is phase-1 valid, phase-2 scripts run. If phase-1 fails, no scripts run, and the transaction is immediately discarded.

Thus, nodes are expected to add processable transactions to a block even if the transactions are not phase-2 valid. This means that either:

• a small amount of uncompensated work is done by a node to find out that a transaction is not processable, but no expensive phase-2 validation is done, or
• the transaction is processable. The node can then perform phase-2 validation of the scripts, tag the transaction accordingly as either phase-2 valid or phase-2 invalid, and add it to a block. In either case, the node will later be compensated for both phases of validation via the fee or collateral collected from this transaction.

The expectation is that phase-2 failure should be rare, because a user submitting a transaction with failing scripts will lose ada while achieving nothing. This is locally predictable, and therefore a preventable event. The phase is a required safeguard to guarantee compensation for scripts’ potentially resource-intensive computation.

Let’s take a closer look at the specifics of each phase.

Phase 1

The first phase of validation must be simple. If this phase fails, a node does not get compensated for the work it has done, as it cannot accept processing fees from unprocessable transactions.

Phase-1 validation verifies two things: that a transaction is constructed correctly, and that it is possible to add it to the ledger. This validation includes the following checks and some additional ones:

• it pays the correct amount of fees and provides the correct amount of collateral (i.e. ada collected in the case of script failure, explained below)
• it includes all the data required for executing Plutus scripts
• it does not exceed any bounds set in the protocol parameters (on output sizes, etc.)
• its inputs refer to UTXOs existing on the ledger
• the stated computational budget for the transaction does not exceed the maximum resource limit per transaction
• integrity hash checks, etc.

Before adding an incoming transaction to the mempool (and eventually, to a block), a node must perform all phase-1 validation checks. If any of these checks fail, the transaction is rejected without being included into a block, and no fees are charged. In previous eras, this was the only validation phase, and Cardano handled all validation failures in this fashion.

Honest, non-compromised nodes are not expected to intentionally produce unprocessable transactions. Nodes can also drop connections performing adversarial dissemination of phase-1 invalid transactions.

Phase 2

The second phase of validation runs Plutus scripts, which can be more computationally expensive. Therefore, fees are charged following either a success or a failure in the second phase. Collected ada goes into the fee pot, and thus compensates nodes for the resources used in the validation process.

Successful phase-1 validation does not guarantee that all of the transaction’s actions are processable, only that collection of the collateral is possible. Phase-2 performs Plutus script validation, and the decision of whether to perform full processing or only collect collateral is made based on the outcome of validation:

• fully apply the transaction (the only possibility prior to Alonzo) – if the Plutus scripts validate all the actions of the transaction, or 

• collect the collateral ada and ignore the rest of the transaction – if one of the Plutus scripts fails 
.

Recall that script validation has a locally predictable outcome and is guaranteed to terminate. Users can locally check script validation outcomes, and there will be no disagreement between honest nodes on how to process a given transaction and the scripts therein.

Collateral

If scripts don't validate, we still need to compensate the nodes for their work. But we can't just take money from the transaction inputs, because those might have been locked with scripts - those that failed! So instead, Alonzo introduces a special provision for this. The collateral of a transaction is the amount of ada that will be collected as a fee in case of a phase-2 script validation failure. In a processable transaction, this amount must be at least a certain percentage of the transaction fee, specified in a protocol parameter.

This amount is specified at the time of constructing the transaction. Not directly, but by adding collateral inputs to the transaction. The total balance in the UTXOs corresponding to these specially marked inputs is the transaction’s collateral amount. These UTXOs must have public key (rather than script) addresses and contain no tokens other than ada.

The collateral inputs get removed from the ledger UTXO only if any script fails phase-2 validation. If all scripts pass, the specified transaction fee amount gets collected, as in previous eras. In particular, the amount comes from the regular, non-collateral inputs, and the collateral inputs are simply ignored. And, good news! It is permitted to use the same inputs as both collateral and regular, since only one of the two sets ever gets removed from the UTXO.

The signatures required to validate the spending of collateral inputs also play an important role in maintaining the integrity of a transaction. They do so by preventing adversaries from altering its contents so that it continues to be processable but fails phase-2 validation. An example of this would be an adversary replacing a redeemer. Collateral key holders’ signatures are required to make such a change. The collateral key holders are also the only users who stand to lose any ada if script validation fails.

Since script evaluation is deterministic, the collateral key holders are able to check locally whether the transaction will pass phase-2 validation on-chain before they sign it. If it does, then they can be sure it will also do so on-chain, and they will definitely not lose their collateral. A user acting in good faith should never lose their collateral. It also means that they can reuse the same collateral inputs for multiple transactions, and be sure the collateral is not collected.

Now that we have launched the public Alonzo testnet, we welcome all users and developers to assess it by building and executing Plutus scripts. You can find out more information in the dedicated Alonzo testnet repository, or discuss Plutus and Alonzo topics with our diverse community.

As the Alonzo hard fork brings core Plutus smart contract capability to Cardano, the ledger evolves to meet the growing need for the deployment of decentralized solutions. Cardano ledger design focuses on high assurance, security, and proven formal verification. In alignment with this strategy, it is also important to ensure that transaction processing is deterministic, meaning that a user can predict its impact and outcome before the actual execution.

The ability to guarantee the cost of transaction execution, and how the transaction behaves on the ledger before it is submitted, becomes even more prominent with the introduction of smart contract support. Unspent Transaction Output (UTXO)-based blockchains, like Cardano, provide such capabilities. Account-based blockchains, like Ethereum, are indeterministic, which means that they cannot guarantee the predictability of the transaction’s effect on-chain. This presents risks of monetary loss, unpredictably high fees, and additional opportunities for adversarial behavior.

In this blog post, we take a closer look at the benefits of Cardano’s deterministic design that allows for secure transaction and script evaluation before execution. In the following blog post, coming later this week, we discuss the two phases of transaction validation on Cardano.

What is transaction determinism and why is it important?

Determinism, in the context of transaction and script processing, is a synonym for predictability. This means that a user can predict locally (off-chain) how their transaction will impact the on-chain state of the ledger, without:

• unexpected script validation outcomes or failures
• unexpected fees
• unexpected ledger or script state updates.

A transaction in a deterministic system might yet be rejected, even if constructed correctly. Rejected means that the transaction could not be applied to the ledger at all, therefore having no effect on its state, so no fees are paid. The reason this would happen is due to ledger changes caused by other transactions processed between the time when the initial transaction is constructed and the time it is processed. This can happen even with simple transactions. For example, another transaction might spend a UTXO that a user was also planning to spend. Determinism ensures that, whenever a transaction is accepted, it will only have predictable effects on the ledger state.

Addressing the issue of indeterminism

Indeterminism means that we cannot predict what effects a transaction will have on the ledger before execution. When designing the ledger, as well as a smart contract interpreter, it is important to foresee conditions in which indeterminism might occur, and make design decisions to avoid them. One of the main hazards in such a case is access to mutable ledger data, that is data that can be changed or altered. When the changes a transaction or a smart contract make to the ledger depend on its state at the time of processing, rather than only the contents of the transaction, indeterminism might become an issue.

Ethereum is notably susceptible to this problem. For example, gas prices, or a decentralized exchange (DEX) rate can fluctuate between the time a user submits a transaction and the time it gets processed. This results in unexpected gas fees, or price changes of assets being purchased. Or a script might simply fail, resulting in high execution costs (hundreds of dollars) and no other effect. This could occur, for instance, if the funds available to cover the gas costs run out mid-execution. Deterministic ledger design eliminates these possibilities.

Other possible sources of indeterminism include allowing scripts to see:

• data in the block containing the transaction, but not included in any transaction, e.g., system randomness, block header, or the current slot number
• data altered or substituted by an adversary, which might change the outcome of script validation, while the transaction itself remains processable.

In most systems, there are ways to mitigate these issues with improved script-writing practices, or layer-2 solutions. Cardano is designed to guarantee predictable outcomes for all scripts and transactions.

How basic UTXO model benefits in terms of determinism

The Cardano ledger is built on a UTXO accounting model, which means that assets are stored on the ledger in unspent outputs, rather than in accounts. Each of these outputs specifies quantities of assets stored therein, together with its address. Unspent outputs are immutable, so a transaction might consume the entire output, but it cannot alter it.

To transfer assets, a transaction consumes one or more outputs and creates new ones, which, in total, contain the same quantities of assets as the ones consumed. These quantities -and their UTXO addresses- are specified in the outputs of the transaction. The only way a transaction can influence the effect of another transaction applied to the ledger is by spending the same UTXO as the later transaction attempts to spend, thus causing the node to reject it. This is the key feature on which the UTXO model relies for maintaining determinism.

A UTXO ledger has both benefits and drawbacks over the account-based model. The latter will encounter fewer instances of transactions blocking one another, for example. Unlike UTXOs, accounts are mutable ledger data. So a transaction might see, for example, a different quantity of assets in an account, depending on whether it was processed before or after another transaction that updates that same account. This circumstance might not cause the transaction to be rejected, but it could result in different – and unpredictable – changes to the ledger.

Spending a UTXO is just one example of an action a transaction can take. Next, we explain what transaction actions are, and how they can be validated. The most significant set of changes introduced in Alonzo are changes to the process of action validation.

Validating actions with signatures and scripts

An important aspect of processing a transaction is validating the actions it is taking. A transaction is taking an action when it contains data in the specific field to that action. For example, a transaction is spending UTXO U when it contains a reference to U in its input field, and it is minting a token X when its mint field contains X.

When the node processes a transaction, it verifies whether it can perform the action it intends to. For this, the author of the transaction must provide relevant pieces of data, e.g., scripts, redeemers, or signatures. A common example of an action that requires validation is spending a UTXO locked with a public key. The transaction must provide a signature from the corresponding private key to perform this action.

Cardano uses scripts to validate actions. These scripts, which are pieces of code, implement pure functions with True or False outputs. Script validation is the process of invoking the script interpreter to run a given script on appropriate arguments.

Script validation can be performed for the following actions:

• Spending a UTXO locked by a script address: the script that is run is the one whose hash forms the address.
• Minting a token: the script that is run is the one whose hash forms the policy ID of the token being minted.
• Reward withdrawal: the script that is run is the one whose hash forms the staking address.
• Applying a certificate: the script that is run is the one whose hash forms the certificate author’s credential.

Besides letting the node know which script to run, all transaction actions indicate how to assemble arguments passed to that script.

Cardano’s multi-asset ledger (Mary) supports simple multisig and timelock scripting languages. These allow users to specify signatures required to perform an action (such as spending a UTXO or minting a non-fungible token (NFT)), and the time interval in which it can be performed. A timelock script can never see the actual slot number in the transaction that includes it. Timelock can only see the validity interval of the carrying transaction. Allowing a timelock script to see the current slot number (i.e., data coming from the block, rather than the author) would break determinism. This is ensured by the fact that a user cannot know the exact slot in which the transaction gets processed, and therefore they cannot predict how the script will behave.

Mary scripts, unlike Plutus contracts in Alonzo, are very limited in what they can express. The Alonzo hard fork ushers in a new era of powerful, stateful contracts that do not compromise the deterministic ledger property.

Plutus scripts

Alonzo introduces a new approach to transaction validation on Cardano due to the implementation of Plutus scripts. The extended unspent transaction output (EUTXO) model, deployed as part of Alonzo, provides the ledger infrastructure to support Plutus contracts. Below, we provide a high-level overview of ledger and transaction changes. For more details about working with the ledger and Plutus scripts, check out the Plutus Pioneer program!

Alonzo changes the data on the ledger as follows:

1. Plutus scripts can lock UTXOs.
2. A new component, added to the contents of the output parts of UTXOs, enables script state-like functionality. In addition to assets and an address, a UTXO locked by Plutus scripts also contains a datum. A datum is a piece of data that can be thought of as an interpretation of the script state.
3. There are a number of new protocol parameters used to impose additional validation requirements on transactions. These include upper limits on computational resources that scripts can consume.

To support Plutus scripts, transactions have been upgraded as follows:

1. For each of its actions, the transaction now carries a user-specified argument, called a redeemer. Depending on the script, a redeemer can serve a different purpose. For example, it can act as the bid the user places in an auction, or the user’s guess in a guessing game, among many other functions.
2. The transaction specifies computational execution budgets for each script.
3. To ensure that a transaction can pay its execution fee, Alonzo introduces additional pieces of data, which we’ll discuss in a follow-up blog post.
4. Transactions contain an integrity hash, needed to ensure that it has not been compromised, outdated, etc.

There are also some changes to the specifics of Alonzo transaction validation as compared to Mary. For each action, the node assembles script arguments expected by the Plutus interpreter, including:

• the datum
• the redeemer
• execution budget
• a summary of the transaction.

The node performs new, Alonzo-specific checks that ensure the transaction is constructed correctly. For example, it must not exceed the maximum execution resource budget. It also invokes the Plutus script interpreter to run the scripts.

Datum objects vs script state

Like mutable accounts, mutable script state falls squarely into the ‘mutable ledger data’ category of indeterminism sources. We already saw that the UTXO model avoids the mutable accounts indeterminism issue. It can also help us reimagine the concept of script state in a way that maintains determinism. If a UTXO is locked by a Plutus script, that UTXO’s script code is associated with its address. The state-analog of this script is the datum stored in that UTXO. When a transaction spends that UTXO, it gets deleted from the ledger, including the datum. However, the contents of the Plutus script could enforce that the transaction carrying it must also create a UTXO containing a specific datum that can be viewed as the updated script state.

Script execution budget

The non-deterministic gas model can charge users unpredictably large fees. In Cardano scripts, this source of indeterminism is addressed by requiring that the resource budget itself, as well as the fee required to cover this budget, are included in the transaction. In Alonzo, a user can predict both locally when constructing the transaction. Script execution necessarily returns either True or False, and will not loop indefinitely. The reason for this is that every operation a script performs takes a non-zero amount of resources, which are tracked by the interpreter. If the budget specified by the transaction is exceeded, script execution terminates and returns False.

Transaction validation in Alonzo

Addressing the possible sources of indeterminism, the following key points make the outcomes of script and transaction validation predictable:

• the script interpreter will always terminate and return the same validation result when applied to the same arguments
• a transaction necessarily fixes all arguments that will be passed to the script interpreter during validation
• a transaction specifies all the actions it is taking that require script validation
• compulsory signatures on a transaction ensure that it cannot be altered by an adversary in a way that causes scripts to fail
• applying a transaction in the EUTXO ledger model is deterministic.

The last point is largely inherited from the UTXO model, as Alonzo ledger protocol updates remain, for the most part, consistent with updates in previous eras (including the delegation scheme, etc.). After the Alonzo upgrade, script validation failure or success does affect how a transaction is processed (more about this in part 2!). However, the True or False outcome, as well as ledger changes associated with either outcome, are predictable for a given transaction.

The deterministic behavior of Cardano’s script and transaction validation is not a natural outcome of using the EUTXO model. To ensure this property, the IOG team had to carefully track the source of every piece of data which a script is allowed to see.

The deterministic evaluation property is formally specified in the Alonzo specification, and the IOG team has also sketched proof that the interpreter gets only those arguments that would not break the property.

In our second blog post, we’ll take a closer look at the 2-phase validation process of Cardano transactions. So, keep an eye out later this week for part two.

Ethereum, custom (user-defined) tokens are implemented using smart contracts to simulate the transfer of custom assets. Our approach with Cardano does not require smart contracts, because the ledger itself supports the accounting of non-ada native assets.

Another difference is that Cardano’s multi-asset ledger supports both fungible and unique, non-fungible tokens without specialized contracts (similar to those required by ERC-20 and ERC-721 tokens). It can store a mix of both fungible and non-fungible tokens in a single output.

Cardano’s native tokens framework is based on four principles:

• lightweight
• affordability
• security
• unified process

Lightweight

The native token framework is based on a multi-asset ledger structure built around token bundles (values), A token bundle can contain a heterogeneous mix of ada and other tokens. These token-containing structures are stored in outputs on the ledger instead of ada, as previously. Each type of token is identified by its asset ID, which includes a hash reference to its minting policy. The minting policy itself is only ever checked during minting or burning, and is not itself stored on the ledger, which makes this approach quite lightweight.

The fungibility relationship is also captured by the asset ID in a lightweight way: tokens with the same asset ID are fungible with each other, but not fungible with those with a different asset ID. Unique tokens have a quantity of exactly 1 associated with their asset ID.

The asset ID identifies each type of token within a single token bundle and across the whole ledger. It also identifies the token’s place in the internal two-level map structure of the token bundle. This internal data structure enables fungible and non-fungible tokens to be represented uniformly. It also gives great flexibility to the kinds of asset use cases that can be tokenized in the system. It is straightforward to represent, for example, timeshares of a single piece of property, or a selection of unique pieces of art scoped under a single minting policy controlled by the artist.

The inherent simplicity of native tokens is further highlighted when we look at how transferring assets between two contracts works in Ethereum’s ERC-20. In this situation, smart contract code is required, which adds complexity, and with it room for error and cost. The structure of token bundles offers a rather lightweight approach to asset transfer, because different types of token can be transacted in a single transaction, with greater speed.

Affordability

In an ERC-20 token environment, transferring any number of tokens between two peers requires the execution of a smart contract, which carries an execution fee (gas). By contrast, in Cardano’s native multi-asset ecosystem, the transfer of assets (tokens, ada, custom currencies, etc) does not require a smart contract, and does not carry any execution fee, which means greater affordability.

Security

Native tokens feature a more lightweight and less costly design than that of Ethereum’s ERC-20 and ERC-721 standards. But these two features would mean nothing without a robust security layer to guarantee the integrity of the system.

In native tokens, system integrity is built around the ledger property of value preservation (that is, that the sum of all the inputs is equal to the sum of the outputs). All native token transfer logic is coded in the ledger, as opposed to user-defined smart contracts. This ensures predictable and uniform behaviour of the system, and does not require users to understand smart contracts, which can often be a point of vulnerability.

While accounting correctness is ensured by the ledger, minting and burning of tokens is regulated by their user-defined minting policies. A minting policy is permanently hash-associated to the tokens scoped under it, and there is no way to change this policy. This guarantees that the policy an issuer chose can never be changed to allow minting or burning of this type of token which was not authorized under the original policy. Whenever a minting transaction is added to the ledger, the policy for each type of token being minted is checked and must be satisfied. Each token in circulation, except ada (as Cardano forbids minting additional ada), necessarily has a minting policy and is guaranteed to have been minted according to that policy.

Therefore the only custom code required to manipulate tokens in Cardano is the policy itself. Tying the policy hash to the asset identifier means that there is no need for a global asset registry, so creating assets is cheap and easy. The system remains simple, lightweight and easy to use.

Unified process

When native tokens are implemented as part of Goguen, the ledger will handle all tokens in the same way. And minting a token can only be done in one way, to reduce ambiguity and possible mistakes or bugs. This simplification in using a unified process will lead to faster development and a better development experience overall.

Pre-production environment incoming

Native token capability will be deployed to the Cardano mainnet following a protocol upgrade in Q1 2021 (known internally by the working name, ‘Mary’) opening up a new world of use cases and opportunity. To onboard new developers ahead of this date, we’re now finalizing the deployment of a pre-production environment for native tokens. So stay close to our social channels for the very latest news on deployment.

If you are a developer and want to get involved early on, visit our developer site, where you can get supporting documentation and resources. We’ll add to this over time; sign up to our developer survey on this page to express your interest and be alerted as soon as everything becomes available.

I started with IOHK in May 2018 as a formal methods developer working on two components of Cardano, neither of which involved writing Haskell code. Because of my expertise in logic, type theory, proof assistants, and theoretical computer science, I became part of the team without having much Haskell experience, even though it is the main language we use. So, I was surprised when my name came up last summer about who would be the teaching assistant for a Haskell course in Ethiopia this year. I had thought I would be a student, but then it became clear that loftier plans were being proposed for me.

While there were other qualified candidates, I imagine not everyone was willing to relocate to Africa for three months. Also, the Ethiopia 2019 class was distinguished from previous versions of the course because it was only open to women. For this reason, the idea of having a female assistant seemed particularly relevant. So, I found myself getting ready to learn, present in class, and assess material that was new to both me and the cohort of students from Ethiopia and Uganda!

I was living in Canada, and had yet to meet any IOHK employees in person. I knew little about Ethiopia. So, I got my inoculations, my one-way ticket — I was not sure when the course was scheduled to end and how much longer I was expected to stay on afterwards — my visa (with my name spelt wrongly), and headed off to Africa for the first time in my life.

Once I got to Addis Ababa, the thing that stood out was the amount of livestock in parts of the city. Donkeys, cows, and goats were grazing, carrying heavy loads, and wandering about the streets. After the first drive through Addis, Lars Brünjes, director of education at IOHK, and John O’Connor, director of Africa operations, and I sat down for a cold drink at the hotel where Lars and I were staying. We had some laughs, talked a bit about ourselves and the course, and it began to seem as if I would be very happy working with my colleagues here for three months — what a relief.

From the first day until the last, which was almost three months, I was really engaged with both the students and the material. The course started at the Ministry of Innovation and Technology and the students showed incredible perseverance. Two had to drop out in the first week, but the rest stayed on, no matter how challenging it got — and no matter the transportation time to class given the crazy Addis traffic!

Most of the students had a computer science degree, some also had a master’s or work experience. However, Haskell is different from anything they would have learned or used before. There were some difficult concepts to grasp, but Lars did an awesome job breaking down the material and providing plenty of relevant examples (as many as the students wanted, which was a lot).

There is a saying that the best way to understand something is to teach it. For me, this was true for the entire duration of the course. I learned a lot, answering questions, delivering lectures, grading work, and especially making up the questions and answers for the final test. It was a special treat to learn about smart contracts, Marlowe, and Plutus in the last two weeks, too — new material for me and a great addition to the course. This part was taught by Phil Wadler, one of the creators of Haskell, and at the very end he delivered a special lecture on propositions as types, which was particularly engaging and open to a wider audience than just the students — it was a very nice way to end the class.

Throughout the three months, but especially at the beautiful graduation ceremony, I really felt the importance of what we were doing — giving people the skills and tools to address their problems locally. The best part was getting to know the wonderfully enthusiastic students and watching them develop their skills in this unusual and interesting programming language. I look forward to having the opportunity to work with some of these women in the future. Finally, getting to know my IOHK colleagues as we arrived in Ethiopia was a real treat as well!

Read Lars Brünjes's blog - Training blockchain developers in Africa