This lab was developed with our partner, Block.one. Your personal information may be shared with Block.one, the lab sponsor, if you have opted-in to receive product updates, announcements, and offers in your Account Profile.
GSP991
Overview
In this lab, you will extend the single node EOSIO blockchain to use multiple nodes. You will install various EOSIO software components to set up your multi node EOSIO blockchain. Finally, you will perform some node operator tasks on your new EOSIO blockchain.
Objectives
In this lab, you will learn how to perform the following tasks:
Load an existing Compute Engine virtual machine.
Install and build the EOSIO system contracts.
Deploy a multi node blockchain with four (4) EOSIO nodes.
Create user accounts on the EOSIO multi node blockchain.
Call an action to transfer tokens among user accounts.
Read these instructions. Labs are timed and you cannot pause them. The timer, which starts when you click Start Lab, shows how long Google Cloud resources are made available to you.
This hands-on lab lets you do the lab activities in a real cloud environment, not in a simulation or demo environment. It does so by giving you new, temporary credentials you use to sign in and access Google Cloud for the duration of the lab.
To complete this lab, you need:
Access to a standard internet browser (Chrome browser recommended).
Note: Use an Incognito (recommended) or private browser window to run this lab. This prevents conflicts between your personal account and the student account, which may cause extra charges incurred to your personal account.
Time to complete the lab—remember, once you start, you cannot pause a lab.
Note: Use only the student account for this lab. If you use a different Google Cloud account, you may incur charges to that account.
How to start your lab and sign in to the Google Cloud console
Click the Start Lab button. If you need to pay for the lab, a dialog opens for you to select your payment method.
On the left is the Lab Details pane with the following:
The Open Google Cloud console button
Time remaining
The temporary credentials that you must use for this lab
Other information, if needed, to step through this lab
Click Open Google Cloud console (or right-click and select Open Link in Incognito Window if you are running the Chrome browser).
The lab spins up resources, and then opens another tab that shows the Sign in page.
Tip: Arrange the tabs in separate windows, side-by-side.
Note: If you see the Choose an account dialog, click Use Another Account.
If necessary, copy the Username below and paste it into the Sign in dialog.
{{{user_0.username | "Username"}}}
You can also find the Username in the Lab Details pane.
Click Next.
Copy the Password below and paste it into the Welcome dialog.
{{{user_0.password | "Password"}}}
You can also find the Password in the Lab Details pane.
Click Next.
Important: You must use the credentials the lab provides you. Do not use your Google Cloud account credentials.
Note: Using your own Google Cloud account for this lab may incur extra charges.
Click through the subsequent pages:
Accept the terms and conditions.
Do not add recovery options or two-factor authentication (because this is a temporary account).
Do not sign up for free trials.
After a few moments, the Google Cloud console opens in this tab.
Note: To access Google Cloud products and services, click the Navigation menu or type the service or product name in the Search field.
Background
This section provides an overview of the concepts covered in this lab.
EOSIO blockchain
An EOSIO blockchain is a highly efficient, deterministic, distributed state machine that can operate in a decentralized fashion. The blockchain keeps track of transactions within a sequence of interchanged blocks. Each block cryptographically commits to the previous blocks along the same chain. It is therefore intractable to modify a transaction recorded on a given block without breaking the cryptographic checks of successive blocks. This simple fact makes blockchain transactions immutable and secure. Block production and block validation are performed by special nodes called Block Producers.
EOSIO consensus
Block validation presents a challenge among any group of distributed nodes. A consensus model must be in place to validate such blocks in a fault tolerant way within the decentralized system. Consensus is the way for such distributed nodes and users to agree upon the current state of the blockchain. Two of the most common consensus models used in blockchains are Proof of Work (PoW) and Proof of Stake (PoS). In Proof of Stake, nodes that own the largest stake or percentage of some asset have equivalent decision power. One interesting variant is Delegated Proof-of-Stake (DPoS) in which a large number of participants or stakeholders elect a smaller number of delegates, which in turn make decisions for them. EOSIO uses Delegated Proof of Stake (DPoS) to elect the active producers who will then be authorized to produce blocks, validate them, and sign them to eventually be added to the blockchain.
EOSIO accounts, keys, and permissions
An account identifies a participant in an EOSIO blockchain. A participant can be an individual or a group depending on the assigned permissions within the account. Accounts also represent the smart contract actors that push and receive actions to and from other accounts in the blockchain. Keys in EOSIO are binary strings represented in Base58 used for signing and verification of transactions, blocks, and other messages. Keys are created within a digital wallet associated with an account. Since account ownership is defined solely by the account name, the keys associated with an account can be updated without compromising security. A novel permission scheme involving accounts, permissions, and authority tables determine what accounts can do and how the actions that make a transaction are authorized. To that end, each account is assigned a hierarchical permission structure and each permission is assigned a pair of public and private keys used for signing and verification.
EOSIO smart contracts
A smart contract is a low-level software library that contains the implementation of the actions that make a transaction. It also defines how the data that is accessed and processed by the actions are stored. In EOSIO, a smart contract is implemented in a high-level language such as C++ and compiled into a WebAssembly (WASM) binary. Thereafter, smart contracts can be deployed and run within an account's sandbox on the blockchain.
Task 1. Create a virtual machine
In the Navigation menu (), click Compute Engine > VM instances.
To create a new instance, click Create Instance.
In the Machine configuration.
Select the following values:
Name: instance-1
For Region and Zone, select the default region and zone assigned by Qwiklabs.
Machine Family - General Purpose (default)
Series - E2
Machine Type - e2-standard-2 (2 vCPU, 8 GB memory)
Click OS and storage.
Click Change to begin configuring your boot disk and select the following values:
Operating system: Ubuntu
Version: Ubuntu 20.04 LTS
Click Select.
Click Security.
Leave the defaults for Identity and API access unmodified.
Leave all other defaults unmodified.
To create and launch the VM, click Create.
Note: The VM can take about two minutes to launch and be fully available for use.
Click Check my progress to verify the objective.
Create a virtual machine
Task 2. Install the EOSIO platform
In the Navigation menu (), click Compute Engine > VM instances.
You will see the VM instance you created.
To open a command prompt on the VM instance-1 instance, click SSH in its row in the VM instances list. This will open a shell terminal window.
First, make sure to update the list of software packages in your VM box:
sudo apt update
Get the EOSIO software binaries. This includes nodeos the node p2p software, cleos the CLI client, keosd the wallet manager, and other tools:
Click Check my progress to verify the objective.
Install the EOSIO platform
Task 3. Create a default wallet
Wallets store the cryptographic keys used by accounts and producers to sign transactions and blocks, respectively. They also provide a secure enclave to perform the signing, so the private keys do not ever leave the wallet. To change state on the blockchain you need a wallet to store cryptographic keys and perform the digital signing.
First, set a large unlock timeout for any wallet managed by keosd, the EOSIO wallet manager:
keosd --unlock-timeout 999999999 &
This will prevent unlocking the wallet every 15 minutes, the default timeout. It will also launch the keosd daemon service. The response should be similar to:
info thread-0 wallet_plugin.cpp:38 plugin_initialize ] initializing wallet plugin
info thread-0 wallet_api_plugin.cpp:84 plugin_startup ] starting wallet_api_plugin
info thread-0 http_plugin.cpp:983 add_handler ] add api url: /v1/wallet/create
...
info thread-0 http_plugin.cpp:983 add_handler ] add api url: /v1/wallet/unlock
info thread-0 http_plugin.cpp:983 add_handler ] add api url: /v1/node/get_supported_apis
Press Enter if needed to get the shell prompt back.
Create a default wallet and output the wallet password to a file named my_wallet_password:
cleos wallet create --file my_wallet_password
This will create a wallet named default. The response should be:
Creating wallet: default
Save password to use in the future to unlock this wallet.
Without password imported keys will not be retrievable.
saving password to my_wallet_password
After the wallet is created, keosd opens it and unlocks it by default.
To view all the managed wallets, execute this command:
cleos wallet list
The response should be:
Wallets:
[
"default *"
]
The asterisk next to the wallet name indicates that the wallet is unlocked.
Click Check my progress to verify the objective.
Create a default wallet
Task 4. Add the private key of the EOSIO system account to the wallet
Every new EOSIO blockchain has a default system account called eosio. This account is used initially to set up the blockchain. It is associated to the following private key by default: 5KQwrPbwdL6PhXujxW37FSSQZ1JiwsST4cqQzDeyXtP79zkvFD3.
Import this private key into your wallet to sign transactions on behalf of the eosio account.
Use the cleos wallet open command to open the default wallet. Note that the wallet is already opened and unlocked. However, you would need to execute the following procedure if the wallet gets locked again:
cleos wallet open
The response should be:
Opened: default
Unlock the default wallet by using the cleos wallet unlock command and passing the wallet password stored in the my_wallet_password file:
cleos wallet unlock <my_wallet_password
The response should be:
password: Unlocked: default
Use the cleos wallet import command to import the eosio private key, 5KQwrPbwdL6PhXujxW37FSSQZ1JiwsST4cqQzDeyXtP79zkvFD3 to the default wallet:
This is the corresponding public key for the imported private key. Ignore any warnings regarding the use of a default wallet. In production deployments, you might want to name your wallet other than using the default one.
Click Check my progress to verify the objective.
Add the private key of the eosio system account to the wallet
Task 5. Install the EOSIO Contract Development Toolkit (CDT)
The EOSIO CDT toolkit includes the CDT C++ compiler, which you can use to build the system contracts and, later on, your own C++ smart contracts. The system contracts are very important since they enable the second layer software functionality that allows additional features to be added and supported.
First, install the build-essential package. This package contains various tools necessary for building C/C++ software on your Linux box.
The response should be the EOSIO CDT compiler version:
eosio-cpp version 1.8.1
Click Check my progress to verify the objective.
Install the EOSIO Contract Development Toolkit
Task 6. Build the system contracts and eosio.boot
To have a functional multi node EOSIO blockchain, you need to deploy the EOSIO system contracts. Among other things, the system contracts provide the layer-2 functionality required for the EOSIO consensus, the ability to stake tokens, vote, create your own tokens, etc. They also allow new features to be supported.
Install CMake, a software dependency used by the eosio.contracts build script:
Build the system eosio.contracts. In this step, you also need to pass the installation directories of eosio and eosio.cdt.
cd eosio.contracts
./build.sh -e /usr/opt/eosio/2.1.0 -c /usr/opt/eosio.cdt/1.8.1
cd ~
Ignore the warnings about missing ricardian contracts, if any. The response should be something similar to (without the warnings):
Using EOSIO.CDT installation at: /usr/opt/eosio.cdt/1.8.1
=========== Building eosio.contracts ===========
...
-- Building eosio.contracts v1.9.2
...
-- Unit tests will not be built. To build tests, set BUILD_TESTS to true.
-- Configuring done
-- Generating done
-- Build files have been written to: ./eosio.contracts/build
[ 11%] Performing build step for 'contracts_project'
[ 18%] Built target eosio.bios
[ 18%] Built target eosio.msig
[ 36%] Built target powup.results
[ 36%] Built target rex.results
[ 54%] Built target eosio.token
[ 90%] Built target eosio.system
[100%] Built target eosio.wrap
[ 22%] No install step for 'contracts_project'
[ 33%] No test step for 'contracts_project'
[ 44%] Completed 'contracts_project'
[100%] Built target contracts_project`
Make note of the location where the EOSIO system contracts are built. You will need it later when deploying specific system contracts: ~/eosio.contracts/build/contracts.
Build the eosio.boot system contract:
git clone https://github.com/EOSIO/eos.git \
--branch release/2.1.x --single-branch
cd eos/contracts/contracts/eosio.boot
cmake .
make
cd ~
Ignore warnings. The response should be something similar to:
-- Configuring done
-- Generating done
-- Build files have been written to: ~/eos/contracts/contracts/eosio.boot
Make a note of the location where the eosio.boot contract is built. You will need to deploy it later: ~/eos/contracts/contracts/eosio.boot.
Click Check my progress to verify the objective.
Build the system contracts and eosio.boot
Task 7. Start the boot node and create system accounts
In this task, you will launch the boot genesis node and create the system accounts. Unlike wallet creation, you need to send a transaction to the blockchain to create an account.
Therefore, at least the boot node must be running. To that end, first you need to create the data directory and the genesis file to be used by all nodes. In a production environment, every node will be hosted on its own server and use its own resources. Here, you will deploy all nodes in the same box.
Create new directory nodes. This is where the blockchain database, the log file, and the configuration file for each node will be stored.
mkdir nodes
Create the default genesis.json file in the nodes directory:
cat >nodes/genesis.json
Copy/paste the following JSON contents to the terminal, then press Ctrl-D to save the genesis.json file:
The "initial_key" indicates the initial development public key used by the eosio account. Here you can override it with your own development key, if needed, which is recommended for production deployments. Such a key must exist in the wallet used by the eosio account. You imported that key in a previous section.
Check that the genesis.json file was indeed created and populated:
cat nodes/genesis.json
You should see the same contents as in the previous step.
Create the boot node directory nodes/00-boot to store all blockchain data related to the boot node:
mkdir nodes/00-boot
Launch the boot node, starting nodeos service daemon as a background task, and redirecting stdout/stderr to the 00-boot/00.log file:
This will launch the boot node with some plugins, producer name eosio, listening on http port 8888, p2p port 9876, and using the default development keys EOS6MRy..., 5KQwrP... for block signing. If not overridden with --signature-provider, the default eosio development keys are used.
Verify that the boot node is running and producing blocks:
tail -f nodes/00-boot/00.log
The output will look as follows.
Output:
...
info 2021-03-08T02:44:11.900 nodeos producer_plugin.cpp:2333 produce_block ] Produced block e0dc6324ce3c8f35... #588 @ 2021-03-08T02:44:12.000 signed by eosio [trxs: 0, lib: 587, confirmed: 0]
The last line reveals the last block number produced and the current last irreversible block (lib) which marks the final block and the end of the blockchain.
Press Ctrl+C and then press Enter to exit the tail program.
Now that the boot node is running, you can get information about the eosio account stored in the chain database:
Note that the eosio account is indeed a privileged account, and the keys associated with both owner and active permissions are the same default development keys that were imported to the wallet. This will allow the eosio account to authorize transactions as the multi node blockchain is being set up.
Create additional system accounts. These accounts will also be stored in the blockchain database.
You assign the same development key pair to all system accounts and to both the owner and active permissions. In production deployments, you would want to assign a different key pair to each permission for added security. You might also use different keys for each system account.
The response for the above commands should be something similar to:
executed transaction: 98f22027c6703e554042b1d4a93beb70b1dc30be0fe65267932155c9d70e6493 200 bytes 153 us
# eosio <= eosio::newaccount {"creator":"eosio","name":"eosio.bpay","owner":{"threshold":1,"keys":[{"key":"EOS6MRyAjQq8ud7hVNYcfn...
warning: transaction executed locally, but may not be confirmed by the network yet ]
...
executed transaction: 2f0cb00110374976f3cd9195bccac81fe0ee330c515b4a656d6d41f6181f636d 200 bytes 189 us
# eosio <= eosio::newaccount {"creator":"eosio","name":"eosio.rex","owner":{"threshold":1,"keys":[{"key":"EOS6MRyAjQq8ud7hVNYcfnV...
warning: transaction executed locally, but may not be confirmed by the network yet ]
Ignore the warnings. You will use these accounts in the next tasks.
Click Check my progress to verify the objective.
Build the system contracts and eosio.boot
Task 8. Deploy the eosio.token, eosio.msig contracts, and create/issue the SYS token
In this task, you will create and issue a fictional SYS token. To that end, you will also need to deploy the eosio.token and eosio.msig system contracts that were built previously.
Deploy the eosio.token contract. This contract allows you to create, issue, and transfer new or existing tokens.
cleos set contract eosio.token ~/eosio.contracts/build/contracts/eosio.token/
The above command pushes a transaction with the create action from the eosio.token contract, authorized by the eosio.token account's active permission.
The response should be something similar to:
Click Check my progress to verify the objective.
Create and deploy the eosio.token, eosio.msig contracts, and SYS token
Task 9. Activate protocol features and deploy eosio.boot and eosio.system contracts
Protocol features bring additional optional capabilities to EOSIO blockchains. In this task, you will activate some protocol features that are highly recommended for EOSIO blockchains. To that end, you will first deploy the eosio.boot contract, then activate these features. Finally, you will deploy the eosio.system contract to complete the second layer of the EOSIO blockchain.
This enables system resources (CPU, RAM, NET), enables tokens to be staked and unstaked, resources to be purchased, potential producers to be registered and subsequently voted on, producer rewards to be claimed, privileges and limits to be set, etc. Protocol features must be activated through EOSIO consensus by a supermajority of producers. In this case, the running boot node is all you need.
First, install the jq tool to parse JSON strings in a more readable form:
sudo apt install jq
Now let's check which protocol features are supported:
curl -X POST http://127.0.0.1:8888/v1/producer/get_supported_protocol_features -d '{"exclude_disabled": false, "exclude_unactivatable": false}' | jq | more
The above command sends an HTTP POST request to the get_supported_protocol_features endpoint provided by the Producer API plugin.
The response will display all protocol features supported by the current EOSIO network (which consists of the boot node so far).
Check the output (press the spacebar multiple times) until you get the shell prompt back.
Activate the PREACTIVATE_FEATURE feature. This allows other features to be enabled, including potential upgrades to the eosio.system contract that might implement new features. Also note that a protocol feature must be activated by passing a feature digest or hashcode for the specific feature.
curl -X POST http://127.0.0.1:8888/v1/producer/schedule_protocol_feature_activations -d '{"protocol_features_to_activate": ["0ec7e080177b2c02b278d5088611686b49d739925a92d9bfcacd7fc6b74053bd"]}'
Note that the feature digest hashcode 0ec7e... can be obtained from the feature_digest field returned by the get_supported_protocol_features endpoint in the previous step. If the activation is successful, the response should be as indicated.
Output:
{"result": "ok"}
If the feature has already been activated, the following response is returned instead, which can be safely ignored.
Output:
{"code": 500, "message": "Internal Service Error", "error": {"code": 3250000, "name": "protocol_feature_exception", "what": "Protocol feature exception", "details": [{"message": "protocol feature with digest '<hex-value>' has already been activated", "file": "controller.cpp", "line_number": 1650, "method": "check_protocol_features"}]}}
Deploy the eosio.boot contract:
cleos set contract eosio ~/eos/contracts/contracts/eosio.boot/
Deploy the eosio.system contract. This will enable second layer features and will allow the transition from a single node to multi node blockchain.
cleos set contract eosio ~/eosio.contracts/build/contracts/eosio.system/
Click Check my progress to verify the objective.
Deploy eosio.boot and eosio.system contracts
Task 10. Create the block producer accounts
In this task, you will create the producer accounts that will eventually take over the EOSIO blockchain and produce blocks. The first step is to designate eosio.msig as a privileged account, so it can authorize on behalf of the eosio account, which later on will resign its authority and let the eosio.prods, account, which represents the group of elected producers, take over the responsibilities of block production and validation.
If the nodes are distributed, each key pair will be stored independently on each server's wallet. In this lab, you store it in the same wallet for simplicity.
Create the block producer accounts with initial resources and public key:
The above command will create new accounts producer.one, producer.two, produc.three, produce.four and associate each key imported in the previous step to each account. It will also assign and stake 50 million SYS for NET and CPU and 8k for RAM.
The response should be something similar to (for each new account):
Note that the number of votes for each producer is still zero since no producers have been voted for yet.
Click Check my progress to verify the objective.
Create the block producer accounts
Task 11. Transition from single-node producer to multi-node producers
In this task, you will finally transition from a single node blockchain to a multi node blockchain. Currently, only the eosio account is privileged and can sign blocks. The goal is to create an EOSIO blockchain managed by a collection of elected producers where consensus is reached after a supermajority of 2/3 + 1 producers confirm a block. Then, all transactions within that block will be final.
Create the remaining node directories nodes/01-node, nodes/02-node, nodes/03-node, nodes/04-node to store all blockchain data related to the new producing nodes 1, 2, 3, 4:
This will launch node 1 with some plugins, producer name producer.one, listening on http port 8889, p2p port 9877, overriding default development keys with its own keys EOS8imf..., 5KLGj1..., and connecting to p2p node 9876 (boot node).
You can inspect the node 1 log for any messages exchanged with the boot node (p2p node 9876) by running the following command:
cat nodes/01-node/01.log | grep :9876 -C 3
The output should look something similar to the following.
Output:
info 2021-10-22T21:56:43.161 nodeos producer_plugin.cpp:421 on_incoming_block ] Received block e79c2cc7df2bd7bc... #3040 @ 2021-10-22T21:56:41.500 signed by eosio [trxs: 0, lib: 3039, conf: 0, latency: 1661 ms]
...
info 2021-10-22T21:56:43.162 net-1 net_plugin.cpp:1604 set_state ] old state lib catchup becoming in sync
info 2021-10-22T21:56:43.162 net-1 net_plugin.cpp:1132 operator() ] Sending handshake generation 2 to 127.0.0.1:9876- 446609d, lib 3040, head 3041, id 5d69f1fa874933cf
info 2021-10-22T21:56:43.163 net-0 net_plugin.cpp:1947 sync_recv_notice ] notice_message, pending 3043, blk_num 3043, id b8f3389c52ff416c...
info 2021-10-22T21:56:43.163 net-0 net_plugin.cpp:1908 verify_catchup ] catch_up while in in sync, fork head num = 3043 target LIB = 3041 next_expected = 3042, id b8f3389c52ff416c..., peer 127.0.0.1:9876 - 446609d
info 2021-10-22T21:56:43.163 net-0 net_plugin.cpp:1604 set_state ] old state in sync becoming head catchup
info 2021-10-22T21:56:43.164 nodeos producer_plugin.cpp:421 on_incoming_block ] Received block 917275dee8c12e57... #3042 @ 2021-10-22T21:56:42.500 signed by eosio [trxs: 0, lib: 3041, conf: 0, latency: 664 ms]
...
Note that node 1 is not producing blocks yet, but receiving/syncing them from other nodes, such as the boot node, for now.
Launch node 2, starting nodeos service daemon as a background task and redirecting stdout/stderr to the 02-node/02.log file:
This will launch node 2 with some plugins, producer name producer.two, listening on http port 8890, p2p port 9878, overriding default development keys with its own keys EOS8imf..., 5KLGj1..., and connecting to p2p node 9876 (boot node) and p2p node 9877 (node 1).
You can inspect the node 2 log for any messages exchanged with say node 1 (p2p node 9877) by running the following command:
cat nodes/02-node/02.log | grep :9877 -C 3
The output should look something similar to the following.
Output:
info 2021-10-22T22:28:31.496 net-1 net_plugin.cpp:1771 start_sync ] Catching up with chain, our last req is 6852, theirs is 6859 peer 127.0.0.1:9877 - 1a245d2
info 2021-10-22T22:28:31.496 net-1 net_plugin.cpp:1717 operator() ] requesting range 6853 to 6859, from 127.0.0.1:9877- 1a245d2
info 2021-10-22T22:28:31.497 nodeos producer_plugin.cpp:421 on_incoming_block ] Received block 3073fa4230d8d232... #6853 @ 2021-10-22T22:28:28.000 signed by eosio [trxs: 0, lib: 6852, conf: 0, latency: 3497 ms]
...
info 2021-10-22T22:28:31.501 net-0 net_plugin.cpp:1604 set_state ] old state lib catchup becoming in sync
info 2021-10-22T22:28:31.501 net-0 net_plugin.cpp:1132 operator() ] Sending handshake generation 3 to 127.0.0.1:9877- 1a245d2, lib 6858, head 6859, id 9a791a880b9b875e
info 2021-10-22T22:28:31.501 net-1 net_plugin.cpp:1947 sync_recv_notice ] notice_message, pending 6860, blk_num 6860, id 4dd2c7c8ae5301ed...
info 2021-10-22T22:28:31.501 net-1 net_plugin.cpp:1908 verify_catchup ] catch_up while in in sync, fork head num = 6860 target LIB = 6859 next_expected = 6860, id 4dd2c7c8ae5301ed..., peer 127.0.0.1:9877 - 1a245d2
info 2021-10-22T22:28:31.501 net-1 net_plugin.cpp:1604 set_state ] old state in sync becoming head catchup
...
Note that node 2 is not producing blocks yet, but receiving/syncing them from other nodes, such as the boot node and node 1, for now.
Launch node 3, starting nodeos service daemon as a background task and redirecting stdout/stderr to the 03-node/03.log file:
This will launch node 3 with some plugins, producer name produc.three, listening on http port 8891, p2p port 9879, overriding default development keys with its own keys EOS5n44..., 5JCStv..., and connecting to p2p node 9876 (boot node), p2p node 9877 (node 1), and p2p node 9878 (node 2).
You can inspect the node 3 log for any messages exchanged with say node 2 (p2p node 9878) by running the following command:
cat nodes/03-node/03.log | grep :9878 -C 3
The output should look something similar to the following.
Output:
info 2021-10-22T22:28:31.496 net-1 net_plugin.cpp:1771 start_sync ] Catching up with chain, our last req is 6852, theirs is 6859 peer 127.0.0.1:`**`9878`** `- 1a245d2`
info 2021-10-22T22:28:31.496 net-1 net_plugin.cpp:1717 operator() ] requesting range 6853 to 6859, from 127.0.0.1:`**`9878`** `- 1a245d2`
info 2021-10-22T22:28:31.497 nodeos producer_plugin.cpp:421 on_incoming_block ] Received block 3073fa4230d8d232... #6853 @ 2021-10-22T22:28:28.000 signed by eosio [trxs: 0, lib: 6852, conf: 0, latency: 3497 ms]
...
info 2021-10-22T22:28:31.501 net-0 net_plugin.cpp:1604 set_state ] old state lib catchup becoming in sync
info 2021-10-22T22:28:31.501 net-0 net_plugin.cpp:1132 operator() ] Sending handshake generation 3 to 127.0.0.1:`**`9878`** `- 1a245d2, lib 6858, head 6859, id 9a791a880b9b875e
info 2021-10-22T22:28:31.501 net-1 net_plugin.cpp:1947 sync_recv_notice ] notice_message, pending 6860, blk_num 6860, id 4dd2c7c8ae5301ed...
info 2021-10-22T22:28:31.501 net-1 net_plugin.cpp:1908 verify_catchup ] catch_up while in in sync, fork head num = 6860 target LIB = 6859 next_expected = 6860, id 4dd2c7c8ae5301ed..., peer 127.0.0.1:`**`9878`** `- 1a245d2
info 2021-10-22T22:28:31.501 net-1 net_plugin.cpp:1604 set_state ] old state in sync becoming head catchup
...
Note that node 3 is not producing blocks yet, but receiving/syncing them from other nodes, such as the boot node, node 1 and 2, for now.
Launch node 4, starting nodeos service daemon as a background task and redirecting stdout/stderr to the 04-node/04.log file:
This will launch node 4 with some plugins, producer name produce.four, listening on http port 8892, p2p port 9880, overriding default development keys with its own keys EOS5y3T..., 5JJjaK..., and connecting to p2p node 9876 (boot node), p2p node 9877 (node 1), p2p node 9878 (node 2), and p2p node 9879 (node 3).
You can inspect the node 4 log for any messages exchanged with say node 3 (p2p node 9879) by running the following command:
cat nodes/04-node/04.log | grep :9879 -C 3
The response should look something similar to:
info 2021-10-22T22:28:31.496 net-1 net_plugin.cpp:1771 start_sync ] Catching up with chain, our last req is 6852, theirs is 6859 peer 127.0.0.1:9879 - 1a245d2
info 2021-10-22T22:28:31.496 net-1 net_plugin.cpp:1717 operator() ] requesting range 6853 to 6859, from 127.0.0.1:9879 - 1a245d2
info 2021-10-22T22:28:31.497 nodeos producer_plugin.cpp:421 on_incoming_block ] Received block 3073fa4230d8d232... #6853 @ 2021-10-22T22:28:28.000 signed by eosio [trxs: 0, lib: 6852, conf: 0, latency: 3497 ms]
...
info 2021-10-22T22:28:31.501 net-0 net_plugin.cpp:1604 set_state ] old state lib catchup becoming in sync
info 2021-10-22T22:28:31.501 net-0 net_plugin.cpp:1132 operator() ] Sending handshake generation 3 to 127.0.0.1:9879 - 1a245d2, lib 6858, head 6859, id 9a791a880b9b875e
info 2021-10-22T22:28:31.501 net-1 net_plugin.cpp:1947 sync_recv_notice ] notice_message, pending 6860, blk_num 6860, id 4dd2c7c8ae5301ed...
info 2021-10-22T22:28:31.501 net-1 net_plugin.cpp:1908 verify_catchup ] catch_up while in in sync, fork head num = 6860 target LIB = 6859 next_expected = 6860, id 4dd2c7c8ae5301ed..., peer 127.0.0.1:9879 - 1a245d2
info 2021-10-22T22:28:31.501 net-1 net_plugin.cpp:1604 set_state ] old state in sync becoming head catchup
...
Note that node 4 is not producing blocks yet, but receiving/syncing them from other nodes, such as the boot node, node 1, 2 and 3, for now.
At this point all new nodes 1, 2, 3, 4, are launched and connected. They receive and synchronize blocks produced by the boot node, but they do not produce yet. To that end, the new producers need to be elected. In a production deployment, users would typically vote for up to 30 producers from a pool of prospective producers in voting command. The top 21 producers get elected as block producers.
Here, you will use the existing producer accounts to elect the only four producers available. After 15% of the SYS token supply has been staked, the top elected producers are then proposed for production. After that block that contains the list of producers is confirmed and becomes final, the new producers begin producing blocks in the next scheduled round.
Use the cleos system voteproducer prods command to vote for specific producers. Account producer.one votes for itself, producers 2, 3, and 4. producer.two votes for itself, producer 3 and 4. produc.three, and produce.four; then produc.three to vote for itself and produce.four; then produce.four to vote for itself.
cleos system voteproducer prods producer.one producer.one producer.two produc.three produce.four
cleos system voteproducer prods producer.two producer.two produc.three produce.four
cleos system voteproducer prods produc.three produc.three produce.four
cleos system voteproducer prods produce.four produce.four
Note that producer 1 got 10% of the votes, then producer 2 followed with 20%, producer 3 got 30%, and finally producer 4 obtained 40%. Since the total number of SYS tokens staked by all producers is greater than 15% of the token supply, the top 21 producers (by default) will get proposed for block production.
Production itself will begin after the block that contains the list of the block producers gets confirmed as final. Since there are only four producers, all of them will produce, and a supermajority will be reached with any three of them.
Check that the new producers are indeed producing blocks:
tail -f nodes/01-node/01.log | grep Produced
The above command should display the last blocks produced by Producer 1. If you don't see any output yet, please wait a few seconds. There are four producers each producing 12 blocks every 6 seconds. So there should be 18 seconds between the last block produced and the first one for the next round.
The response should look something similar to:
info 2021-10-28T15:31:47.904 nodeos producer_plugin.cpp:2333 produce_block ] Produced block adeb43a9249e0ef8...` **`#7035`** `@ 2021-10-28T15:31:48.000 signed by` **`producer.one`** `[trxs: 0, lib: 6998, confirmed: 36]
info 2021-10-28T15:31:48.400 nodeos producer_plugin.cpp:2333 produce_block ] Produced block 991c7f7addb6006e...` **`#7036`** `@ 2021-10-28T15:31:48.500 signed by` **`producer.one`** `[trxs: 0, lib: 6998, confirmed: 0]
...
info 2021-10-28T15:31:52.901 nodeos producer_plugin.cpp:2333 produce_block ] Produced block c947dfab144600f7...` **`#7045`** `@ 2021-10-28T15:31:53.000 signed by` **`producer.one`** `[trxs: 0, lib: 6998, confirmed: 0]
info 2021-10-28T15:31:53.300 nodeos producer_plugin.cpp:2333 produce_block ] Produced block c871f71b3ebe9151...` **`#7046`** `@ 2021-10-28T15:31:53.500 signed by` **`producer.one`** `[trxs: 0, lib: 6998, confirmed: 0]`
If you wait 18 more seconds you should see the next round of 12 blocks produced, and so on. You may also check that the other three nodes are indeed producing blocks by replacing the 01s in the above command with 02s, 03s, 04s.
Press Ctrl+C to exit the tail command.
Check that the boot node is not producing blocks anymore:
tail -f nodes/00-boot/00.log | grep Produced
Note that the above command should not produce any output, which demonstrates that the boot node relinquished block production to the new producers that were elected.
Now that you have a fully functioning blockchain with four producers, you should strip the default eosio and related eosio.* accounts of any powers to sign transactions and modify state on the blockchain. To that end, you need to resign the keys used by the eosio and eosio.* accounts by resetting them to null.
Get the eosio account information. Note the keys associated with the owner and active permissions:
Note that the eosio account no longer has the default development keys associated with the owner and active permissions. Instead, the permissions are now child permissions of the parent eosio.prods@active permission. This is the same account group that represents all the active block producers.
Therefore, now the eosio account is essentially controlled by the eosio.prods account.
Resign the keys used by the eosio.* accounts and link their permissions to the eosio account:
Now the eosio.* accounts no longer have the default development keys associated with the owner and active permissions. Instead, the permissions are now child permissions of the parent eosio@active permission, which in turn is a child permission of the eosio.prods@active account.
Therefore, now all the system accounts, including eosio, are essentially controlled by the eosio.prods account, which represents the account group of all active block producers. Finally, now you have built a fully operational multi node EOSIO blockchain!
Click Check my progress to verify the objective.
Transition from single node producer to multi node producers
Congratulations!
In this lab, you extended the single node EOSIO blockchain deployed previously to a multiple blockchain network. You also installed various EOSIO software components to set up your multi node EOSIO blockchain. Finally, you performed some node operator tasks on your new EOSIO blockchain.
...helps you make the most of Google Cloud technologies. Our classes include technical skills and best practices to help you get up to speed quickly and continue your learning journey. We offer fundamental to advanced level training, with on-demand, live, and virtual options to suit your busy schedule. Certifications help you validate and prove your skill and expertise in Google Cloud technologies.
Manual Last Updated December 12, 2024
Lab Last Tested October 22, 2024
Copyright 2025 Google LLC. All rights reserved. Google and the Google logo are trademarks of Google LLC. All other company and product names may be trademarks of the respective companies with which they are associated.
Labs create a Google Cloud project and resources for a fixed time
Labs have a time limit and no pause feature. If you end the lab, you'll have to restart from the beginning.
On the top left of your screen, click Start lab to begin
Use private browsing
Copy the provided Username and Password for the lab
Click Open console in private mode
Sign in to the Console
Sign in using your lab credentials. Using other credentials might cause errors or incur charges.
Accept the terms, and skip the recovery resource page
Don't click End lab unless you've finished the lab or want to restart it, as it will clear your work and remove the project
This content is not currently available
We will notify you via email when it becomes available
Great!
We will contact you via email if it becomes available
One lab at a time
Confirm to end all existing labs and start this one
Use private browsing to run the lab
Use an Incognito or private browser window to run this lab. This
prevents any conflicts between your personal account and the Student
account, which may cause extra charges incurred to your personal account.
In this lab, you will extend the single node EOSIO blockchain to use multiple nodes.