Liam Vu's Blog

Liam Vu's Blog

Run EC2 with more than 2 TiB of storage for BSC full node

Run EC2 with more than 2 TiB of storage for BSC full node

Blockchain full nodes store a complete copy of the distributed ledger and are responsible for the reliability of the stored data. Almost full nodes help the network by accepting transactions and blocks from other full nodes, validating those transactions and blocks, and then relaying them to further full nodes. Some chains require a huge amount of storage for their full nodes, e.g. Binance Smart Chain. However, EC2 AWS instance's root volume is limited to 2 TiB by default due to MBR partition. This post resolves a problem that full nodes require more than 2 TiB of storage on EC2 AWS.

MBR vs GPT

MBR uses a 32-bit data structure to store block addresses. This means that each data block is mapped with one of 232 possible integers. The block size for MBR volumes is conventionally limited to 512 bytes. Therefore, 2 TiB is the limitation for MBR volumes.

GPT uses a 64-bit data structure to store block addresses. This means that each data block is mapped with one of 264 possible integers. The block size for GPT volumes is commonly 4096 bytes so the maximum size of GPT volume is 64 ZiB.

Reference: docs.aws.amazon.com/AWSEC2/latest/UserGuide..

gpt-vs-mbr.png

Convert EC2's default MBR to GPT

Reference: aws.amazon.com/premiumsupport/knowledge-cen..

1. Launch an instance from an Amazon Machine Image (AMI) running Ubuntu (default 8GiB storage).

Screen Shot 2021-08-31 at 1.17.31 AM.png

2. Launch a second instance in the same Availability Zone as the first instance and from the same Ubuntu AMI with a 3 TiB root volume.

Note: If you already have a Ubuntu instance, you don't need to launch a new Ubuntu instance. For Ubuntu, modify the root volume in order to extend its size to over 2 TiB.

Screen Shot 2021-08-31 at 3.35.33 AM.png

3. Stop the instance with the 3 TiB root volume.

Screen Shot 2021-08-31 at 3.41.28 AM.png

4. Detach the root volume (/dev/xvda or /dev/sda1) from the stopped instance. Then, attach it to the instance you created in step 2 as /dev/sdf.

Screen Shot 2021-08-31 at 3.48.09 AM.png

Screen Shot 2021-08-31 at 3.50.58 AM.png

5. Connect to the running instance (created in step 1) using SSH. Use the lsblk command to view the root partition of /dev/sdf. The root partition of /dev/sdf is only 2 TiB size, as shown in the following example:

$ lsblk
NAME    MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
(snip)
xvdf    202:80    0   3T  0 disk 
└─xvdf1 202:81    0   2T  0 part 
(snip)

6. Use the gdisk tool to convert the partition table from MBR to GPT.

$ sudo gdisk /dev/xvdf
GPT fdisk (gdisk) version 1.0.5

Partition table scan:
  MBR: MBR only
  BSD: not present
  APM: not present
  GPT: not present

***************************************************************
Found invalid GPT and valid MBR; converting MBR to GPT format
in memory. THIS OPERATION IS POTENTIALLY DESTRUCTIVE! Exit by
typing 'q' if you don't want to convert your MBR partitions
to GPT format!
***************************************************************

7. Enter the following commands at the prompts to enter Expert mode and set the sector alignment value. Note: Make sure that you change the sector alignment value from the default 8 to 1. Using the 8 bytes alignment might cause issues when you create the GPT partition.

Command (? for help): x                                            
Expert command (? for help): l                                     
Enter the sector alignment value (1-65536, default = 2048): 1      
Expert command (? for help): m

8. Enter the following commands at the prompts to create a GPT partition. Type enter at the Last sector prompt to use the default sector number 2047.

Command (? for help): n                                                                                                        
Partition number (2-128, default 2): 128                                                                                
First sector (34-6291455966, default = 4294967296) or {+-}size{KMGTP}: 34                     
Last sector (34-2047, default = 2047) or {+-}size{KMGTP}:                                                  
Current type is 'Linux filesystem'
Hex code or GUID (L to show codes, Enter = 8300): ef02                                                     
Changed type of partition to 'BIOS boot partition'

Note: In the preceding example, ef02 is the BIOS boot partition number.

9. Enter the following commands at the prompts to delete the root partition:

Command (? for help): d                                                                                                         
Partition number (1-128): 1

10. Enter the following commands at the prompts to recreate the root partition to 3 TB. In the First sector, Last sector, and Hex code or GUID prompts, type enter to use the default settings.

Command (? for help): n                                                                                                         
Partition number (1-128, default 1): 1                                                                                     
First sector (2048-6291455966, default = 2048) or {+-}size{KMGTP}:                                  
Last sector (2048-6291455966, default = 6291455966) or {+-}size{KMGTP}:                       
Current type is 'Linux filesystem'
Hex code or GUID (L to show codes, Enter = 8300):                                                             
Changed type of partition to 'Linux filesystem'

11. Enter the following commands at the prompts to save the GPT partition table:

Command (? for help): w                                                                                                        
Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!
Do you want to proceed? (Y/N): y                                                                                          
OK; writing new GUID partition table (GPT) to /dev/xvdf.
The operation has completed successfully.

Run the lsblk command to view the new volume information:

$ lsblk
NAME    MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvdf    202:80   0   3T  0 disk 
└─xvdf1 202:81   0   3T  0 part

12. Use the fsck tool to check that the file system of device /dev/xvdf1 is correct. For more information on the fsck tool, see fsck on the Ubuntu man website.

$ sudo e2fsck -f /dev/xvdf1
e2fsck 1.42.13 (17-May-2015)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
cloudimg-rootfs: 57524/262144000 files (0.0% non-contiguous), 16648272/536870655 blocks

13. Run the resize command to resize the file system in order to extend it to 3TB. Note: It might take from 10 to 20 seconds to resize the file system.

$ sudo resize2fs /dev/xvdf1
resize2fs 1.42.13 (17-May-2015)
Resizing the filesystem on /dev/xvdf1 to 786431739 (4k) blocks.
The filesystem on /dev/xvdf1 is now 786431739 (4k) blocks long.

14. Install Grub on the device /dev/xvdf1 and configure it using the following commands for Ubuntu 20.04:

sudo mount /dev/xvdf1 /mnt 
sudo mount --bind /proc /mnt/proc
sudo mount --bind /sys /mnt/sys
sudo mount --bind /dev /mnt/dev
sudo chroot /mnt /bin/bash
grub-install /dev/xvdf
grub-mkdevicemap
echo "GRUB_DISABLE_OS_PROBER=true" > /etc/default/grub
echo "GRUB_FORCE_PARTUUID=" > /etc/default/grub.d/40-force-partuuid.cfg
update-grub
exit
sudo umount -l /mnt/dev
sudo umount -l /mnt/sys
sudo umount -l /mnt/proc
sudo umount -l /mnt

15. Detach the volume /dev/xvdf from the running instance and attach the volume /dev/xvdf back to its original instance (stopped instance) as /dev/xvda or /dev/sda1.

16. Start the original instance using SSH and run the lsblk command to verify that the root volume on your original instance now has 3 TiB of space.

 
Share this