The Linux BSP (Board Support Package) for the NXP i.MXRT SoCs supports booting from SD card. By default, the root file system is a part of the bootable Linux image and resides in SDRAM when the target board is running.

This application note explains how to configure the Linux project and prepare the SD card to move the root file system to a separate, ext4 formatted partition on the SD card. This provides the following benefits:

No need to keep all files in the init RAM disk inside the bootable image;
Use a larger file system without affecting resources in RAM;
Update the file system on the per-file basis without need to rebuild the entire bootable image.

1. Building the Project

Step through the following procedure:

On the Linux development host, install the BSP distribution and activate the cross-build environment as described in TBD Installing and Activating Cross Development Environment:
```
$ tar -jxf linux-cm-imxrt-<bsp>-<release>.tar.bz2
$ cd linux-cm-imxrt-<bsp>-<release>
$ . ./ACTIVATE.sh
```
Change to the rootfs project directory:
```
$ cd projects/rootfs
```
Update the Linux configuration in order to not include the initial RAM disk into the bootable uImage. For this, run make kmenuconfig in the project directory and uncheck (disable) the Initial RAM filesystem... option:
```
General setup --->
[ ] Initial RAM filesystem and RAM disk (initramfs/initrd) support
```
Enable building the tarball with the root file system outside of the bootable uImage. For this, define the special FS_IMAGES option as $(SAMPLE).tar.gz in Makefile. Open the projects/rootfs/Makefile file in a text editor and uncomment the following line:
```
FS_IMAGES := $(SAMPLE).tar.gz
```

Build the project:

$ make
...
SAMPLE=rootfs  \
        ../rfs-builder.py create-rfs-dir \
        rootfs.initramfs `pwd`/rootfs-build-tmpdir
tar -C rootfs-build-tmpdir -zcf rootfs.tar.gz --owner=0 --group=0 .

Check that the following images required to boot the system with the root file system mounted on SD Card has been created:
- rootfs.uImage - a kernel image without the root file system in it;
- rootfs.tar.gz - a tarball containing the root filesystem.
```
$ ls -1 rootfs.uImage rootfs.tar.gz
rootfs.tar.gz
rootfs.uImage
$
```

2. Partitioning SD Card

The Emcraft Linux software distribution for the i.MX RT BSPs assumes the following utilization of the SD card storage:

The first 1MB is a service area to store the partition table and the bootloader files;
The next 100MB are allocated to the Boot partition to store the bootable Linux image;
The remainder of the SD card space is allocated to the RootFS partition to store the root file system.

Step through the following procedure:

Use the fdisk utility to prepare the SD card on a Linux host.

As all operations are made with the root permissions, be careful, and verify the destination /dev/sdX to avoid corruption of the system drives:

$ sudo fdisk /dev/sdx

Welcome to fdisk (util-linux 2.37.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Command (m for help): p
Disk /dev/sdb: 7,39 GiB, 7932477440 bytes, 15493120 sectors
Disk model: STORAGE DEVICE
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xc36d1c99

Device     Boot  Start      End  Sectors  Size Id Type
/dev/sdb1         2048   196607   194560   95M  b W95 FAT32
/dev/sdb2       196608 15493119 15296512  7,3G 83 Linux

Remove all partitions using the d command:

Command (m for help): d
Partition number (1,2, default 2):

Partition 2 has been deleted.

Command (m for help): d
Selected partition 1
Partition 1 has been deleted.

Command (m for help): p
Disk /dev/sdb: 7,39 GiB, 7932477440 bytes, 15493120 sectors
Disk model: STORAGE DEVICE
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xc36d1c99

Use the n command to create new primary (p) partitions. The first partition size is 100M, the second one size is all free space left on SD card:

Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p): p
Partition number (1-4, default 1):
First sector (2048-15493119, default 2048):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-15493119, 
default 15493119): +100M

Created a new partition 1 of type 'Linux' and of size 100 MiB.
Partition #1 contains a vfat signature.

Do you want to remove the signature? [Y]es/[N]o: Y

The signature will be removed by a write command.

Command (m for help): t
Selected partition 1
Hex code or alias (type L to list all): b
Changed type of partition 'Linux' to 'W95 FAT32'.

Command (m for help): n
Partition type
   p   primary (1 primary, 0 extended, 3 free)
   e   extended (container for logical partitions)
Select (default p): p
Partition number (2-4, default 2):
First sector (206848-15493119, default 206848):
Last sector, +/-sectors or +/-size{K,M,G,T,P} 
(206848-15493119, default 15493119):

Created a new partition 2 of type 'Linux' and of size 7,3 GiB.

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.
$

Install U-Boot to the beginning of the SD card (the prebuilt U-Boot SPL and TPL images can be obtained from the Emcraft web site):

$ sudo dd if=SPL of=/dev/sdb bs=1k seek=1 conv=notrunc; sync
$ sudo dd if=u-boot.img of=/dev/sdb bs=1k seek=128 conv=notrunc; sync

Format the Boot partition with a FAT file system:

$ sudo mkfs.vfat /dev/sdb1
mkfs.fat 4.2 (2021-01-31)
$

Install the Linux bootable image to the Boot partition on the SD card:

$ sudo mount /dev/sdb1 /mnt
$ sudo cp ${INSTALL_ROOT}/projects/rootfs/roofts.uImage /mnt
$ sudo umount /mnt

Format the RootFS partition with the EXT4 file system:

$ sudo mkfs.ext4 -E lazy_itable_init=0,lazy_journal_init=0 /dev/sdb2
mke2fs 1.46.5 (30-Dec-2021)
Creating filesystem with 1910784 4k blocks and 478608 inodes
Filesystem UUID: ba548d3d-f65e-4353-a08b-d6bc711922a9
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632

Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done
$

The lazy_itable_init=0,lazy_journal_init=0 options to mkfs.ext4 are required to disable "lazy initialization", due to issues with this EXT4 feature on the no-MMU targets in kernel 5.15.

Mount the RootFS partition, change to the mounted partition directory and unpack the rootfs.tar.gz image here (the command can require the root privileges):
```
$ sudo mount /dev/sdb2 /mnt
$ cd /mnt
$ sudo tar zxf ${INSTALL_ROOT}/projects/rootfs/rootfs.tar.gz
$ cd -
$ sudo umount /mnt
```
Set up the boot jumper on the target board to boot from SD Card.
Insert the prepared card into the SD Card holder on the target.

Update the bootargs environment variable in U-boot to tell the kernel to mount the root file system from the dedicated partition on the SD Card. For this, power on the target board and stop booting in the U-Boot command monitor by pressing any key:

U-Boot SPL 2023.04 (Dec 12 2023 - 13:24:53 +0000)
Trying to boot from MMC1


U-Boot 2023.04 (Dec 12 2023 - 13:24:53 +0000)

Model: NXP imxrt1170-evk board
DRAM:  960 KiB (effective 64.9 MiB)
Core:  72 devices, 15 uclasses, devicetree: separate
MMC:   FSL_SDHC: 0
Loading Environment from MMC... OK
In:    serial@4007c000
Out:   serial@4007c000
Err:   serial@4007c000
Net:   eth0: ethernet@40424000
Hit any key to stop autoboot:  0
=>

Update the bootargs environment variable to use /dev/mmcblk0p2 matching the 2nd partition on the SD Card:

=> setenv bootargs root=/dev/mmcblk0p2 rw rootwait
=> saveenv
Saving Environment to MMC... Writing to MMC(0)... OK
=>

Reset the board to boot Linux with the root filesystem on the SD Card. Wait for the shell prompt and check that the system is booted and the mount utility shows that the root file system resides on the SD Card partition /dev/mmcblb0p2:

U-Boot SPL 2023.04 (Dec 12 2023 - 13:24:53 +0000)
Trying to boot from MMC1


U-Boot 2023.04 (Dec 12 2023 - 13:24:53 +0000)

Model: NXP imxrt1170-evk board
DRAM:  960 KiB (effective 64.9 MiB)
Core:  72 devices, 15 uclasses, devicetree: separate
MMC:   FSL_SDHC: 0
Loading Environment from MMC... OK
In:    serial@4007c000
Out:   serial@4007c000
Err:   serial@4007c000
Net:   eth0: ethernet@40424000
Hit any key to stop autoboot:  0
4984553 bytes read in 427 ms (11.1 MiB/s)
## Booting kernel from Legacy Image at 80007fc0 ...
   Image Name:   Linux-6.1.22
   Image Type:   ARM Linux Multi-File Image (uncompressed)
   Data Size:    4984489 Bytes = 4.8 MiB
   Load Address: 80008000
   Entry Point:  80008001
   Contents:
      Image 0: 4972544 Bytes = 4.7 MiB
      Image 1: 11933 Bytes = 11.7 KiB
   Verifying Checksum ... OK
## Flattened Device Tree from multi component Image at 80007FC0
   Booting using the fdt at 0x804c600c
Working FDT set to 804c600c
   Loading Multi-File Image
   Loading Device Tree to 2032a000, end 2032fe9c ... OK
Working FDT set to 2032a000

Starting kernel ...

Booting Linux on physical CPU 0x0
Linux version 6.1.22 (sasha@workbench.emcraft.com) (arm-none-eabi-gcc 
(GNU Arm Embedded Toolchain 10.3-2021.10) 10.3.1 20210824 (release), GNU ld 
(GNU Arm Embedded Toolchain 10.3-2021.10) 2.36.1.20210621) #207 Tue Dec 12 
12:30:27 UTC 2023
...
mmc0: new high speed SDHC card at address 59b4
mmcblk0: mmc0:59b4 USD   7.39 GiB
 mmcblk0: p1 p2
EXT4-fs (mmcblk0p2): recovery complete
EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. 
Quota mode: disabled.
VFS: Mounted root (ext4 filesystem) on device 179:2.
devtmpfs: mounted
Freeing unused kernel image (initmem) memory: 92K
This architecture does not have kernel memory protection.
Run /sbin/init as init process
[80] Jan 01 00:00:02 Running in background
/ # RTL8201F Fast Ethernet 40424000.ethernet-1:00: attached PHY driver 
(mii_bus:phy_addr=40424000.ethernet-1:00, irq=POLL)
fec 40424000.ethernet eth0: Link is Up - 100Mbps/Full - flow control off
random: crng init done
/ # echo Hello
Hello
/ # mount
/dev/root on / type ext4 (rw,relatime)
devtmpfs on /dev type devtmpfs (rw,relatime)
proc on /proc type proc (rw,relatime)
sysfs on /sys type sysfs (rw,relatime)
devpts on /dev/pts type devpts (rw,relatime,gid=5,mode=620,ptmxmode=000)
/ # dmesg | grep "Mounted root"
VFS: Mounted root (ext4 filesystem) on device 179:2.
/ # ls -l /dev/mmcblk0p2 
brw-------    1 root     root      179,   2 Jan  1 00:00 /dev/mmcblk0p2
/ #

Boot the board with the root file system on the SD Card. Wait for the shell prompt and check that root file system is mounted in the Read/Write mode:

...
Run /sbin/init as init process
[80] Jan 01 00:00:01 Running in background
/ # RTL8201F Fast Ethernet 40424000.ethernet-1:00: 
attached PHY driver (mii_bus:phy_addr=40424000.ethernet-1:00, irq=POLL)
random: crng init done

/ # cp /bin/busybox /root/
/ # reboot
The system is going down NOW!
Sent SIGTERM to all processes
[80] Jan 01 00:11:54 Early exit: Terminated by signal
Terminated
Requesting system reboot
reboot: Restarting system

U-Boot SPL 2023.04 (Dec 12 2023 - 13:24:53 +0000)
Trying to boot from MMC1
...
/ # md5sum /bin/busybox /root/busybox
fc95be9f30c93e98beda14b2ec2c818f  /bin/busybox
fc95be9f30c93e98beda14b2ec2c818f  /root/busybox
/ #

Booting Linux with root Filesystem on SD Card

1. Building the Project

2. Partitioning SD Card