Rock 5B+ NVMe Migration & Storage Architecture Guide

This document details the complete process of migrating a Rock 5B+ (RK3588) system from an SD card to an NVMe SSD, specifically covering partition planning and operations, technical principles, troubleshooting guides, and SPI Flash expansion operations.

Platform: Rock 5B+ / RK3588 Development Board

Core Objectives:

1. Performance Leap: Migrate the Root Filesystem (Rootfs) to NVMe SSD (Read/Write speeds of 2000MB/s+).
2. Storage Tiering: NVMe handles hot system data + compilation cache; SD card handles cold data backups.
3. High Stability: Resolve PCIe dropouts and boot sequence Race Conditions.

1: Storage Media & Partitioning Strategy

This is the foundation of the system architecture. Proper planning avoids future expansion troubles and memory overflows.

1.1 Partition Design Philosophy

Linux systems typically require only one root partition, but in embedded high-performance scenarios, physical isolation is recommended:

• Rootfs (System Area): Stores the OS and software libraries. Doesn’t need to be huge, but must be fast.
• Swap (Swap Area): Critical Point! Compiling large C++ projects (like LLVM/Chromium) on RK3588 can easily cause Out Of Memory (OOM) errors. NVMe random read/write far exceeds SD cards, so allocating Swap here acts as “cheap memory”.
• Data (Data Area): Stores user code, Docker images, and AI models. Isolating this ensures data safety during system reinstalls.

1.2 Recommended Partition Table (NVMe 256GB Example)

1.3 Practical Operation Commands

Use cfdisk (GUI-like) or fdisk (Command line).

Step A: NVMe Partitioning & Formatting

# 1. Enter partitioning tool
sudo cfdisk /dev/nvme0n1
# Action: Delete old partitions -> New -> Input size (64G) -> Type select Linux filesystem
# Action: New -> Input size (16G) -> Type select Linux swap
# Action: New -> Remaining space -> Type select Linux filesystem
# Action: Write (type yes) -> Quit

# 2. Format filesystems (Critical)
# Format system partition (p1)
sudo mkfs.ext4 -L rootfs_nvme /dev/nvme0n1p1

# Make and enable Swap (p2)
sudo mkswap /dev/nvme0n1p2
sudo swapon /dev/nvme0n1p2

# Format data partition (p3)
sudo mkfs.ext4 -L data_nvme /dev/nvme0n1p3

2: System Cloning & Data Migration

This phase involves “hot migrating” the running OS to the new drive, essentially performing surgery on the system.

2.1 Mounting Environment Preparation

# Create temporary mount point
sudo mkdir -p /mnt/nvme_root
# Mount new system partition
sudo mount /dev/nvme0n1p1 /mnt/nvme_root

2.2 Execute Rsync Mirror Cloning (Core Operation)

Technical Principle: rsync handles file synchronization. You must use --exclude to exclude virtual filesystems like /proc, /sys, etc. These directories contain files dynamically generated by the kernel in memory; copying them directly causes infinite recursion or errors.

Full Command:

sudo rsync -axHAWXS --numeric-ids --info=progress2 \
  --exclude=/proc \
  --exclude=/sys \
  --exclude=/dev \
  --exclude=/run \
  --exclude=/mnt \
  --exclude=/media \
  --exclude=/lost+found \
  --exclude=/tmp \
  / /mnt/nvme_root/

2.3 Rebuild Virtual Directory Structure (Common Pitfall)

Although we excluded the contents of virtual directories, we must preserve empty folders as mount points, otherwise the system cannot mount hardware interfaces during boot, causing the init process to crash.

sudo mkdir -p /mnt/nvme_root/{proc,sys,dev,run,mnt,media,tmp,data}
sudo chmod 1777 /mnt/nvme_root/tmp

3: Boot Configuration

Now that the system has moved, we need to modify its “ID card” and “road signs”.

3.1 Get UUIDs

Device names in Linux (/dev/nvme0n1) can change; UUIDs are the unique identifiers.

lsblk -f
# Record UUIDs for NVMe p1, p2, p3

3.2 Modify Fstab (New System’s Mount Table)

Edit the file inside the new drive: sudo nano /mnt/nvme_root/etc/fstab

# <file system>    <mount point>   <type>  <options>       <dump>  <pass>
# Root directory points to NVMe p1
UUID=bd10...       /               ext4    defaults        0       1
# Enable NVMe Swap
UUID=2792...       none            swap    sw              0       0
# Mount Data drive
UUID=30c7...       /data           ext4    defaults        0       2

3.3 Modify Extlinux (Boot Signpost)

This is the configuration read by the Bootloader. Edit the file on the SD card (since we are booting from SD): sudo nano /boot/extlinux/extlinux.conf

Key Modifications:

1. root target: Change to NVMe p1 UUID.
2. rootwait (Must Add): Prevents the kernel from booting too fast before NVMe initializes, avoiding Gave up waiting for root.
3. PCIe Slowdown (Optional): If you encounter LTSSM is 0x0 errors, it indicates unstable signals; force downgrade to PCIe 2.0.

Final Configuration Line Example:

append root=UUID=bd10... console=ttyFIQ0 ... rootwait rk_pcie.link_speed=2 rw init=/sbin/init

4: SD Card Reclamation & Secondary Use

After successful migration, the old system partition on the SD card becomes junk data.

4.1 Operation Steps

1. Delete Old Partition: Use cfdisk /dev/mmcblk1 to delete the largest partition (p3).

2. Create New Data Partition: Create a new ext4 partition in place.

3. Recover “Lost” Space (Technical Point):

   • Phenomenon: A 238G card only has 220G left after formatting.
   • Principle: ext4 defaults to reserving 5% space for the root user (Reserved Blocks). For a pure data disk, this is wasteful.
   • Command:

   # -m 0 means reserve 0%
   sudo tune2fs -m 0 /dev/mmcblk1p3

4. Auto Mount: Add nofail parameter in fstab (prevents boot failure if card is removed).

5: Expansion - SPI Flash Boot (Ditching the SD Card)

If you wish to completely remove the SD card and boot directly from NVMe, you need to flash the Bootloader into the onboard SPI Flash.

5.1 Principle

Rock 5B+ Boot Order: SPI Flash -> eMMC -> SD Card. By default, SPI is empty, so an SD card is required (acting as a key). After flashing SPI, the board has its own key and can unlock NVMe directly.

5.2 Flashing Steps (High Risk)

1. Download Image: Get the official Radxa spi-image.img that supports NVMe boot.

2. Check Device: Confirm /dev/mtdblock0 exists.

3. Flash Command:

   # Clear old data at the head
   sudo dd if=/dev/zero of=/dev/mtdblock0 bs=1M count=10
   # Write new Bootloader
   sudo dd if=spi-image.img of=/dev/mtdblock0 bs=1K status=progress
   sync

4. Verify: Power off, remove SD card, and power on. If it boots into the system, congratulations on completely detaching from the SD card.

Troubleshooting Checklist