Flashing an Orin Nano (8Gb) NVME + initrd_flash questions

[paroque28]

Hi guys,

So my questions started because I was trying to flash a p3767-0001-p3509-a02 with only and NVME (no SDcard connected, and as I understand no EPPROM as well) with a Mender based image (A/B)

When I ran the initrd flash script I got the following error:

== Step 2: Boot Jetson via RCM at 2024-08-07T20:46:06+02:00 ==
Found Jetson device in recovery mode at USB 1-5
== Step 3: Sending flash sequence commands at 2024-08-07T20:46:08+02:00 ==
Waiting for USB storage device flashpkg from 0353fe09.......[/dev/sda]
Device size in blocks: 262144
Unmounted /dev/sda.
== Step 4: Writing partitions on internal storage device at 2024-08-07T20:46:31+02:00 ==
Waiting for USB storage device nvme0n1 from 0353fe09...[/dev/sda]
No partition definitions found in initrd-flash.xml
ERR: write failure to internal storage at 2024-08-07T20:46:33+02:00

The thing is that I noticed that flash_t234_qspi_nvme.xml has both the SPI and NVME partitions, but when tegra-bootfiles recipe runs, it strips out the rootfs part leaving only the boot one

See code:
https://github.com/OE4T/meta-tegra/blob/kirkstone/recipes-bsp/tegra-binaries/tegra-bootfiles_35.5.0.bb#L159

if [ "${TEGRAFLASH_NO_INTERNAL_STORAGE}" = "1" ]; then
  install -m 0644 ${PARTITION_FILE} ${D}${datadir}/tegraflash/bupgen-${PARTITION_LAYOUT_TEMPLATE}
  nvflashxmlparse --extract -t boot --output=${D}${datadir}/tegraflash/${PARTITION_LAYOUT_TEMPLATE} ${PARTITION_FILE}
  chmod 0644 ${D}${datadir}/tegraflash/${PARTITION_LAYOUT_TEMPLATE}

And why is this a problem?
When initrd_flash runs make-sdcard the xml file only contains one device (SPI). The nvme part was removed in tegra-bootfiles.
Then the script fails because it needs at least two devices.

https://github.com/OE4T/meta-tegra/blob/kirkstone/recipes-bsp/tegra-binaries/tegra-helper-scripts/make-sdcard.sh#L430

mapfile PARTS < <("$here/nvflashxmlparse" -t rootfs "$cfgfile")
if [ ${#PARTS[@]} -eq 0 ]; then
    echo "No partition definitions found in $cfgfile" >&2
    exit 1
fi

So my question is, if we have an Orin nano with an nvme and a A/B image.

What do we expect the flash to be? Meaning, what should the flash.xml.in look like, is it an internal or external flash or both? Should I connect an SDcard?

What should my .env.initrd-flash variables + flash.xml.in look like?

I ask about them because if I know what the output should look like then I can change my machine to match this configuration.

I hope this is clear enough, otherwise let me know

Some of my current variables (.env.initrd+Yocto):

BOOTDEV="nvme0n1p1"
ROOTFS_DEVICE="nvme0n1"
CHIPID="0x23"
DTBFILE="tegra234-p3767-0001-p3509-a02.dtb"
EMMC_BCTS="tegra234-p3767-0000-sdram-l4t.dts"
BOOT_PARTITIONS_ON_EMMC=0
NO_INTERNAL_STORAGE=1
TNSPEC_BOOTDEV_DEFAULT = "nvme0n1p1"
TNSPEC_BOOTDEV = "nvme0n1p1"
EXTERNAL_ROOTFS_DRIVE=0 # also tried with EXTERNAL_ROOTFS_DRIVE=1
PARTITION_LAYOUT_TEMPLATE_DEFAULT = "flash_t234_qspi_nvme.xml"
PARTITION_LAYOUT_EXTERNAL = "flash_l4t_nvme_rootfs_ab.xml"

Thanks a lot for any information

[paroque28]

What confuses me right now is that I don't know if I should do internal or external flash.

[dwalkes]

I was trying to flash a p3767-0001-p3509-a02 with only and NVME

So you've added a MACHINE for this at https://github.com/OE4T/meta-tegra/tree/kirkstone/conf/machine or in your own layer?

Meaning, what should the flash.xml.in look like, is it an internal or external flash or both?

I think flash.xml.in will only have the internal flash/spi. If I look at the resulting tegraflash dir for a scarthgap build of jetson-orin-nano-devkit-nvme.conf that is what I see. I'd try comparing a similar working MACHINE as a starting point.

There's a separate external-flash.xml.in which contains the external flash partitions I believe. See

meta-tegra/classes/image_types_tegra.bbclass

Lines 444 to 448 in a715a1e

	if [ "${TEGRAFLASH_ROOTFS_EXTERNAL}" = "1" ]; then
	rm -rf "${WORKDIR}/tegraflash/external" external-flash.xml.in
	mkdir "${WORKDIR}/tegraflash/external"
	tegraflash_create_flash_config "${WORKDIR}/tegraflash/external" ${LNXFILE} ${STAGING_DATADIR}/tegraflash/${PARTITION_LAYOUT_EXTERNAL}
	mv external/flash.xml.in ./external-flash.xml.in

Based on the logs it looks like you are ending up in this path of initrd-flash, which is happening because EXTERNAL_ROOTFS_DRIVE is not being set. I can see this is setup in my scarthgap config for jetson-orin-nano-devkit-nvme in the .env.initrd-flash section. Haven't traced where this is set up.

If I were going to try to dig further on this I'd compare to a working similar example, probably jetson-orin-nano-devkit-nvme.

[paroque28]

The machine is on layer and it looks like this:

require conf/machine/p3509-a02-p3767-0000.conf

KERNEL_DEVICETREE = "tegra234-p3767-0001-p3509-a02.dtb"
TEGRA_BOARDSKU = "0001"
NVPMODEL = "nvpmodel_p3767_0001"
TNSPEC_BOOTDEV_DEFAULT = "nvme0n1p1" # set to mmc if  TEGRAFLASH_ROOTFS_EXTERNAL = 1 is needed
TNSPEC_BOOTDEV = "nvme0n1p1"

PARTITION_LAYOUT_TEMPLATE_DEFAULT = "flash_t234_qspi_nvme.xml"
# PARTITION_LAYOUT_EXTERNAL = "flash_l4t_nvme_rootfs_ab.xml"

TEGRAFLASH_SDCARD_SIZE ?= "64G" # Fixes Mender issue with tegra_mender_calc_total_size
TEGRA_SPIFLASH_BOOT ?= "1"

# From Mender https://github.com/mendersoftware/meta-mender-community/blob/kirkstone/kas/jetson-orin-nano-devkit.yml#L15
UBOOT_EXTLINUX = "1"
USE_REDUNDANT_FLASH_LAYOUT_DEFAULT = "1"

Thanks for your suggestion Dan. I'll build for jetson-orin-nano-devkit-nvme and try, unfortunately I don't have that device in my hands but the resulting files should give me some clues

[paroque28]

@dwalkes

This was the right combination that made this work for me:

require conf/machine/p3509-a02-p3767-0000.conf
TEGRA_BUPGEN_SPECS:append = " fab=300;boardsku=0001;boardrev="


# From README: Linux_for_Tegra/tools/kernel_flash/README_initrd_flash.txt
TEGRA_FAB = "300"
TEGRA_BOARDSKU = "0001"
TEGRA_BOARDREV=""
NVPMODEL = "nvpmodel_p3767_0001"

TNSPEC_BOOTDEV_DEFAULT = "mmcblk0p1"
TNSPEC_BOOTDEV = "nvme0n1p1"

KERNEL_DEVICETREE = "tegra234-p3767-0001-p3509-a02.dtb"
EMMC_BCT = "tegra234-p3767-0001-sdram-l4t.dts"

TEGRAFLASH_NO_INTERNAL_STORAGE = "0"

PARTITION_LAYOUT_TEMPLATE_DEFAULT = "flash_t234_qspi.xml"
PARTITION_LAYOUT_EXTERNAL="flash_l4t_t234_nvme.xml"

# Mender tegra_mender_calc_total_size workaround
# Setting MENDER_STORAGE_TOTAL_SIZE_MB to 16Gb and no EMMC
EMMC_SIZE = "0"
MENDER_STORAGE_TOTAL_SIZE_MB = "16384"

# Mender KAS configuration
UBOOT_EXTLINUX = "1"
USE_REDUNDANT_FLASH_LAYOUT_DEFAULT = "1"