diff --git a/web/lukegbcom/posts/2023-08-08-uefi-boot-for-mochabin.md b/web/lukegbcom/posts/2023-08-08-uefi-boot-for-mochabin.md new file mode 100644 index 0000000000..9579b34af1 --- /dev/null +++ b/web/lukegbcom/posts/2023-08-08-uefi-boot-for-mochabin.md @@ -0,0 +1,269 @@ +--- +title: "UEFI Boot for Mochabin" +date: 2023-08-08 +layout: Post +hero: https://images.unsplash.com/photo-1506619216599-9d16d0903dfd +hero credit: https://unsplash.com/photos/XtUd5SiX464 +hero credit text: "Jakub Dziubak" +classes: + header: header-black-gradient +--- + +MOCHAbin is a pretty capable ARM board - it has a quad core ARMv8 Cortex-A72 @ +1400MHz, 8GB of RAM, 16GB of onboard eMMC, not to mention a _bunch_ of Ethernet +connectivity (1x 10Gb SFP+ cage, 1x 1Gb SFP cage, a WAN RJ45 port with PoE in, +and 4x LAN ports connected to an onboard switch chip). + +The main downside for me, however, was the boot firmware. Out of the box, it +ships with a [pretty +ancient](https://github.com/globalscaletechnologies/u-boot-marvell/tree/u-boot-2020.10-gti) +build of U-Boot, which fails to properly support UEFI. + +--- + +There are other options: there are references to people using what I can only +assume to be some variant of Marvell's fork of EDK II to provide UEFI support. +However, I went down the route of trying to get a more modern version of U-Boot +working. + +To cut a long story short, I got [Tow-Boot](https://tow-boot.org), a user-friendly +distribution of U-Boot with a pretty decent build system, to boot properly including +proper UEFI support. + +## What works + +- Environment storage in SPI flash +- Reading (and booting) from: + - eMMC + - USB +- EFI boot + +## What's not tested + +- The SFP+/SFP cages +- The WAN NIC +- SATA +- PCIe + +## What doesn't work + +- Any of the 4 "LAN" ports -- I think they require bringup of the switch chip, + and I haven't tried configuring that properly yet +- Using the actual `hw_info` SPI block that's supposed to hold the Ethernet MAC + addresses and PCB serial number. + +## Building an image + +If you want to build an image yourself, you'll need a Linux system with a +working install of [Nix](https://nixos.org). This doesn't need to be NixOS, Nix +on Debian should do just fine. You can even (probably) use Nix inside a Docker +container if you'd like. + +Got that together? Alright: + +1. Grab https://github.com/lukegb/Tow-Boot. The `lukegb/globalscale-mochabin` + branch should be the default, and it's what you'll need. +1. Run `nix-build -A globalscale-mochabin-8gb` (if you have the 8GB RAM + variant), or `nix-build -A globalscale-mochabin-4gb` (if you have the 4GB + variant). +1. Wait. +1. You should now have a `result` symlink that points to a directory with some + files in it. The one you probably want is + `result/binaries/Tow-Boot.spi.bin`, which is the build of Tow-Boot that uses + the SPI flash for storing the U-Boot environment. + +At this point, you can use the instructions in "Recovering" below to use +`mvebu64boot` to just boot this once, to see what it's like, or continue below +to flash it as your main bootloader. + +## Flashing + +Apologies in advance: many of these steps _should_ be automated, but _aren't_. +There are various steps where you should take notes or backups of things so you +can restore your device to a working state later. Please do that, and copy them +somewhere safe! + +You will need: + +- a USB stick that you can wipe +- a micro-USB cable to use to connect to the MOCHAbin via USB-serial + +### Getting set up + +1. Wipe your memory stick. Put a ext2\* filesystem on it, and copy + `result/binaries/Tow-Boot.spi.bin` to it. +1. Turn off the MOCHAbin. +1. Unplug any other USB devices from the MOCHAbin, and connect your memory + stick. +1. Start your terminal emulator. The MOCHAbin runs at 115200 bps. +1. Turn on the MOCHAbin. When prompted to interrupt boot, press a key. +1. You should now have a `Marvell>>` prompt. + +### Making sure everything is in place + +1. Run `usb start`. You should get some output ending in `scanning usb for storage devices... 1 Storage Device(s) found`. +1. Run `usb part`. You should see your single, ext2 filesystem. +1. Run `ext2ls usb 0:1`. You should see the content of your filesystem, + including `Tow-Boot.spi.bin`. + +### Backing up the SPI flash + +1. Run `sf probe`. This should report `SF: Detected w25q32bv with page size 256 Bytes, erase size 4 KiB, total 4 MiB`. This is the information about + your SPI flash. *If your SPI flash is not 4MiB, stop!* +1. Run `sf read $kernel_addr_r 0 0x400000`. This should report `device 0 whole chip` and then hang for 10s or so. +1. Run `ext4write usb 0:1 $kernel_addr_r /backupspi.img 0x400000`. This + should report `File System is consistent`, and then complete after between + 6 and 15s. + +### Backing up the environment block + +1. Run `env export -t $kernel_addr_r`. This will appear to do nothing. +1. Run `ext4write usb 0:1 $kernel_addr_r /backupenv.txt $filesize`. This + should again report `File System is consistent`, then complete after a few + seconds. + +### Noting down key environment variables + +1. Run `env print ethaddr eth1addr eth2addr pcb_sn`. +1. Copy and save the output somewhere. **You'll need this later.** + +### Flashing the new image + +1. Erase your SPI flash: run `sf erase 0 0x400000`. This ensures you don't + have any of the existing U-Boot environment lingering around. This will + appear to hang for a few tens of seconds, then report `SF: 4194304 bytes @ 0x0 Erased: OK`. +1. Flash the new image: run `bubt Tow-Boot.spi.bin spi usb`. + +### Rebooting and setting the environment again + +1. Turn the power off, then on again +1. When prompted to interrupt boot, hit Escape or press Ctrl-C. +1. You'll now be presented with the Tow-Boot menu. Scroll down to `Firmware Console` using the arrow keys and hit enter. +1. Now instead of `Marvell>> `, you should get a `=> ` prompt. +1. For each of `ethaddr`, `eth1addr`, `eth2addr` and `pcb_sn`, run `env set $VARIABLE_NAME $THE_VARIABLE_VALUE_YOU_NOTED_DOWN_EARLIER` +1. Run `env save`. This should eventually print `OK`. +1. Turn the power off, then on again. + +### Congratulations! + +You now have Tow-Boot. If you have a UEFI-compatible boot medium (either on the +eMMC or over USB, for instance), it will Just Work(TM). + +1. Take the memory stick and back up `backupspi.img` and `backupenv.txt` + somewhere safe. + +\* Technically this could be ext4. I suggest ext2 because modern +Linux distros will enable some ext4 extensions that won't work by default, and +this is very temporary. + +## Using an up-to-date DTB + +U-Boot will automatically discover new DTBs from the boot partition. I have the +following setup (my EFI System Partition is mounted at `/boot`; adjust +appropriately if yours is `/boot/efi`): + +``` +/boot + /boot/dtb + /boot/dtb/marvell + /boot/dtb/marvell/armada-7040-mochabin.dtb +``` + +I copy this file from the DTBs shipped with my Linux kernel build on update; if +you don't do this then you will (I think?) inherit the DTB used by U-Boot. + +## Recovering + +Oops! Something went wrong and you need to recover. No sweat. The Marvell +BootROM supports sending it an image via X-Modem, and you can do that over the +USB-Serial interface. + +The tool I've been using for doing this is +[`mvebu64boot`](https://github.com/pali/mvebu64boot), which is super simple. If +you have Nix, then this is available in recent `nixpkgs-unstable`, but this is +super trivial to build without Nix: just clone that somewhere and use `make`. + +Make sure you don't have any terminal emulators open on the port, and run: + +```txt +$ mvebu64boot -t -b path/to/flash-image.img /dev/ttyUSB0 + +# for instance: +$ mvebu64boot -t -b backupspi.img /dev/ttyUSB0 + +# or: +$ mvebu64boot -t -b result/binaries/Tow-Boot.spi.bin /dev/ttyUSB0 + +# or, using the bootloader images available from Globalscale's FTP (ftp://76.80.10.5/Downloads/Mochabin/bootloader_for_mochabin_hw-rev-1-5-0_20220905/): +$ mvebu64boot -t -b mochabin-bootloader-ddr4-8g-mvddr-41927ee-atf-277d4b6b-uboot-b794de0054-20220905-rel.bin /dev/ttyUSB0 +``` + +(where ttyUSB0 is replaced with whatever device name you have for the flash +image you're booting) + +Once this prints `Sending boot pattern...`, powercycle the MOCHAbin. You should +then get `BootROM is ready for image file transfer`, followed by a +rapidly-increasing percentage. This will take a few minutes - first the image +prolog needs to be sent, then there's a `Waiting for BootROM...` stage, then +the image itself. You'll then be presented with a serial terminal. Use `Ctrl-\ c` to exit. + +This works for booting the Tow-Boot images you built, or the SPI flash backup +image I repeatedly reminded you to make. + +To return to the previous image, boot to the U-Boot console (either the +built-in Marvell one or a Tow-Boot one), insert a memory stick containing the +image to flash from (I suggest ext2 again, for the same reasons as above) and +run `bubt backupspi.img spi usb`. + +## Getting things working + +NOTE: if you're just interested in building your own image, you can ignore +everything from this point on. + +Getting things working was all relatively simple: upstream U-Boot 2022.07 and +2023.07 have pretty much all the hardware support that's required to boot this +board. The main thing missing is a device tree. + +### Device trees + +Device trees are effectively a description of what hardware is where: this +avoids some of the automatic configuration, and simplifies a bunch of +customization required to get things booting. The Linux kernel has a bunch of +these - and we're in luck, because Globalscale have been paying Sartura to +upstream a bunch of the hardware enablement required to get the MOCHAbin to +boot well +(https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/boot/dts/marvell/armada-7040-mochabin.dts). + +There's one problem here, however: this DTS doesn't work inside the U-Boot +tree. U-Boot tries to be compatible with whatever Linux is doing so that device +trees are as portable as possible from the Linux source to the U-Boot tree, but +the specific naming of the `compatible` arguments is important for drivers, and +some of the includes are in different places. + +Instead, I opted to take the device tree [from their version of +U-Boot](https://github.com/globalscaletechnologies/u-boot-marvell/blob/u-boot-2020.10-gti/arch/arm/dts/armada-7040-mochabin.dts), +and hack it together until it worked on a more recent version. + +### Porting their device tree forwards + +To give Globalscale credit, this mostly worked without a hitch. There are a few +differences (`phy-mode = "sfi";` needed to become `phy-mode = "10gbase-r";`), +but for the most part this... just worked. + +I haven't tested a bunch of the functionality that I didn't need: in particular, +I don't plan on PXE booting or using any network functionality at startup. I also +don't have any drives connected over the internal M.2 SATA interface, so I couldn't +test that either. + +### Future work + +I'm hoping to, eventually: + +- Try to get the Linux DTS working on U-Boot and upstream it. +- Get the 4 RJ45 LAN ports working. +- Check that the SATA M.2 port actually works. +- Get writable EFI variables working? +- Upstream my build to Tow-Boot proper. +- Getting the flashing steps automated in the Tow-Boot installer. + +So stay tuned. Maybe.