Unable to boot Linux - userspace cannot write to console, no busybox shell about linux-on-litex-rocket

[gsomlo]

On Wed, Mar 22, 2023 at 08:45:41AM -0700, Norbert Kremeris wrote: I'm building LiteX SoC with Rocket on a digilent nexys video board, which is listed as one of the supported platforms here.

If you're using the pre-built binaries for software, chances are that MMIO registers placement has changed in more recent gateware builds. If you're putting together your own software, make sure your devicetree .dts matches the way your gateware was built (use `--csr-csv ./csr-csv` to ensure that information is captured during build). HTH, --Gabriel PS. please be warned that cpu variant naming is about to be changed pretty significantly, which will make the "linux-on-litex-rocket/README.md" (even more) obsolete, and in need of an update (next on my list, I promise :) )

[n-kremeris]

Hi Gabriel! Sorry, i accidentally posted the issue without writing all the info and then frantically started editing it. Didn't expect such a quick reply!

Please take a look again at the updated info when you have the time. I am indeed using the generated (but slightly modified) dts.

I am using my own build of linux, bbl, and tried a custom build of busybox/initramfs.cpio as well as the prebuilt one.

[roryt12]

Sounds like the issues I had with the irq of liteuart? Take a look at https://github.com/roryt12/qmtech_wukong_debian_on_litex_rocket . It hanged exactly at this point if I enable irq vs pooling?

Also try an alternative , ie the hvc0 driver, at my try with naxvriscv https://github.com/roryt12/qmtech_wukong_debian_on_litex_rocket seems to work fine (needs more options in kernel and in dts )

[gsomlo]

I haven't built anything using BBL in a very long time, having been discouraged from doing so by upstream who now favor
opensbi. The latter doesn't support emulating the FPU in M-mode, so it only works on boards that can accomodate at least one FPU-enabled Rocket core (85k ecp5, so no versa board; nexys* boards and better xilinx-equipped ones should still work).

I need to update the README to reflect this, but I'm also trying to get upstream LiteX to generate the full *.dts for a rocket-based system automatically, so I don't have to screw around with sample device tree files anymore :)

In the mean time, if you have a nexys_video, I'd like to propose that you try the following and let me know how it goes:

On the very latest upstream litex, add #define CONFIG_BIOS_NO_BOOT to the top of litex/soc/software/bios/main.c to inhibit it from automatically trying to boot from sdcard (you can still manually issue whatever boot command you want from the LiteX bios prompt). Then, build for the nexys_video:

litex-boards/litex_boards/targets/digilent_nexys_video.py --build \
    --cpu-type rocket --cpu-variant full --cpu-num-cores 2 --cpu-mem-width 2 --sys-clk-freq 50e6 \
    --with-ethernet --with-sdcard \
    --with-sata --sata-gen 1 --with-sata-pll-refclk \
    --csr-csv ./csr.csv

It's OK if you don't have a SATA adapter, but building it in anyway will make the MMIO registers line up properly with the DTS sample I'm about to link you to :)

Build the litex-rebase branch of the litex-hub kernel tree:

make clean
make ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu-  litex_rocket_defconfig
make ARCH=riscv CROSS_COMPILE=riscv64-unknown-linux-gnu- -j8

This gets you an arch/riscv/boot/Image kernel file.

Unpack nexys_video_binaries.zip There will be a copy of Image already in there, pre-built. Feel free to use it or ignore it if you built your own. There's also a satatest4imafdc.dts sample in there. Compile it with:

dtc -O dtb satatest4imafdc.dts -o satatest4imafdc.dtb

Then build fw_jump.bin from source (or use the included one), like so:

make CROSS_COMPILE=riscv64-unknown-linux-gnu- PLATFORM=generic \
    FW_FDT_PATH=~/...path/to/.../satatest2imafdcbkph.dtb FW_JUMP_FDT_ADDR=0x82400000

Use the included initrd_bb (which should not be significantly different from previous sets of instructions).

Place the included boot.json, initrd_bb, Image, and fw_jump.bin into a tftp server directory (or on your FAT formatted sdcard), and issue the appropriate boot command (netboot or sdcardboot) from the LiteX bios.

LMK if this works for you (it does for me, and it's how things will be done from now on -- no more BBL).

All of this is also perfectly capable to boot e.g. Fedora (with an appropriately modified DTS and some doctoring of the official raw image to get it installed to the sdcard :) ).

LMK how it goes -- good luck!

[gsomlo]

note the edited --cpu-variant linux --cpu-num-cores 4 --cpu-mem-width 2 command line. If you go for full (now with [H]ypervisor support), there's only room for 2 such cores, and I gave you the wrong DTS for that :)

[gsomlo]

final edit: use --cpu-type rocket --cpu-variant full --cpu-num-cores 2 to get H support, and be able to use the enclosed binary opensbi blob (updated the source DTS to reflect the two larger h-capable full cores vs. the previous 4 lighter linux ones). The updated/edited commend containing build instructions is now correct and should reference the correct zip file.

[n-kremeris]

Hi @gsomlo!

Thanks very much for the detailed updated instructions. I have successfully managed to boot linux on the digilent nexys video by following your comments, with slight modifications to suit my needs!

I have had some weird issues with my litex_setup.py - doing "--update" failed to pull the latest pythondata-cpu-rocket versions so the required configuration did not exist! To remedy, i had to delete everything related to litex, clone a fresh copy of litex and re-run litex_setup.py --install --size full --user.

I have built the bitstream using ./litex-boards/litex_boards/targets/digilent_nexys_video.py --build --cpu-type rocket --cpu-variant full --cpu-num-cores 1 --cpu-mem-width 2 --sys-clk-freq 50e6 --with-ethernet --with-sdcard --with-sata --sata-gen 1 --with-sata-pll-refclk --csr-csv ./csr.csv

I have additionally modified the provided dts to only have 1 cpu core, as well as update the initramfs end address to fit my new bigger busybox image (initrd end = initrd_start + sizeof(initramfs file) + 12814)

linux,initrd-end = <0x8220320E>; /* end initrd.gz + 12814 (?) bytes */
otherwise, the boot gets stuck at Waiting for root device /dev/ram0...

I have built busybox at tag 1_36_0, and for anyone else looking to rebuild the initramfs/initrd, here's how i did it (mostly matches the original instructions)

#!/bin/bash
rm -rf initramfs
rm -rf initramfs.cpio
mkdir initramfs
pushd initramfs
mkdir -p bin sbin lib etc dev home proc sys tmp mnt nfs root \
          usr/bin usr/sbin usr/lib
sudo mknod -m 622 dev/console c 5 1
sudo mknod -m 622 dev/tty0 c 4 0
cp ../busybox_git/busybox bin/
ln -s bin/busybox ./init
cat > etc/inittab <<- "EOT"
::sysinit:/bin/busybox mount -t proc proc /proc
::sysinit:/bin/busybox mount -t devtmpfs devtmpfs /dev
::sysinit:/bin/busybox mount -t tmpfs tmpfs /tmp
::sysinit:/bin/busybox mount -t sysfs sysfs /sys
::sysinit:/bin/busybox --install -s
/dev/console::sysinit:-/bin/ash
EOT
fakeroot <<- "EOT"
find . | cpio -H newc -o > ../initramfs.cpio
EOT
popd

Then copy the initramfs.cpio to tftp dir as initrd_bb.

Some extra details incase anyone is looking to replicate this:

riscv64-linux-gnu-gcc (Ubuntu 12.1.0-2ubuntu1~22.04) 12.1.0
riscv64-unknown-elf-gcc (g1ea978e3066) 12.1.0
DISTRIB_DESCRIPTION="Ubuntu 22.04.2 LTS"
opensbi @ c6a092cd80112529cb2e92e180767ff5341b22a3
litex-hub/linux (litex-rebase) @ b73e060b3b04cba84983a3786845a6d16c77bf1f

EDIT: perhaps you'd like for me to try update the readme to match current status?