attila-lendvai / openwrt-auto-extroot

Use the OpenWRT ImageBuilder to build a firmware that automatically formats and moves extroot to any (!) inserted storage device

What

It's a script to build a customized OpenWrt firmware image using ImageBuilder.

If the generated image is flashed on a router, then during its boot process it will try to automatically set up extroot on any (!) storage device plugged into the USB port (/dev/sda), including your already working extroot pendrive if you plug it in too late in the boot process.

Why

So that e.g. customers can buy a router on their own, download and flash our custom firmware, plug in a pendrive, and manage their SIP (telephony) node from our webapp.

I've extracted the generic parts from the above mentioned auto-provision project because I thought it's useful enough for making it public.

It also serves me well on my own routers ever since then.

How

You can read more about the underlying technology on the OpenWrt wiki: see e.g. the ImageBuilder page, or the page that lists some other ImageBuilder frontends.

As for the actual mechanism: custom scripts are baked into the boot process of the flashed firmware. If the extroot overlay is properly set up, then these scripts get hidden by it; i.e. they will only be run when the extroot has failed to mount early in the boot process.

Keep in mind that this will automatically erase/format any inserted storage device while the router is in the initial setup phase! Unfortunately there's little that can be done at that point to ask the user for confirmation.

Building

OpenWrt's ImageBuilder only works on Linux x86_64. To build a firmware, issue the following command: ./build.sh architecture variant device-profile, e.g.:

• ./build.sh ath79 generic tplink_tl-wr1043nd-v1
• ./build.sh ath79 generic tplink_archer-c6-v2
• ./build.sh ath79 generic tplink_tl-wdr4300-v1
• ./build.sh bcm53xx generic dlink_dir-885l

Results will be under build/openwrt-imagebuilder-${release}-${architecture}-${variant}.Linux-x86_64/bin/.

To see a list of available targets, run make info in the ImageBuilder dir.

If you want to change which OpenWrt version is used, then try editing the relevant variable(s) in build.sh. It's not guaranteed to work across OpenWrt releases, therefore we keep git branches for the past releases.

Setup stages

Blinking leds show which phase the extroot setup scripts are in. Consult the sources for details: autoprovision-functions.sh.

Stage 1: setup extroot

When the custom firmware first boots, the autoprovision script will wait for anything (!) in /dev/sda to show up (that is >= 512M), then erase it and set up a swap, an extroot, and a datafilesystem (for the remaining space), and then reboot.

Stage 2: download and install some packages from the internet

Once it rebooted into the new extroot, it will continuously keep trying to install some OpenWrt packages until an internet connection is set up on the router. You need to do that manually either by using ssh or the web UI (LuCI).

Stage 3, optional

We also have a 3rd stage, written in Python, but it's commented out here. Search for autoprovision-stage3.py to see how it's done.

Login

After flashing the firmware the router will have the standard 192.168.1.1 IP address.

By default the root passwd is not set, so the router will start telnet with no password. If you want to set up a password, then edit the stage 2 script: autoprovision-stage2.sh.

If a password is set, then telnet is disabled by OpenWrt and SSH will listen using the keys specified in authorized_keys.

Once connected, you can read the log with logread -f.

Status

This is more of a template than something standalone, but I use it for my home routers as is. For more specific applications you most probably want to customize this script here and there; search for CUSTOMIZE for places of interest.

Most importantly, set up a password and maybe add your ssh key by adding it to image-extras/common/etc/dropbear/authorized_keys.

None of this script is hardware specific except setLedAttribute, which is used to provide feedback about the progress of the initial setup phase. At the time of writing it only works on a few routers (mostly ath79 ones), but without this everything should work fine, if only a bit less convenient.

Troubleshooting

Which file should I flash?

You should consult the OpenWrt documentation. The produced firmware files should be somewhere around ./build/openwrt-imagebuilder-21.02.0-ath79-generic.Linux-x86_64/bin/targets/ath79/generic/.

In short:

• You need a file with the name -factory.bin or -sysupgrade.bin. The former is to be used when you first install OpenWrt, the latter is when you upgrade an already installed OpenWrt.

• You must carefully pick the proper firmware file for your hardware version! I advise you to look up the wiki page for your hardware on the OpenWrt wiki, because most of them have a table of the released hardware versions with comments on their status (sometimes new hardware revisions are only supported by the latest OpenWrt, which is not released yet).

Help! The build has finished but there's no firmware file!

If the build doesn't yield a firmware file (*-factory.bin and/or *-sysupgrade.bin): when there's not enough space in the flash memory of the target device to install everything then the OpenWrt ImageBuilder prints a hardly visible error into its flow of output and silently continues. Look into build.sh and try to remove some packages that you can live without.

Extroot is not mounted after a sysupgrade

In short, this is an OpenWrt issue, and the solution is to mount the extroot somewhere, and delete /etc/.extroot-uuid. More details are available in this issue, and a way to deal with it can be found in this blog post. You may also want to check out the official OpenWrt wiki on this topic.

openwrt-auto-extroot's Issues

This could be added to the official OpenWrt project github, couldn't it?

It's an very useful addon!!
I'd ask for it.
Thanks!

pause and wait for the press of a button before formatting /dev/sda

it would be nice to get a confirmation before formatting the plugged in storage device, but there aren't many convenient ways in this context to interact with the user...

Suppport for WD My Net n750 LEDs

I have OpenWRT already running on a WD My Net n750 but it is an older build, 19.07.9. Due to a issue with newer builds and a possible solution of using extroot to get around it. I would like to use this to build a extroot enabled OpenWRT image.

I know the LEDs are named differently for this model but I'm not sure where to get the proper names. The only names I see so far are:

ls /sys/class/leds/
ath9k-phy0                ath9k-phy1                mynet-n750:blue:internet  mynet-n750:blue:power     mynet-n750:blue:wireless  mynet-n750:blue:wps

Does that mean I would insert something like the following into the autoprovision-functions.sh file?

    *mynet-n750*)
        autoprovisionUSBLed="blue:wps"
        autoprovisionStatusLed="blue:wireless"
        ;;

Does not boot from USB after sysupgrade

Hey,

I have created an image with your script for the TL-WDR4300.

I have used the -sysupgrade.img to maintain my previous configuration.
After rebooting, I had the USB stick mounted and the system has been upgraded.
The problem is that / is not mounted to the USB stick.

Since I could not find any other possibility to contact you, I am trying this, even if this is not the proper way to ask such questions :)

./build.sh: line 86: pushd: bin/ar71xx/: No such file or directory

Tried with lede-imagebuilder-17.01.2 and 01,same result all time.
TP-Link TL-MR3420 v2
these packages disabled
#PREINSTALLED_PACKAGES+=" ppp ppp-mod-pppoe ppp-mod-pppol2tp ppp-mod-pptp kmod-ppp kmod-pppoe"
#PREINSTALLED_PACKAGES+=" luci"

TL MR3040 V2

Hi!
In
$make info | grep 3040
TLMR3040:
TP-LINK TL-MR3040
i found only TL-MR3040.
Is it means that TL-MR3040 V2 is not supported by this script?

UPDATE:
After successful compilation there a 2 versions of bins for V1 and V2 hardware separately
Nice!!!
Thx

port it over to https://lede-project.org/

back up master branch in a branch called 'openwrt', and replace master with the 'lede' branch.

the lede branch was reported to work by one user at around the end of 2016.

rename the project to lede-auto-extroot.

How to reset / firstboot / jffs2reset?

I've tried the methods on the Failsafe Mode, Factory Reset, and Recovery Mode page without luck.

Both the soft reset firstboot && reboot now and the hard reset umount /overlay && jffs2reset && reboot now don't seem to have any effect.

How can I reset the extroot to factory / defaults?

Update build script to reflect openwrt-lede remerge

I have a small change to the build.sh script to update it to reflect the openwrt-lede remerge, and also to update the default version number to the current one.

--- a/build.sh
+++ b/build.sh
@@ -26,7 +26,7 @@ BUILD=`absolutize $BUILD`
 ### chose a release
 ###
 #RELEASE="15.05.1"
-RELEASE="17.01.1"
+RELEASE="17.01.4"
 
 IMGBUILDER_NAME="lede-imagebuilder-${RELEASE}-${TARGET_ARCHITECTURE}-${TARGET_VARIANT}.Linux-x86_64"
 IMGBUILDER_DIR="${BUILD}/${IMGBUILDER_NAME}"
@@ -35,7 +35,7 @@ IMGBUILDER_ARCHIVE="${IMGBUILDER_NAME}.tar.xz"
 IMGTEMPDIR="${BUILD}/openwrt-build-image-extras"
 #https://downloads.lede-project.org/snapshots/targets/ar71xx/generic/lede-imagebuilder-ar71xx-generic.Linux-x86_64.tar.xz
 #https://downloads.lede-project.org/snapshots/targets/ar71xx/generic/lede-imagebuilder-ar71xx-generic.Linux-x86_64.tar.xz
-IMGBUILDERURL="https://downloads.lede-project.org/releases/${RELEASE}/targets/${TARGET_ARCHITECTURE}/${TARGET_VARIANT}/${IMGBUILDER_ARCHIVE}"
+IMGBUILDERURL="https://downloads.openwrt.org/releases/${RELEASE}/targets/${TARGET_ARCHITECTURE}/${TARGET_VARIANT}/${IMGBUILDER_ARCHIVE}"
 
 if [ -z ${TARGET_DEVICE} ]; then
     echo "Usage: $0 architecture variant device-profile"

how to change to disk layout ?

thanks for this wonderful application.
I use this to build my own 19.07.3 openwrt image , it's perfect !

now it's 512M in / , and remain space goes to /mnt/data
like I want more space in root , like 8G or more
can I just modify autoprovision-stage1.sh to make my own disk layout
or is there any other script should bt modified ?

Is this project to be used on linux machines only?

hi!

i have os x and i tried to build an openwrt image for tp-link mr3020 and everything went well (minor modifications of mkdir -pv and from make to gmake in build.sh file) until I got:

        fi
/Users/sindhus/openwrt-auto-extroot/build/OpenWrt-ImageBuilder-15.05-ar71xx-generic.Linux-x86_64/staging_dir/host/bin/find: line 4: /Users/sindhus/openwrt-auto-extroot/build/OpenWrt-ImageBuilder-15.05-ar71xx-generic.Linux-x86_64/staging_dir/host/bin/bundled/lib/ld-linux-x86-64.so.2: cannot execute binary file
/Users/sindhus/openwrt-auto-extroot/build/OpenWrt-ImageBuilder-15.05-ar71xx-generic.Linux-x86_64/staging_dir/host/bin/find: line 4: /Users/sindhus/openwrt-auto-extroot/build/OpenWrt-ImageBuilder-15.05-ar71xx-generic.Linux-x86_64/staging_dir/host/bin/bundled/lib/ld-linux-x86-64.so.2: Undefined error: 0
/Users/sindhus/openwrt-auto-extroot/build/OpenWrt-ImageBuilder-15.05-ar71xx-generic.Linux-x86_64/staging_dir/host/bin/opkg: line 4: /Users/sindhus/openwrt-auto-extroot/build/OpenWrt-ImageBuilder-15.05-ar71xx-generic.Linux-x86_64/staging_dir/host/bin/bundled/lib/ld-linux-x86-64.so.2: cannot execute binary file
/Users/sindhus/openwrt-auto-extroot/build/OpenWrt-ImageBuilder-15.05-ar71xx-generic.Linux-x86_64/staging_dir/host/bin/opkg: line 4: /Users/sindhus/openwrt-auto-extroot/build/OpenWrt-ImageBuilder-15.05-ar71xx-generic.Linux-x86_64/staging_dir/host/bin/bundled/lib/ld-linux-x86-64.so.2: Undefined error: 0
/Applications/Xcode.app/Contents/Developer/usr/bin/make package_install

Installing packages...
IPKG_NO_SCRIPT=1 IPKG_TMP="/Users/sindhus/openwrt-auto-extroot/build/OpenWrt-ImageBuilder-15.05-ar71xx-generic.Linux-x86_64/tmp/ipkgtmp" IPKG_INSTROOT="/Users/sindhus/openwrt-auto-extroot/build/OpenWrt-ImageBuilder-15.05-ar71xx-generic.Linux-x86_64/build_dir/target-mips_34kc_uClibc-0.9.33.2/root-ar71xx" IPKG_CONF_DIR="/Users/sindhus/openwrt-auto-extroot/build/OpenWrt-ImageBuilder-15.05-ar71xx-generic.Linux-x86_64/tmp" IPKG_OFFLINE_ROOT="/Users/sindhus/openwrt-auto-extroot/build/OpenWrt-ImageBuilder-15.05-ar71xx-generic.Linux-x86_64/build_dir/target-mips_34kc_uClibc-0.9.33.2/root-ar71xx" /Users/sindhus/openwrt-auto-extroot/build/OpenWrt-ImageBuilder-15.05-ar71xx-generic.Linux-x86_64/staging_dir/host/bin/opkg -f /Users/sindhus/openwrt-auto-extroot/build/OpenWrt-ImageBuilder-15.05x" /Users/sindhus/openwrt-auto-extroot/build/OpenWrt-ImageBuilder-15.05-ar71xx-generic.Linux-x86_64/staging_dir/host/bin/opkg -f /Users/sindhus/openwrt-auto-extroot/build/OpenWrt-ImageBuilder-15.05-ar71xx-generic.Linux-x86_64/repositories.conf --force-depends --force-overwrite --force-postinstall --cache /Users/sindhus/openwrt-auto-extroot/build/OpenWrt-ImageBuilder-15.05-ar71xx-generic.Linux-x86_64/dl --offline-root /Users/sindhus/openwrt-auto-extroot/build/OpenWrt-ImageBuilder-15.05-ar71xx-generic.Linux-x86_64/build_dir/target-mips_34kc_uClibc-0.9.33.2/root-ar71xx --add-dest root:/ --add-arch all:100 --add-arch ar71xx:200 install /Users/sindhus/openwrt-auto-extroot/build/OpenWrt-ImageBuilder-15.05-ar71xx-generic.Linux-x86_64/packages/base/libc_0.9.33.2-1_ar71xx.ipk
/Users/sindhus/openwrt-auto-extroot/build/OpenWrt-ImageBuilder-15.05-ar71xx-generic.Linux-x86_64/staging_dir/host/bin/opkg: line 4: /Users/sindhus/openwrt-auto-extroot/build/OpenWrt-ImageBuilder-15.05-ar71xx-generic.Linux-x86_64/staging_dir/host/bin/bundled/lib/ld-linux-x86-64.so.2: cannot execute binary file
/Users/sindhus/openwrt-auto-extroot/build/OpenWrt-ImageBuilder-15.05-ar71xx-generic.Linux-x86_64/staging_dir/host/bin/opkg: line 4: /Users/sindhus/openwrt-auto-extroot/build/OpenWrt-ImageBuilder-15.05-ar71xx-generic.Linux-x86_64/staging_dir/host/bin/bundled/lib/ld-linux-x86-64.so.2: Undefined error: 0
make[2]: *** [package_install] Error 1
make[1]: *** [_call_image] Error 2
make: *** [image] Error 2

am guessing the Makefile in the build dir is trying to execute the binaries of openwrt?

How do I use this project on OS X?

Thanks!

URL doesn't work for some targets, e.g. Dlink-DIR885l

converting PR #15 into this issue, so that i can close it.

i opened an OpenWRT feature request that hopes to resolve this upstream: openwrt/openwrt#6618

USB formats on reboot

Hi,

I've created image for Asus RT-AC88U. Everything worked until one random reboot and USB that I use for extroot got formatted again. And it happened again randomly after poweroff/reboot. Not sure why this happens. I can provide more information if you direct me to what you need.

OpenWrt 23.05.2

tnx

Support for 18.06.0?

I cloned the repository, edited the build.sh to have RELEASE="18.06.0" and then run:

./build.sh ar71xx generic tl-wdr3600-v1

But I get this error:

./build.sh: line 84: pushd: /home/user/openwrt-auto-extroot-2018-07/build/lede-imagebuilder-18.06.0-ar71xx-generic.Linux-x86_64: No such file or directory

Does openwrt-auto-extroot work with 18.06.0 ?

Not able to generate image even with minimum packages

PREINSTALLED_PACKAGES+=" blkid mount-utils swap-utils e2fsprogs fdisk" add to the size of the image and leave very little space so one can have functionality (like USB Storage PREINSTALLED_PACKAGES="block-mount kmod-usb-storage kmod-fs-exfat").

I commented out:

#PREINSTALLED_PACKAGES+=" kmod-usb-uhci kmod-usb-ohci"

# the following packages are optional, feel free to (un)comment them
#PREINSTALLED_PACKAGES+=" wireless-tools firewall iptables"
#PREINSTALLED_PACKAGES+=" kmod-usb-storage-extras kmod-mmc"
#PREINSTALLED_PACKAGES+=" ppp ppp-mod-pppoe ppp-mod-pppol2tp ppp-mod-pptp kmod-ppp kmod-pppoe"
#PREINSTALLED_PACKAGES+=" luci"

and yet I get errors:

[mktplinkfw] *** error: images are too big

and of course no .bin images are generated as a result.

Because I had the same issue of NAND memory running out on router trying to install USB support manually (opkg install), I figured your project would help. Of course, am still stuck in the same problem as before.

How do I go about this?

tl-mr3020 support

is this version supported... already tried make info and it wasnt listed, yet in your project i can see a folder with its name

Device not mounted after sysupgrade (UUID mismatch)

After upgrading from 17.01.1 to 17.01.4 (extroot to extroot), there was an issue: The overlay was not properly mounted.

Kernel will log something like [ 11.680000] block: extroot: UUID mismatch (root: 57a0acc6-ecc0a5e7-2ea71070-845539a7, overlay: d578bcbf-510a3603-823f14c1-4e16d34e)

This takes a look at what might be the underlying issue - I ended fixing the issue by mounting the USB-stick and deleting /etc/.extroot-uuid as explained here.

Perhaps there is a way this could be automated in the future...

Another issue I ran into after the sysupgrade was, that I couldn't install any packages, as they were expecting a newer kernel. I could only solve this issue by manually wgeting the current kernel-ipkg and installing it. Perhaps also something that should be considered by the provisioning-script when performing a sysupgrade...

Support TP-Link TL-WDR3600 v1

I have a couple of TP-Link TL-WDR3600 v1 routers. I'm posting here because of a strange quirk I noticed and I'm not sure how to debug for. Hoping someone here has an idea what could be up.

When trying the build generated from this project it seems to work but requires a manual power cycle after the USB storage is setup.

What I've observed is that the router will turn on and with the below LED code flash the appropriate LEDs showing that the script is setting up along with the USB thumb drive flashing itself. After seeing the router reboot the first time both the router and USB will flash LEDs more signifying that it's installing packages, then a second reboot happens. It's at the second reboot where it seems to hang. The USB LED will not come on but the LEDs for router power and connected Ethernet ports do come on. It seems to get into a odd state where internet is "passing through" like a unmanaged switch but it doesn't bother with the USB or respond to SSH or WebUI requests, it's not accessible over 192.168.1.1.

If I manually cycle power at this state the router reboots normally and everything appears to work as if nothing is wrong. I can at this point even run a reboot command from he WebUI LUCI.

Any thoughts what might be happening? I'm hazarding a guess that after setup and manually power cycling once that it is good to go.

LED code

    *tl-wdr3600*)
        autoprovisionUSBLed="green:wlan2g"
        autoprovisionStatusLed="green:wlan5g"
        ;;