ubports / ubports-pdk

Platform Development Kit for Ubuntu Touch

Platform Development Kit

A way to help the development of Ubuntu Touch. Runs an Ubuntu Touch emulator that has little pre-installed, yet (no indicators or no core apps so far). It allows you to build, install and test system components.

Goals:

• Make debugging platform components easy
• Don't worry about porting
• Lower barrier to entry for contributors

Installation:

• Linux with Snap:
  • sudo snap install --beta ubports-pdk
  • sudo snap connect ubports-pdk:kvm
  • sudo snap connect ubports-pdk:network-control
• macOS & other Linux: Clone this repository or download a copy of it from GitHub

On Linux, make sure that your user account is in the kvm group:

• Ubuntu 18.04 and earlier:
  • sudo groupadd kvm
  • sudo usermod -aG kvm $USER
• Ubuntu 20.04 and up:
  • sudo usermod -aG kvm $USER

Usage:

For the snap version, use the command ubports-pdk.pdk instead of ubuntu-touch-pdk.

• ubuntu-touch-pdk setup: Sets up the environment and mounting capabilities for your development needs
• ubuntu-touch-pdk pull: Download the freshest development image (based on focal)
• ubuntu-touch-pdk run: Run the development VM instance

Changes within a VM are persistent until you run the pull command again. You can have separate VMs by adding --name=my_custom_vm to the pull and run commands.

For more options, please run the command with the -h flag.

Tutorial: Fetching package sources

When logged into the VM instance (user: root, password: root) you're able to clone UBports repositories using ubports-clone. Let's try that with indicator-network.

root@linux:~# ubports-clone indicator-network

This will download the sources from Git and make them available in your PDK workspace (configured during the ubuntu-touch-pdk setup step).

Within the VM instance you're able to access those files in /pdk/sources.

Tutorial: Building package sources

Still logged into the VM instance, you're able to trigger a build of your cloned sources using ubports-build, ie for indicator-network:

root@linux:~# ubports-build indicator-network

This will copy your sources over to the VM, build the source code and publish Debian packages easily accessible from within your PDK workspace. For indicator-network those files will be located in $PDK_WORKSPACE/sources/indicator-network

The same Debian packages generated by the ubports-build command can be installed into the VM, like: sudo apt install /pdk/sources/indicator-network/*.deb

Troubleshooting

Running on distributions other than Ubuntu is possible, but not guaranteed yet.

Reset PDK

To reset the PDK completely, remove the environment directory you chose on setup as well as your configuration file, which is located in one of the following:

• Linux with Snap: $SNAP_USER_COMMON/config.sh (e.g. $HOME/snap/ubports-pdk/common/config.sh)
• Other Linux: $HOME/.config/UbuntuTouchPdk/config.sh
• Mac: $HOME/Library/Caches/UbuntuTouchPdk/config.sh

Nvidia GPU Support Note

There are currently known issues with NVIDIA GPUs. If the QEMU window remains black after starting the PDK and logging into the VM instance, this may be the case.

In order to check which video card you are using, you can run the following command on host side:

glxinfo|egrep "OpenGL vendor|OpenGL renderer"

And if a discrete Nvidia graphics card is used, and there is onboard Intel and PRIME is supported, executing sudo prime-select intel to switch and then rebooting may help.

Technical details:

• Preconfigured QEMU with OpenGL support
• Focus on Linux and macOS as host systems first
• Intel images on Intel hardware, ARM64 on ARM64 hardware (Mac M1)
• 20.04 off-the-shelf kernel
• Rootfs builds taken straight from Debos
• Utilities that wrap around Git and make it easier to pull sources
• Utilities around building and testing UT components

ubports-pdk's Issues

ubports-pdk.pdk run yields "qemu-system-x86_64: OpenGL is not supported by the display"

This is on a not-so-old Lenovo X230T Notebook. glxgears runs fine, so OpenGL obviously is supported by my display. Attaching output of glxinfo.
What am I doing wrong?
glxinfo.txt

set different number of trying when pulling

I have very low bandwidth stability.
Unfortunately when I did the pull I reached the twentieth attempt when I was at 96%, therefore it left the process when it was one step away from the end.
is there any way to change the maximum number of pull attempts?
is there a way to clone and store locally the development image as backup version so to avoid to have to pull again the image?

Used the snap version 0.0.0-rc0 2021-07-18 (143)

Change license from 'none' to 'GPLv3'

We didn't have a license set up for this project before, so to make it easy and understandable what people are dealing with,
let's make this a GPLv3 licensed project.

Tagging existing contributors to ask for agreement to change the license to GPLv3:

• @just-carlod

Please agree or disagree to change the license below in a comment.

Mounting PDK source dir fails for snap

Only for the snap version, mounting the PDK source dir fails.

# journalctl -u ubports-pdk-mount.service
-- Logs begin at Sun 2021-07-11 09:23:36 UTC, end at Sun 2021-07-11 09:27:16 UTC. --
Jul 11 09:23:39 linux systemd[1]: Starting Mount PDK host directory...
Jul 11 09:23:39 linux bash[512]: mount: /pdk/settings: WARNING: device write-protected, mounted read-only.
Jul 11 09:23:39 linux bash[505]: /pdk/settings still mounted
Jul 11 09:23:39 linux bash[520]: read: Connection reset by peer
Jul 11 09:23:39 linux systemd[1]: ubports-pdk-mount.service: Main process exited, code=exited, status=1/FAILURE
Jul 11 09:23:39 linux systemd[1]: ubports-pdk-mount.service: Failed with result 'exit-code'.
Jul 11 09:23:39 linux systemd[1]: Failed to start Mount PDK host directory.

Sources pulled as described in the README do not end up in the PDK environment directory therefore.

ubports-clone fails on snap rev 157, because of missing "curl" package

You need to do an apt update and apt install curl to make ubports-clone work.

can't set up in non-Snap Linux environment

cobalt@ideapad-flex-5:~/ubports-pdk$ ./ubuntu-touch-pdk setup
#####################################
Ubuntu Touch Platform Development Kit
#####################################

Executing tasks:
	Setup? yes
	Clear cache? no
	List? no
	Pull image? no
	Run an image? no

WARNING: You havent set up your environment yet. Continuing with setup...
Installing prerequisites
sudo: snap: command not found

XDG_CURRENT_DESKTOP not set

XDG_CURRENT_DESKTOP should be set to "Lomiri". Since Ayatana Indicators are desktop-agnostic, we need to know what we're running on.

Even without Ayatana, it would be a good idea to have this pre-set in the image.

Use other partition

hi,
would like to request to add support that allow us to boot the img /data from the other partition

yes there's an option where can we put the data , but when trying to add it , it said that cannot create a directory (Permission denied)

Disk size update.

How can ı update my disk size. when ı creare mydisk.hdd it's creating 15GB outomaticaly. ı want to add more size.

Package as DEB for Debian?

I wonder if it would make sense to have the UBports PDK available as a DEB package in Debian testing/unstable (possibly with an RC bug blocker to prohibit releasing the DEB to Debian stable).

No sshd/settings.raw

I'm not sure if I miss some steps. I did:
setup (on ../qemu)
pull
reboot
run

but I get an error saying I have no settings.raw (which is true)

#####################################
Ubuntu Touch Platform Development Kit
#####################################

Executing tasks:
	Setup? no
	Clear cache? no
	List? no
	Pull image? no
	Create custom image? no
	Run an image? yes


Name of the environment: default

ERROR: ld.so: object 'libgtk3-nocsd.so.0' from LD_PRELOAD cannot be preloaded (failed to map segment from shared object): ignored.
qemu-system-x86_64: -drive if=virtio,format=raw,file=../qemu/sshd/settings.raw,readonly=on: Could not open '../qemu/sshd/settings.raw': No such file or directory

if I create it will complain saying:
[...]ubports-pdk/scripts/images.sh: línia 74: [...]/ubports-pdk/deps/rootfs-builder-debos/debos-docker: file or folder doesn't exist`

Is it supposed to be set up under the main ubports-pdk directory and not outside?

Add check if user is in kvm group

The pull command shouldn't work if the user isn't in the kvm group. The priority is low because I run it outside of snap.

nanu@Maulwurf:/media/nanu/home2/src/ut/ubports-pdk$ ./ubuntu-touch-pdk pull
#####################################
Ubuntu Touch Platform Development Kit
#####################################

Executing tasks:
	Setup? no
	Clear cache? no
	List? no
	Pull image? yes
	Run an image? no

--2021-07-11 20:56:55--  https://ci.ubports.com/job/Platform%20Development%20Kit/job/pdk-vm-image-amd64/lastSuccessfulBuild/artifact/ubuntu-touch-pdk-amd64.raw.xz
Auflösen des Hostnamens ci.ubports.com (ci.ubports.com) … 172.67.150.210, 104.21.0.97, 2606:4700:3033::ac43:96d2, ...
Verbindungsaufbau zu ci.ubports.com (ci.ubports.com)|172.67.150.210|:443 … verbunden.
HTTP-Anforderung gesendet, auf Antwort wird gewartet … 200 OK
Länge: 718382184 (685M) [application/x-xz]
Wird in ‘/media/nanu/home2/src/ut/pdk/pdk-image-cache/default/ubuntu-touch-pdk-amd64.raw.xz’ gespeichert.

ubuntu-touch-pdk-amd64.raw.xz                       100%[=================================================================================================================>] 685,10M  15,4MB/s    in 45s     

2021-07-11 20:57:40 (15,4 MB/s) - ‘/media/nanu/home2/src/ut/pdk/pdk-image-cache/default/ubuntu-touch-pdk-amd64.raw.xz’ gespeichert [718382184/718382184]

Unpacking the archive
Pull successful!
nanu@Maulwurf:/media/nanu/home2/src/ut/ubports-pdk$ ./ubuntu-touch-pdk run
#####################################
Ubuntu Touch Platform Development Kit
#####################################

Executing tasks:
	Setup? no
	Clear cache? no
	List? no
	Pull image? no
	Run an image? yes


Name of the environment: default

qemu-system-x86_64: warning: dbind: Couldn't connect to accessibility bus: Failed to connect to socket /tmp/dbus-WrX2pMLhwJ: No such file or directory

Please migrate this repository over to gitlab.com/ubports

@fredldotme Do you see any blocker for migrating this repo over to gitlab.com? If not, please ping me and we'll see to its migration (including all open PRs and issues).

ubuntu-touch-pdk not in PATH after snap install

I installed the PDK via sudo snap install --edge ubports-pdk. ubuntu-touch-pdk is not in PATH though. Interestingly there is ubuntu-touch-pdk in PATH (from /snap/bin/ubports-pdk.pdk). Running that seems to do the same as running the tool with from its absolute path /snap/ubports-pdk/126/bin/ubuntu-touch-pdk.

OS: Ubuntu 20.04

Unable to sign-in on linux non-snap host

I see that the login credentials given in the README are root root but it just says that they're incorrect and doesn't allow me to login

Pull image fails

On first run the pull command fails with

...
SSH enabled successfully!
grep: /home/jonatan/.ssh/authorized_keys: Datei oder Verzeichnis nicht gefunden
Inserting ssh key into authorized keys list
Generating image with settings
100+0 Datensätze ein
100+0 Datensätze aus
104857600 Bytes (105 MB, 100 MiB) kopiert, 0,379751 s, 276 MB/s
mkfs.fat 4.1 (2017-01-24)
Copying settings to image
Using mtools to create a settings image
mkdir: das Verzeichnis »“ kann nicht angelegt werden: Datei oder Verzeichnis nicht gefunden

On second run the issue does not occur anymore.

PDK Snap package

Tracking progress in the snap-confined branch.

Road to greatness:

• Confined snap packaging
• Shipping all necessary dependencies
• Allowing shared directory mounting
• Launching QEMU

SDL display doesn't work on amd GPUs

SDL display method doesn't work on my machine with ryzen 4500u and vega 6 iGPU. From my researches, this error may appear on more amd GPUs.

Fails to run on Solus, Unknown package manager, GLIBC_2.33 not found

git clone https://github.com/ubports/ubports-pdk

./ubuntu-touch-pdk setup

WARNING: You haven't set up your environment yet. Continuing with setup...
Installing prerequisites
snap "qemu-ut-pdk" is already installed, see 'snap help refresh'
KVM group all set up, good.
Please create and enter the directory path you want to set up.
This directory will contain your VM images and source code.
Path: ubports
WARNING: The OpenSSH server seems to be missing or not activated, please install it using your package manager.
Would you like to do that automatically now [y/n]?
Unknown package manager used, please add support for it in UBports PDK. <- Solus uses eopkg
Inserting ssh key into authorized keys list
Generating image with settings
100+0 records in
100+0 records out
104857600 bytes (105 MB, 100 MiB) copied, 0.838064 s, 125 MB/s
mkfs.fat 4.1 (2017-01-24)
Copying settings to image
Using mtools to create a settings image

./ubuntu-touch-pdk pull

Pull successful!

./ubuntu-touch-pdk run

Name of the environment: default

WARNING: cgroup v2 is not fully supported yet, proceeding with partial confinement
qemu-system-x86_64: warning: dbind: Couldn't connect to accessibility bus: Failed to connect to socket /tmp/dbus-Z0DWnohoda: No such file or directory
Couldn't open libEGL.so.1: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /var/lib/snapd/lib/gl/libEGL.so.1)

OpenSSH server and Glibc_2.33 is already installed

Failing to run with recent qemu version (6.2.0)

When running ubuntu-touch-pdk run, it fails to actually run.

$> -device virtio-vga,virgl=on: Property 'virtio-vga.virgl' not found

My qemu version is 6.2.0.

$> qemu-system-x86_64 --version 
QEMU emulator version 6.2.0 (qemu-6.2.0-2.fc36)
Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers

According a blog post, the parameters have changed -> https://www.kraxel.org/blog/2021/05/virtio-gpu-qemu-graphics-update/

Below patch fixes running on my host.
However, this will break older qemu versions.
We should implement a qemu version check.

From 44fce280d5c5a4807639697b86eeb5a1eb900930 Mon Sep 17 00:00:00 2001
From: Alexander Martinz <[email protected]>
Date: Wed, 19 Jan 2022 15:20:48 +0100
Subject: [PATCH] scripts: images: use new virgl=on replacement

 $> -device virtio-vga,virgl=on: Property 'virtio-vga.virgl' not found

See: https://www.kraxel.org/blog/2021/05/virtio-gpu-qemu-graphics-update/

Change-Id: I4a6fd542d7eb8b74a1058894856cdf806a02ba25
Signed-off-by: Alexander Martinz <[email protected]>
---
 scripts/images.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/images.sh b/scripts/images.sh
index b43fd40..3b88272 100644
--- a/scripts/images.sh
+++ b/scripts/images.sh
@@ -29,9 +29,9 @@ function initImageVars {
         fi
 
         if [ "$HOST_ARCH" == "$ARCH" ]; then
-            QEMU_ARGS="-enable-kvm -device virtio-vga,virgl=on -display gtk,gl=on $QEMU_ARGS"
+            QEMU_ARGS="-enable-kvm -device virtio-vga-gl -display gtk,gl=on $QEMU_ARGS"
         else
-            QEMU_ARGS="-machine virt -device virtio-gpu-pci,virgl=on -display gtk,gl=on $QEMU_ARGS"
+            QEMU_ARGS="-machine virt -device virtio-gpu-gl-pci -display gtk,gl=on $QEMU_ARGS"
         fi
         QEMU_ARGS="-device virtio-keyboard-pci -device virtio-mouse-pci $QEMU_ARGS"
     elif [ "$(uname -s)" == "Darwin" ]; then
-- 
2.35.0.rc1

Provide more Lomiri components by default

I wonder if we should not provide more packages installed by default in the PDK (e.g. lomiri-url-dispatcher, indicators, lomiri-system-settings, etc.).

What do others think?

Mouse not working with snap version on manjaro linux

When I run the pdk, the login screen is shown, together with mouse cursor, but the cursor won't move.
Keyboard works, can login using that and run terminal.

Note: I have a touchscreen and nor touch is working in the virtual environment.

Build tutorial from README fails due to dependency errors

The build tutorial from the README fails due to dependency errors:

# ubports-build indicator-network
...
dh_builddeb
dpkg-deb: building package 'lomiri-indicator-network-build-deps' in '../lomiri-indicator-network-build-deps_1.0.0_amd64.deb'.

The package has been created.
Attention, the package has been created in the current directory,
not in ".." as indicated by the message above!
Selecting previously unselected package lomiri-indicator-network-build-deps.
(Reading database ... 68188 files and directories currently installed.)
Preparing to unpack lomiri-indicator-network-build-deps_1.0.0_amd64.deb ...
Unpacking lomiri-indicator-network-build-deps (1.0.0) ...
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Correcting dependencies...Starting pkgProblemResolver with broken count: 1
Starting 2 pkgProblemResolver with broken count: 1
Investigating (0) libqt5quick5-gles:amd64 < 5.12.8-0ubuntu1 @ii mK Ib >
Broken libqt5quick5-gles:amd64 Conflicts on libqt5quick5:amd64 < none -> 5.12.8-0ubuntu1 @un uN >
  Considering libqt5quick5:amd64 67 as a solution to libqt5quick5-gles:amd64 68
  Added libqt5quick5:amd64 to the remove list
  Fixing libqt5quick5-gles:amd64 via keep of libqt5quick5:amd64
Investigating (0) qtdeclarative5-dev:amd64 < none -> 5.12.8-0ubuntu1 @un uN Ib >
Broken qtdeclarative5-dev:amd64 Depends on libqt5quick5:amd64 < none | 5.12.8-0ubuntu1 @un uH > (= 5.12.8-0ubuntu1)
  Considering libqt5quick5:amd64 67 as a solution to qtdeclarative5-dev:amd64 0
  Holding Back qtdeclarative5-dev:amd64 rather than change libqt5quick5:amd64
Investigating (0) lomiri-indicator-network-build-deps:amd64 < 1.0.0 @iU mK Nb Ib >
Broken lomiri-indicator-network-build-deps:amd64 Depends on qtdeclarative5-dev:amd64 < none | 5.12.8-0ubuntu1 @un uH >
  Considering qtdeclarative5-dev:amd64 0 as a solution to lomiri-indicator-network-build-deps:amd64 -2
  Removing lomiri-indicator-network-build-deps:amd64 rather than change qtdeclarative5-dev:amd64
Done
 Done
Starting pkgProblemResolver with broken count: 0
Starting 2 pkgProblemResolver with broken count: 0
Done
The following packages will be REMOVED:
  lomiri-indicator-network-build-deps
0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded.
1 not fully installed or removed.
After this operation, 10.2 kB disk space will be freed.
(Reading database ... 68192 files and directories currently installed.)
Removing lomiri-indicator-network-build-deps (1.0.0) ...
mk-build-deps: Unable to install lomiri-indicator-network-build-deps at /usr/bin/mk-build-deps line 457.
mk-build-deps: Unable to install all build-dep packages

Ability to increase partition size

From https://t.me/ubports_devel/70273 when someone asked about getting a bigger rootfs (assumes linux env and not using the snap):

disk_img=~/.config/UbuntuTouchPdk/data/pdk-image-cache/default/hdd.raw
root_par_start_sector=976563
root_par_offset=$(($root_par_start_sector * 512))

dd if=/dev/zero bs=1G count=15 >> $disk_img
fdisk $disk_img <<EOF
p
d
3
n
p
3
$root_par_start_sector

n
p
w
EOF

loop_dev=$(sudo losetup --offset=$root_par_offset --find --show $disk_img)
sudo e2fsck -fy $loop_dev
sudo resize2fs $loop_dev
sudo losetup --detach $loop_dev

about the "magic numbers" 976563 is the Start sector of the 13.5G ext4 root partition as shown by fdisk -l <path to hdd.raw>, and 500000256 (root_par_offset) is the start sector * sector size (512 bytes), so the partition start offset in bytes

Not sure if this is super useful since not many (if any) people have needed it so far, but gonna leave this here just in case it might be.

Setup failed. run hook "configure": cannot query current apparmor profile: Invalid argument

Operating System: openSUSE Tumbleweed 20210806
Kernel Version: 5.3.18-57-default (64-bit)
AppArmor Version: 3.0.1-9.1

sudo snap install --beta ubports-pdk
sudo snap connect ubports-pdk:kvm
sudo snap connect ubports-pdk:network-control
sudo usermod -aG kvm $USER
ubports-pdk.pdk setup

results in

cannot query current apparmor profile: Invalid argument

and

git clone https://github.com/ubports/ubports-pdk
cd ubports-pdk
./ubuntu-touch-pdk setup

results in

#####################################
Ubuntu Touch Platform Development Kit
#####################################

Executing tasks:
        Setup? yes
        Clear cache? no
        List? no
        Pull image? no
        Run an image? no
which: no mtools in (/usr/local/bin:/usr/bin:/bin:/snap/bin)

WARNING: You haven't set up your environment yet. Continuing with setup...
Installing prerequisites
[sudo] password for root: 
error: cannot perform the following tasks:
- Run configure hook of "qemu-ut-pdk" snap if present (run hook "configure": cannot query current apparmor profile: Invalid argument)

Question:
Is there any option for a more verbose output or the source of the qemu-ut-pdk snap?

Eliminate unsupported CPU features warning when running on AMD CPU

When I run PDK unpatched on my AMD laptop, the code forces -cpu Haswell-v4 upon me, which causes a bunch of qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.<varies> on the screen before we start.

I was going to propose changing that to -cpu host, but then I realized that PDK supports running foreign architecture, in which case -cpu host won't work. What's the best way to solve that here? Using -cpu host when [ $HOST_ARCH = $ARCH ]?

Can't find gitlab's development packages when using ubports-clone command

Tried to

# ubports-clone nuntium

and it didn't find the packages on gitlab.

That's because it didn't search in https://gitlab.com/ubports/development/core/, but in https://gitlab.com/ubports/core/

Issue on Mac Os with M1 (Monterey 12.6)

everything works except running XD

please see below

Detect if no vm exists

The error should be explaining that pull has to be run and ask if it should do it now :)

Name of the environment: default

Cache directory for image 'default' doesn't exist.