lima-vm / lima Goto Github PK

#124 showed an example of the qemu process no longer running, and I just experienced the same after a reboot.

It looks like store.Inspect is already supposed to detect this (and set the status to "Broken"), but for some reason it doesn't seem to work:

$ l ls
NAME       STATUS     SSH                ARCH      DIR
default    Running    127.0.0.1:60022    x86_64    /Users/jan/.lima/default
$ l delete -f default
INFO[0000] Sending SIGKILL to the QEMU process 51073
ERRO[0000] no such process
INFO[0000] Sending SIGKILL to the host agent process 51072
ERRO[0000] no such process
INFO[0000] Removing *.pid *.sock under "/Users/jan/.lima/default"
INFO[0000] Removing "/Users/jan/.lima/default/ga.sock"
INFO[0000] Removing "/Users/jan/.lima/default/ha.pid"
INFO[0000] Removing "/Users/jan/.lima/default/qemu.pid"
INFO[0000] Removing "/Users/jan/.lima/default/qmp.sock"
INFO[0000] Removing "/Users/jan/.lima/default/serial.sock"
INFO[0000] Removing "/Users/jan/.lima/default/ssh.sock"
INFO[0000] Deleted "default" ("/Users/jan/.lima/default")

Suggest replacing qemu with Virtualization.framework

(base) tylerjaacks in ~/Desktop/lima on master λ make
mkdir -p _output/bin
cp -a ./cmd/lima _output/bin/lima
CGO_ENABLED=0 go build -ldflags="-s -w -X github.com/AkihiroSuda/lima/pkg/version.Version=v0.1.0-2-g0ebf6f0" -o _output/bin/limactl ./cmd/limactl
pkg/limayaml/template.go:4:2: package embed is not in GOROOT (/usr/local/go/src/embed)
pkg/cidata/cidata.go:4:2: package io/fs is not in GOROOT (/usr/local/go/src/io/fs)
make: *** [Makefile:35: _output/bin/limactl] Error 1

macOS instances of GHA are slow and quite flaky.

Maybe we should use other CI such as Cirrus https://cirrus-ci.org/guide/macOS/

I would like to extend mounts in 2 ways:

1. Allow different host and guest locations
2. Allow mounting single files instead of directories

The first seems easy enough, but it is not clear to me if this is going to work with sshfs. Also given that #118 is WIP, this may be wasted effort (unless it is rather simple to implement).

Any hints of making single file mounts work with sshfs? And if not, will those be possible with samba?

One use case (I have others) would be adding custom CA certs to the guest:

mounts:
- host: ~/company/root.crt
  guest: /usr/local/share/ca-certificates/company.crt
  writable: false
provision:
- mode: system
  script: |
    #!/bin/bash
    update-ca-certificates

I guess I would like to have something similar to the capabilities of the docker -v option to define mounts.

I ran limactl start default without adding the hypervisor entitlements to qemu (see #4) and then connected via lima bash. The mounted directories /tmp/lima (and /Users/jan) work as expected for the normal user:

jan@lima-default:/$ ls -l /tmp/
total 24
drwxr-xr-x 1 root root  192 May 15 00:55 lima
drwx------ 3 root root 4096 May 15 00:53 snap.lxd
drwx------ 2 jan  jan  4096 May 15 00:53 ssh-VWQD1U87aO
drwx------ 3 root root 4096 May 15 00:53 systemd-private-193c8c72bfc547b5bd03fbf7a3776fda-systemd-logind.service-BsE9tL
drwx------ 3 root root 4096 May 15 00:53 systemd-private-193c8c72bfc547b5bd03fbf7a3776fda-systemd-resolved.service-cfZvr4
drwx------ 3 root root 4096 May 15 00:53 systemd-private-193c8c72bfc547b5bd03fbf7a3776fda-systemd-timesyncd.service-fAE656
jan@lima-default:/$ touch /tmp/lima/foo

But the root users doesn't seem to have write/execute permission to them:

jan@lima-default:/$ sudo ls -l /tmp/
ls: cannot access '/tmp/lima': Permission denied
total 20
d????????? ? ?    ?       ?            ? lima
drwx------ 3 root root 4096 May 15 00:53 snap.lxd
drwx------ 2 jan  jan  4096 May 15 00:53 ssh-VWQD1U87aO
drwx------ 3 root root 4096 May 15 00:53 systemd-private-193c8c72bfc547b5bd03fbf7a3776fda-systemd-logind.service-BsE9tL
drwx------ 3 root root 4096 May 15 00:53 systemd-private-193c8c72bfc547b5bd03fbf7a3776fda-systemd-resolved.service-cfZvr4
drwx------ 3 root root 4096 May 15 00:53 systemd-private-193c8c72bfc547b5bd03fbf7a3776fda-systemd-timesyncd.service-fAE656
jan@lima-default:/$ sudo touch /tmp/lima/foo
touch: cannot touch '/tmp/lima/foo': Permission denied

https://twitter.com/_AkihiroSuda_/status/1403403845842075648

I'm planning to support privileged ports using authbind (https://github.com/Castaglia/MacOSX-authbind) if it works.

Would be nice if lima was on Homebrew!

I've been running into this while adding tests to #114, and I don't understand how this is even possible.

This experiment was done with current master, not with my #114 related changes.

Start lima with the default instance and add a couple of port forwards using netcat:

$ limactl start --tty=false examples/default.yaml
[...]
$ lima
jan@lima-default:~$ for port in 4001 4002 4003; do (nc -l 127.0.0.1 $port &); done

This produces the following output in ~/.lima/default/ha.stderr.log:

{"level":"debug","msg":"guest agent event: {Time:2021-07-23 01:40:13.814921119 +0000 UTC LocalPortsAdded:[{IP:127.0.0.1 Port:4001} {IP:127.0.0.1 Port:4002} {IP:127.0.0.1 Port:4003}] LocalPortsRemoved:[] Errors:[]}","time":"2021-07-22T18:40:13-07:00"}
{"level":"info","msg":"Forwarding TCP port 4001","time":"2021-07-22T18:40:13-07:00"}
{"level":"info","msg":"Forwarding TCP port 4002","time":"2021-07-22T18:40:13-07:00"}
{"level":"info","msg":"Forwarding TCP port 4003","time":"2021-07-22T18:40:13-07:00"}

Now I kill all netcat instances:

jan@lima-default:~$ pkill -x nc

Which adds these entries to the log:

{"level":"debug","msg":"guest agent event: {Time:2021-07-23 01:40:28.814399265 +0000 UTC LocalPortsAdded:[{IP:127.0.0.1 Port:4001} {IP:127.0.0.1 Port:4002} {IP:127.0.0.1 Port:4003}] LocalPortsRemoved:[{IP:127.0.0.1 Port:4001} {IP:127.0.0.1 Port:4002} {IP:127.0.0.1 Port:4003}] Errors:[]}","time":"2021-07-22T18:40:28-07:00"}
{"level":"info","msg":"Stopping forwarding TCP port 4001","time":"2021-07-22T18:40:28-07:00"}
{"level":"info","msg":"Stopping forwarding TCP port 4002","time":"2021-07-22T18:40:28-07:00"}
{"level":"info","msg":"Stopping forwarding TCP port 4003","time":"2021-07-22T18:40:28-07:00"}
{"level":"info","msg":"Forwarding TCP port 4001","time":"2021-07-22T18:40:28-07:00"}
{"level":"info","msg":"Forwarding TCP port 4002","time":"2021-07-22T18:40:28-07:00"}
{"level":"info","msg":"Forwarding TCP port 4003","time":"2021-07-22T18:40:28-07:00"}

Why are the removed ports still included in LocalPortsAdded? I've been staring at the comparePorts function for a long time now, and this shouldn't be possible: https://github.com/AkihiroSuda/lima/blob/master/pkg/guestagent/guestagent_linux.go#L34-L59

Any ideas what could cause this?

should we use https://cloud-images.ubuntu.com/minimal/daily/hirsute/current/ instead of https://cloud-images.ubuntu.com/hirsute/current/ ?

the minimal is half the size

2021/06/30 14:16:47 Stopping tail as file no longer exists: /Users/jan/.lima/std/ha.stdout.log
2021/06/30 14:16:47 Stopping tail as file no longer exists: /Users/jan/.lima/std/ha.stderr.log
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0x144e18b]

goroutine 1 [running]:
github.com/AkihiroSuda/lima/pkg/hostagent/api.WatchEvents(0x169c9e8, 0xc00006ed20, 0xc000022990, 0x22, 0xc0000229c0, 0x22, 0xc02f55a7b59fbcc8, 0x1bceac, 0x1993120, 0xc000175b88, ...)
	/Users/jan/suse/lima/pkg/hostagent/api/eventwatcher.go:46 +0x24b

INFO[0000] Sending SIGINT to hostagent process 25149
INFO[0000] Waiting for the host agent and the qemu processes to shut down
INFO[0000] [hostagent] Received SIGINT, shutting down the host agent
INFO[0000] [hostagent] Shutting down the host agent
INFO[0000] [hostagent] Unmounting "/Users/jan"
WARN[0000] [hostagent] connection to the guest agent was closed unexpectedly
INFO[0000] [hostagent] Shutting down QEMU with ACPI
INFO[0000] [hostagent] Sending QMP system_powerdown command
INFO[0010] [hostagent] Forwarding "/run/user/501/lima-guestagent.sock" (guest) to "/Users/jan/.lima/std/ga.sock" (host)
WARN[0010] [hostagent] failed to setting up forward from "/run/user/501/lima-guestagent.sock" (guest) to "/Users/jan/.lima/std/ga.sock" (host)
WARN[0010] [hostagent] connection to the guest agent was closed unexpectedly
INFO[0020] [hostagent] Forwarding "/run/user/501/lima-guestagent.sock" (guest) to "/Users/jan/.lima/std/ga.sock" (host)
WARN[0020] [hostagent] failed to setting up forward from "/run/user/501/lima-guestagent.sock" (guest) to "/Users/jan/.lima/std/ga.sock" (host)
WARN[0020] [hostagent] connection to the guest agent was closed unexpectedly
INFO[0030] [hostagent] Forwarding "/run/user/501/lima-guestagent.sock" (guest) to "/Users/jan/.lima/std/ga.sock" (host)
WARN[0030] [hostagent] failed to setting up forward from "/run/user/501/lima-guestagent.sock" (guest) to "/Users/jan/.lima/std/ga.sock" (host)
WARN[0030] [hostagent] connection to the guest agent was closed unexpectedly
2021/06/30 14:16:47 Stopping tail as file no longer exists: /Users/jan/.lima/std/ha.stdout.log
2021/06/30 14:16:47 Stopping tail as file no longer exists: /Users/jan/.lima/std/ha.stderr.log
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0x144e18b]

goroutine 1 [running]:
github.com/AkihiroSuda/lima/pkg/hostagent/api.WatchEvents(0x169c9e8, 0xc00006ed20, 0xc000022990, 0x22, 0xc0000229c0, 0x22, 0xc02f55a7b59fbcc8, 0x1bceac, 0x1993120, 0xc000175b88, ...)
	/Users/jan/suse/lima/pkg/hostagent/api/eventwatcher.go:46 +0x24b
main.waitForHostAgentTermination(0x169c9b0, 0xc000028a68, 0xc000163ea0, 0xc02f55a7b59fbcc8, 0x1bceac, 0x1993120, 0x0, 0x0)
	/Users/jan/suse/lima/cmd/limactl/stop.go:91 +0x23c
main.stopInstanceGracefully(0xc000163ea0, 0x15e6a5b, 0x5)
	/Users/jan/suse/lima/cmd/limactl/stop.go:69 +0x2f5
main.stopAction(0xc00003d040, 0x2, 0x2)
	/Users/jan/suse/lima/cmd/limactl/stop.go:54 +0x169
github.com/urfave/cli/v2.(*Command).Run(0x1986d40, 0xc00003cf00, 0x0, 0x0)
	/Users/jan/go/pkg/mod/github.com/urfave/cli/[email protected]/command.go:163 +0x4dd
github.com/urfave/cli/v2.(*App).RunContext(0xc0001be000, 0x169c9b0, 0xc000028a60, 0xc00001e1b0, 0x3, 0x3, 0x0, 0x0)
	/Users/jan/go/pkg/mod/github.com/urfave/cli/[email protected]/app.go:313 +0x810
github.com/urfave/cli/v2.(*App).Run(...)
	/Users/jan/go/pkg/mod/github.com/urfave/cli/[email protected]/app.go:224
main.main()
	/Users/jan/suse/lima/cmd/limactl/main.go:15 +0x67

I suspect that the Lines channel is returning nil instead of a *Line pointer, but haven't looked further. Also not sure if this is worth spending time on, except maybe by throwing an if line != nil condition in there?

The user-data.TEMPLATE is becoming too long.
Eventually we should just place the scripts as plain files in the cidata drive.
(And we can execute scripts from the cidata drive directly, without copying them into /var/lib/lima-guestagent.)

Can be another PR, though.

Originally posted by @AkihiroSuda in #52 (comment)

Hey!

I’m asking myself whether there is an config option to customize the used qemu parameters?
For example to add -nographic and other settings.

Thx and greets Nico

is it possible to add custom user-data? (i.e. to trigger additional guest configurations, like package installations)
if not, would the maintainers be open for a pr?

I'd be interested in passing a user data string like you can with cloud providers (see below). I would imagine two modes: One that adds to the default user-data template and one that overwrites the default template.

Example from digitalocean droplets:

$ doctl compute droplet create --help
[...]
      --user-data string            User-data to configure the Droplet on first boot

Example from aws ec2:

$aws ec2 run-instances help
[...]
       --user-data (string)
          The  user  data to make available to the instance. For more informa-
          tion, see Running commands on your Linux instance at launch  (Linux)
          and  Adding  User  Data  (Windows).  If you are using a command line
          tool, base64-encoding is performed for you, and  you  can  load  the
          text  from  a file. Otherwise, you must provide base64-encoded text.
          User data is limited to 16 KB.

I tried on both latest Catalina and Big Sur on Intel Macs to run Lima with an ARM image. The only changes I made in the config YAML was setting arch="aarch64" and cpus=1. It starts off like this:

But then the screen blanks and nothing further seems to be happening (there is still a lot of CPU usage in the qemu process though):

In desperation I tried to set legacyBIOS=true, but that didn't work at all.

Is this working for anybody else?

On my machine/configuration SSH seems to not be able to bind to port 60022 and therefore things fail to start. I've checked whether I already have anything else listening on that port on the localhost interface but the port is indeed available.

The image to boot is the default one, and I'm running macOS 11.3.1. QEMU comes from MacPorts and entitlements are correctly installed. Debug log follows:

agatti@tardigrade ~ % limactl --debug start
DEBU[0000] interpreting argument "default" as an instance name "default"
INFO[0000] Using the existing instance "default"
INFO[0000] Starting QEMU
DEBU[0000] qCmd.Args: [/opt/local/bin/qemu-system-x86_64 -cpu Haswell-v4 -machine q35,accel=hvf -smp 4,sockets=1,cores=4,threads=1 -m 4096 -drive if=pflash,format=raw,readonly,file=/opt/local/share/qemu/edk2-x86_64-code.fd -drive file=/Users/agatti/.lima/default/diffdisk,if=virtio -boot c -cdrom /Users/agatti/.lima/default/cidata.iso -net nic,model=virtio -net user,net=192.168.5.0/24,hostfwd=tcp:127.0.0.1:60022-:22 -device virtio-rng-pci -device virtio-vga -device virtio-keyboard-pci -device virtio-mouse-pci -parallel none -name lima-default -pidfile /Users/agatti/.lima/default/qemu-pid]
INFO[0000] SSH: 127.0.0.1:60022
INFO[0000] Waiting for the essential requirement 1 of 3: "ssh"
DEBU[0000] executing script "ssh"
DEBU[0000] executing ssh for script "ssh": /usr/bin/ssh [ssh -o ControlMaster=auto -o ControlPath=/Users/agatti/.lima/default/ssh.sock -o ControlPersist=5m -o StrictHostKeyChecking=no -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -p 60022 127.0.0.1 -- /bin/bash]
DEBU[0000] stdout="", stderr="ssh: connect to host 127.0.0.1 port 60022: Connection refused\r\n", err=failed to execute script "ssh": stdout="", stderr="ssh: connect to host 127.0.0.1 port 60022: Connection refused\r\n": exit status 255

[then keeps looping on waiting for ssh]

Is there anything else I should check to get this to at least start?

...
+ limactl shell archlinux nerdctl info
time="2021-06-29T09:48:49Z" level=warning msg="environment variable XDG_RUNTIME_DIR is not set, see https://rootlesscontaine.rs/getting-started/common/login/"
time="2021-06-29T09:48:49Z" level=fatal msg="rootless containerd not running? (hint: use `containerd-rootless-setuptool.sh install` to start rootless containerd): environment variable XDG_RUNTIME_DIR is not set, see https://rootlesscontaine.rs/getting-started/common/login/"
exit status 1
+ rm -rf /home/testuser/lima-test-tmp

Detected in #95

I was curious to see what sort of filesystem performance gains there were compared to Docker for Mac. I wanted to test a repo that would make a lot of filesystem writes, but in my tests I encountered an error saying the volume was mounted read-only. My apologies for my ignorance, but is this intentional? Should containers launched via lima with volumes be mounted as read-only volumes? For reference, here's how to reproduce:

cd path/to/somewhere
git clone [email protected]:withspectrum/spectrum.git
cd spectrum
lima nerdctl run -it --rm -v "$PWD":/usr/src/app -w /usr/src/app node:14 node shared/install-dependencies.js

This will eventually print the following error:

error Could not write file "/usr/src/app/yarn-error.log": "EROFS: read-only file system, open '/usr/src/app/yarn-error.log'"
error An unexpected error occurred: "EROFS: read-only file system, mkdir '/usr/src/app/node_modules'".

The docker analog that works ok:

docker run -it --rm -v "$PWD":/usr/src/app -w /usr/src/app node:14 node shared/install-dependencies.js

(base) tylerjaacks in ~ λ limactl start default
INFO[0027] Attempting to download the image from "file:///Users/tylerjaacks/Downloads/hirsute-server-cloudimg-amd64.img"
curl: (37) Couldn't open file /Users/tylerjaacks/Downloads/hirsute-server-cloudimg-amd64.img
INFO[0027] Attempting to download the image from "https://cloud-images.ubuntu.com/hirsute/current/hirsute-server-cloudimg-amd64.img"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 548M 100 548M 0 0 7790k 0 0:01:12 0:01:12 --:--:-- 8069k
INFO[0099] Starting QEMU
INFO[0099] SSH: 127.0.0.1:60022
INFO[0099] Waiting for the essential requirement 1 of 3: "ssh"
INFO[0109] Waiting for the essential requirement 1 of 3: "ssh"
INFO[0119] Waiting for the essential requirement 1 of 3: "ssh"
INFO[0129] Waiting for the essential requirement 1 of 3: "ssh"
INFO[0139] Waiting for the essential requirement 1 of 3: "ssh"
INFO[0149] Waiting for the essential requirement 1 of 3: "ssh"
INFO[0159] Waiting for the essential requirement 1 of 3: "ssh"

Start command needs terminal to be left open to work and sending interrupt signal kills QEMU.

There is place for a few improvements:

• Make start command to daemonize process by default: the process detaches making QEMU running in background.
• Add stop command that makes VM gracefully shutdown. Can be implemented by communicating with guest agent and calling shutdown command on guest.

https://gitlab.com/qemu-project/qemu/-/issues/334

Because the logging just goes to the regular logger and not the hostagent logger: https://github.com/AkihiroSuda/sshocker/blob/56aa3c7/pkg/ssh/ssh.go#L98

sshfs could be replaced with 9pfs

The current test suite is written in bash and perl, but we should rewrite tests in Go

There should be an option to use custom QEMU installation that is located in some directory. It is not always convenient to have system-global QEMU.

This setting needs to be configured per VM (in config).

Currently there's 3 things that are used from QEMU:

• qemu-system-
• qemu-img
• firmware (UEFI)

QEMU supports nested virtualisation, any reason LIMA wouldn't also support that use case?

I know Multipass doesnt (https://discourse.ubuntu.com/t/nested-virtualization-on-macos/19345) but -cpu host and -enable-kvm should get us there and I would really love something that gives me Linux host with the capabilities to spin guests for Infrastructure-as-code type affairs.

Previous discussion: #62 (comment)

With this qemu patch: https://lore.kernel.org/qemu-devel/[email protected]/ https://github.com/AkihiroSuda/vde_vmnet

github.com/lima is already taken by another project, so we have to come up with another name, e.g., github.com/linuxmachines.

We can also consider renaming lima itself, as there has been also Lima GPU driver project (https://gitlab.freedesktop.org/lima). (But finding an unique/short/rememberable name is not easy 😅 )

EDIT Created https://github.com/lima-vm . Thanks to @jandubois for suggestion.

Hi,

I'm very excited about this project, so I tried using with my company's current stack.
We're using docker with docker-compose. Working mainly with php-fpm and nginx.

docker-compose up works fine, but as soon as I try to install any dependency within a docker container, I get a permission denied.
Same goes for reading, I managed to forward 443 port on my host machine but nginx (running in docker, running in Lima) cannot read the index.php file.

Tested on a 2017 macbook pro, running on a 2,5 GHz Intel Core i7. macOS version is 10.14.16.

Following is my lima.yaml file, I only made the home directory writable:

  # Arch: "default", "x86_64", "aarch64".
  # "default" corresponds to the host architecture.
  arch: "default"
  
  # An image must support systemd and cloud-init.
  # Ubuntu and Fedora are known to work.
  # Default: none (must be specified)
  images:
    # Try to use a local image first.
    - location: "~/Downloads/hirsute-server-cloudimg-amd64.img"
      arch: "x86_64"
    - location: "~/Downloads/hirsute-server-cloudimg-arm64.img"
      arch: "aarch64"
  
    # Download the file from the internet when the local file is missing.
    - location: "https://cloud-images.ubuntu.com/hirsute/current/hirsute-server-cloudimg-amd64.img"
      arch: "x86_64"
    - location: "https://cloud-images.ubuntu.com/hirsute/current/hirsute-server-cloudimg-arm64.img"
      arch: "aarch64"
  
  # CPUs: if you see performance issues, try limiting cpus to 1.
  # Default: 4
  cpus: 4
  
  # Memory size
  # Default: "4GiB"
  memory: "4GiB"
  
  # Disk size
  # Default: "100GiB"
  disk: "100GiB"
  
  # Expose host directories to the guest
  # Default: none
  mounts:
    - location: "~"
      # CAUTION: `writable` SHOULD be false for the home directory.
      # Setting `writable` to true is possible, but untested and dangerous.
      writable: true
    - location: "/tmp/lima"
      writable: true
  
  ssh:
    # A localhost port of the host. Forwarded to port 22 of the guest.
    # Currently, this port number has to be specified manually.
    # Default: none
    localPort: 60022
  
  firmware:
    # Use legacy BIOS instead of UEFI.
    # Default: false
    legacyBIOS: false
  
  video:
    # QEMU display, e.g., "none", "cocoa", "sdl".
    # As of QEMU v5.2, enabling this is known to have negative impact
    # on performance on macOS hosts: https://gitlab.com/qemu-project/qemu/-/issues/334
    # Default: "none"
    display: "none"
  
  containerd:
    # Enable system-wide (aka rootful)  containerd and its dependencies (BuildKit, Stargz Snapshotter)
    # Default: false
    system: false
    # Enable user-scoped (aka rootless) containerd and its dependencies
    # Default: true
    user: true
  
  # Provisioning scripts need to be idempotent because they might be called
  # multiple times, e.g. when the host VM is being restarted.
  # provision:
  #   # `system` is executed with the root privilege
  #   - mode: system
  #     script: |
  #       #!/bin/bash
  #       set -eux -o pipefail
  #       export DEBIAN_FRONTEND=noninteractive
  #       apt-get install -y vim
  #   # `user` is executed without the root privilege
  #   - mode: user
  #     script: |
  #       #!/bin/bash
  #       set -eux -o pipefail
  #       cat <<EOF > ~/.vimrc
  #       set number
  #       EOF
  
  # probes:
  #  # Only `readiness` probes are supported right now.
  #  - mode: readiness
  #    description: vim to be installed
  #    script: |
  #       #!/bin/bash
  #       set -eux -o pipefail
  #       if ! timeout 30s bash -c "until command -v vim; do sleep 3; done"; then
  #         echo >&2 "vim is not installed yet"
  #         exit 1
  #       fi
  #    hint: |
  #      vim was not installed in the guest. Make sure the package system is working correctly.
  #      Also see "/var/log/cloud-init-output.log" in the guest.

Any help would be appreciated 🙂

Hello. I would like to give some ideas and advices about filesystem sharing.

So, since we are using QEMU, lets see which options do we have:

• VirtioFS. Looking very cool, seems to have really good performance, but works only on Linux hosts. It is very optimised for using in virtual machines, it even uses DAX (direct access) for files, so there's no need to copy files over network, they're just in the shared RAM between VM and host.
• VirtFS (9P). I've tried to use it, but it's incredible slow. Really. Using just git status in shared directory with middle size project takes at least half a minute. I would rather just place files in VM and access them via some remote file access protocol and use vscode with remote access (sad, but they're proprietary). I think that it is so slow because it is sync. Whenever you read some file, do stat call etc. you have to wait for this operation to end.
• Just sync filesystem state between VM and host.
• Write custom FUSE driver with asynchronous protocol and multithreading support. Theoretically this can be more performant than 9P, but not sure.

follow-up #18

$ brew info libslirp
libslirp: stable 4.6.1 (bottled)

$ limactl start default
? Creating an instance "default" Proceed with the default configuration
INFO[0001] Downloading "https://github.com/containerd/nerdctl/releases/download/v0.9.0/nerdctl-full-0.9.0-linux-amd64.tar.gz" 
INFO[0001] Using cache "/Users/nicolas/Library/Caches/lima/download/by-url-sha256/4c048fab91ba88527ad58a6b4dc4c3bfa35e828b8147565456b76d362ec2e6cb/data" 
INFO[0004] Attempting to download the image from "~/Downloads/hirsute-server-cloudimg-amd64.img" 
INFO[0004] Attempting to download the image from "https://cloud-images.ubuntu.com/hirsute/current/hirsute-server-cloudimg-amd64.img" 
INFO[0006] Using cache "/Users/nicolas/Library/Caches/lima/download/by-url-sha256/e1fed960ebd29619676c7ab7535bc83f7fb2ad71739edb6fde4e17bce0b61a47/data" 
INFO[0006] [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/nicolas/.lima/default/serial.log") 
INFO[0006] SSH Local Port: 60022                        
INFO[0007] [hostagent] Waiting for the essential requirement 1 of 4: "ssh" 
...
INFO[0578] [hostagent] Waiting for the essential requirement 1 of 4: "ssh" 
INFO[0588] [hostagent] Waiting for the essential requirement 1 of 4: "ssh" 
INFO[0598] [hostagent] Waiting for the essential requirement 1 of 4: "ssh" 
FATA[0607] did not receive an event with the "running" status 

serial.log

lima --version
limactl version 0.4.0

output of limactl ls

➜  ~ limactl ls
NAME       STATUS     SSH                ARCH      DIR
default    Running    127.0.0.1:60022    x86_64    /Users/fanbingxin/.lima/default

output of limactl --debug stop default

limactl --debug stop default
INFO[0000] Sending SIGINT to hostagent process 14565
ERRO[0000] no such process
INFO[0000] Waiting for the host agent and the qemu processes to shut down
DEBU[0000] received an event                             event="{2021-07-21 21:04:04.079921 +0800 CST {false false false [] 60022}}"
DEBU[0000] received an event                             event="{2021-07-21 21:05:19.355825 +0800 CST {true false false [] 60022}}"
FATA[0180] did not receive an event with the "exiting" status

The QEMU process no longer exists, but Lima thinks the virtual machine is still running.
How to restart Lima instance?

Thank you!

Here's the error I get:

failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:722: waiting for init preliminary setup caused: unexpected 123 != 0: unknown

Reproduced with docker-compose and nerdctl:

NB : I use a Macbook pro 13 with the M1 ship.

Thanks in advance.

Right now lima adds all ~/.ssh/*.pub keys as authorized keys to the guest OS and lets ssh pick a key to authenticate.

lima will not work unless the user has at least one key pair, and asks them to run ssh-keygen when there isn't one.

For embedding lima in Rancher Desktop, we would want to create a key pair independently of what the user does, and not store it under ~/ssh because that should only be done by the user themselves.

I'm proposing to authorize a key from $LIMA_HOME/_config/user.pub and use $LIMA_HOME/_config/user.key as the identity file, if those files exist, and fall back on the current behaviour if they don't.

I've choosen $LIMA_HOME/_config because there seemed to be tentative agreement already in #63 on using $LIMA_HOME/_cache for the cache.

I'm happy to implement this, but wanted to make sure first that this approach is agreeable.

I might be interested in adding Arch Linux. What does it take and where do I look?

$ limactl shell alpine ls /sys/fs/cgroup

/sys/fs/cgroup has to be mounted for running container applications.

Thanks for this amazing project!

I wonder if it would make sense devise a roadmap to support high-performance graphics in the style of https://github.com/microsoft/wslg ?
"WSLg is short for Windows Subsystem for Linux GUI and the purpose of the project is to enable support for running Linux GUI applications (X11 and Wayland) on Windows in a fully integrated desktop experience."

There were some discussions about Wayland on macOS (Miguel de Icaza https://twitter.com/migueldeicaza/status/1254929025988005889). XQuartz worked great before retina display was introduced but AFAIK support for full resolution was never completed (https://gitlab.freedesktop.org/xorg/xserver/-/issues/187).

Can you add advantages of using this to the readme?

speed? Is it all about speed? Can i run a dev env that includes node_modules that wont be super slow?

#88 (comment)

This makes is harder to parse it using JSON.parse from JavaScript:

$ limactl ls --json
{"name":"alpine","status":"Running","dir":"/Users/jan/.lima/alpine","arch":"x86_64","sshLocalPort":60020,"hostAgentPID":16450,"qemuPID":16451}
{"name":"default","status":"Running","dir":"/Users/jan/.lima/default","arch":"x86_64","sshLocalPort":60022,"hostAgentPID":13199,"qemuPID":13200}

Is there a reason not to wrap the objects in an array, making the output a valid JSON document?

Hypothetical situation: I use a company laptop, and device management software forces me to use some LDAP identity as the local user name, say JDubois. But for all my VMs and lab machines I use jan as the user name. Therefore I create an ~/.ssh/config with explicit entries for a couple of official servers using my LDAP identify, and a fallback for my lab:

Host *
     User jan
     ForwardAgent yes
     UseKeychain yes
     AddKeysToAgent yes
     IdentityFile ~/.ssh/jan
     IdentitiesOnly yes

This currently breaks lima because it would create a JDubois user in the guest VM, but tries to connect with a jan user because of the config definition.

Always specifying the user in ssh commands avoids this potential error situation.

I've installed qemu via brew. As soon as I invoke codesign with the parameters specified by the README.md, qemu no longer works and just prints Killed: 9 regardless of which options are specified, e.g.

$ cp /usr/local/bin/qemu-system-x86_64 .

$ ./qemu-system-x86_64 --version
QEMU emulator version 5.2.0
Copyright (c) 2003-2020 Fabrice Bellard and the QEMU Project developers

$ codesign -v -s - --entitlements entitlements.xml ./qemu-system-x86_64
./qemu-system-x86_64: signed Mach-O thin (x86_64) [qemu-system-x86_64-5555494422424fa5348c3b5a936d67231b727a83]

$ ./qemu-system-x86_64 --version
Killed: 9

$ cat entitlements.xml
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <!-- for OS X 10.10 - macOS 10.15 -->
    <key>com.apple.vm.hypervisor</key>
    <true/>
    <!-- for macOS 11 and later -->
    <key>com.apple.security.hypervisor</key>
    <true/>
</dict>
</plist>

$ codesign -dv qemu-system-x86_64
Executable=/Users/jan/Downloads/qemu-system-x86_64
Identifier=qemu-system-x86_64-5555494422424fa5348c3b5a936d67231b727a83
Format=Mach-O thin (x86_64)
CodeDirectory v=20100 size=100716 flags=0x2(adhoc) hashes=3139+5 location=embedded
Signature=adhoc
Info.plist=not bound
TeamIdentifier=not set
Sealed Resources=none
Internal requirements count=0 size=12

$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.15.7
BuildVersion:	19H1030

The failure is due to the entitlement, not qemu: when I sign a different binary with the same XML file, it fails the same way. Same thing with just the macOS 10.10+ entitlement:

<dict>
    <key>com.apple.vm.hypervisor</key>
    <true/>
</dict>

I can sign with a different entitlement without problem, e.g. this works just fine:

<dict>
    <key>com.apple.security.cs.disable-library-validation</key>
    <true/>
</dict>

I'm trying to run lima on my Macbook pro, but lima is complaining about a missing command. Do I need some sort of additional dependency to run this?

Actually we don't need to see QEMU windows which always just shows the same image:

This can be removed with -nographic option and omitting -vga none and -device ramfb options.

The second thing it is worth to mention is boot (BIOS) timeout (3 seconds) which can be bypassed using -boot "splash-time=0,menu=on" option.

But these options should be configurable by start command flag and (maybe) by VM config. They're pretty useful for troubleshooting.

==> /Users/runner/.lima/debian/serial.log <==
57
[    0.056000] Call Trace:
58
[    0.056000]  dump_stack+0x6b/0x83
59
+ exit 1
60
[    0.056000]  panic+0x101/0x2d7
61
+ limactl delete -f debian
62
[    0.056000]  setup_IO_APIC+0x80e/0x855
63
[    0.056000]  ? _raw_spin_unlock_irqrestore+0x16/0x20
64
[    0.056000]  ? clear_IO_APIC_pin+0x173/0x240
65
[    0.056000]  x86_late_time_init+0x20/0x30
66
[    0.056000]  start_kernel+0x4eb/0x5a8
67
[    0.056000]  secondary_startup_64_no_verify+0xb0/0xbb
68
[    0.056000] ---[ end Kernel panic - not syncing: IO-APIC + timer doesn't work!  Boot with apic=debug and send a report.  Then try booting with the 'noapic' option. ]---

https://github.com/AkihiroSuda/lima/blob/eec56822fe197c15a9ac40a0ca736c3a1f5f833e/examples/debian.yaml