Coder Social home page Coder Social logo

podman-desktop-e2e's Introduction

podman-desktop-e2e

This project define a set of e2e tests around podman desktop, this is a complementary set of e2e tests for those user stories which requires interaction beyond the podman desktop app itself with some third party apps (i.e installers from extensions).

Overview

This project is intended for running without any extra dependecy, tests are self contained into a binary wich then will be run on a target host where podman dektop should be accessible.

To accomplish this dependent-less runtime the project uses goax which uses OS native accessibility APIs to interact with UX elements (not only the ones from podman desktop but any other UX element rendered by the OS).

Build

Currently the main two target OSs for the projects are Windows and MacOS: windows binary can be built on any platform, for building MacOS binary we need to build the binary on a MacOS machine (to ensure compatibilty it is recommended to build it on MacOS 12 Monterrey).

Following commands will build the binary (windows arm64 not supported):

# Build for mac amd64 (This need to be run on a MacOS)
ARCH=amd64 make build-darwin
# Binary will be located at
out/darwin-amd64/pd-e2e

# Build for mac arm64 (This need to be run on a MacOS)
ARCH=arm64 make build-darwin
# Binary will be located at
out/darwin-arm64/pd-e2e

# Build for windows amd64 
ARCH=amd64 make build-windows
# Binary will be located at
out/windows-amd64/pd-e2e.exe

Run

The binary can be executed localy on the target hosts, it requires some parameters (on Windows):

  • pdPath: Set the path where the podman desktop executable is located.
  • pdUrl: Set the url where podman desktop executable will be downloaded.
  • user-password: Set the password for the currrent user (this is needed for running installers which require elevated permissions).
  • junit-filename: This is an optional parameter in case we want to set the name of the junit resulting file (Default: junit_report.xml).

Following command will run the tests on a windows host:

pd-e2e.exe --pdPath /Users/rhqp/pd.exe --user-password MyPassword --junit-filename pd-e2e.xml

Also the project is intended to be executed from a CI/CD system, here is a full explanation on how to use it.

Extend

The project is intended as an isolated project where e2e scenario can be defined and they would be implemented by interacting within the podman desktop application through goax.

Here are some guidelines on how current suite of tests are defined and how this can be extended to cover more functionality from podman desktop.

podman-desktop-e2e's People

Contributors

adrianriobo avatar benoitf avatar odockal avatar renovate[bot] avatar xbabalov avatar

Stargazers

avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

benoitf odockal dgolovin adrianriobo screwtsw xbabalov

podman-desktop-e2e's Issues

[enhancement] Avoid installing a distribution within wsl installation

Currently a default distro is being installed within the installation of wsl, to avoid it we can install it without distribution:

wsl --install --no-distribution

This can be fixed at

podman-desktop-e2e/oci/lib/windows/run.ps1

Line 30 in 5140a09

wsl --install

We can take advantage to wrap the wsl installation logic inside a param wslInstallFix so we can avoid that logic to be executed if we are sure wsl2 is already installed on the machine

Implement a E2E test scenario that will install OpenShift Local extension and installs CRC using GOAX testing framework

We want to implement and automate a scenario, where the Podman Desktop is spawned, OpenShift local extension is installed and finally, a CRC is installed. This has to be done using GOAX testing framework, as the crc msi installer is windows/mac native application and cannot be operated from playwright.

Follow up test (using playwright) can initialize and start the machine using openshift preset. Microshift preset will be implemented in another issue.

Automation can be also covered in another issue, but this issue should be a placeholder. First OS to aim for an automation is windows, followed up with a Mac OS. On linux, crc has no installer and so the scenario can be implemented using playwright.

Integrate a podman functional E2E tests into a pipeline that install podman through Podman Desktop

Issue to track @adrianriobo's idea to combine a podman e2e tests that are run against a podman (and podman machine) installed by Podman Desktop.

We right now have a workflow that provides this basic testing functionality (install podman through podman desktop) that all is happening on windows machines, next action could be to integration also tweaked (so they run against podman machine on windows) podman e2e tests into this workflow and collect a results.

Implement E2E test case covering Podman Onboarding workflow using GOAX

There is a new way how to install Podman, that is so called onboarding workflow. It is a Notification on a Dashboard asking user to setup Podman and Podman Machine, or from Settings->Resources -> Setup Button.

It downloads msi installed and install podman, then it offers to create, initialize and start podman machine in the same workflow. It is basically a set of forms or pages that guides the user from installing podman to started podman machine.

Let's start with windows for this purpose.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

dockerfile
oci/Containerfile
  • quay.io/rhqp/deliverest v0.0.5
github-actions
.github/workflows/build.yaml
  • actions/checkout v4
  • actions/setup-go v5
  • actions/upload-artifact v4
  • actions/checkout v4
  • actions/setup-go v5
  • actions/upload-artifact v4
  • actions/checkout v4
  • actions/download-artifact v4
  • redhat-actions/podman-login v1
  • actions/checkout v4
  • actions/download-artifact v4
  • redhat-actions/podman-login v1
.github/workflows/desktop-e2e-podman-windows-matrix.yaml
  • mikepenz/action-junit-report v4
  • actions/upload-artifact v4
.github/workflows/desktop-e2e-test-job-windows.yaml
  • mikepenz/action-junit-report v4
  • actions/upload-artifact v4
.github/workflows/podman-desktop-e2e-nightly-windows.yaml
  • mikepenz/action-junit-report v4
  • actions/upload-artifact v4
.github/workflows/podman-desktop-e2e-windows.yaml
  • mikepenz/action-junit-report v4
  • actions/upload-artifact v4
.github/workflows/podman-e2e-windows.yaml
  • mikepenz/action-junit-report v4
  • actions/upload-artifact v4
.github/workflows/sso-e2e-nightly-windows.yaml
  • mikepenz/action-junit-report v4
  • actions/upload-artifact v4
.github/workflows/tkn-bundle.yaml
  • actions/checkout v4
  • redhat-actions/podman-login v1
gomod
go.mod
  • go 1.20
  • github.com/adrianriobo/goax v0.1.1
  • github.com/onsi/ginkgo/v2 v2.17.1
  • github.com/onsi/gomega v1.32.0
  • github.com/spf13/pflag v1.0.5
  • Check this box to trigger a request for Renovate to run again on this repository

Cannot compile podman desktop repo on windows on our CI

Ref job: https://github.com/containers/podman-desktop-e2e/actions/runs/7042690597/job/19167242385

Error:

$ cross-env MODE=production npm run build && electron-builder build --config .electron-builder.config.cjs --dir --config.asar=false

> [email protected] build
> npm run build:main && npm run build:preload && npm run build:preload-docker-extension && npm run build:renderer && npm run build:extensions


> [email protected] build:main
> cd ./packages/main && vite build

node:internal/modules/cjs/loader:1340
  return process.dlopen(module, path.toNamespacedPath(filename));
                 ^

Error: The specified module could not be found.

\\?\C:\Users\rhqp\pd-e2e\podman-desktop\node_modules\@rollup\rollup-win32-x64-msvc\rollup.win32-x64-msvc.node
    at Module._extensions..node (node:internal/modules/cjs/loader:1340:18)
    at Module.load (node:internal/modules/cjs/loader:1119:32)
    at Module._load (node:internal/modules/cjs/loader:960:12)
    at Module.require (node:internal/modules/cjs/loader:1143:19)
    at require (node:internal/modules/cjs/helpers:119:18)
    at Object.<anonymous> (C:\Users\rhqp\pd-e2e\podman-desktop\node_modules\rollup\dist\native.js:60:48)
    at Module._compile (node:internal/modules/cjs/loader:1256:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
    at Module.load (node:internal/modules/cjs/loader:1119:32)
    at Module._load (node:internal/modules/cjs/loader:960:12) {
  code: 'ERR_DLOPEN_FAILED'
}

Setup a cron trigger (schedule) on Podman Desktop Playwright E2E workflows

For new we have a workflow-dispatch triggers on playwright e2e related workflows. We also want that workflows are triggered nightly - so we need to add a schedule trigger with a cron specification.

We need to reflect an input parameters default value into a workflow triggered on schedule.
(What is defined on workflow dispatch needs to be also part of schedule-based workflow.)

Windows Workflow with E2E tests that are run against a podman machine with a `--user-mode-networking` flag set

After encountering a containers/podman#20921 we could make use of a workflow, that would prevent a regression.

Idea could be as follows:
A host machine installs podman, then 3 options are provided as a parameter in order to prepare a podman machine for e2e tests to run against.

  • Rootless machine is initialized and started
  • Rootfull machine is initialized and started
  • A machine with (--user-mode-networking) flag enabled with the init command. There is also alternative to first init the machine and then call set --user-mode-networking=true afterwards.

Remove code to handle wsl installation

wsl installation requires a machine restart, as so the flow executed within podman-desktop-e2e does not control that, as so this should be handle as a pre-requisite on the target machine instead of being solved within podman-desktop-e2e.

Currently there is some code to fix the wsl installaiton which should be removed as it should not be trusted

Prove of concept for Mac based workflows

Investigate and propose a PoC for Mac Based workflows.

Discussed in #62.

Brief summary on how I was planning to use the qenvs support:

Mac machine should be running for 24 hours...and qenvs does not support auto prune after that...this means we need to run create and destroy 24 hours later...

So there should be 2 scheduled flows mac-test and mac-test-destroy (scheduled 24 hours after the create flow)

During the 24 hours period qenvs can spin multiple os versions so the idea is create a template for the flow and reuse mulitple times on the main flow (mac-test) that template:

will request a machine (i.e. version 14)
will run the tests on it
destroy the machine
check the state for the dedicated mac

This template can be executed multiple times during the 24 hours period testing mac 12,13,14...

I would add that we need to incorporate a way to securely pass secrets among jobs, as every workflow that will be running on mac needs to get access information.

[DOC] Add notes around setup for Mac target machines

Recently we on boarded several new target Mac machines to run podman-desktop-e2e. There are several setup which need to be in place in order to podman-desktop-e2e can be executed; for some of them there are already automation in place for other we still need to figure it out how to automate without manual intervention (that would be required for dynamic cloud provisioning of mac machines)

Latest workflow runs cannot create new azure instances - Operation could not be completed as it results in exceeding approved LowPriorityCores quota

Create an instance step has an error:

azure-native:compute:VirtualMachine awd-vm creating (1s) error: autorest/azure: Service returned an error. Status=\u003cnil\u003e Code=\"OperationNotAllowed\" Message=\"Operation could not be completed as it results in exceeding approved LowPriorityCores quota. Additional details - Deployment Model: Resource Manager, Location: westus2, Current Limit: 350, Current Usage: 344, Additional Required: 8, (Minimum) New Limit Required: 352. Submit a request for Quota increase at [https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity](https://aka.ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota.ReactView/Parameters/%7B%22subscriptionId%22:%22***%22)

Add flow name as tag to qenvs

In case something goes wrong with decomission it could increase the cloud cost, adding the flow name as label to qenvs will allow us to track easily where the issue comes from

Avoid to cleanup settings of Podman Desktop

The tool is cleaning up folders like

exec.Command("/bin/sh", "-c", "rm -rf ${HOME}/.local/share/containers/podman-desktop")

at https://github.com/adrianriobo/podman-desktop-e2e/blob/main/test/extended/podman-desktop/podman-desktop_darwin.go

but Podman Desktop has support now for a custom settings directory

So it would be better to use a custom directory when running tests so it never collapse with other runs/results or the developer's environment

The env variable is used in playwright e2e tests. It's called PODMAN_DESKTOP_HOME_DIR

so all logs, extensions being installed, etc, they're all going to this folder.

so it would allow to easily run tests on a developer environment

[bug] On windows 10 a reboot is need after install wsl

We are seeing failures when trying to podman machine init on windows 10 when no reboot has been executed after install wsl until we get adrianriobo/qenvs#71 fixed the best option is to include a step on our windows flows to handle wsl2 installation + reboot + ensure machine is running again

This requires #30 to avoid installing it within pd-e2e

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.