unoplatform / uno.uitest

Unified UI Testing Framework for Uno Platform based applications

Uno.UITest for Uno Platform

Welcome to the Uno.UITest repository, a framework which enables unified UI Testing of Uno Platform apps, using NUnit 3.x.

This library provides a set of APIs to interact with an app, and assess its behavior using device simulators and browsers. The API set is based on Xamarin.UITest, which makes the migration and patterns very familiar.

The testing is available through :

• Selenium for WebAssembly apps, using Chrome
• Xamarin.UITest and AppCenter for iOS and Android apps.

The following target platforms are not yet supported:

• SkiaSharp backends (GTK, WPF and Tizen )
• Xamarin.macOS
• Windows

Build status

Target	Branch	Status	Recommended Nuget packages version
development	master

Getting started with Uno.UITest

Visit our documentation on how to get started with Uno.UITest.

uno.uitest's Issues

[iOS] Enable support for iOS UI Testing

I'm submitting a...

• Feature request

Current behavior

UI Testing for iOS is not functioning properly because of microsoft/appcenter#421

Expected behavior

iOS UI Testing is working

Environment

Package Version(s): 

Visual Studio
- [ ] 2017 (version: )
- [ ] 2017 Preview (version: )
- [x] for Mac (version: )

Update:

• Testing with VS4mac 8.2, and Xamarin.UITest 3.0.1-dev1 and dev2 does not change the outcome of the build.

Needs support for advance custom pointer gestures

Feature request

We should be able to emulate advance pointer gestures, with multiple pointers at once, and also have to ability to run check while pointer are pressed.

Current behavior

We only have some pre-baked gesture / pointer manipulations like "Drag and drop", "pinch", etc.

Expected behavior

We should be able to:

• Select the pointer type
• Control each pointer individually, including its location and its pressed state
• Be able to take a screenshot or run some checks while a pointer is pressed

Minimal reproduction of the problem with instructions

https://github.com/unoplatform/uno/pull/2496/files/8d9e491df98abf9d8420c145918d7d10fdffec61#diff-0d9db92fc7555ca8b9e81df6d0cf3ac3

Environment

Package Version(s): _current_
Visual Studio: _irrelevant_

Readme steps don't end up in runnable WASM test

I'm submitting a...

• Bug report (I searched for similar issues and did not find one)

Current behavior

See the attached repro that was created using the steps in the readme. When running the test, I get a NullReferenceError because App is null:

Expected behavior

Steps in readme should lead to runnable test on WASM.

Minimal reproduction of the problem with instructions

Repro project created using steps in readme:

App1.zip

Environment

Package Version(s): 

Uno.UI.*: 3.5.1
Uno.UITests.Helpers: 1.1.0-dev.11 

Visual Studio
- [x] 2019 (version:  16.8.5)
- [ ] 2017 (version: )
- [ ] 2017 Preview (version:)
- [ ] for Mac (version: )

CyberModz

Needs support of PinchToZoomInCoordinates for WASM

Feature request

Add support of PinchToZoomInCoordinates for WASM

Current behavior

PinchToZoomInCoordinates throws NotImplemented on WASM

Expected behavior

PinchToZoomInCoordinates does a "pinch to zoom" gesture

Minimal reproduction of the problem with instructions

https://github.com/unoplatform/uno/pull/2445/files/f85d235b368ecfd6a2042f9773e85b48ccfa6899#diff-76289c95a7b3ef52727fd43e7f498de2

Environment

Package Version(s): current
Visual Studio: irrelevant

Create a ActivePlatforms/IgnoredPlatforms Attribute for test methods/classes

I'm submitting a

Feature Request

Current behavior

Currently we would need to something like the following for our test methods and classes.

if (AppInitializer.GetLocalPlatform() == Platform.Android
    || AppInitializer.GetLocalPlatform() == Platform.iOS))
 {
   // Do something
 }
else if  (AppInitializer.GetLocalPlatform() == Platform.Browser)
{
  // Do something else
}

Desired behavior

What would be nice and cleaner is some sort of ActivePlatforms attribute for test methods and classes.
So we can mark classes and methods by doing something like:

[ActivePlatforms(Platform.Android, Platform.iOS)]
public void MyTest() { }

Or maybe even an IgnoredPlatforms?

[IgnoredPlatforms(Platform.Browser)]
public void MyTest() { }

[UWP] Enable support for UWP UI testing

Being able to run UI tests on UWP would be a useful 'source of truth' for Uno Platform tests.

I'm submitting a...

Feature request

IAppResult of controls have Text set to null

I'm submitting a...

• Bug report (I searched for similar issues and did not find one) -->

Current behavior

Using the following test and XAML, the IAppResult returns null instead of the text of the control.

<TextBlock x:Name="MyTextBlock" Text="Some text" />

var result= App.WaitForElement("MyTextBlock");
Assert.IsTrue(errorMessageResult.Length > 0);
Assert.IsTrue(errorMessageResult[0].Text == "Some text"); // This line fails as Text is null.

Expected behavior

Text should report the actual text of the control if its a TextBlock.

Minimal reproduction of the problem with instructions

See code above.

Environment

Package Version(s): 

Uno.UITest.*: 1.1.0-dev11

Visual Studio
- [x] 2019 (version: 16.10.0)
- [ ] 2019 Preview (version: )
- [ ] for Mac (version: )

No way to start app without clearing App data

I'm submitting a...

• Documentation issue or request

Current behavior

There are 2 ways to start an app

1. through AppInitializer.ColdStartApp() which clears all app data
2. Through AppInitializer.AttachToApp() which doesn't restart the app just attaches to the open app

Is there a way to start the app but without clearing the App data first?

Expected behavior

Minimal reproduction of the problem with instructions

Environment

Package Version(s): 

Visual Studio
- [ ] 2019 (version: )
- [ ] 2019 Preview (version: )
- [ ] for Mac (version: )

Needs support for DoubleTapCoordinates

I'm submitting a Feature request:

Needs support for DoubleTapCoordinates

Current behavior

throws

Expected behavior

works

Minimal reproduction of the problem with instructions

Invoke DoubleTapCoordinates

Environment

Package Version(s): current
Visual Studio: irrelevant

[Wasm] Add support for DragCoordinate

I'm submitting a...

• Feature request

Current behavior

IApp.DragCoordinate not implemented for WebAssembly

Expected behavior

IApp.DragCoordinate works.

readme out of sync with sample

• Documentation issue or request

here https://github.com/unoplatform/Uno.UITest/blob/master/src/Sample/Sample.UITests/TestBase.cs we have

		protected IApp App
		{
			get => _app;
			private set
			{
				_app = value;
				Uno.UITest.Helpers.Queries.Helpers.App = value;
			}
		}

		[SetUp]
		public void StartApp()
		{
			App = AppInitializer.AttachToApp();
		}

but in the readme we have

	protected IApp App { get; set; }

	[SetUp]
	public void StartApp()
	{
		// Attach to the running application, for better performance
		App = AppInitializer.AttachToApp();
	}

I was just wondering if the readme should also have Uno.UITest.Helpers.Queries.Helpers.App being initialized? and if so would you like a PR for it?

SeleniumAppQuery doesn't implement IAppQuery.Index()

I'm submitting a...

• Feature request

Minimal reproduction of the problem with instructions

		private void ClickCheckBoxAt(int i)
		{
			_app.Marked("CheckBox").AtIndex(i).Tap();
		}

Current behavior

Throws NotImplementedException

Expected behavior

Taps i'th checkbox

Environment

Package Version(s): 

Visual Studio
- [ ] 2017 (version: )
- [ ] 2017 Preview (version: )
- [ ] for Mac (version: )

ChromeDriver not getting installed for newer Chrome versions

Attempting to run the WASM UI Tests with a fresh machine with Chrome above v114.0.5735.90 will not work.

chromedriver.exe is no longer available using the following method:

Uno.UITest/src/Uno.UITest.Puppeteer/SeleniumApp.cs

Lines 129 to 138 in 0b4d1ce

	// Chrome driver selection: http://chromedriver.chromium.org/downloads/version-selection
	var chromeDriverLatestVersionUri = $"https://chromedriver.storage.googleapis.com/LATEST_RELEASE_{chromeVersion.Major}";
	Console.WriteLine($"Fetching Chrome driver version for Chrome [{chromeRawVersion}]");
	var driverVersion = new WebClient().DownloadString(chromeDriverLatestVersionUri);
	var chromeDriverVersionUri = $"https://chromedriver.storage.googleapis.com/{driverVersion}/chromedriver_win32.zip";
	var tempZipFileName = Path.GetTempFileName();

Notice the alert on the ChromeDriver download page

We should now be using the Chrome for Testing availability Dashboard which provides a list of JSON API Endpoints for automation scripts to download the proper chromedriver versions

fyi @jeromelaban @nickrandolph: this was the cause of Rafael's issues with UITesting the WASM apps

Improve Screenshot speed

Feature request

Improve performance of screenshot which highly impacts the duration of the build on CI

Current behavior

App.TakeScreenshot takes at least 3 sec

Expected behavior

... much faster 🚀!

Minimal reproduction of the problem with instructions

Wait for a build of Uno

Can not run tests against WASM head: Chrome does not start

I'm submitting a...

• Bug report (I searched for similar issues and did not find one) -->

Current behavior

When creating a new App and adding the Uno.UITest projects, I would expect to be able to run a test against the WASM head. However Chrome never gets started. Updating the Uno.UITest.* dependencies to the latest pre-release does not fix this.

For some reason. cloning this repository and running tests agains the SampleApp work fine though, are the packages and the source code that different?

Expected behavior

Running against WASM should work.

Minimal reproduction of the problem with instructions

1. Create a new project
2. Add Uno.UITest project
3. Deploy app
4. Update Constants.cs
5. Try to run tests against WASM -> Chrome does not start
6. Update to 1.1.0-dev24 versions
7. Try to run tests again -> Chrome does not start

Environment

Package Version(s): 

Uno.UITest.* : 1.1.0-dev11 or 1.1.0-dev24 (both failed)



Visual Studio
- [x] 2019 (version: 16.10.0)
- [ ] 2019 Preview (version: )
- [ ] for Mac (version: )

IAppRect is updated

Bug report

IAppRect does not have the same behavior on WASM and Xamarin

Current behavior

On WASM the IAppRect is updated when you move the element, while on Xamarin it's an immutable object captured only once.

Expected behavior

IAppRect should be immutable on all platforms

Minimal reproduction of the problem with instructions

1. Get the rect of an element
2. Move it using a RenderTransform

Environment

Package Version(s): latest
Visual Studio: irrelevant

Invoking `IApp.Dispose()` does not terminate the webdriver

Invoking:

Uno.UITest/src/Uno.UITest.Puppeteer/SeleniumApp.cs

Line 48 in 9669fd2

_driver.Close();

does not terminate the web driver, which can cause issues in multi-build environments.

Remove "Delete after 3.5 release" comment in readme

I'm submitting a...

• Documentation issue or request

Current behavior

Readme states "remove this block after Uno.UI 3.5 is released" here which happened now so it can be released.

Expected behavior

Probably best to not point it (out that way) since 3.5 was released.

Minimal reproduction of the problem with instructions

N/A

Environment

Package Version(s): 
N/A
Visual Studio
- [ ] 2017 (version: )
- [ ] 2017 Preview (version: )
- [ ] for Mac (version: )

[Wasm] Add support for Raw queries

I'm submitting a...

• Feature request

Current behavior

Wasm does not allow for selecting arbitrary items in the visual tree, e.g. items in a list view.

Expected behavior

It is possible to select items.

Minimal reproduction of the problem with instructions

https://github.com/unoplatform/Uno.Xamarin.Forms.Platform/blob/7005a07037ca4cca531403da02697ca8fea45efa/Xamarin.Forms.Core.Windows.UITests/WinQuery.cs#L25

Environment

Package Version(s): 

Visual Studio
- [ ] 2017 (version: )
- [ ] 2017 Preview (version: )
- [ ] for Mac (version: )

Improve support & documentation for UI testing on a mac

I'm submitting a...

• Documentation issue or request

Current behavior

UI Testing on a Mac is difficult and poorly documented or supported.
I have previously gotten it to work but updates (that I'm not entirely sure about) mean it no longer does 😢.

It seems that there are many factors that can influence how/if this works and having the right combination of library versions to match the versions of OS X, XCode, VS4Mac, & more(?) are required.

There is also a lack of resources to help debug related issues.

Expected behavior

Documentation of how to run UI tests on a mac would be really helpful.
Ideally, these will include:

• A guide to UI testing on a mac (how to get started, what settings are needed, etc.)
• Known good configurations of libraries, tooling, and OS versions.
• Debugging guide for when things don't work as desired/expected.
• Documentation of the scripts used for UI testing in the main Uno project.

Minimal reproduction of the problem with instructions

n/a

Environment

Package Version(s): 

Visual Studio
- [ ] 2019 (version: )
- [ ] 2019 Preview (version: )
- [x] for Mac (version: 8.9.7)

Handle Chrome in Program Files folder

I'm submitting a...

• Feature request

Current behavior

UI tests expect Chrome.exe in Program Files (x86). When Chrome is installed in Program Files, the following error appears:

Unable to determine the chrome driver version. 
The used path was [C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe], found [].

Expected behavior

Both x86 and non-x86 Program Files could be handled automatically.

Minimal reproduction of the problem with instructions

Environment

Package Version(s): 

Visual Studio
- [ ] 2017 (version: )
- [ ] 2017 Preview (version: )
- [ ] for Mac (version: )

expose the Selenium RemoteWebDriver

• Feature request

I would like to access to underlying RemoteWebDriver from SeleniumApp to unlock the full features of the Selenium API

Current behavior

SeleniumApp does not expose the RemoteWebDriver field _driver.

Expected behavior

expose the RemoteWebDriver field _driver.

Android and IOS setup documentation

I'm submitting a documentation request

Looking for documentation on setting up UI tests for Android and IOS but I couldn't find docs any in this repo
Also is Uno.UITest supported in AppCenter Test?

Running Browser tests using Uno.UiTest outside of development code

I'm submitting a...

Sample app request
Documentation issue or request

Current behavior

Uno.UiTest has documentation regarding how to create tests inside of the same solution where host code lies, but in a lot of cases, UI Automation developers are a part of QA and create their test code elsewhere in a solution outside of development

Expected behavior

If this is possible, have examples of how you could have a standalone Uno.UiTest solution to execute browser based tests

Sorry if this is not allowed, I tried asking a question on the Stackoverflow forum but no response and my team has invested a lot of time in using Selenium to create tests against our uno brower version.

The original SO post --> https://stackoverflow.com/questions/69976830/running-browser-tests-using-uno-uitest-outside-of-development-code

[WASM] IAppRect might be offseted by one pixel

Bug report

[WASM] IAppRect might be offseted by one pixel

Current behavior

On WASM when we get the Rect of an element in the UI tests, the actual content might by offseted by 1 pixel.

Expected behavior

When rect is X=100 an Y=100 the pixel at 100,100 should be the colored by the background of the control!

Minimal reproduction of the problem with instructions

Remove .WithPixelTolerance(x: 1, y: 1) in DoubleAnimation_Tests.TransformGroup.cs

Environment

Package Version(s): latest
Visual Studio: irrelevant

EnterText does not work on Android emulator

I'm submitting a...

• Bug report (I searched for similar issues and did not find one) -->

Current behavior

(at least on my machine): When creating a new app that contains a TextBox and trying to use EnterText on it does not actually enter text when using the emulator. When running the app on a real device, EnterText works though.

Expected behavior

EnterText should enter text regardless if using emulator or not.

Minimal reproduction of the problem with instructions

Environment

Package Version(s): 

Uno.UITest.*: 1.1.0-dev11

Visual Studio
- [x] 2019 (version: 16.10.0)
- [ ] 2019 Preview (version: )
- [ ] for Mac (version: )

Selenium 4.3 and later breaks TapCoordinates

For some unknown reason, Selenium 4.3 and later breaks TapCoordinates:

Uno.UITest/src/Uno.UITest.Puppeteer/SeleniumApp.cs

Lines 302 to 308 in 7befa53

	void IApp.DoubleTapCoordinates(float x, float y)
	{
	PerformActions(a => a
	.MoveToElement(_driver.FindElement(By.TagName("body")), 0, 0)
	.MoveByOffset((int)x, (int)y)
	.DoubleClick());
	}

Tap with an explicit element is not impacted.

[WASM] Collapsed view is still considered present

I'm submitting a...

• Bug report (I searched for similar issues and did not find one)

Minimal reproduction of the problem with instructions

Set element TargetView to Visibility = Collapsed.

Call _app.WaitForNoElement("TargetView"); in test.

Current behavior

Call will time out on WASM.

Expected behavior

When view is collapsed it will be considered 'not there', matching iOS and Android.

Environment

Package Version(s): 

Visual Studio
- [ ] 2017 (version: )
- [ ] 2017 Preview (version: )
- [ ] for Mac (version: )

XamarinAppRect does not implement value comparison.

I'm submitting a...

• Feature request

Current behavior

XamarinAppRect does not implement value comparison, only the object's default referential comparison.

Expected behavior

XamarinAppRect should implement value comparison.

Minimal reproduction of the problem with instructions

var target = _app.Marked("TargetTextBox");
var rect1 = target.FirstResult().Rect;
var rect2 = target.FirstResult().Rect;

rect1.Should().Be(rect2); // fails