flaui / uiautomation-interop

Interop wrappers for UIAutomationCore and UIAutomationClient

Badges

What	Badge
Chat
Build
Tests
Libraries (NuGet)
CI Artefacts	FlaUI CI

Introduction

FlaUI is a .NET library which helps with automated UI testing of Windows applications (Win32, WinForms, WPF, Store Apps, ...).
It is based on native UI Automation libraries from Microsoft and therefore kind of a wrapper around them.
FlaUI wraps almost everything from the UI Automation libraries but also provides the native objects in case someone has a special need which is not covered (yet) by FlaUI.
Some ideas are copied from the UIAComWrapper project or TestStack.White but rewritten from scratch to have a clean codebase.

Sponsoring

If you appreciate my work, feel free to support me by sponsoring on github or with a one-time payment over at PayPal.

Why another library?

There are quite some automation solutions out there. Commercial ones like TestComplete, Ranorex, CodedUI just to name a few. And also free ones which are mainly TestStack.White.
All of them are based on what Microsoft provides. These are the UI Automation libraries. There are three versions of it:

• MSAA
  • MSAA is very obsolete and we'll skip this here (some like CodedUI still use it)
• UIA2: Managed Library for native UI Automation API
  • UIA2 is managed only, which would be good for C# but it does not support newer features (like touch) and it also does not work well with WPF or even worse with Windows Store Apps.
• UIA3: Com Library for native UI Automation API
  • UIA3 is the newest of them all and works great for WPF / Windows Store Apps but unfortunately, it can have some bugs with WinForms applications (see FAQ) which are not existent in UIA2.

So, the commercial solutions are mostly based on multiple of those and/or implement a lot of workaround code to fix those issues. TestStack.White has two versions, one for UIA2 and one for UIA3 but because of the old codebase, it's fairly hard to bring UIA3 to work. For this, it also uses an additional library, the UIAComWrapper which uses the same naming as the managed UIA2 and wraps the UIA3 com interop with them (one more source for errors). FlaUI now tries to provide an interface for UIA2 and UIA3 where the developer can choose, which version he wants to use. It should also provide a very clean and modern codebase so that collaboration and further development is as easy as possible.

Usage

Installation

To use FlaUI, you need to reference the appropriate assemblies. So you should decide, if you want to use UIA2 or UIA3 and install the appropriate library from NuGet. You can of course always download the source and compile it yourself.

Usage in Code

The entry point is usually an application or the desktop so you get an automation element (like a the main window of the application). On this, you can then search sub-elements and interact with them. There is a helper class to launch, attach or close applications. Since the application is not related to any UIA library, you need to create the automation you want and use it to get your first element, which then is your entry point.

using FlaUI.UIA3;

var app = FlaUI.Core.Application.Launch("notepad.exe");
using (var automation = new UIA3Automation())
{
	var window = app.GetMainWindow(automation);
	Console.WriteLine(window.Title);
	...
}

using FlaUI.Core.AutomationElements;
using FlaUI.UIA3;

// Note: Works only pre-Windows 8 with the legacy calculator
var app = FlaUI.Core.Application.Launch("calc.exe");
using (var automation = new UIA3Automation())
{
	var window = app.GetMainWindow(automation);
	var button1 = window.FindFirstDescendant(cf => cf.ByText("1"))?.AsButton();
	button1?.Invoke();
	...
}

Further Resources

YouTube Tutorials

Have a look at H Y R Tutorials. This channel provides some videos to get you started with FlaUI.

FlaUI UITests

FlaUI itself contains quite some UI tests itself. Browse to the code of them here and look how they work.

Chat

Head over to the chat to ask your specific questions.

Contribution

Feel free to fork FlaUI and send pull requests of your modifications.
You can also create issues if you find problems or have ideas on how to further improve FlaUI.

Donors and Sponsors

• Thank you Gehtsoft USA LLC for the generous donation

Acknowledgements

JetBrains

Thanks to JetBrains for providing a free license of ReSharper.

AppVeyor

Thanks to AppVeyor for providing a free CI build system for FlaUI.

TestStack.White

Thanks to the creators and maintainers (especially to @JakeGinnivan and @petmongrels) for their work and inspiration for this project.

Microsoft

Thanks to Microsoft for providing great tools which made developing this project possible.

uiautomation-interop's Issues

I can't Find first Parent element by use IUIAutomationElement.FindFirst() or IUIAutomationElement3.FindFirst() .. and so on

At first: I created a CUIAutomationClass or CUIAutomation8Class instance and cacheRequest

            uia = new CUIAutomationClass();
           cacheRequest = AutomationHelper.UIA.CreateCacheRequest();
            cacheRequest.AddProperty(UIA_PropertyIds.UIA_ClassNamePropertyId);
            cacheRequest.AddProperty(UIA_PropertyIds.UIA_NamePropertyId);
            cacheRequest.AddProperty(UIA_PropertyIds.UIA_ControlTypePropertyId);
            cacheRequest.AddProperty(UIA_PropertyIds.UIA_ProcessIdPropertyId);
            cacheRequest.AddProperty(UIA_PropertyIds.UIA_BoundingRectanglePropertyId);
            cacheRequest.TreeScope = TreeScope.TreeScope_Element;

Second: I get a element from Desktop MousePoint

           User32.GetCursorPos(out POINT point))                
           element = uia.ElementFromPointBuildCache(new tagPOINT { x = point.X, y = point.Y }, cacheRequest);

Tips：right now ,the element is a Notpad.exe's " Edit Area"
Then: here is the problem coming....
I want to get the parent of the element, so I code like this:

        var res = element.FindFirst(TreeScope.TreeScope_Parent,AutomationHelper.UIA.ControlViewCondition);

The res is always null event through I Change to TreeScope.TreeScope_Ancestors, niether convert the element to IUIAutomationElement3 too.

SomeBody who knows ?
Thanks all

Confusing issue regarding tagRECT

So, I am currently programming an application that automatically opens, screenshots, and closes .nif files by opening the files in a program called NifSkope that you can find on here (specifically, Nifskope dev 2 version 7). TLDR: I need to allow the user to automatically adjust the camera in Nifskope because the .nifs don't always load in an ideal position. That means I need access to the buttons.
So I have written code to retrieve each button in an element array, but for some confusing reason, when I attempt to get the bounding rectangle property of a button, instead of returning a tagRECT, it returns an array of doubles. The documentation says that tagRECTS has four integer values, so it's confusing as to why it would return a double array. But it's a bad thing-integers are not floating point numbers, but are used as parameters for both points, rectangles and sizes in c#, for one. Doubles aren't. For another, it's causing a runtime exception. I'd appreciate some explanation as to what is occurring. I'm writing this in c# using windows forms.

CUIAutomation8Class.ElementFromPointBuildCache(....) throw out a UnauthorizedAccessException

I started a Win32 App, and it can be found by use inspect.exe.
but when I access it in my C# code by use CUIAutomation8Class.ElementFromPointBuildCache([this is screen point]),
It throw me a UnauthorizedAccessException,
Anybody know the reason?