microsoft / powerapps-testengine

Power Apps Test Engine

This is currently an experimental project.

Overview

Power Apps Test Engine is an open source project with the aim of providing makers with a single automated testing platform for all Power Apps apps. Test Engine has the following benefits:

• Power Fx test authoring - Makers can author tests in YAML format using the familiar Power Fx language.
• DOM abstraction - Tests are authored using references to control names that are defined at design-time. Test authors do not need to write JavaScript, and do not need to be familiar with the browser DOM of the app's rendered output.
• Connector mocking - Test authors can optionally create mocks of network calls, typically used when Power Apps make calls to connectors. This allows the app to be tested without modification to the app itself while avoiding any unwanted side-effects of the external APIs.
• Screenshot and video recording support - Test Engine can take screenshots at any point during your test execution, and records videos of the test run. This can be very helpful to diagnose failed tests and to understand what the actual experience of the failed test case was.

Build this project using the instructions below. This will create a local executable that can be used to run tests from your machine.

Test Engine uses Playwright to orchestrate the tests.

Test Engine currently supports Power Apps canvas apps.

Getting Started

To get started, you will need to clone the Test Engine code from GitHub, locally build the project, and install the browser(s) you wish to use to execute the tests. Once you have the Test Engine executable built, the repo contains a library of sample test plans and apps you can use to exercise the tool.

Prerequisites for building Test Engine

1. Install .NET Core 6.0.x SDK
2. Ensure that your MSBuildSDKsPath environment variable is pointing to .NET Core 6.0.x SDK.
3. Make sure PowerShell is installed.

Build locally

Run the commands below in PowerShell. These commands will clone the repo to your desktop, will build the executable, and will install browser prerequisites needed for the tests to execute.

# Clone GitHub repo
git clone https://github.com/microsoft/PowerApps-TestEngine.git

# Change to the PowerAppsTestEngine folder
cd PowerApps-TestEngine\src\PowerAppsTestEngine

# Build
dotnet build

# Install browsers required by Playwright
.\bin\Debug\PowerAppsTestEngine\playwright.ps1 install

If you face trouble running dotnet build try Clean locally first.

Clean locally

Run the command below in PowerShell to clean untracked files. Please make sure to run this in the root folder.

# Clean
.\clean.cmd

Once done, continue following Build locally

Using the provided samples

Test Engine includes a library of samples you can use to get started. Each sample consists of solution that you can directly import into your Power Apps environment, and a test plan file that corresponds to the Power Apps canvas app that is in the solution.

To use the samples, import the solution into your environment and then execute the corresponding test plan file with Test Engine.

See Samples Introduction for more sample solutions.

1. Import a sample solution

Choose one of the packaged solutions within the Samples folder (e.g., PowerApps-TestEngine\samples\basicgallery\BasicGallery_1_0_0_3.zip). You can import solutions using the Dataverse portal, or by using the PAC CLI command line tool. Remember the environment that you imported the solution to.

2. Set up the config file

You can use a config file to specify values needed to run the tests, or you can provide these same values on the command line when executing the app. This example shows using the config file.

Create a config.dev.json file inside the PowerAppsTestEngine folder.

Here is an example of its contents (a file config.json is provided in the repo as an example):

{
  "environmentId": "",
  "tenantId": "",
  "testPlanFile": "",
  "outputDirectory": ""
}

Fill in the required properties:

• environmentId: The ID of the Environment into which you imported the solution
• tenantId: The ID of your tenant

Both environmentId and tenantId can be found by opening the Settings > Session details dialog from within the environment:

• testPlanFile: Path to the test plan YAML filethat you wish to run. (e.g., ../../samples/basicgallery/testPlan.fx.yaml)
• outputDirectory: Path to folder where test output/results will be placed.

For more information about the config and the inputs to the command, please view this link.

Set up user authentication

This refers to the account that Test Engine will use to execute the test.

Test Engine does not support multi-factor authentication. Use an account that requires only a username and password to sign in for your tests.

Test credentials cannot be stored in test plan files. Rather, they are stored in PowerShell environment variables. The test plan file contains references to which environment variables are used for credentials. For example, the following snippet indicates that the user1Email and user1Password environment variables will be used:

environmentVariables:
  users:
    - personaName: User1
      emailKey: user1Email
      passwordKey: user1Password

Please view the YAML/Users reference page for more information.

Run test

Once the config.dev.json and credentials are configured, you are ready to run the test. Use the following command:

# Run test
dotnet run

When the run is complete, check the folder specified in the outputDirectory configuration setting for test run results.

Check Samples Introduction for more sample solutions.

Languages and regions that use period as the decimal separator

The syntax of Power Fx can differ based on your system's language settings. Use the locale property in the testSettings section of the test plan file to specify the locale in which your Power Fx is written. (See locale in test settings). This is useful if you are working across regions that use different decimal or thousands separators. For example, , instead of . for decimals and ; instead of ,.

See the following samples that have the locale property specified as examples of its usage:

1. ; instead of , for separator - testPlanForRegionUsePeriodAsDecimalSeparator.fx.yaml
2. , instead of . for separator - testPlanWithCommaForDecimal.fx.yaml.fx.yaml

NOTE: Since the integration tests that use these samples run in an en-US locale, the testPlanWithCommaForDecimal sample represents absolute numbers with decimals as , but their string representations of the decimals are still . to match the locale these tests are running in. Additionally, this is also because the corresponding sample app is also hosted in an en-US locale. But for real-world applications make sure the syntax of the tests represent the locale for the app and environment as well.

What to do next

Option 1 - Author your own test plan: Modify the testPlan.fx.yaml of a provided sample to run tests created on your own. You can also modify the sample Power App apps and create new tests for your updated app. Check Power Fx for writing functions. The sample test plan will be here.

Option 2 - Download recorded tests from Test Studio: If you have tests that you have recorded in Test Studio, you can download them from Test Studio to reuse in Test Engine.

• Make use of the "Download suite" button available in Test Studio to download the test plan. Choose the test suite to download if you have multiple test suites.
• Alternatively you can make use of the "Download" button available under each test suite.
• Make sure you update the config file and user configurations if you are using a different tenant or environment for this app.
• Now you should be ready to run the test with dotnet run

More about the test plan

Yaml Format

Power Fx

How apps are referenced in test plan files

The way that the test plan files refer to the target app differs depending on whether or not the app is in a solution or not. We recommend using solutions whenever possible because they provide greater portability.

• Working with apps within Solutions - Test plan files for apps that are part of solutions are portable across environments. For solution-based apps, the test plan refers to the target app with a logical name (the appLogicalName property) which does not change if the app moves to a different environment.

  1. Locate the App Logical name for the app
     1. In the Solutions tab, open the solution that contains the app
     2. Select Apps
     3. Note the Name column. It is the app logical name (Not the Display name)
  2. Update your test plan file
     1. Open the test plan YAML file for the app
     2. Fill in the appLogicalName value with the new App logical name

• Working with apps outside of Solutions - If you move an app that is not part of a solution to a new environment, you will need to manually update the test plan file to refer to the app. How to update a test plan file for a non-solution based app:

  1. Locate the App ID for the app in its new location
     1. In the Apps list, locate the app and open the context menu
     2. Select Details
     3. Note the App ID GUID on the Details pane
  2. Update your test plan file
     1. Open the test plan YAML file for the app
     2. Fill in the appId with the new App ID

Power Platform CLI usage

To get started, download and install the Microsoft Power Platform CLI.

Review this documentation regarding the execution of Test Engine in Power Platform CLI here.

Known limitations

See the Test Engine documentation for known limitations.

Frequently asked questions

1. The test execution failed. What should I do?

We suggest checking the logs and recording in the test result folder to see what caused the error. Common issues are listed below:

• Entering incorrect authentication information
• Requiring authorization to certain controls or features (you will need to manually open the app and grant the access before using test engine to run tests on the app)
• Having a bad network connection (which causes Test Engine to not load the app)

Occasionally, you might get a timeout error due to the app taking longer to load than the default 30 second timeout. Most of the time, re-running the program will solve this problem. If this error still happens, you will probably want to check the recording as mentioned. If your app takes a while to load, you can also modify the timeout limit in test settings to give it more time.

If these steps don't help, you can run

# Run test with trace logs
dotnet run -l trace

and ask for support by adhering to this.

2. What is the difference between the settings passed in via command line/config.json vs settings located inside the YAML?

The settings passed in via command line or config.dev.json are settings that either start off the test (link to the test plan) or they are settings that are likely to change due to the environment the app being test in is located.

Settings located in the YAML should be able to be "imported" with the solution, so another person could take the solution and corresponding test plan and use the two of them without any modifications.

Example: environmentId changes if the app is imported to a another tenant/environment, and so it is located as a command line or config.json setting.

How to report issues

You are invited to contribute corrections to both code and documentation. See the below section entitled "Contributing to Test Engine code and documentation" for further info. You may also file GitHub issues for Test Engine to make Microsoft and the community aware of issues.

Contributing to Test Engine code and documentation

This project welcomes contributions and suggestions to both code and documentation. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

Note: We are not accepting contributions for content within the JS folder.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

powerapps-testengine's Issues

Login goes to time out on the organization page

I am trying to set up automation testing for our projects and I started with samples tests provided, starting with the basic gallery.

For sign in I am taken to the organization login page and the login goes into timeout.

debugLogs.txt

ae10575b97a223e805108b740663e62d.webm

Allow for queryParams in config.json

We now allow for queryParams in the commandline, but should in the future allow in config.json as well.

Handle screen navigation

Test what happens when you navigate to a different screen and want to test controls in a different screen. It is likely that we will need to fetch a new set of control object models.

Exit criteria:

• Support navigating to other screens in a test
• Samples provided
• Docs updated if necessary

Implement timeout

https://github.com/microsoft/PowerApps-TestEngine/blob/main/src/Microsoft.PowerApps.TestEngine/PowerApps/PowerAppFunctions.cs
In all the places where there is a comment to implement a timeout, respect the timeout property in the TestSettings (or use default) and exit the while loop if the timeout is reached.

This is to prevent an infinite loop if the Power App is stuck loading.

Exit criteria

• Infinitely loading Power App should fail the test rather than be stuck forever
• Add to the testSettings documentation, information about the timeout property

Fill in support.md

Determine our support channels and fill in support.md with correct information

Exit criteria:

• Support.md updated

Create github action for running YAML tests

Create a github action to run the YAML tests. This should run on every PR and merge to master.

It should run through all the apps in the samples.

Exit criteria:

• Sample tests are run as part of the PR build

Support nested galleries

Exit criteria

• Able to test nested galleries
• Docs updated if necessary
• Sample provided

Not all Date properties work for Wait()

Certain date properties do not work as of now, due to some type ambiguity.

The 'DefaultDate' property works, because we are handling DateTimeValues.
However, 'SelectedDate' does not, because we are not handling DateValues.

The goal would be to handle both of these, yet we cannot because, when we create a Wait() function overload for both FormulaType.DateTime (which has a DateTimeValue) and FormulaType.Date (which has a DateValue), FormulaType.Date uses it's correct overload, but FormulaType.DateTime uses the FormulaType.Date overload.

Once PowerFX is changed in a way that FormulaType.DateTime will not consider itself a FormulaType.Date, we will be able to overload and use both successfully.

Support containers

Ensure that controls inside containers can be tested

Exit criteria:

• Samples updated
• Controls inside containers (and containers inside galleries/components) can be tested
• Docs updated if applicable

Sample Apps fail on Assert. The formula contains 'ParenClose' where 'Error' is expected

Hi,

I've just downloaded and build the repo. I have currently tested 3 different apps.

The error I get is on each test case the same:
[Error]: System.AggregateException: One or more errors occurred. (Errors: Error 48-49: Unexpected characters. The formula contains 'ParenClose' where 'Error' is expected.
Error 48-49: Unexpected characters. Characters are used in the formula in an unexpected way.)

Log: debugLogs.txt

I can see that the screenshot action and the Select action are working correctly. The calculator app is opened and played correctly. All buttons are pressed as you can see in the attached movie.

bee8d2cb2dab61aa1fb7f6c5e20c79c5.webm

Implement Power FX overlay

We have a way in the test settings to enable the Power FX overlay that highlights the command currently being run. The setting is currently not being read. We need to implement the overlay to display the command.

This is pending a Power FX update in order to support this.

Exit criteria:

• If Power FX overlay is enabled, the overlay is displayed to indicate the command that is being run.

Update Wait function syntax

Wait function is currently like this, Wait(Label1, "Text", "hello")

We would like it to be like this, Wait(Label1.Text = "hello")

This requires some changes on the Power FX side.

Exit criteria:

• Wait function updated to new syntax
• Samples updated
• Docs updated

Localization

Make localization work with Test Engine

Handle number of tests running at a time

In https://github.com/microsoft/PowerApps-TestEngine/blob/3ee79f3d236ae22130b8ef790a22023d7220c064/src/Microsoft.PowerApps.TestEngine/TestEngine.cs, we currently run all tests in parallel.

We should handle a max number of tests being run in parallel and handle queueing where necessary

Exit criteria:

• Able to specify the number of tests to run in parallel (with a default being some number of tests -> follow playwright config for guidance)
• Update documentation on how to specify the number of tests to run in parallel.

Timed out during login attempt

I have used 2 different samples, Calculator and Many Screens, and I am getting the same issue on both. The recording shows the login process working successfully, however the debugLogs state this did not work. Nothing then happens after the login process and none of the steps are actioned.

We have tried both UK and US sandbox environments.

debugLogs.txt

Race condition between registering TestEnginePlugin and making a request.

Encountered a race condition between registering the plugin and making a request.

Test Engine is taking dependencies on some Player/Runtime internal implementation details (e.g. DOM elements, events, etc) that will most likely break going forward with various projects that the Player and Runtime teams are working on.

PowerApps-TestEngine/CanvasAppSdk.js at main · microsoft/PowerApps-TestEngine (github.com)

We have already had a plan for the long-term integration of the js into the web player/published app. We should make sure that the plan we come up with is in sync with projects that are ongoing

Update JS SDK to new model reviewed by the team

Object model:
interface IPowerAppObjectModel {
IPowerAppControlObject[] Controls;
}

interface IPowerAppControlObject {
string Name;
IPowerAppProperty[] Properties;
IPowerAppObject[] Children;
int? ItemCount;
}

interface IPowerAppControlProperty {
string Name;
PowerAppType Type;
}

Item path:
interface IItemPath {
string controlName;
int? index;
IItemPath? childControl;
string? propertyName;
}

Functions:
getAppStatus()
buildObjectModel()
getValue(itemPath)
setValue(itemPath, inputValue)
select(itemPath)

[Error]: Access to the path '/Users/User/PowerApps-TestEngine/samples/basicgallery' is denied

I am trying to set up TestEngine but I ran into the following error while running command "dotnet run" to start the tests.

Unhandled exception. System.UnauthorizedAccessException: Access to the path '/Users/User/PowerApps-TestEngine/samples/basicgallery' is denied.
--->System.IO.IOException: Permission denied
--- End of inner exception stack trace ---

I set the variables in Power Shell to the email and password I use for PowerApps.

I understand this is an access/rights issue, but I don't know how to set the right access.
This is my personal machine I am using and TestEngine folder is stored locally.

I am running the command from where the project is located - /Users/User/PowerApps-TestEngine/src/PowerAppsTestEngine .
I am running it on a Mac. I tried to run it as root and got the same error message.

Review logs generated

Ensure we log enough to aid us in debugging
Build config for user to disable the logging if they so choose

Exit criteria:

• User can disable the logs
• More verbose logging
• Make Sure Logs are Readable
• Should we log before logger is created?
• Should we throw exceptions even after logging an error?
• Confirm logs are adequate for figuring out a test failure in 95% of cases

Timeline of development and multiple projects

Hi,

Is there a timeline of when new features will be ready? For example when will we it be possible to select controls inside components?

Also, how can I best approach test automation on multiple projects? Like how do I fill this in in the config/testplan?

Thanks in advance.

Kr,
Jordi

[Open discussion] Model-driven apps support

Hi team 👋🏼

First, thank you for the amazing work you have already done on this project 👍🏼

Based on the introduction of the Overview section of the README file of the project, I am making the assumption that one of your goals for this project is to also cover model-driven apps.

I have done some tests and explored a bit the source code of PowerApps-TestsEngine the day the announcement blog article was published, and I arrived at the conclusion that currently the tool does not cover model-driven apps (or even cover only canvas apps).

First, am I wrong or PowerApps-TestsEngine does not currently cover model-driven apps?
If the answer is "You are wrong, model-driven apps are already covered", I will be really happy but will need some help to make it work 😅 The good news in this situation would be that we would be able to quickly close this issue 😉

But if you confirm I am not wrong and model-driven are not already covered, could you provide some answers to the question below please?

• Is model-driven apps' support already in your backlog or even in your roadmap? If yes, do you have (even at a high level) an ETA for this part?
• Are you looking for some contributions / help to help you bring model-driven apps support to PowerApps-TestsEngine?

Thank you again for all your work on this project and also for taking the time to take a look to this issue.

Build supported control matrix

Create a matrix of supported controls document

• List out all controls currently in Studio
• List out whether the control is testable
• Include whether control can be tested within components and galleries
• Include sample (if applicable)

This issue is to just create the matrix. Consider breaking up additional issues for actually doing the work to validate each control and update the matrix.

Exit criteria:

• Supported controls matrix document created
• Test plan created for all supported controls

Support components

Able to interact with input/output properties of components and the internal controls within the components as well.

Exit criteria:

• Components can be tested
• Samples provided
• Documentation updated

Mimic playwright config for video and screenshots

Playwright config has settings to only record video and screenshots on failure, all the time or never. We should follow the same options as well.

Exit criteria:

• Additional options provided for video and screenshots
• Docs updated
• Samples updated -> samples should be updated to only record on failure so we don't use as much storage space

Have to wrap control properties with Text() before using it

Example: Assert(Text(Label1.Text) = "1", "Counter should be incremented to 1");

This is a limitation of Power FX. If Power FX is able to support detecting that Label1.Text can be directly compared with "1", that would prevent the need for this extra code.

Exit criteria:

• No need for Text() to wrap control properties
• Update docs
• Update samples

Create nuget for Microsoft.PowerApps.TestEngine dll

Create publishing pipeline to publish a nuget for Microsoft.PowerApps.TestEngine.dll

Exit criteria:
Nuget uploaded to nuget feed.

dotnet build fails error NU1301: Unable to load the service index for source https://api.nuget.org/v3/index.json.

WHen following script to build the test engine (donet build) the scrip continually fails error NU1301: Unable to l
oad the service index for source https://api.nuget.org/v3/index.json.

Support PCF

Able to test input/output properties of PCF.
Out of scope: Ability to modify internal PCF controls states

Exit criteria:

• Test PCF controls
• Samples updated
• Documentation updated

Support test suites

Enable running suites of tests

Exit criteria:

• Able to run a suite of different tests
• Documentation updated
• Samples provided

Support before and after setup for tests

Common test practices have a before setup and after teardown.

Exit criteria

• Before and after steps supported
• Samples updated
• Docs updated

Figure out console logging

Once the test logging provider is registered, nothing gets logged to the console (See https://github.com/microsoft/PowerApps-TestEngine/blob/3716ea76accf0ed1e85f6dffe995981094b94606/src/PowerAppsTestEngine/Program.cs).

Figure out why this is happening. Absolute worst case, we also can implement our own console logging in the test logging provider.

Exit criteria:

• Logs get surfaced in the console

[Question] Integration with Azure DevOps / Pipeline

Hello,

I'm in the middle of selecting test framework for canvas app.
Looks like this project covers all my needs - easy to use, tests from studio are portable, simple configuration.

My next step is to integrate it with CI in Azure DevOps.
I couldn't find any instruction how to do this correctly.
Have you tired this so you can provide some simple guid?

Make SetProperty() work for Galleries

Make SetProperty(), that was added in PR #38 to solve issue #5 , work for Galleries.

Acceptance Criteria:

• Works for Galleries
• Works for Nested Galleries
• Unit Tested

Move JS SDK into Power Apps

Move the javascript SDK into Power Apps code base

Exit criteria:

• Min version of Power Apps supported documented
• All JS in this repo deleted

Support network request mocking

Add a section in the test that enables network request mocking.

Suggested format:

test:
mocks:
- request: "url of request"
data: "path to mock data"

Ensure dataverse and sharepoint are supported

Exit criteria:

• Dataverse and Sharepoint connectors can be mocked
• Documentation updated
• Samples provided

Support galleries

Able to interact with controls within galleries

Exit criteria:

• Controls inside galleries can be tested
• Samples updated
• Documentation updated

System AggregateException when trying to run sample test

Describe the bug

When trying to execute dotnet run after setting up the test sample, it provides the System AggregateException with the following message:

[Error]: System.AggregateException: One or more errors occurred. (Errors: Error 39-40: Unexpected characters. The formula contains 'ParenClose' where 'Error' is expected.
Error 39-40: Unexpected characters. Characters are used in the formula in an unexpected way.)
 ---> System.InvalidOperationException: Errors: Error 39-40: Unexpected characters. The formula contains 'ParenClose' where 'Error' is expected.
Error 39-40: Unexpected characters. Characters are used in the formula in an unexpected way.
   at Microsoft.PowerFx.CheckResult.ThrowOnErrors()
   at Microsoft.PowerFx.RecalcEngine.EvalAsync(String expressionText, CancellationToken cancel, RecordValue parameters, ParserOptions options)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at System.Threading.Tasks.Task`1.get_Result()
   at Microsoft.PowerFx.RecalcEngine.Eval(String expressionText, RecordValue parameters, ParserOptions options)
   at Microsoft.PowerApps.TestEngine.PowerFx.PowerFxEngine.Execute(String testSteps) in C:\Users\CAS00\PowerApps-TestEngine\src\Microsoft.PowerApps.TestEngine\PowerFx\PowerFxEngine.cs:line 123
   at Microsoft.PowerApps.TestEngine.PowerFx.PowerFxEngine.ExecuteWithRetryAsync(String testSteps) in C:\Users\CAS00\PowerApps-TestEngine\src\Microsoft.PowerApps.TestEngine\PowerFx\PowerFxEngine.cs:line 70
   at Microsoft.PowerApps.TestEngine.SingleTestRunner.RunTestAsync(String testRunId, String testRunDirectory, TestSuiteDefinition testSuiteDefinition, BrowserConfiguration browserConfig, String queryParams) in C:\Users\CAS00\PowerApps-TestEngine\src\Microsoft.PowerApps.TestEngine\SingleTestRunner.cs:line 142

To Reproduce

Steps to reproduce the behavior:

1. dotnet run on BasicGallery_1_0_0_2

Expected Behavior: Correctly running the tests
Actual Behavior: System AggregateException

Additional context

Theres no issue when the engine is trying to set up browser, test infrastructure or navigating to the correct URL. It happens specifically when trying to run the test cases.
I've attempted changing properties in the sample on PowerApps website, and changing the test steps to be more forgiving. Even when trying to run the test cases on Test Studio, theres no issues whatsoever. It only occurs in Test Engine.
The sample is: BasicGallery_1_0_0_2. I havent tried using any other samples yet.

Make SetProperty() work for Components

Make SetProperty(), that was added in PR #38 to solve issue #5 , work for Components.

Acceptance Criteria:

• Works for Components
• Unit Tested

Implement SetProperty function

Implement a SetProperty function so that you can set TextInput.Text

Example: SetProperty(TextInput.Text, "Hello")

Exit criteria:

• SetProperty function implemented for TextInput
• Documentation updated
• Samples provided

changing config domain value to make.powerapps.com failed

Changing make.powerapps.com as the domain in the config file broke the test with a page not found in the portal

*screenshot of the recording*

giving a timeout as a result:

config file
{
"environmentId": "694072a8-690a-xxxxxxxxxxxxxxxxxx",
"tenantId": "ced2c552-7d1f-4731-xxxxxxxxxxxxxxx",
"testPlanFile": "../../samples/calculator/testPlan.fx.yaml",
"outputDirectory": "../../samples/calculator/",
"logLevel": "",
"domain": "make.powerapps.com",
"queryParams": ""
}

[Error]: Waiting timed out.

[Error]: System.TimeoutException: The operation has timed out.
at Microsoft.PowerApps.TestEngine.Helpers.PollingHelper.CheckIfTimedOut(DateTime startTime, Int32 timeout, ILogger logger) in C:\Users\carlo\testengine\PowerApps-TestEngine\src\Microsoft.PowerApps.TestEngine\Helpers\PollingHelper.cs:line 73
at Microsoft.PowerApps.TestEngine.Helpers.PollingHelper.PollAsync[T](T value, Func2 conditionToCheck, Func1 functionToCall, Int32 timeout, ILogger logger) in C:\Users\carlo\testengine\PowerApps-TestEngine\src\Microsoft.PowerApps.TestEngine\Helpers\PollingHelper.cs:line 44
at Microsoft.PowerApps.TestEngine.PowerApps.PowerAppFunctions.LoadPowerAppsObjectModelAsync() in C:\Users\carlo\testengine\PowerApps-TestEngine\src\Microsoft.PowerApps.TestEngine\PowerApps\PowerAppFunctions.cs:line 142
at Microsoft.PowerApps.TestEngine.PowerFx.PowerFxEngine.UpdatePowerFxModelAsync() in C:\Users\carlo\testengine\PowerApps-TestEngine\src\Microsoft.PowerApps.TestEngine\PowerFx\PowerFxEngine.cs:line 142
at Microsoft.PowerApps.TestEngine.SingleTestRunner.RunTestAsync(String testRunId, String testRunDirectory, TestSuiteDefinition testSuiteDefinition, BrowserConfiguration browserConfig, String domain, String queryParams) in C:\Users\carlo\testengine\PowerApps-TestEngine\src\Microsoft.PowerApps.TestEngine\SingleTestRunner.cs:line 112

Added Functions are not marked as causing side effects when they cause side effects.

The functions added in Setup() of PowerFxEngine are not marked as having side effects, these functions are SelectOneParamFunction, SelectTwoParamsFunction, SelectThreeParamsFunction. I don't know if the ScreenshotFunction is imperative, but I think AssertWithoutMessageFunction and AssertFunction also should be marked as imperative.

Sample Test Fails When Dynamics Asks For Feedback

Describe the bug

Running the basicgallery sample with dotnet run results in a timeout. Running with dotnet run -l trace works just fine

To Reproduce

Steps to reproduce the behavior:

1. dotnet run -l trace
2. That works
3. dotnet run
4. Throws the exception stack below

``c:\code\PowerApps-TestEngine\src\PowerAppsTestEngine\bin\Debug\net6.0.playwright\package\lib\server\dispatchers\dispatcher.js:281
stack: validMetadata.stack,
^

TypeError: Cannot read properties of undefined (reading 'stack')
at DispatcherConnection.dispatch (c:\code\PowerApps-TestEngine\src\PowerAppsTestEngine\bin\Debug\net6.0.playwright\package\lib\server\dispatchers\dispatcher.js:281:28)
at PipeTransport.transport.onmessage (c:\code\PowerApps-TestEngine\src\PowerAppsTestEngine\bin\Debug\net6.0.playwright\package\lib\cli\driver.js:61:57)
at Immediate. (c:\code\PowerApps-TestEngine\src\PowerAppsTestEngine\bin\Debug\net6.0.playwright\package\lib\protocol\transport.js:89:34)
at processImmediate (node:internal/timers:464:21)
[Error]: Waiting timed out.

[Error]: System.TimeoutException: The operation has timed out.
at Microsoft.PowerApps.TestEngine.Helpers.PollingHelper.CheckIfTimedOut(DateTime startTime, Int32 timeout, ILogger logger) in c:\code\PowerApps-TestEngine\src\Microsoft.PowerApps.TestEngine\Helpers\PollingHelper.cs:line 74
at Microsoft.PowerApps.TestEngine.Helpers.PollingHelper.PollAsync[T](T value, Func2 conditionToCheck, Func1 functionToCall, Int32 timeout, ILogger logger) in c:\code\PowerApps-TestEngine\src\Microsoft.PowerApps.TestEngine\Helpers\PollingHelper.cs:line 44
at Microsoft.PowerApps.TestEngine.PowerApps.PowerAppFunctions.LoadPowerAppsObjectModelAsync() in c:\code\PowerApps-TestEngine\src\Microsoft.PowerApps.TestEngine\PowerApps\PowerAppFunctions.cs:line 142
at Microsoft.PowerApps.TestEngine.PowerFx.PowerFxEngine.UpdatePowerFxModelAsync() in c:\code\PowerApps-TestEngine\src\Microsoft.PowerApps.TestEngine\PowerFx\PowerFxEngine.cs:line 142
at Microsoft.PowerApps.TestEngine.SingleTestRunner.RunTestAsync(String testRunId, String testRunDirectory, TestSuiteDefinition testSuiteDefinition, BrowserConfiguration browserConfig, String domain, String queryParams) in C:\Users\david.potts\OneDrive - British Engineering Services
Limited\code_bes\PowerApps-TestEngine\src\Microsoft.PowerApps.TestEngine\SingleTestRunner.cs:line 112

[Error]: Connection closed (Process exited)

Unhandled exception. Microsoft.Playwright.PlaywrightException: Connection closed (Process exited)
at Microsoft.Playwright.Transport.Connection.InnerSendMessageToServerAsync[T](String guid, String method, Object args) in //src/Playwright/Transport/Connection.cs:line 107
at Microsoft.Playwright.Transport.Connection.WrapApiCallAsync[T](Func`1 action, Boolean isInternal) in //src/Playwright/Transport/Connection.cs:line 479
at Microsoft.Playwright.Core.BrowserContext.CloseAsync() in /_/src/Playwright/Core/BrowserContext.cs:line 175
at Microsoft.PowerApps.TestEngine.TestInfra.PlaywrightTestInfraFunctions.EndTestRunAsync() in c:\code\PowerApps-TestEngine\src\Microsoft.PowerApps.TestEngine\TestInfra\PlaywrightTestInfraFunctions.cs:line 226
at Microsoft.PowerApps.TestEngine.SingleTestRunner.RunTestAsync(String testRunId, String testRunDirectory, TestSuiteDefinition testSuiteDefinition, BrowserConfiguration browserConfig, String domain, String queryParams) in C:\Users\david.potts\OneDrive - British Engineering Services
Limited\code_bes\PowerApps-TestEngine\src\Microsoft.PowerApps.TestEngine\SingleTestRunner.cs:line 197
at Microsoft.PowerApps.TestEngine.TestEngine.RunOneTestAsync(String testRunId, String testRunDirectory, TestSuiteDefinition testSuiteDefinition, BrowserConfiguration browserConfig, String domain, String queryParams) in c:\code\PowerApps-TestEngine\src\Microsoft.PowerApps.TestEngine\TestEngine.cs:line 143
at Microsoft.PowerApps.TestEngine.TestEngine.RunTestByWorkerCountAsync(String testRunId, String testRunDirectory, String domain, String queryParams) in c:\code\PowerApps-TestEngine\src\Microsoft.PowerApps.TestEngine\TestEngine.cs:line 136
at Microsoft.PowerApps.TestEngine.TestEngine.RunTestAsync(String testConfigFile, String environmentId, String tenantId, String outputDirectory, String domain, String queryParams) in c:\code\PowerApps-TestEngine\src\Microsoft.PowerApps.TestEngine\TestEngine.cs:line 99
at Program.

$(String[] args) in c:\code\PowerApps-TestEngine\src\PowerAppsTestEngine\Program.cs:line 160
at Program.(String[] args)``

Support test settings and environment variables in separate files

Instead of having to support test settings and environment inline in the test, allow it to be a pointer to a file so they can be reused across multiple tests

Exit criteria:

• test settings and environment variables can be in different YAML files
• Documentation updated
• Samples provided

Implement polling/retry on fetching the Power Apps object model

If Label1 hasn't loaded, we should be able to refetch the Power Apps object model and re-run that expression to see if it has loaded a little later. We can poll for a few times and only after some time has elapsed then we fail the test. Right now we would fail it immediately.

Exit criteria:

• Can rerun Power FX expression on failure in case there are delays with load

Test occasionally gets stuck on login screen

Need to debug to find out why this is happening. Possibly a race condition somewhere.

Exit criteria:

• Improved reliability on tests

Fix test browser not popping out

It was working but it doesn't pop out now. The tests still execute without error and videos are recorded.

Exit criteria:

• Test browser should pop out during the test execution.

Unexpected characters. The formula contains 'ParenClose' where 'Error' is expected.

I cloned the repo yesterday and installed it on 2 different machines (Windows 10 and 11).
After that I ran unmodified tests from the samples in various US- and Europe based production environments:

• browseGallery
• buttonclicker, and
• calculator.

All tests throw errors stating "Unexpected characters. The formula contains 'ParenClose' where 'Error' is expected".
The only teststeps that seem to work are Screenshot() and simple Select(Control) statements.

I installed the .net sdk 6.0.402 twice on both machines,

Attached you will find the logs of a browseGallery test run.
testresults.zip

Playwright Navigate Exception

A Playwright navigate exception is printed to console whenever dotnet run command is entered, but does not fail or cause issues with the program at all. Prints regardless, whether test is going to pass or fail. This prints at the beginning, so the other logs and the confirmation of the success of the run is printed after this exception, thereby burying this exception eventually, and ending on a success.

`System.NullReferenceException: Object reference not set to an instance of an object.` error with Combo Box

• Please attach the test suite YAML & .msapp which demonstrates the issue you are experiencing and any logs written to the specified outputDirectory. (Please do not attach any file which may contain sensitive personal or company data.)

App:
SampleAppforTestEngine_20221026141239.zip

Test Suite:

testSuite:
  testSuiteName: SampleTestSuite
  testSuiteDescription: Verify the functionality of ComboBox in Canvas App
  persona: User1
  appLogicalName: <REDACTED>

  testCases:
    - testCaseName: NavigateToSecondScreen
      testSteps: |
        = Select('Button1');
          SetProperty('ComboBox1'.SelectedItems, Table({'Name':"Record 1"}));          
testSettings:
  recordVideo: true
  browserConfigurations:
    - browser: Chromium

environmentVariables:
  users:
    - personaName: User1
      emailKey: user1Email
      passwordKey: user1Password

Describe the bug

Test Engine throws System.NullReferenceException: Object reference not set to an instance of an object. error when setting a value to Combo box control in Power Apps Canvas App. Here, SetProperty() function is used to set the value.

To Reproduce

Steps to reproduce the behavior:

1. Insert a Combo Box control in Power Apps Canvas App.
2. Use a Dataverse table or a collection as a datasource for the above created Combo Box control. The column name of the table should be anything other than Value.
3. Write a test case that sets the value of Combo Box as below:

SetProperty('ComboBox1'.SelectedItems, Table({'Name':"Sample App"}));

Expected Behavior:

Test Engine should set the value of Combo Box as defined in the SetProperty() function.

Actual Behavior:

The test has failed with the below errors:

PS /<REDACTED>/PowerApps-TestEngine/src/PowerAppsTestEngine> dotnet run
Test results will be stored in: TestOutput/2fed15



---------------------------------------------------------------------------
RUNNING TEST SUITE: SampleTestSuite
---------------------------------------------------------------------------

Browser configuration: {"Browser":"Chromium","Device":null,"ScreenWidth":null,"ScreenHeight":null,"ConfigName":null}

Browser setup finished

Browser context created

Test infrastructure setup finished

Successfully navigate page to target URL.

---------------------------------------------------------------------------
RUNNING TEST CASE: NavigateToSecondScreen
---------------------------------------------------------------------------

[Warning]: Syntax check failed. Now attempting to execute lines step by step

------------------------------

Executing SetProperty function.

[Error]: System.AggregateException: One or more errors occurred. (Exception has been thrown by the target of an invocation.)
 ---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.AggregateException: One or more errors occurred. (Object reference not set to an instance of an object.)
 ---> System.NullReferenceException: Object reference not set to an instance of an object.
   at Microsoft.PowerApps.TestEngine.PowerApps.PowerAppFunctions.SetPropertyTableAsync(ItemPath itemPath, TableValue value) in /<REDACTED>/PowerApps-TestEngine/src/Microsoft.PowerApps.TestEngine/PowerApps/PowerAppFunctions.cs:line 239
   at Microsoft.PowerApps.TestEngine.PowerApps.PowerAppFunctions.SetPropertyAsync(ItemPath itemPath, FormulaValue value) in /<REDACTED>/PowerApps-TestEngine/src/Microsoft.PowerApps.TestEngine/PowerApps/PowerAppFunctions.cs:line 186
   at Microsoft.PowerApps.TestEngine.PowerFx.Functions.SetPropertyFunction.SetProperty(RecordValue obj, StringValue propName, FormulaValue value) in /<REDACTED>/PowerApps-TestEngine/src/Microsoft.PowerApps.TestEngine/PowerFx/Functions/SetPropertyFunction.cs:line 44
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at Microsoft.PowerApps.TestEngine.PowerFx.Functions.SetPropertyFunction.Execute(RecordValue obj, StringValue propName, FormulaValue value) in /<REDACTED>/PowerApps-TestEngine/src/Microsoft.PowerApps.TestEngine/PowerFx/Functions/SetPropertyFunction.cs:line 32
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Span`1& arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Microsoft.PowerFx.ReflectionFunction.Invoke(FormulaValue[] args)
   at Microsoft.PowerFx.ReflectionFunction.<GetTexlFunction>b__7_0(FormulaValue[] args)
   at Microsoft.PowerFx.CustomSetPropertyFunction.InvokeAsync(FormulaValue[] args, CancellationToken cancellation)
   at Microsoft.PowerFx.EvalVisitor.TryHandleSetProperty(CallNode node, EvalVisitorContext context)
   at Microsoft.PowerFx.EvalVisitor.Visit(CallNode node, EvalVisitorContext context)
   at Microsoft.PowerFx.ParsedExpression.EvalAsync(RecordValue parameters, CancellationToken cancel)
   at Microsoft.PowerFx.RecalcEngine.EvalAsync(String expressionText, CancellationToken cancel, RecordValue parameters, ParserOptions options)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at System.Threading.Tasks.Task`1.get_Result()
   at Microsoft.PowerFx.RecalcEngine.Eval(String expressionText, RecordValue parameters, ParserOptions options)
   at Microsoft.PowerApps.TestEngine.PowerFx.PowerFxEngine.Execute(String testSteps) in /<REDACTED>/PowerApps-TestEngine/src/Microsoft.PowerApps.TestEngine/PowerFx/PowerFxEngine.cs:line 123
   at Microsoft.PowerApps.TestEngine.PowerFx.PowerFxEngine.ExecuteWithRetryAsync(String testSteps) in /<REDACTED>/PowerApps-TestEngine/src/Microsoft.PowerApps.TestEngine/PowerFx/PowerFxEngine.cs:line 70
   at Microsoft.PowerApps.TestEngine.SingleTestRunner.RunTestAsync(String testRunId, String testRunDirectory, TestSuiteDefinition testSuiteDefinition, BrowserConfiguration browserConfig, String queryParams) in /<REDACTED>/PowerApps-TestEngine/src/Microsoft.PowerApps.TestEngine/SingleTestRunner.cs:line 142

---------------------------------------------------------------------------
SampleTestSuite TEST SUMMARY
---------------------------------------------------------------------------

Total cases: 1

Cases passed: 0

Cases skipped: 0

Cases failed: 1


Test results can be found here: TestOutput/2fed15/Results_2fed15dd-f34b-4247-9fe5-7f71a62c55d6.trx

Additional context

I believe that the source code is expecting the column name of table to be Value. The tests ran successfully when we made the below changes:

1. Updated the column name in the table to Value
2. Updated the SetProperty as follows SetProperty('ComboBox1'.SelectedItems, Table({'Value':"Record 1"}));

Additionally, if the column name is set to Value in SetProperty() function, without changing it in the Datasource (Dataverse), below error is thrown:

[Error]: System.AggregateException: One or more errors occurred. (Exception has been thrown by the target of an invocation.)
 ---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.AggregateException: One or more errors occurred. (Error: Contract failure: Unspecified.
    at Z (https://content.powerapps.com/resource/app/5ov1l0job96ha/publish/js/pa.core.bundle3.js:62:78486)
    at Object.u [as checkValue] (https://content.powerapps.com/resource/app/5ov1l0job96ha/publish/js/pa.core.bundle3.js:62:76240)
    at ControlWidget.interactWithControlAsync (https://content.powerapps.com/resource/app/2lc91ftsgjsce/publish/js/pa.common.bundle1.js:4:146461)
    at interactWithControl (/<REDACTED>/PowerApps-TestEngine/src/Microsoft.PowerApps.TestEngine/JS/PublishedAppTesting.js:186:146)
    at eval (eval at onMessage (https://pa-static-ms.azureedge.net/resource/webplayerdynamic/publishedapp/preloadindex?preloadIndexPath=https%3A%2F%2Fcontent.powerapps.com%2Fresource%2Fapp%2Fmu7r2b8lrlh44%2Fpreloadindex.web.html&PowerAppsLanguage=en-US&loader=inline&lv=d5r9l51q3u44d&serviceWorkerUrl=https%3A%2F%2Fpa-static-ms.azureedge.net%2Fresource%2Fwebplayer%2Fhashedresources%2Fp681v2nkcinh5%2Fjs%2FPowerAppsServiceWorker.PublishedApp.js&unregisterServiceWorkersHash=61al1u62ufj72&piv=48DEC944&featureGates={%22publishedAppServiceWorker%22:false}:1:10728), <anonymous>:3:26)
    at onMessage (https://pa-static-ms.azureedge.net/resource/webplayerdynamic/publishedapp/preloadindex?preloadIndexPath=https%3A%2F%2Fcontent.powerapps.com%2Fresource%2Fapp%2Fmu7r2b8lrlh44%2Fpreloadindex.web.html&PowerAppsLanguage=en-US&loader=inline&lv=d5r9l51q3u44d&serviceWorkerUrl=https%3A%2F%2Fpa-static-ms.azureedge.net%2Fresource%2Fwebplayer%2Fhashedresources%2Fp681v2nkcinh5%2Fjs%2FPowerAppsServiceWorker.PublishedApp.js&unregisterServiceWorkersHash=61al1u62ufj72&piv=48DEC944&featureGates={%22publishedAppServiceWorker%22:false}:255:10759))
 ---> Microsoft.Playwright.PlaywrightException: Error: Contract failure: Unspecified.
    at Z (https://content.powerapps.com/resource/app/5ov1l0job96ha/publish/js/pa.core.bundle3.js:62:78486)
    at Object.u [as checkValue] (https://content.powerapps.com/resource/app/5ov1l0job96ha/publish/js/pa.core.bundle3.js:62:76240)
    at ControlWidget.interactWithControlAsync (https://content.powerapps.com/resource/app/2lc91ftsgjsce/publish/js/pa.common.bundle1.js:4:146461)
    at interactWithControl (/<REDACTED>/PowerApps-TestEngine/src/Microsoft.PowerApps.TestEngine/JS/PublishedAppTesting.js:186:146)
    at eval (eval at onMessage (https://pa-static-ms.azureedge.net/resource/webplayerdynamic/publishedapp/preloadindex?preloadIndexPath=https%3A%2F%2Fcontent.powerapps.com%2Fresource%2Fapp%2Fmu7r2b8lrlh44%2Fpreloadindex.web.html&PowerAppsLanguage=en-US&loader=inline&lv=d5r9l51q3u44d&serviceWorkerUrl=https%3A%2F%2Fpa-static-ms.azureedge.net%2Fresource%2Fwebplayer%2Fhashedresources%2Fp681v2nkcinh5%2Fjs%2FPowerAppsServiceWorker.PublishedApp.js&unregisterServiceWorkersHash=61al1u62ufj72&piv=48DEC944&featureGates={%22publishedAppServiceWorker%22:false}:1:10728), <anonymous>:3:26)
    at onMessage (https://pa-static-ms.azureedge.net/resource/webplayerdynamic/publishedapp/preloadindex?preloadIndexPath=https%3A%2F%2Fcontent.powerapps.com%2Fresource%2Fapp%2Fmu7r2b8lrlh44%2Fpreloadindex.web.html&PowerAppsLanguage=en-US&loader=inline&lv=d5r9l51q3u44d&serviceWorkerUrl=https%3A%2F%2Fpa-static-ms.azureedge.net%2Fresource%2Fwebplayer%2Fhashedresources%2Fp681v2nkcinh5%2Fjs%2FPowerAppsServiceWorker.PublishedApp.js&unregisterServiceWorkersHash=61al1u62ufj72&piv=48DEC944&featureGates={%22publishedAppServiceWorker%22:false}:255:10759)
   at Microsoft.Playwright.Transport.Connection.InnerSendMessageToServerAsync[T](String guid, String method, Object args) in /_/src/Playwright/Transport/Connection.cs:line 164
   at Microsoft.Playwright.Transport.Connection.WrapApiCallAsync[T](Func`1 action, Boolean isInternal) in /_/src/Playwright/Transport/Connection.cs:line 479
   at Microsoft.Playwright.Core.Frame.EvaluateAsync[T](String script, Object arg) in /_/src/Playwright/Core/Frame.cs:line 496
   at Microsoft.PowerApps.TestEngine.TestInfra.PlaywrightTestInfraFunctions.RunJavascriptAsync[T](String jsExpression) in /<REDACTED>/PowerApps-TestEngine/src/Microsoft.PowerApps.TestEngine/TestInfra/PlaywrightTestInfraFunctions.cs:line 278
   at Microsoft.PowerApps.TestEngine.PowerApps.PowerAppFunctions.SetPropertyTableAsync(ItemPath itemPath, TableValue value) in /<REDACTED>/PowerApps-TestEngine/src/Microsoft.PowerApps.TestEngine/PowerApps/PowerAppFunctions.cs:line 249
   at Microsoft.PowerApps.TestEngine.PowerApps.PowerAppFunctions.SetPropertyAsync(ItemPath itemPath, FormulaValue value) in /<REDACTED>/PowerApps-TestEngine/src/Microsoft.PowerApps.TestEngine/PowerApps/PowerAppFunctions.cs:line 186
   at Microsoft.PowerApps.TestEngine.PowerFx.Functions.SetPropertyFunction.SetProperty(RecordValue obj, StringValue propName, FormulaValue value) in /<REDACTED>/PowerApps-TestEngine/src/Microsoft.PowerApps.TestEngine/PowerFx/Functions/SetPropertyFunction.cs:line 44
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at Microsoft.PowerApps.TestEngine.PowerFx.Functions.SetPropertyFunction.Execute(RecordValue obj, StringValue propName, FormulaValue value) in /<REDACTED>/PowerApps-TestEngine/src/Microsoft.PowerApps.TestEngine/PowerFx/Functions/SetPropertyFunction.cs:line 32
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Span`1& arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Microsoft.PowerFx.ReflectionFunction.Invoke(FormulaValue[] args)
   at Microsoft.PowerFx.ReflectionFunction.<GetTexlFunction>b__7_0(FormulaValue[] args)
   at Microsoft.PowerFx.CustomSetPropertyFunction.InvokeAsync(FormulaValue[] args, CancellationToken cancellation)
   at Microsoft.PowerFx.EvalVisitor.TryHandleSetProperty(CallNode node, EvalVisitorContext context)
   at Microsoft.PowerFx.EvalVisitor.Visit(CallNode node, EvalVisitorContext context)
   at Microsoft.PowerFx.ParsedExpression.EvalAsync(RecordValue parameters, CancellationToken cancel)
   at Microsoft.PowerFx.RecalcEngine.EvalAsync(String expressionText, CancellationToken cancel, RecordValue parameters, ParserOptions options)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at System.Threading.Tasks.Task`1.get_Result()
   at Microsoft.PowerFx.RecalcEngine.Eval(String expressionText, RecordValue parameters, ParserOptions options)
   at Microsoft.PowerApps.TestEngine.PowerFx.PowerFxEngine.Execute(String testSteps) in /<REDACTED>/PowerApps-TestEngine/src/Microsoft.PowerApps.TestEngine/PowerFx/PowerFxEngine.cs:line 130
   at Microsoft.PowerApps.TestEngine.PowerFx.PowerFxEngine.ExecuteWithRetryAsync(String testSteps) in /<REDACTED>/PowerApps-TestEngine/src/Microsoft.PowerApps.TestEngine/PowerFx/PowerFxEngine.cs:line 70
   at Microsoft.PowerApps.TestEngine.SingleTestRunner.RunTestAsync(String testRunId, String testRunDirectory, TestSuiteDefinition testSuiteDefinition, BrowserConfiguration browserConfig, String queryParams) in /<REDACTED>/PowerApps-TestEngine/src/Microsoft.PowerApps.TestEngine/SingleTestRunner.cs:line 142

Note: These test cases work fine without any issues in Test Studio.

Due to this issue, I couldn't proceed further on testing.

Update SetProperty syntax

Need to match test studio syntax