Evaluate saucelabs.com for UI testing about manageiq-ui-service

[himdel]

The site is so covered in marketing-speak that it doesn't actually say anything specific on how it works and what it does :) .. But judging from https://wiki.saucelabs.com/ it seems it's a simple selenium-in-the-cloud service and not much more...

So I wonder if we shouldn't be doing exactly the same thing as in the ops ui.. QE already has some selenium tests in https://github.com/ManageIQ/integration_tests and we probably shouldn't be doing things too differently, should we?

[chriskacerguis]

I guess it depends on what exactly the Ops ui is doing. Are you testing on travis build, or just when QE is doing their mojo?

[himdel]

The second .. IIRC it takes way too long to run for every PR.

Unless you're proposing we write these tests, in which case, umm... I'm not the right person to ask, I've used selenium, and not planning to do that again :) (But seriously, I had high hopes for https://www.cypress.io/ in this regard, but they still haven't released anything...)

[chriskacerguis]

Yes, I'm suggesting that we write our own tests (at least for the SUI).

[himdel]

OK, in that case, I'd be strongly for a solution that involves writing the tests in Javascript, and which can handle things like waiting for promises before continuing and in general has a decent async support.

Selenium can't really do these things that well, and if we're writing something new...

(Unfortunately, I only know about that cypress project, but maybe there are other candidates..)

[chriskacerguis]

Well SUPPOSEDLY SauceLabs can do that (it's built for Angular apps)

[himdel]

Allright, can't see the how in the docs, but maybe it's there somewhere :) if so .. I'm happy :)

[chriskacerguis]

@chalettu can you comment on @himdel requirement?

[chalettu]

The tests we will be writing in Javascript. As part of my evaluation I will be sure to test out waiting for a page to finish coming up. I will test a handful of use cases but async requests are common fair for this type of testing framework. Anything that can be done on a webpage can be replicated in tests.

[himdel]

OK, @chalettu when doing that, could you please share those test samples here? Just so we can see how it actually looks.. :)

[chalettu]

Sure, I will post links to the videos taken from the sample tests.

On Fri, Sep 30, 2016 at 11:52 AM, Martin Hradil [email protected]
wrote:

OK, @chalettu https://github.com/chalettu when doing that, could you
please share those test samples here? Just so we can see how it actually
looks.. :)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#193 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAOElccTT5N7HP9trup0bGMDhMrO2QIgks5qvT4wgaJpZM4KHky2
.

Chris Hale

Senior Software Engineer
[email protected]
512-585-7286
https://bluejeans.com/chhale

[himdel]

@chalettu I'm more interested in the code itself, a video could be interesting too, but .. it's not what I'm asking for :).

[martinpovolny]

I am quite sure we should not duplicate QE work here. So writing selenium tests for SSUI makes no sense.

If we have problem with low coverage of SSUI by QE tests then we should document what is missing and talk to them.

Even if we decided to write our own selenium tests, we should contribute to the QE repository and not try to use our own set of tools. That's waste of time.

[chriskacerguis]

@martinpovolny do you not see any value in running tests (and taking at least some of the workload from the QE team)? As far as contributing, yes I agree we should contribute as much as possible.

[martinpovolny]

@chriskacerguis : Running tests the QE run? From time to time that may be helpful.

Taking their workload? No. We have our workload. We try to make their life easier by simplifying their testing. E.g. ManageIQ/manageiq#11390. But doing their work? No.

[dajohnso]

• @psav

So this, SSUI, is something that as of today we have little support around in terms of test automation. We are getting there but it has not quite landed yet as we needed to refactor the test automation to handle the multiple end points that the product is beginning to offer. This is all part of the FW3.0 effort that we have been mentioning.

The same can be said with regards to saucelabs. We have done some work here, we have seen it work, it is slightly slower, and we have some jenkins jobs that run against ops UI. As part of the effort to refactor the test automation framework, we are working towards building in support to better handle which tests need to be parameterized for sauce browser testing.

----- Original Message -----

From: "Martin Povolny" [email protected]
To: "ManageIQ/manageiq-ui-self_service" [email protected]
Cc: "Dave Johnson" [email protected], "Mention" [email protected]
Sent: Monday, October 3, 2016 9:29:23 AM
Subject: Re: [ManageIQ/manageiq-ui-self_service] Evaluate saucelabs.com for UI testing (#193)

@chriskacerguis : Running tests the QE run? From time to time that may be
helpful.

Taking their workload? No. We have our workload. We try to make their life
easier by simplifying their testing. E.g.
ManageIQ/manageiq#11390. But doing their work? No.

You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#193 (comment)

[chriskacerguis]

@dajohnso thanks for the update. I really like to do more testing at build time, and yes it is a bit slower, but being able to do checks as we do PR's will make things more stable. Since you have very little, can our teams work together and put SauceLabs support in our repo for testing? thoughts?

[dajohnso]

@chriskacerguis , of course, I am completely in favour of more collaboration between the two teams. "Our repo" is a little ambiguous but sure, qe wants to try and assist if possible although the majority of our focus will be towards adding the support in our existing framework than to start creating a new one.

We are making more progress towards opening up our qe test automation framework to appropriately switch between the ops ui and the service ui. Once that is in place, adding tests should be straight forward and the framework already supports sauce.

Feel free to get a meeting scheduled sometime this upcoming week if you want to further discuss this endeavour. Late morning EST is the preference.