RFD - Upgrade Integration tests - WIP about governance

[kcpevey]

There was some discussion around replacing cypress with playwright. I messed around with it for a bit today and was able to work through the first part of authenticating via google (which included working in incognito mode on chromium and it handled the redirects). Here is the code I was using as a POC:

from playwright.sync_api import sync_playwright
import dotenv
import os

dotenv.load_dotenv()

url = 'https://nebari.quansight.dev/user/[email protected]/lab'
google_email = os.environ['GOOGLE_EMAIL']
google_password = os.environ['GOOGLE_PASSWORD']

with sync_playwright() as p:
    browser = p.chromium.launch(headless=False, slow_mo=20)
    page = browser.new_page()
    page.goto(url)
    print(page.title())
    locator = page.get_by_role("button", name="Sign in with Keycloak")
    locator.hover()
    locator.click()

    page.locator("#social-google").click()  # get via element id

    # fill in email and click next
    page.get_by_label("Email or phone").fill(google_email)
    page.get_by_role("button", name="Next").click() 

    # fill in password and click next
    page.get_by_label("Enter your password").fill(google_password)
    page.get_by_role("button", name="Next")

    browser.close()

I'm happy to investigate this further or start setting something up.

[viniciusdc]

@kcpevey I think one interesting thing we could test out is login in a Nebari deployment and run a jupyter notebook within playwright

[kcpevey]

I think one interesting thing we could test out is login in a Nebari deployment and run a jupyter notebook within playwright

I could do that, its just a few additional steps.

[iameskild]

Thanks a lot @viniciusdc for thorough RFD!! I definitely agree that improving our integration tests (along with CI) will go a long way to improving and speeding up the release process so I'm all for it!

To me, @kcpevey's example of playwright shows a lot of promise and would make adding and maintaining tests a lot easier. Although Cypress can perform many of these same types of tests, I feel resistance to adding new or improving existing tests mostly because of my lack of JS experience. Just like Cypress, we can run these tests during CI (kubernetes test) and during the integration tests!

I like the idea of making sure that, after each deployed stage, things are working as expected. We currently have a few checks in place that run after each stages, so I wonder if expanding on those would be sufficient. Perhaps instead of --stop-at we just run the full deployment but we can enable different level of checks.

This might looks something like: nebari deploy -c ... --checks none | basic | full

• none (currently possible with --disable-checks)
• basic (default and what we have now)
• full (the additions you are proposing and enabled for CI / IT)

[viniciusdc]

I am closing this, as we already have considerations around CI integration enchantments. This RFD does not serve a direct purpose. I will summarize these points in a follow-up to the original tracking issues.