jupyterlab / galata Goto Github PK

Description

I'm seeing false positives, described at spacetelescope/jdaviz#543 (comment) . The capture of the output cells seems ok, but it seems to be capturing a part of the Jupyter Lab 3.0 interface (with different portions grey each time). I'm not sure this is a user bug or a Galata bug.

When using Galata with JupyterLab 3.0.9 I am having the following error in the terminal running Galata:

yarn run v1.22.10
$ galata
 LOG  Deleting existing test output directory 'test'
Determining test suites to run...

 LOG  Using Galata built for JupyterLab version 3.0.3
 LOG  Failed to inject galataip object into browser context

 RUNS  tests/widgets.test.ts
 LOG  jest output not found
 LOG  galata output not found
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

And at the same moment I am getting the following error in the terminal running jupyterlab:

[E 2021-03-29 15:03:37.662 ServerApp] Uncaught exception GET /lab/api/settings/@jupyterlab/docmanager-extension:plugin?1617023016487 (::1)
    HTTPServerRequest(protocol='http', host='localhost:8888', method='GET', uri='/lab/api/settings/@jupyterlab/docmanager-extension:plugin?1617023016487', version='HTTP/1.1', remote_ip='::1')
    Traceback (most recent call last):
      File "/home/martin/miniconda3/envs/bqplot/lib/python3.9/site-packages/tornado/web.py", line 1704, in _execute
        result = await result
    tornado.iostream.StreamClosedError: Stream is closed

Currently Galata targets 3.0.7:

We should consider updating to 3.0.16.

A more robust fix could be to retrieve all the items programmatically instead of manually overriding them with overrides["id"] = "left";

Originally posted by @jtpio in #45 (comment)

Description

I'm able to run galata fine as long as I don't specify my own browser. Otherwise the process just hangs. I've tried running:

node ./node_modules/.bin/galata "--browser-url" "http://localhost:9222" --browser-type chromium

This might be related to this issue: microsoft/playwright#4054
But I've tried replacing connect in Galata source code with connectOverCDP but without success.

When running --update-references for the first time, the command fails with:

Reference directory './reference-output' does not exist.
error Command failed with exit code 1.

Maybe Galata could automatically create the reference directory if it doesn't exist?

With jupyterlab/jupyterlab#10796 now merged upstream, we should be able to archive this repo to clearly communicate where the main development is happening.

TODO

• Transfer relevant issues to https://github.com/jupyterlab/jupyterlab
• Add an archive header to the README.md: #82
• Archive the repo

Problem

For now the set of tab bar ids is defined here:

It would be great if this could be made configurable, or be able to pass any id.

For example the @jupyterlab/git extension adds a panel to the left area, and this extension could use galata for end to end testing.

Proposed Solution

Allow arbitrary ids (as strings)?

Additional context

Related to #45

Since releasing Galata only consists in publishing the package to npm, it should be possible to automate this workflow.

Either with a simple GitHub Actions workflow.

Or using the Jupyter Releaser: https://github.com/jupyter-server/jupyter_releaser

Problem

playwright is an amazing tool with a strong support from Microsoft and the community. So we can only expect it to grow in reliability and ease of usage. In particular with v1.12 a test runner was released with the builtin ability to compare image, generate trace, create videos, running tests in parallel, fixtures, reporters,... . I think we should onboard that as there is no value for us to try keeping up the pace by hiding the tool.

Proposed Solution

• Embrace playwright test runner
  • Port current report generator to be a valid playwright reporter
  • Unlock the ability to create test from the code generator tool by copy-paste

while keeping the nice helpers functions.

As mentioned in #48, we should adopt ESLint and Prettier to format the code base, so it's consistent with the tooling already in place in the main JupyterLab repo.

Opening a new issue to discuss potential ideas for the next iterations.

I've been trying out playwright lately and it's been really nice. It's also great that chromium, firefox and webkit are all supported by default.

Since the playwright api is very similar to puppeteer (and the same for the 3 browser), maybe it's worth considering it in the future?

The change would most likely be about updating imports, and maybe also about simplifying selectors. The galata cli could also be extended to configure browsers, so they could be set as matrix items on CI.

Support for side-by-side view, swipe and transition like GitHub PR image diff would be great

For now it looks like galata uses puppeteer-core:

https://github.com/mbektasbbg/galata/blob/91f99885a0be2da7c048d587d9ccf697773b180a/packages/galata/src/jlabtest.ts#L4

Which requires specifying the path to a Chrome binary.

Should there also be support for running galata in environments where an existing Chrome might not be available? For example on CI.

I was under the impression this was already the case since this is a new repo, but it could be that it was created a while ago when main was not the default on GitHub yet.

How about renaming the default master branch to main? It should be possible to do that easily via the settings menu.

Now would be a good time since there is no open PRs so it shouldn't be too disruptive.

Really nice library, thanks for putting that together!

At the moment it looks like the chrome path defaults to the one for Mac when using galata on Linux:

https://github.com/mbektasbbg/galata/blob/91f99885a0be2da7c048d587d9ccf697773b180a/packages/galata/bin/cli.js#L142-L144

Wondering whether it should be the default? Or maybe default to /usr/bin/chromium or which chromium?

Otherwise a small section in the documentation / readme could be enough as a starting point.

It would be nice to be able to select the theme with a command-line option, something like:

galata --theme=dark

We should publish galata to npm, so it can easily be added as a dependency in other projects, list the galata example:

Things to check before releasing:

• Release strategy: how often and when (depending on the JupyterLab release cycle?)
• Versioning
• API breaking changes

To make the tooling consistent across the @jupyterlab repositories, we should consider using yarn or jlpm instead of npm to manage dependencies and run scripts.

This would then also mean replacing the existing package-lock.json by a new yarn.lock.

Problem

It would be great if galata could also be used to test alternative JupyterLab distributions such as RetroLab.

Proposed Solution

At the moment it looks like the default page hits the /lab endpoint:

There is also this setup logic using /lab by default:

Additional context

A potential candidate for this would be RetroLab: jupyterlab/retrolab#135

Description

The top-level version added in #72 was used mostly as a placeholder so it can be npm pack.

However it ended as the version added to the changelog:

Reproduce

In the automated changelog entry in #73, it picked up 0.1.0 as the version instead of 3.0.11-3 as specified in the workflow.

Probably we could sync the lerna version with the top-level one in package.json

We should add a RELEASE.md at the top level of the repo, that will list the steps for releasing a new version of Galata.

The file would also explain how to version Galata, and when to upgrade for new version of JupyterLab (#29 could be a reference).

Description

Toggling simple mode doesn't always wait enough for switch widget's animation to end.

Reproduce

Randomly reproduces with toggleSimpleMode API method.

Expected behavior

Should wait for switch animation to end.

Description

notebook.createNew is not waiting enough if a name is set