joelgriffith / navalia Goto Github PK

Hey folks,

Looking to dust off this project a bit. Wondering if we should can the GraphQL interface? I'm thinking about a separate project for that since it seems to be not directly related to offering a decent API.

Thoughts?

Hi !
Thanks for this tool 👍
Just requesting a feature: add a way to pass custom headers to goto.
Not sure how it works, but there's something about headers in Network.Request

I am using Chrome stable v60 (not Canary). The code is below

import { Chrome } from 'navalia';
const chrome = new Chrome();

async function screenshot() {
  await chrome.goto('https://www.google.com');
  await chrome.screenshot('body');
  chrome.done();
}

screenshot();

I ran fine without error but no screenshot file either.

Any help?

Hi, thanks for the great work.
I'm trying to test some elements that have a onMouseDown function bound to it with the Navalia click method, but the event is not triggered.

Don't know if the best option is to create a new Navalia method called mouseDown or make click work with onClick, onMouseDown and other types of events like touch.

First, awesome project. I have seen notes about this project is or will be deprecated. I do hope rumors of its demise will be proven false. :)

The Chrome instance is easy enough to use however the navalia usage seems to have a problem. I haven't dug into code yet though.

This example fails:

process.env.DEBUG = "navalia";

import { Navalia } from "navalia";

const navalia: Navalia = new Navalia();

(async () => {
  await navalia.run(async chrome => chrome.goto("http://joelgriffith.net"));
})();

Output:

Wed, 14 Feb 2018 22:01:20 GMT navalia starting, using up to 1 instances
Wed, 14 Feb 2018 22:01:20 GMT navalia instances are available and starting
(node:22949) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: unexpected server response (404)
(node:22949) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

I have a scenario that I would like to assert the current URL, today I could not find how to do this.

Is there any way today using navalia.js to do this?

I have to set some local storage values to chrome and use auth0 or oauth2 authentication, is there a way to have auth support oauth2? or to set local storage values before doing chrome.goto(url)?

It seems a little heavy (and opinionated) to achieve what Nightmare.js does.

Currently, this library seems to rely on chrome-launcher when it comes to connecting to chrome instances. Would it make sense to have alternative way of connecting to already running instances during Navalia initialization? I wouldnt mind working on this because i have huge need for this and i really like navalia's api.

I happen to have use case where i have number of VMs running headless chrome and i can connect to them over the remote debugging protocol. It becomes useful when using chrome from AWS Lambda functions, for example.

Anyway, thanks for the fresh approach in browser driver landscape. I've been looking forward to this for some time by now. Cheers!

Hi there, and thanks for this amazing library!

I'm trying to resize the browser window in order to capture automated full page screenshots for various screen sizes, but I'm seeing an issue where screenshots are always blank after resizing the browser window.

I'm using navalia 1.2.0 on mac os 10.12.06 with chrome 61.0.3163.100

I've created a minimal reproduction case here:

const { Chrome } = require('navalia')
const chrome = new Chrome()
const fs = require('fs')

chrome
  .goto('https://google.com')
  .then(() => chrome.screenshot('body'))
  .then((img) => fs.writeFileSync('yep.png', img))
  .then(() => chrome.size(640,640))
  .then(() => chrome.screenshot('body'))
  .then((img) => fs.writeFileSync('nope.png', img))
  .then(() => chrome.done())

And here is yep.png:

and nope.png:

I'd be happy to submit a PR to fix this, but am at a bit of a loss as to how to debug since I'm not sure how to get some visibility into the execution of the chromedriver process. Any tips there would be much appreciated!

Need to implement network requests. API might look like:

tab.onNetworkRequest((src: string) => {});

I think having distinct methods for this type of eventing are more obvious (especially with TS), vs using EventEmitter and emitting events on a particular action.

After updating to Chrome Version 62.0.3202.62 tests started failing.

    unexpected server response (404)

      at ClientRequest._req.on (node_modules/chrome-remote-interface/node_modules/ws/lib/WebSocket.js:651:26)
      at emitOne (events.js:115:13)
      at ClientRequest.emit (events.js:210:7)
      at HTTPParser.parserOnIncomingClient (_http_client.js:565:21)
      at HTTPParser.parserOnHeadersComplete (_http_common.js:116:23)
      at Socket.socketOnData (_http_client.js:454:20)
      at emitOne (events.js:115:13)
      at Socket.emit (events.js:210:7)
      at addChunk (_stream_readable.js:266:12)
      at readableAddChunk (_stream_readable.js:253:11)

Debug log:

  navalia starting, using up to 1 instances +0ms
  navalia:chrome-helper using up to infinite tabs +1ms
  navalia instances are available and starting +2ms
  navalia:chrome-helper starting chrome +0ms
  navalia instances are available and starting +226ms
  ChromeLauncher Waiting for browser. +9ms
  ChromeLauncher Waiting for browser... +0ms
  ChromeLauncher Waiting for browser..... +512ms
  ChromeLauncher Waiting for browser.....✓ +2ms

How do I access to Emulation.setVisibleSize to set the window size before taking screenshot?

Love this repo and switching all our E2E tests to using Navalia!

Bumped into this issue with screenshot, my situation is the same as this closed issue: #63

However no matter how large I resize the browser to (I tried chrome.size(1600,900), chrome.size(800,800), chrome.size(2500, 1200), etc), once chrome.size is called, the screenshot taken is blank.

Also, it's interesting that the png generated is four times as large as the dimension given. For example, if I call chrome.size(1600, 900), the generated blank png is of dimension (3200, 1800).

Chrome just released code-coverage stuff in there devtools. I can see this being extremely useful in CI scenarios where consumers want to assert usage thresholds.

There's enough meta-data inside of Chrome to generate a webpage with only the resources it uses. This is fairly easy to achieve with the css, but gets tricker with js as the API for interaction with those resources is different (and the file-size of JS can be quite large).

MVP would:

• Generate an asset of some kind (zip, gzip, tar) containing all the resources necessary.
• We would automatically trim out unused css or JS.
• Potentially make a lazy-loader where the remaining unused assets can be loaded async. This would help in getting time to first paint, but still preserve full functionality since it's hard to itemize all user cases.

I actually came across this last week or so and couldn't really figure out what it was. Just now I found out that it's like chromeless (which is like Nightmare.js), except that it's actually feature complete and not just an empty v1.0

"Navalia" doesn't communicate anything to me and makes me think of some frontend framework, like Aurelia or something.

When I use await chrome.wait(3000); the test crashes with the code

 Timeout - Async callback was not invoked within timeout specified by jasmine.DEFAULT_TIMEOUT_INTERVAL.

the error persists even if I put the parameter timeout...

        await chrome.wait(3000, 5000);

        await chrome.wait(3000, { timeout: 5000 });

        chrome = new Chrome({ timeout: 5000 });

Getting this error : Unhandled promise rejection (rejection id: 1): Error: unexpected server response (500) and sometimes
Error: 'Target.createBrowserContext' wasn't found

Using Chrome Version 59.0.3071.115

Just tried this example:

chrome
    .start()
    .then((tab) =>
        tab.navigate('https://www.google.com/')
        .then(() => tab.screenshot('google.png'))
        .then(() => tab.done())
 );

Also tried other examples , getting the same 500 error.
A quick help would be appreciable.

Hello All, awesome project, so much easier to setup than PhanstomJs + CasperJs.

I am attempting to get a POC of End-to-End testing using Navalia. This is in an effort to modernize our frontend testing process. We are wanting to use Javascript style syntax as is aligns better with our projects.

I get Navalia running in the terminal...

Then I attempt to execute the following basic test:

./test_1.js:

const { Chrome } = require('navalia');
const chrome = new Chrome();

chrome
    .goto('https://amazon.com')
    .type('input', 'Kindle')
    .click('.buy-now')
    .end()
    .then((responses) => {
    console.log(responses); // ['https://www.amazon.com/', true, true, true]
});

...but now what? How do I tell node to execute the ./test_1.js again Navalia and this get a chrome session processing the commands.

Sorry if this is super-noob, my background as a server-side engineer leaves my client side knowledgte lacking.

Thank you in advance for the assistance.

After checking the backlog of issue this is a dependant issue of #72

Awesome project!

Does it make sense to support starting Chrome via Docker? It would allow us to use navalia to start the same Chrome rendering on both our local machines (using MacOS) and our integration environment (Travis + Linux).

This example:

const { Chrome } = require('navalia');
const chrome = new Chrome();

chrome
  .goto('https://amazon.com')
  .type('input', 'Kindle')
  .click('.buy-now')
  .end()
  .then(responses => {
    console.log(responses); // ['https://www.amazon.com/', true, true, true]
  });

causes the following error:

(node:13803) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Goto failed to load in the timeout specified
(node:13803) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

no way to work around it. Can't use navalia at all. Please help!

The graphql version referenced by [email protected] and the jest version referenced by [email protected] seem to be incompatible with one another. Gives an error message:

 FAIL  src/react.spec.js
  ● Test suite failed to run

    /Users/sgreene/src/tutorials/test-navalia/node_modules/graphql/index.mjs:2
    export { graphql, graphqlSync } from './graphql';
    ^^^^^^

Steps to reproduce:

1. Install [email protected] (current stable version as of this writing)
2. create-react-app test-navalia
3. Create the app Login-Page.js and the first test react.spec.js from the blog post https://codeburst.io/composable-end-to-end-tests-for-react-apps-2ec82170af62.
4. yarn test.

package.json looks like this:

{
  "name": "test-navalia",
  "version": "0.1.0",
  "private": true,
  "dependencies": {
    "react": "^16.2.0",
    "react-dom": "^16.2.0",
    "react-scripts": "1.1.1"
  },
  "scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test --env=jsdom",
    "eject": "react-scripts eject"
  },
  "devDependencies": {
    "navalia": "^1.3.0"
  }
}

Here's the code I'm trying to run, compiled from two different examples in the docs:

const { Chrome } = require('navalia');
const chrome = new Chrome({chromeOptions: {
    flags: {
      headless: false
    }
  }});
chrome
  .goto('https://amazon.com')
  .type('input', 'Kindle')
  .click('.buy-now')
  .then((responses) => {
    console.log(responses); // ['https://www.amazon.com/', true, true, true]
  });

But when I run it it works as normal, like it did before I added the noheadless arg. Anything obvious I'm doing wrong?

Here is my dream api:

// in jest
describe('app', () =>
  it('should not change', async () => {
    const myComponentLib = 'localhost:6006'
    const filepath = __dirname
    // an image?
    const actual = await chrome.goto(myComponentLib)
      .then(() => chrome.screenshot(filepath))
      .then(() => chrome.done());
    const expected = await theMostRecentImageOnDisk
    expect(actual).toEqual(expected)
    // if this throws, prompt me with a comparison of the two images side by side
    // let me approve the overwrite or accept the failure
    // ala jest snapshots
  })
)

i guess the screenshot could be enough if i had a good diffing library, but the hard part is still prompting users with the comparison side-by-side, and then continuing the tests after input is received.

this feels like an overlap of concerns between the test runner, the assertion lib, and the driver, but nobody is doing this well right now. jest html snapshots have the perfect workflow, it's just that they're nonsense text output instead of pictures.

I'm learning navalia, and have a test that looks on the page for a certain element, like this:

.then(() => chrome.exists('[#ns-leftdrawer-list-item--atlas]'))

it succeeds, and my test then attempts to click on a child element, like this:

chrome.click('[#ns-leftdrawer-list-item--atlas > a]')

this fails, and the chatter from navalia says this:

navalia:chrome :WARN > Retrying 1 time(s) due to issue: 'Error: [object Object]' +10ms

My question is: how do I get visibility into what the error actually is? Is there a way to dump/format the [object Object]?

When trying to change the value of a dropdown using select chrome.select returns ReferenceError: option is not defined

async function test() {
    await chrome.goto('http://localhost:3000/pva');
    const x = await chrome.exists('#accountViewSelector') //true
    const y = await chrome.html('#accountViewSelector') //returns correct dropdown <select ..
    const z = await chrome.select('#accountViewSelector', '0'); // ReferenceError: option is not defined
    chrome.done();
}

test();

Sole component being rendered

const PortfolioValueAnalysis = () => {
    return (
        <select id='accountViewSelector' className="form-control">
            <option value='0'>0</option>
            <option value='1'>1</option>
            <option value='2'>2</option>
        </select>
    )
}

export default PortfolioValueAnalysis;

Is there any API to clear all cookies?

Running the following code results into an error because it's accessing a property from an undefined variable:

async function run () {
  const browser = new Chrome()
  await browser.goto('https://google.com')
  const res = await browser.evaluate(() => window.__UNKNOWN_VARIABLE__.x)
  console.log(res)
  await browser.done()
}

Error:

Error
    at Chrome.<anonymous> (D:\...\node_modules\navalia\build\Chrome.js:141:23)
    at Generator.next (<anonymous>)
    at fulfilled (D:\dev\...\node_modules\navalia\build\Chrome.js:4:58)
    at <anonymous>

Is there anyway we could forward chrome's error messages when .evaluate()ing?

Hi! Awesome work, and inspiring too!

Any plans to support API testing, specially JSON responses?

Thanks again!

Hello,

Firefox exists in headless mode. I am looking for a library to handle both chrome and firefox. Would it be possible to include both in navalia?

Thank you

When trying to use Navalia, I don't seem to be able to connect to chrome as any operation simply timesout. At first I thought this was an issue with my chrome install, but I can connect to it if I install and use chrome-launcher seperately. I'm using ubuntu 16 with version 60 of google-chrome-stable.

Any ideas what might be going wrong?
Edit: I can get Selenium working with chrome so I don't think it's anything to do with the chrome install.

Hi,

I am trying to figure out how to even run this application. I know it runs on headless chrome but I have no idea on how to actually run the test script(s) that I have made.

Thanks!

I've been evaluating navalia for integration testing on a new project. I really like the api and found it very easy to get it up and running, so great job!

I have a basic test that just makes sure our React app has actually loaded:

const { Chrome } = require('navalia')

const testUrl = process.env.TEST_URL || 'http://localhost:3000'

console.log(`TEST_URL: ${testUrl}`)

describe('Entry App', () => {

  let chrome = {}

  beforeEach(() => {
    chrome = new Chrome()
  })

  afterEach(() => chrome.done())

  it('should load entry page', async () => {
    await chrome.goto(testUrl)
    await chrome.wait(500)
    expect(await chrome.exists('#root')).toBe(true)
  }, 30000)
})

I'm having an issue where the test runs fine on my machine (macOS Sierra 10.12.5, Chrome 59.0.3071.115) but it's not working on our CI server. The CI is running in a docker container (RHEL7, Chrome 59.0.3071.115) and the error it shows is:

connect ECONNREFUSED 127.0.0.1:38502
  at Object.exports._errnoException (util.js:1022:11)
  at exports._exceptionWithHostPort (util.js:1045:20)
  at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1146:14)

Interestingly, the test is also failing on another dev's machine (macOS Sierra 10.12.5, Chrome 60.something - don't have the exact version on hand) with a similar error but instead of 127.0.0.1 it has ::1 (IPv6).

So far the only machine it works on is mine (we have only tried these 3 but will be seeking others tomorrow to test), but I have no idea what I've done to be so special.

Any insights you may have are very much appreciated.

I've only recently started looking into web scrapers and I've been trying to get some headway with them. Until I stumbled across Navalia I was trying to learn Nightmarejs, but since Navalia can work with multiple tabs I think it's a better scraper for me to cut my teeth on.

What has frustrated me with Nightmare and Navalia is a shortage of simple code examples (looping through pagination links, scraping thumbnail URLs, downloading an image, etc). What tutorials/forum discussions I've found assume a lot of previous knowledge or they're too simple and they just open a page and close it. The programming structure is also very different from what I (and I'd say most people) are used to (asynchronous). So in addition to pouring through the API you also have to get used to the way the programming flow works. This makes for a steep learning curve.

But this all brings me to something that I'd like to contribute to the project: code examples. I'm willing to tinker and toil for weeks if I have to in order to get something to work, so I'm not asking for hand-holding. Once I know how to do a few concrete things I can start to get a sense of how things work and start making my own improvements on them.

I'd like to create code snippets for doing basic tasks on specific websites. casperjs comes with a lot of code examples like this, which helped me a lot. If anyone here is willing to answer basic questions now & then I'm willing to do the work of creating code examples from the perspective of a noob in order to help others get started with Navalia.

Hi,

I would really like to be able to control the mobile prop of the captureScreenshot call.

Hi,

Thanks for a great tool. Is it possible for Navalia (or chromeless) to support reCaptchas? This is a screenshot of a search engine using navalia:

This project hasn't been updated in a while. Is it still being maintained? Trying to decide if we should use it for our next project. Thanks!

this happens when using setWindowSize. The only way I found to get around this is to insert an artificial waiting time (around 10s) or no to have the windows headeless and resizing by hand... as soon as it moves a pixel it re-renders the webpage and everything gets on the right place

Currently, exists will wait up to 10 seconds for the element to appear. If it's not there, the timer runs out and Navalia will throw. When a timer is being utilized, exists should "catch" the timeout error and return false, which is the spirit of this API (true when present, false if not).

Hello I was trying out Navalia for the first time and it's awesome 👍 I'm just having one problem. Whenever I add chrome.size(1024, 768) to a test it'll timeout on this part navalia:chrome :screenshot() > saving screenshotpage

I'm using Navalia 1.2.0 and Jest 21.0.1 with Node 8.4.0.

My test:

import { Chrome } from 'navalia'
import { toMatchImageSnapshot } from 'jest-image-snapshot'

expect.extend({ toMatchImageSnapshot })

jasmine.DEFAULT_TIMEOUT_INTERVAL = 30000

describe('Navalia', () => {
  let chrome

  beforeEach(() => {
    chrome = new Chrome()
  })

  afterEach(() => {
    chrome.done()
  })

  it('should create a screenshot', async () => {
    await chrome.goto('http://localhost:3000')
    await chrome.size(1024, 768)

    const image = await chrome.screenshot()

    expect(image).toMatchImageSnapshot()
  })
})

I've tried increasing the timeout interval to 5 minutes but it just stays stuck on the saving screenshot part until it times out. However as soon as I uncomment the chrome.size(1024, 768) part it'll work without any problems.

I'm not sure if I'm doing anything wrong or if it's an issue with Navalia, any idea what it could be?

I am not sure why my chrome instance is coming up with a blank window when I run navalia code, but it is. I am simply running the examples provided in this article:
https://codeburst.io/capturing-unused-application-code-2b7594a9fe06

As well, I am also running some code based on the code in that article. Nothing on the screen but tests are passing. Is there anyone else who has seen this problem and can possibly share what they've done to fix it? Thanks in advance!

This issue is to (hopefully) give folks better visibility into the merge status of chromeless. Below is a fairly rough roadmap of what will transpire:

1. Testing and stability of chromeless (~2 weeks or so).
2. Merging of pending PR's + features (week).
3. Moving over of Navalia into chromeless (another week or two).

Again, this is a really rough outline of what will happen based upon my current observations at this point in time. It's possible that any one of these points could take longer or happen faster. I'll try and keep folks updated via this issue thread.

As we're transitioning I'll be doing bugfixes and small requests, but the API is pretty locked down at this point in time. Feel free to create an issue in chromeless https://github.com/graphcool/chromeless/issues for any new feature requests. Thanks!

Really like this idea. Some thoughts I have:

• No clear way to "clone" a tab in Chrome...
• Would probably need to have some limitation on the amount of parallelism in requests.
• Technically might be challenging to orchestrate all of it due to the async nature of the API and how all of these pieces interact.

Hi there,

I have started to implement Navalia, it works great on production but fails when trying on localhost.

Here is my test case:

const { Chrome } = require('navalia');
const pageUrl = 'http://localhost:3000';

describe('Homepage', () => {
  let chrome = {};

  // Setup a clean instance for each test
  beforeEach(() => {
    chrome = new Chrome({
      timeout: 100000,
      flags: { headless: false }
    });
  });

  // Tear down for each test
  afterEach(() => {
    return chrome.done();
  });

  it('should load', () => {
    return chrome.goto(pageUrl)
      .then(() => chrome.exists('[data-test="username"]'))
      .then((exists) => expect(exists).toEqual(true));
  });
});

When I use my production URL instead of localhost one it works fine. But with localhost I can see chrome opening, the page loads and the close before testing. Here are the logs

  navalia:chrome :then() > Executing 1 actions +0ms
  navalia:chrome :getChromeCDP() > starting chrome +2ms
  navalia:chrome :getChromeCDP() > chrome launched on port 57985 +1s
  navalia:chrome :goto() > going to http://localhost:3000 +0ms
  navalia:chrome :goto() > waiting for pageload on http://localhost:3000 +2s
  navalia:chrome :done() > finished +2s
  navalia:chrome :done() > closing chrome +0ms

Not sure how to debug that?

Thanks,

Love this repo! I'm thinking about switching over one of my projects from NightmareJS to this but I need to use authenticated proxies. I know with the chrome command line flag you can pass in a proxy via --proxy-server=PROXY_URL_HERE. However, this only works for non-authenticated proxies. In NightmareJS they handle this by listening for an authentication event and then logging in.

How could I implement something similar for this repo?

If you can provide me some guidance, I'd be willing to implement the PR.

For example, when entering text into a search I'd like to press enter afterwards. Sometimes there's no button to press to confirm a text entry.