Coder Social home page Coder Social logo

pending-xhr-puppeteer's Introduction

Pending XHR Puppeteer

npm version Build Status

| Introduction | Installation | Usage | Contribute |

Introduction

Pending XHR Puppeteer is a tool that detect when there is xhr requests not yet finished. You can use it to have a xhr requests count or to wait for all xhr requests to be finished.

Installation

To install with yarn :

yarn add pending-xhr-puppeteer -D

To install with npm :

npm install pending-xhr-puppeteer --save-dev

Usage

wait for all xhr requests to be finished

const puppeteer = require('puppeteer');
const { PendingXHR } = require('pending-xhr-puppeteer');

const browser = await puppeteer.launch({
  headless: true,
  args,
});

const page = await browser.newPage();
const pendingXHR = new PendingXHR(page);
await page.goto(`http://page-with-xhr`);
// Here all xhr requests are not finished
await pendingXHR.waitForAllXhrFinished();
// Here all xhr requests are finished

Get the number of pending xhr

const puppeteer = require('puppeteer');
const { PendingXHR } = require('pending-xhr-puppeteer');

const browser = await puppeteer.launch({
  headless: true,
  args,
});

const page = await browser.newPage();
const pendingXHR = new PendingXHR(page);
await page.goto(`http://page-with-xhr`);
console.log(pendingXHR.pendingXhrCount());
// Display the number of xhr pending

Usage with Promise.race

If you need to wait xhrs but not longer than a specific time, You can race pending-xhr-puppeteer and setTimeout in a Promise.race.

const puppeteer = require('puppeteer');
const { PendingXHR } = require('pending-xhr-puppeteer');

const browser = await puppeteer.launch({
  headless: true,
  args,
});

const page = await browser.newPage();
const pendingXHR = new PendingXHR(page);
await page.goto(`http://page-with-xhr`);
// We will wait max 1 seconde for xhrs
await Promise.race([
  pendingXHR.waitForAllXhrFinished(),
  new Promise(resolve => {
    setTimeout(resolve, 1000);
  }),
]);
console.log(pendingXHR.pendingXhrCount());
// May or may not have pending xhrs

Wait for all xhr triggered by all the events of the page

You can use this lib to wait for xhr triggered by any event from the UI (click, typing, ...).

Exemple :

const pendingXHR = new PendingXHR(page);
await page.goto(`http://page-with-xhr`);
await page.click('.my-selector'); // This action will trigger some xhr
// Here all xhr requests triggered by the click are not finished
await pendingXHR.waitForAllXhrFinished();
// Here all xhr requests triggered by the click are finished
// You can then perform an other xhr producer event
await page.click('.my-selector2'); // This action will trigger some xhr
// You can rewait them
await pendingXHR.waitForAllXhrFinished();

This mode is usefull to test SPA, you d'ont have to recreate a new instance at each time. The request listeners will be deleted when you leave the page.

Wait for all xhr triggered by an event of the page

with waitOnceForAllXhrFinished you can wait until all the xhr are finished and them remove the listeners. This is usefull when waitForAllXhrFinished has a leaking behaviour for you.

Exemple :

const pendingXHR = new PendingXHR(page);
await page.goto(`http://page-with-xhr`);
await page.click('.my-selector'); // This action will trigger some xhr
// Here all xhr requests triggered by the click are not finished
await pendingXHR.waitOnceForAllXhrFinished();
// Here all xhr requests triggered by the click are finished
// All pendingXHR listeners are remove here too

Contribute

git clone https://github.com/jtassin/pending-xhr-puppeteer.git
cd pending-xhr-puppeteer
yarn
yarn test

Merge requests and issues are welcome.

pending-xhr-puppeteer's People

Contributors

jtassin avatar mendelgusmao avatar renovate-bot avatar renovate[bot] avatar sdenardi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

pending-xhr-puppeteer's Issues

How to get data after waitForAllXhrFinished();

Hi,

I'm a newbie and and I want to get data from HTTPRequest number 2. But after the code await pendingXHR.waitForAllXhrFinished(), how do I get the data and close the browser? Something like this?

page.on('response', async (response) => { return response } }); await browser.close();

Thank you for help

How to get this to work on puppeteer-cluster?

There's a puppeteer parallel version called puppeteer-cluster: https://github.com/thomasdondorf/puppeteer-cluster

How can we get pending-xhr to work with it?

This is a sample puppeteer-cluster code:

const { Cluster } = require('puppeteer-cluster');

(async () => {
  const cluster = await Cluster.launch({
    concurrency: Cluster.CONCURRENCY_CONTEXT,
    maxConcurrency: 2,
  });

  await cluster.task(async ({ page, data: url }) => {
    await page.goto(url);
    const screen = await page.screenshot();
    // Store screenshot, do something else
  });

  cluster.queue('http://www.google.com/');
  cluster.queue('http://www.wikipedia.org/');
  // many more pages

  await cluster.idle();
  await cluster.close();
})();

Suggestion

Firstly, this is a great module!

I've used it recently but had to modify it for my purposes - I didn't want to monitor XHR requests but did need to monitor images and fonts in my case.

I was thinking of releasing the code as a separate module called pending-requests-puppeteer, whereby a user instantiates the Class and passes through the resource type(s) they want to monitor.
Seeing as though the changes are fairly minimal (constructor changes, method names and type checking) I wanted to check if you wanted me to PR them into your module?

Action Required: Fix Renovate Configuration

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: undefined. Note: this is a nested preset so please contact the preset author if you are unable to fix it yourself.

can it catch late response?

hi ! first of all thanks for good library

i have some questions

if i click some element and that element has following function
()=>{ setTimeout(()=>{ ajax.get(...) },5000) }

in this situation this library can catch it properly ??

Dependency is throwing error in ubuntu machine

Running node service using babel and pm2 on ubuntu (Ubuntu 16.04.4 LTS)
Node version - 6.x

/srv/deployments/profile-service/node_modules/pending-xhr-puppeteer/lib/index.js:19
);
^
SyntaxError: Unexpected token )
at createScript (vm.js:56:10)
at Object.runInThisContext (vm.js:97:10)
at Module._compile (module.js:549:28)
at Module._extensions..js (module.js:586:10)
at Object.require.extensions.(anonymous function) [as .js] (/srv/deployments/profile-service/node_modules/babel-register/lib/node.js:152:7)
at Module.load (module.js:494:32)
at tryModuleLoad (module.js:453:12)
at Function.Module._load (module.js:445:3)
at Module.require (module.js:504:17)
at require (internal/module.js:20:19)
at Object. (/srv/deployments/profile-service/commons/utils/DarbyHelper.js:9:1)
at Module._compile (module.js:577:32)
at loader (/srv/deployments/profile-service/node_modules/babel-register/lib/node.js:144:5)
at Object.require.extensions.(anonymous function) [as .js] (/srv/deployments/profile-service/node_modules/babel-register/lib/node.js:154:7)
at Module.load (module.js:494:32)
at tryModuleLoad (module.js:453:12)

TypeError: request.resolver is not a function

I get these warning messages constantly on my program when I use await pendingXHR.waitForAllXhrFinished();
It doesn't crash my program but console logs are filled with the same error. If this isn't a bug, how can I fix that?

My versions:
"pending-xhr-puppeteer": "^2.1.0",
"puppeteer": "^1.20.0",

(node:3760) UnhandledPromiseRejectionWarning: TypeError: request.resolver is not a function
    at Page.PendingXHR.requestFinishedListener (C:\Users\omerf\Desktop\folder\project\node_modules\pending-xhr-puppeteer\lib\index.js:86:25)
    at Page.emit (events.js:228:7)
    at Page.EventEmitter.emit (domain.js:475:20)
    at NetworkManager.<anonymous> (C:\Users\omerf\Desktop\folder\project\node_modules\puppeteer\lib\Page.js:113:76)
    at NetworkManager.emit (events.js:223:5)
    at NetworkManager.EventEmitter.emit (domain.js:475:20)
    at NetworkManager._onLoadingFinished (C:\Users\omerf\Desktop\folder\project\node_modules\puppeteer\lib\NetworkManager.js:291:10)
    at CDPSession.emit (events.js:223:5)
    at CDPSession.EventEmitter.emit (domain.js:475:20)
    at CDPSession._onMessage (C:\Users\omerf\Desktop\folder\project\node_modules\puppeteer\lib\Connection.js:200:12)
    at Connection._onMessage (C:\Users\omerf\Desktop\folder\project\node_modules\puppeteer\lib\Connection.js:112:17)
    at WebSocket.<anonymous> (C:\Users\omerf\Desktop\folder\project\node_modules\puppeteer\lib\WebSocketTransport.js:44:24)
    at WebSocket.onMessage (C:\Users\omerf\Desktop\folder\project\node_modules\ws\lib\event-target.js:120:16)
    at WebSocket.emit (events.js:223:5)
    at WebSocket.EventEmitter.emit (domain.js:475:20)
    at Receiver.receiverOnMessage (C:\Users\omerf\Desktop\folder\project\node_modules\ws\lib\websocket.js:789:20)
(node:3760) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 65)

Edge Cases?

Lets say ajax request happen after clicking button after 3-5 sec ? Do it work ? Or this is based on particular instant like because sometime ajax can delay lets say after 2-3 sec and this plugin may check before ajax happen?

I was trying to use as below code usually don't work. Thanks.


await Promise.all([page.waitForNavigation({waitUntil: 'networkidle2'})], btnHandler.click())

Let me know if we have other solutions too :)

Question about situation when there are multiple XHR requests

page.on('request', request => {
if (request.resourceType() === this.resourceType) {
this.pendingXhrs.add(request);
if (!this.promise) {
this.promise = new Promise(resolve => {
this.resolver = resolve;
});
}
}
});

From my understanding of code, it seems that if there are two pending XHRs, only the first one will trigger a creation of promise and link its resolve() to resolver. The second one will not have its own promise object and resolver. How can you make sure each request is handled properly in this way. Correct me if I am wrong :)

Support multiple resource types

Currently there's support for single only resourceType, but I want to wait when several resourceTypes have finished such as document, script, fetch

await needs async

await needs async error
Would appreciate if there is live example of usage

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

  • chore(deps): update dependency eslint-plugin-jest to v25.7.0
  • chore(deps): update dependency lint-staged to v12.5.0
  • chore(deps): update dependency release-it to v14.14.3
  • chore(deps): update dependency rollup-plugin-typescript2 to v0.36.0
  • chore(deps): update dependency tslib to v2.6.2
  • chore(deps): update jest monorepo (@types/jest, jest)
  • chore(deps): update actions/checkout action to v4
  • chore(deps): update actions/setup-node action to v4
  • chore(deps): update dependency @keplr/eslint-config-backend-node to v3
  • chore(deps): update dependency @release-it/conventional-changelog to v8
  • chore(deps): update dependency @types/node to v20
  • chore(deps): update dependency @types/puppeteer to v7
  • chore(deps): update dependency delay to v6
  • chore(deps): update dependency eslint to v9
  • chore(deps): update dependency eslint-config-prettier to v9
  • chore(deps): update dependency eslint-plugin-jest to v28
  • chore(deps): update dependency husky to v9
  • chore(deps): update dependency lint-staged to v15
  • chore(deps): update dependency prettier to v3
  • chore(deps): update dependency puppeteer to v22
  • chore(deps): update dependency release-it to v17
  • chore(deps): update dependency rollup to v4
  • chore(deps): update dependency typescript to v5
  • chore(deps): update jest monorepo to v29 (major) (@types/jest, jest, ts-jest)
  • ๐Ÿ” Create all rate-limited PRs at once ๐Ÿ”

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

github-actions
.github/workflows/test.yml
  • actions/checkout v2
  • actions/setup-node v2
  • actions/checkout v2
  • actions/setup-node v2
npm
package.json
  • @keplr/eslint-config-backend-node 2.5.1
  • @release-it/conventional-changelog 3.3.0
  • @types/express 4.17.13
  • @types/jest 27.0.3
  • @types/node 16.11.11
  • @types/puppeteer 5.4.4
  • auto-changelog 2.3.0
  • delay 5.0.0
  • eslint 7.32.0
  • eslint-config-prettier 8.3.0
  • eslint-plugin-jest 25.3.0
  • husky 7.0.4
  • jest 27.4.3
  • lint-staged 12.1.2
  • prettier 2.5.0
  • puppeteer 5.5.0
  • release-it 14.11.8
  • rollup 2.60.2
  • rollup-plugin-typescript 1.0.1
  • rollup-plugin-typescript2 0.31.1
  • ts-jest 27.0.7
  • tsc 2.0.3
  • tslib 2.3.1
  • typescript 4.3.5

  • Check this box to trigger a request for Renovate to run again on this repository

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.