gwuhaolin / chrome-render Goto Github PK

View Code? Open in Web Editor NEW

95.0 9.0 10.0 61 KB

general server render base on headless chrome

JavaScript 100.00%

server-rendering headless-chrome

chrome-render's People

Contributors

Stargazers

Watchers

Forkers

agdolla tracyshen skellla shilin8805 luckyrockyma sermonis chelestewang kekewind geekwolverine

chrome-render's Issues

Project still alive? Latest updates for nodejs and Chrome?

Hi there

Curious to know if this project is still alive?

Is it tested with latest nodejs and Chrome?

I got this error after running the sample code:

const ChromeRender = require('chrome-render');
// ChromeRender.new() return a Promise, you can use async function in this way:
// const chromeRender = await ChromeRender.new();
ChromeRender.new({}).then(async(chromeRender)=>{
    const htmlString = await chromeRender.render({
       url: 'http://qq.com',
    });
    console.log(htmString)
});

ubuntu@ip-10-0-0-196:~/chrome-render$ node server
(node:18206) UnhandledPromiseRejectionWarning: Error: chrome-render timeout
    at Object.<anonymous> (/home/ubuntu/node_modules/chrome-render/index.js:6:28)
    at Module._compile (internal/modules/cjs/loader.js:678:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:689:10)
    at Module.load (internal/modules/cjs/loader.js:589:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:528:12)
    at Function.Module._load (internal/modules/cjs/loader.js:520:3)
    at Module.require (internal/modules/cjs/loader.js:626:17)
    at require (internal/modules/cjs/helpers.js:20:18)
    at Object.<anonymous> (/home/ubuntu/chrome-render/server.js:1:84)
    at Module._compile (internal/modules/cjs/loader.js:678:30)
(node:18206) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)
(node:18206) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

thanks!

How to setup a Docker container to make it work with this?

What would a Docker setup look to make chrome-render work in a container?

Started to get "Could not find node with given id"

Started to get this error. No clear context/ reason.

{ method: 'Page.enable', params: undefined }
{ method: 'DOM.enable', params: undefined }
{ method: 'Network.enable', params: undefined }
{ method: 'Console.enable', params: undefined }
{ method: 'Network.setCookie',
  params:
   { url: 'https://go2cinema.com/movies/my-cousin-rachel-2017-3128208',
     name: 'GO2CINEMA_RENDERER',
     value: '1' } }
{ method: 'Page.addScriptToEvaluateOnLoad',
  params: { scriptSource: '\nObject.defineProperty(window, \'isPageReady\', {\n  set: function() { console.log(\'P_R\'); },\n})' } }
{ method: 'Network.setExtraHTTPHeaders',
  params: { headers: { 'x-chrome-render': '1.2.0' } } }
{ method: 'Emulation.setDeviceMetricsOverride',
  params:
   { deviceScaleFactor: 1,
     fitWindow: false,
     height: 1080,
     mobile: false,
     width: 1920 } }
{ method: 'Page.navigate',
  params:
   { url: 'https://go2cinema.com/movies/my-cousin-rachel-2017-3128208',
     referrer: undefined } }
{ method: 'DOM.getDocument', params: undefined }
{ method: 'DOM.getDocument', params: undefined }
{ method: 'DOM.getOuterHTML', params: { nodeId: 1 } }
{ method: 'DOM.getOuterHTML', params: { nodeId: 5 } }
error true { code: -32000, message: 'Could not find node with given id' }
{ method: 'Page.navigate', params: { url: 'about:blank' } }
/Users/gajus/Documents/dev/applaudience/go2cinema-com-renderer/src/bin/index.js:15
  throw error;
  ^

Error: Could not find node with given id
    at /Users/gajus/Documents/dev/applaudience/go2cinema-com-renderer/node_modules/chrome-remote-interface/lib/chrome.js:83:28
    at Chrome.handleMessage (/Users/gajus/Documents/dev/applaudience/go2cinema-com-renderer/node_modules/chrome-remote-interface/lib/chrome.js:273:13)
    at WebSocket.<anonymous> (/Users/gajus/Documents/dev/applaudience/go2cinema-com-renderer/node_modules/chrome-remote-interface/lib/chrome.js:250:27)
    at emitTwo (events.js:125:13)
    at WebSocket.emit (events.js:213:7)
    at Receiver._receiver.onmessage (/Users/gajus/Documents/dev/applaudience/go2cinema-com-renderer/node_modules/ws/lib/WebSocket.js:143:54)
    at Receiver.dataMessage (/Users/gajus/Documents/dev/applaudience/go2cinema-com-renderer/node_modules/ws/lib/Receiver.js:385:14)
    at extension.decompress (/Users/gajus/Documents/dev/applaudience/go2cinema-com-renderer/node_modules/ws/lib/Receiver.js:354:40)
    at _inflate.flush (/Users/gajus/Documents/dev/applaudience/go2cinema-com-renderer/node_modules/ws/lib/PerMessageDeflate.js:279:12)
    at afterWrite (_stream_writable.js:430:3)
    at onwrite (_stream_writable.js:421:7)
    at afterTransform (_stream_transform.js:102:3)
    at TransformState.afterTransform (_stream_transform.js:75:12)
    at Zlib.callback (zlib.js:471:7)

No Chrome environment, please install Chrome 59+!

I already have chrome installed on the server
[root@izj6c1h8vqjcmblprak6fvz chrome-render]# chrome --version
Google Chrome 66.0.3359.117

However, when the project is running, it prompts
Error: chrome-render timeout
at Object. (/data/www/novel-nodeServer/node_modules/chrome-render/index.js:6:28)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
at Module.require (module.js:596:17)
at require (internal/module.js:11:18)
at Object. (/data/www/novel-nodeServer/node_modules/express-middleware-seo/core.js:2:22)
at Module._compile (module.js:652:30)
No Chrome environment, please install Chrome 59+!

How can I solve it?

Add ability to configure browser window size

This is required to render different content for responsive websites.

How to get the whole page content (the part render by js)

I use chrome-render to fetch some page, but the html returned by chrome-render doesn't include the part rendered by js........
......how to get the whole page content?......Thanks...

is it fully port of prerender? but for headless chrome

Join efforts with usus?

Hi @gwuhaolin,

chrome-render and https://github.com/gajus/usus have similar goals. Would you be interested to come as a collaborator to the latter project instead of duplicating efforts?

Use document event instead of console.log in useReady

Hello again. As of crome-remote-interface API all console messages that were logged before are evaluated when Console.messageAdded method is called. So if you have the same chrome instance or tab on each render that's called it will increase those console messages which isn't very good.

I have a proposition to change setting the variable isPageReady to listening to event on document like:

document.addEventListener('crPageRendered', ...)

I've tested it locally and it seems that it's working much better that listening to console messages. It fires only once per render and doesn't produce console messages that could probably lead to potential memory leak (because for now there is no method for clearing them, the only option destroy chrome tab).

Chrome processes remain hanging after the program is shutdown

Example, start a program with:

import ChromeRender from 'chrome-render';

const renderPage = async (chromeRenderer) => {
  const html = await chromeRenderer.render({
    url: 'https://go2cinema.com/'
  });

  console.log('html', html);
};

const renderPages = async () => {
  const chromeRenderer = await ChromeRender.new({
    maxTab: 1
  });

  await renderPage(chromeRenderer);
};

renderPages();

After the program completes, the Chrome process remains hanging.

Configure host/port

I'm not seeing a way to configure the Chrome Headless server's host and port. This is important since the headless server is not always running on same machine as the pre-render module.

The options will need to be updated in 'chrome-pool' as well to pass into CDP.

cookies setting (it appears) causes chrome-remote-interface to crash

Code:

chromeRenderer.render({
  cookies: {
    // This cookie is used by the @applaudience/go2cinema-web-app to identify
    // the prerender.
    // eslint-disable-next-line id-match
    GO2CINEMA_RENDERER: 1
  },
  deviceMetricsOverride,
  url,
  useReady: true
});

Here is the log:

{ method: 'Page.enable', params: undefined }
{ method: 'DOM.enable', params: undefined }
{ method: 'Network.enable', params: undefined }
{ method: 'Console.enable', params: undefined }
{ method: 'Network.setCookie',
  params:
   { url: 'https://go2cinema.com',
     name: 'GO2CINEMA_RENDERER',
     value: 1 } }
{ method: 'Page.addScriptToEvaluateOnLoad',
  params: { scriptSource: '\nObject.defineProperty(window, \'isPageReady\', {\n  set: function() { console.log(\'P_R\'); },\n})' } }
{ method: 'Network.setExtraHTTPHeaders',
  params: { headers: { 'x-chrome-render': '1.2.0' } } }
{ method: 'Emulation.setDeviceMetricsOverride',
  params:
   { deviceScaleFactor: 1,
     fitWindow: false,
     height: 1080,
     mobile: false,
     width: 1920 } }
{ method: 'Page.navigate',
  params: { url: 'https://go2cinema.com', referrer: undefined } }
error true

Not sure whats the issue yet.

support inject http header

Destroy Chrome instance upon SIGINT

chrome-render should have an option for automatically destroying the allocated Chrome instances upon receiving SIGINT.

const chromeRenderer = await ChromeRender.new({
  maxTab: 1
});

process.on('SIGINT', async () => {
  debug('terminating the associated Chrome instance');

  await chromeRenderer.destroyRender();

  // eslint-disable-next-line no-process-exit
  process.exit();
});

I don't think this should be enabled by default, as there could valid use cases for wanting to keep Chrome running after Node.js process terminates. However, a built-in option that enables this behaviour would be nice.