11ty / api-screenshot

A service to add web page screenshots to your Eleventy sites.

Screenshot API

A runtime service to use live website screenshots on your site.

Read the Blog post: Building an Automated Screenshot Service on Netlify in ~140 Lines of Code.

Deploy

• You will need to set an environment variable in the Netlify App UI AWS_LAMBDA_JS_RUNTIME with the value nodejs12.x. Read more at Issue #17.

Usage

Image URLs have the formats:

/:url/
/:url/:size/
/:url/:size/:aspectratio/
/:url/:size/:aspectratio/:zoom/

• url must be URI encoded.
• Valid size values:
  • small: 375×___ (default)
  • medium: 650×___
  • large: 1024×___
    • aspectratio of 9:16 is not supported (throws an error)
  • opengraph: always 1200×630, works with zoom
    • aspectratio is ignored (no errors thrown)
• Valid aspectratio values:
  • 1:1 (default)
  • 9:16
• Valid zoom values:
  • bigger (1.4 devicePixelRatio)
  • smaller (0.71 devicePixelRatio)

Advanced Options

Manual Cache Busting

If the screenshots aren’t updating at a high enough frequency you can pass in your own cache busting key using an underscore prefix _ after your URL.

This can be any arbitrary string tied to your unique build, here’s some examples that use today’s date:

/:url/_20210802/
/:url/:size/_20210802/
/:url/:size/:aspectratio/_20210802/
/:url/:size/:aspectratio/:zoom/_20210802/

Custom Wait Conditions

You can customize the conditions with which the headless browser will wait to take the screenshot. At a low level, this controls the waitUntil property in Puppeteer’s goto call. The options are:

• DOMContentLoaded wait:0
• Load event wait:1 (default)
• Load event and there have been no network connections for 500ms: wait:2
• Load event and there are fewer than two network connections for 500ms: wait:3

/:url/_wait:0/
/:url/_wait:1/
/:url/_wait:2/
/:url/_wait:3/

Custom Timeout

Number of seconds to wait before the request times out. We will attempt to simulate the stop button and return the screenshot that exists up to that point. Worst case, a default Eleventy logo is returned.

• Minimum: 3
• Maximum: 9

/:url/_timeout:3/
/:url/_timeout:9/

Combine these options

You can use any of these advanced options together, like /:url/_20210802_wait:0_timeout:2/. Order only matters to the uniqueness of the URL caching on the CDN: /:url/_20210802_wait:0/ and /:url/_wait:0_20210802/ will be functionally equivalent but make two different screenshot requests.

api-screenshot's Issues

Handling timeout

  // We need to return 200 here or Firefox won’t display the image
 // HOWEVER a 200 means that if it times out on the first attempt it will stay the default image until the next build.

Could you do a 302 to the default image instead? On a second render, the browser will check it again, since it's a temporary redirect.

Build breaking due to incorrect dependency resolution

Building fails since a8b6b69 due to chrome-aws-lambda requiring puppeteer-core 10.1.*.

npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR! 
npm ERR! While resolving: eleventy-api-screenshot@1.0.0
npm ERR! Found: puppeteer-core@13.4.1
npm ERR! node_modules/puppeteer-core
npm ERR!   puppeteer-core@"^13.4.0" from the root project
npm ERR! 
npm ERR! Could not resolve dependency:
npm ERR! peer puppeteer-core@"^10.1.0" from chrome-aws-lambda@10.1.0
npm ERR! node_modules/chrome-aws-lambda
npm ERR!   chrome-aws-lambda@"^10.1.0" from the root project
npm ERR! 
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force, or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
npm ERR! 
npm ERR! See /Users/alexandersandberg/.npm/eresolve-report.txt for a full report.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/alexandersandberg/.npm/_logs/2022-03-06T11_11_34_128Z-debug-0.log

Solution

Downgrade puppeteer-core.

Font Weight is not preserved

The font weight does not seem to be preserved.

I tried the manual cache invalidation and custom wait conditions too but get the same results.
This might be related to issue #9

Example

Original	11ty Screenshot
Original URL	11ty API URL wait:2
Original URL	11ty API URL

I'm using Bootstrap v5 and all custom CSS is minified inline. The NavBar and other styles from Bootstrap seem to load.
I'm using the default font from bootstrap and am changing the font weight using custom CSS.

SourceCode

CSS features seem to be unsupported on headless Chromium

I am pretty sure that this is the wrong place to open this issue and not a problem with your scripts, but somehow it might be helpful to see whatever solution this brings for others: I am using a specific CSS3-feature that seems to be broken in the screenshots.

Screenshot:

Original (with Chrome browser loaded):

Could you maybe point me in a direction what exactly is the issue here?

Problems:

• dark theme is default, light theme is used (cache? I just changed today to the dark theme as default, but adding random characters to the URL won't load a dark theme screenshot)
• background-clip is not working - code

Is the headless Chrome somehow in-capacitated in regards to specific features? I thought the used browser in puppeteer is the latest version...

chrome-aws-lambda not compatible with Node 16, using a replacement is too large to deploy

After cloning this repo and deploying a new Netlify site, my site function wasn't generating images even when using exactly the same URL syntax as used on 11ty.dev.

Here's my original question on the 11ty Discord forum:
https://discord.com/channels/741017160297611315/1039318195644928060

The problem appears to be chrome-aws-lambda not running correctly on Node 16. LewisDaleUK attempted to submit a PR ( #14 ) using @sparticuz/chromium as a better-maintained replacement, but changing the dependency increases the package size over the 50 MB limit for functions.

Because Netlify is switching all new deploys to Node 16, this effectively means this project can't be used to launch new sites on Netlify.

Docs: explain how to deal with CORS

I've installed this on my own domain screenshots.nicolas-hoizey.com hosted on Netlify.

It works well when run in the browser:
https://screenshots.nicolas-hoizey.com/https%3A%2F%2Fcloudfour.com%2Fthinks%2Fsvg-icon-stress-test%2F/large/

But I get a CORS error when I try to use this image in a page on another domain:

I tried to add this to my netlify.toml file, but it doesn't fix the issue:

[[headers]]
  for = "/*"
    [headers.values]
    Access-Control-Allow-Origin = "*"

Is there something specific to do because it's a Netlify Function?

There might be something here: https://stackoverflow.com/a/57974136/717195

Error: Invalid `url`: .netlify

Is there something I have to configure in Netlify to have it skip the .netlify/functions/screenshots when making a request for a screenshot?

Calling the function with this in the path results in an error Invalid url: .netlify

Thanks.

Feature: allow setting valid `size` and `aspectratio` values in a config file

As I'm trying to use it on my own domain, I would like to use different size and aspectratio values, but would like to set them without modifying the code, to keep upgrading easy.

I can work on a PR.

Chromium 50MB AWS Lambda Node 12 Mega Issue

Spawned from #13. Related to #15.

Ideally we could upgrade the node runtime https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html but newer versions of puppeteer blow past the 50MB AWS Lambda limit. I haven’t tested all versions of Puppeteer to see if there are some that work in Node 14 but don’t hit the 50MB limit yet.

Right now the workaround/requirement is to put an environment variable in your Netlify app UI setting AWS_LAMBDA_JS_RUNTIME to nodejs12.x. This cannot be set via netlify.toml (https://answers.netlify.com/t/netlify-functions-and-env-variables-from-netlify-toml/4404/28)

PR #14 uses the new package from Sparticuz: Sparticuz/chromium#8

There is a hard deadline on this one (March 31, 2023) as Node 12 in AWS is going away: https://aws.amazon.com/blogs/developer/announcing-the-end-of-support-for-node-js-12-x-in-the-aws-sdk-for-javascript-v3/

Related:

• Build plugin may workaround the netlify.toml issue https://github.com/bencao/netlify-plugin-inline-functions-env#readme
• https://github.com/puppeteer/puppeteer/blob/da92055e9c187f4d0cce6ca548bc69689c452b6d/docs/troubleshooting.md
• https://answers.netlify.com/t/chrome-aws-lambda-stopped-working-possibly-due-to-different-version-of-netlify-build/48847
• https://answers.netlify.com/t/netlify-function-with-puppeteer-breaks-if-i-make-any-changes/76924

Discord Fails to show screenshots

When linking to an 11ty screenshot image using either a direct link or an embed there is no image.
Screenshot url: https://v1.screenshot.11ty.dev/https%3A%2F%2Fsplatoon.catgirlin.space%2Fopengraph%2F/opengraph/_wait:2_202249--1162/

Link: https://splatoon.catgirlin.space/battles/VnNIaXN0b3J5RGV0YWlsLXUtYXFha3g1N21va3NzZWlic3pubW06UkVDRU5UOjIwMjIxMjExVDAyMjAwMl9jNDJmZWFhZi03NDA2LTRkY2YtYWY3MC0wNTNjZGMzYmE1Yjg/
Screenshot url: https://v1.screenshot.11ty.dev/https%3A%2F%2Fsplatoon.catgirlin.space%2Fopengraph%2Fbattles%2FVnNIaXN0b3J5RGV0YWlsLXUtYXFha3g1N21va3NzZWlic3pubW06UkVDRU5UOjIwMjIxMjExVDAyMjAwMl9jNDJmZWFhZi03NDA2LTRkY2YtYWY3MC0wNTNjZGMzYmE1Yjg%2F/opengraph/_wait:2_202249/

In both cases, the image was already generated, so it shouldn't be a timeout issue. These links work fine on their own, and I haven't had an issue with open graph images anywhere else.

Add file format option to public API

I follow Zach's approach of using the screenshot service to generate OpenGraph images, which works nicely (and is far simpler than my previous approaches).

However, my OpenGraph images are pretty simple, with simple colours, shapes, and illustrations. Unfortunately, that means the screenshot service's defaulting to JPEGs makes them look pretty crunchy, with plenty of visible artifacts and noise. It's not terrible, but it is noticeable.

Having the option to generate a PNG without needing to edit and deploy a customised version would be nice to have.

Feature: add Referer header

To show in analytics when the requests are coming from the screenshot service.

Remove the `main` branch

Make the latest versioned branch on GitHub the primary branch (right now it’s v1)

Missing emoji

via the original blog post:

Update (July 30, 2021): The other issue I noticed with using Puppeteer in a Lambda is that emoji are not available to the rendered content. So if a site is using Emoji they do not render. It looks like Matic Jurglič may have a workaround to solve this.

https://www.zachleat.com/web/screenshots/

I found that Chinese characters cannot be displayed correctly in screenshots while using.

I found that Chinese characters cannot be displayed correctly in screenshots while using.
I tried to make some modifications based on the prompts from ChatGPT (downloaded a Chinese font and placed it in the same directory as the script), but it doesn't seem to work.

我在使用过程中发现中文字符在截图中无法正常显示，
我尝试着通过 ChatGPT 的提示做了一些修改（下载了一个中文字体放在脚本的同目录下），但是似乎并不起作用。

await page.evaluate(() => {
    const font = new FontFace('Microsoft YaHei', 'url(./微软雅黑.ttf)'); // 这里使用相对路径引用字体文件
    document.fonts.add(font);
    document.body.style.fontFamily = 'Microsoft YaHei,sans-serif !important'; // 设置使用自定义字体
});

Explicitly identify service by User-Agent

My static site is hosted through Cloudflare and I've found that the service bumps captchas by screenshoting them instead of the actual page. I tried to write a rule to explicitly allow requests from this service to bypass the protection. But I came across the fact that from the systems point of view these requests are practically no different from requests from other bots:

It would be nice to change at least the User-Agent to something that explicitly identifies the service as a screenshot robot, maybe 11tyScreenshoter or something. This would make Cloudflare cool its heels and make it easier to detect and filter all requests with this User-Agent as explicitly allowed by custom rules.

unnecessary code?

I feel like this piece is doing nothing...

api-screenshot/functions/screenshot.js

Lines 43 to 45 in 36407cf

	if(format === "jpeg") {
	quality = 80;
	}

Feature: Support #anchor links in passed URL

An in-page anchor link would navigate to show that section in the viewport before taking the screenshot.