venkatarao-palaparthi / third-party-web

Data on third party entities and their impact on the web.

Third Party Web

Check out the shiny new web UI https://www.thirdpartyweb.today/

Data on third party entities and their impact on the web.

This document is a summary of which third party scripts are most responsible for excessive JavaScript execution on the web today.

Table of Contents

1. Goals
2. Methodology
3. NPM Module
4. Updates
5. Data
   1. Summary
   2. How to Interpret
   3. Third Parties by Category
      1. Advertising
      2. Analytics
      3. Social
      4. Video
      5. Developer Utilities
      6. Hosting Platforms
      7. Marketing
      8. Customer Success
      9. Content & Publishing
      10. CDNs
      11. Tag Management
      12. Consent Management Provider
      13. Mixed / Other
   4. Third Parties by Total Impact
6. Future Work
7. FAQs
8. Contributing

Goals

1. Quantify the impact of third party scripts on the web.
2. Identify the third party scripts on the web that have the greatest performance cost.
3. Give developers the information they need to make informed decisions about which third parties to include on their sites.
4. Incentivize responsible third party script behavior.
5. Make this information accessible and useful.

Methodology

HTTP Archive is an inititiave that tracks how the web is built. Every month, ~4 million sites are crawled with Lighthouse on mobile. Lighthouse breaks down the total script execution time of each page and attributes the execution to a URL. Using BigQuery, this project aggregates the script execution to the origin-level and assigns each origin to the responsible entity.

NPM Module

The entity classification data is available as an NPM module.

const {getEntity} = require('third-party-web')
const entity = getEntity('https://d36mpcpuzc4ztk.cloudfront.net/js/visitor.js')
console.log(entity)
//   {
//     "name": "Freshdesk",
//     "homepage": "https://freshdesk.com/",
//     "categories": ["customer-success"],
//     "domains": ["d36mpcpuzc4ztk.cloudfront.net"]
//   }

Updates

2019-02-01 dataset

Huge props to WordAds for reducing their impact from ~2.5s to ~200ms on average! A few entities are showing considerably less data this cycle (Media Math, Crazy Egg, DoubleVerify, Bootstrap CDN). Perhaps they've added new CDNs/hostnames that we haven't identified or the basket of sites in HTTPArchive has shifted away from their usage.

2019-03-01 dataset

Almost 2,000 entities tracked now across ~3,000+ domains! Huge props to @simonhearne for making this massive increase possible. Tag Managers have now been split out into their own category since they represented such a large percentage of the "Mixed / Other" category.

2019-05-06 dataset

Google Ads clarified that www.googletagservices.com serves more ad scripts than generic tag management, and it has been reclassified accordingly. This has dropped the overall Tag Management share considerably back down to its earlier position.

2019-05-13 dataset

A shortcoming of the attribution approach has been fixed. Total usage is now reported based on the number of pages in the dataset that use the third-party, not the number of scripts. Correspondingly, all average impact times are now reported per page rather than per script. Previously, a third party could appear to have a lower impact or be more popular simply by splitting their work across multiple files.

Third-parties that performed most of their work from a single script should see little to no impact from this change, but some entities have seen significant ranking movement. Hosting providers that host entire pages are, understandably, the most affected.

Some notable changes below:

Third-Party	Previously (per-script)	Now (per-page)
Beeketing	137 ms	465 ms
Sumo	263 ms	798 ms
Tumblr	324 ms	1499 ms
Yandex APIs	393 ms	1231 ms
Google Ads	402 ms	1285 ms
Wix	972 ms	5393 ms

Data

Summary

Across top ~4 million sites, ~2700 origins account for ~57% of all script execution time with the top 50 entities already accounting for ~47%. Third party script execution is the majority chunk of the web today, and it's important to make informed choices.

How to Interpret

Each entity has a number of data points available.

1. Usage (Total Number of Occurrences) - how many scripts from their origins were included on pages
2. Total Impact (Total Execution Time) - how many seconds were spent executing their scripts across the web
3. Average Impact (Average Execution Time) - on average, how many milliseconds were spent executing each script
4. Category - what type of script is this

Third Parties by Category

This section breaks down third parties by category. The third parties in each category are ranked from first to last based on the average impact of their scripts. Perhaps the most important comparisons lie here. You always need to pick an analytics provider, but at least you can pick the most well-behaved analytics provider.

Overall Breakdown

Unsurprisingly, ads account for the largest identifiable chunk of third party script execution.

Advertising

These scripts are part of advertising networks, either serving or measuring.

Rank	Name	Usage	Average Impact
1	GumGum	2,984	22 ms
2	ExoClick	1,213	43 ms
3	SublimeSkinz	1,121	58 ms
4	MailMunch	5,107	60 ms
5	Geniee	1,164	68 ms
6	District M	3,096	69 ms
7	Salesforce.com	1,020	70 ms
8	Rakuten Marketing	1,210	73 ms
9	BlueKai	2,331	75 ms
10	PushCrew	2,927	76 ms
11	FreakOut	3,828	76 ms
12	AppNexus	1,849	78 ms
13	Constant Contact	1,315	80 ms
14	Scorecard Research	7,851	83 ms
15	Gemius	4,559	87 ms
16	Tynt	23,214	88 ms
17	Rubicon Project	40,208	90 ms
18	Nativo	1,174	101 ms
19	Popads	2,898	103 ms
20	Unbounce	3,591	105 ms
21	Amazon Ads	26,048	106 ms
22	Outbrain	8,216	107 ms
23	i-mobile	3,228	110 ms
24	LINE Corporation	5,184	113 ms
25	Sharethrough	1,670	114 ms
26	Adroll	24,347	115 ms
27	Intent Media	8,019	116 ms
28	Bing Ads	15,836	117 ms
29	Index Exchange	5,527	122 ms
30	JuicyAds	1,498	125 ms
31	DTSCOUT	12,711	132 ms
32	Yahoo! JAPAN Ads	4,367	137 ms
33	Attentive	1,206	146 ms
34	WordAds	18,479	149 ms
35	OpenX	3,701	160 ms
36	Smart AdServer	4,272	162 ms
37	Skimbit	15,827	167 ms
38	OptiMonk	1,602	173 ms
39	Teads	3,464	177 ms
40	Cxense	4,375	179 ms
41	TrafficStars	2,040	187 ms
42	Adform	6,868	187 ms
43	sovrn	4,758	190 ms
44	Tribal Fusion	1,064	190 ms
45	Unruly Media	1,193	193 ms
46	Bizible	1,300	194 ms
47	Perfect Market	1,597	218 ms
48	GetResponse	1,073	219 ms
49	Sortable	1,551	220 ms
50	Yahoo!	2,071	230 ms
51	Criteo	62,765	235 ms
52	Adyoulike	1,235	242 ms
53	Permutive	9,558	243 ms
54	Refersion	1,323	249 ms
55	VigLink	7,100	309 ms
56	Privy	13,047	309 ms
57	Yandex Ads	25,906	313 ms
58	Pubmatic	52,655	319 ms
59	Media.net	14,562	342 ms
60	Taboola	24,645	364 ms
61	LongTail Ad Solutions	3,444	365 ms
62	Klaviyo	14,595	392 ms
63	piano	1,183	459 ms
64	DoubleVerify	1,776	552 ms
65	RevJet	1,211	553 ms
66	MGID	8,214	575 ms
67	Infolinks	3,699	583 ms
68	33 Across	1,197	611 ms
69	Vidible	1,148	624 ms
70	Admixer for Publishers	1,043	642 ms
71	Mediavine	6,490	655 ms
72	Bridgewell DSP	14,079	791 ms
73	Moat	6,917	811 ms
74	LoyaltyLion	1,301	830 ms
75	Adloox	5,759	921 ms
76	Integral Ad Science	8,328	994 ms
77	Sizmek	1,487	1046 ms
78	LKQD	1,337	1448 ms
79	Google/Doubleclick Ads	680,998	1457 ms
80	fam	2,135	1807 ms

Analytics

These scripts measure or track users and their actions. There's a wide range in impact here depending on what's being tracked.

Rank	Name	Usage	Average Impact
1	StatCounter	3,414	59 ms
2	Treasure Data	4,857	62 ms
3	WordPress Site Stats	3,746	64 ms
4	Roxr Software	1,631	66 ms
5	Amplitude Mobile Analytics	2,430	69 ms
6	Heap	2,609	72 ms
7	Mixpanel	7,528	75 ms
8	Google Analytics	1,202,015	77 ms
9	Quantcast	4,253	78 ms
10	Chartbeat	4,685	79 ms
11	Hotjar	177,526	84 ms
12	Parse.ly	3,361	90 ms
13	Searchanise	4,304	91 ms
14	Smart Insight Tracking	1,162	93 ms
15	etracker	1,643	94 ms
16	Snowplow	5,845	95 ms
17	CallRail	7,019	101 ms
18	Marchex	2,512	104 ms
19	Nielsen NetRatings SiteCensus	11,880	104 ms
20	Baidu Analytics	10,740	106 ms
21	Crazy Egg	4,994	110 ms
22	Evidon	1,067	111 ms
23	ContentSquare	1,368	134 ms
24	VWO	4,728	158 ms
25	Trust Pilot	15,202	164 ms
26	Net Reviews	2,537	176 ms
27	PageSense	1,294	180 ms
28	FullStory	7,655	184 ms
29	Segment	9,541	191 ms
30	Kampyle	1,095	206 ms
31	Optimizely	19,584	224 ms
32	Nosto	1,947	225 ms
33	UserReport	1,309	229 ms
34	BounceX	1,387	235 ms
35	mPulse	13,190	246 ms
36	PowerReviews	1,043	261 ms
37	Marketo	1,389	357 ms
38	Histats	13,545	361 ms
39	Inspectlet	5,605	361 ms
40	Bazaarvoice	1,845	397 ms
41	Snapchat	13,345	410 ms
42	Salesforce	10,894	479 ms
43	Lucky Orange	7,531	491 ms
44	Feefo.com	1,686	502 ms
45	Yandex Metrica	292,542	580 ms
46	Gigya	2,262	580 ms
47	Ezoic	1,331	581 ms
48	Revolver Maps	1,146	608 ms
49	AB Tasty	3,010	1594 ms

Social

These scripts enable social features.

Rank	Name	Usage	Average Impact
1	AddToAny	24,501	87 ms
2	Pinterest	17,278	89 ms
3	TikTok	2,380	90 ms
4	LinkedIn	14,057	111 ms
5	VK	13,492	122 ms
6	Twitter	214,251	139 ms
7	Kakao	18,109	158 ms
8	Instagram	9,444	184 ms
9	Facebook	1,461,908	229 ms
10	ShareThis	40,215	234 ms
11	SocialShopWave	2,044	302 ms
12	AddThis	118,289	404 ms
13	Disqus	1,252	995 ms
14	LiveJournal	3,680	1327 ms
15	PIXNET	15,434	1508 ms
16	Tumblr	7,974	2048 ms

Video

These scripts enable video player and streaming functionality.

Rank	Name	Usage	Average Impact
1	Vimeo	10,406	355 ms
2	Brightcove	6,617	809 ms
3	YouTube	408,343	849 ms
4	Wistia	13,083	929 ms
5	Twitch	1,068	1898 ms

Developer Utilities

These scripts are developer utilities (API clients, site monitoring, fraud detection, etc).

Rank	Name	Usage	Average Impact
1	Key CDN	1,470	55 ms
2	LightWidget	2,329	66 ms
3	Siteimprove	1,510	76 ms
4	Trusted Shops	5,639	79 ms
5	New Relic	13,101	95 ms
6	Accessibe	3,561	99 ms
7	GetSiteControl	1,815	99 ms
8	Riskified	1,074	106 ms
9	Affirm	2,501	113 ms
10	iubenda	12,784	124 ms
11	Swiftype	1,544	135 ms
12	Seznam	1,655	135 ms
13	Bold Commerce	13,136	144 ms
14	Cookiebot	20,839	147 ms
15	Sift Science	1,080	148 ms
16	Other Google APIs/SDKs	580,694	168 ms
17	Amazon Pay	5,751	169 ms
18	TrustArc	1,374	179 ms
19	MaxCDN Enterprise	2,415	199 ms
20	GitHub	1,661	232 ms
21	Fraudlogix	2,245	263 ms
22	Fastly	6,806	273 ms
23	PayPal	15,436	343 ms
24	Stripe	23,539	389 ms
25	Cloudflare	53,332	465 ms
26	Google Maps	267,444	520 ms
27	AppDynamics	1,283	526 ms
28	Secomapp	2,151	546 ms
29	Bugsnag	8,688	569 ms
30	Rambler	7,714	690 ms
31	GoDaddy	6,687	700 ms
32	Sentry	9,062	715 ms
33	Signyfyd	1,691	867 ms
34	Mapbox	5,206	877 ms
35	Yandex APIs	27,480	1080 ms
36	POWr	16,407	1364 ms
37	Esri ArcGIS	1,692	4750 ms

Hosting Platforms

These scripts are from web hosting platforms (WordPress, Wix, Squarespace, etc). Note that in this category, this can sometimes be the entirety of script on the page, and so the "impact" rank might be misleading. In the case of WordPress, this just indicates the libraries hosted and served by WordPress not all sites using self-hosted WordPress.

Rank	Name	Usage	Average Impact
1	Blogger	23,690	327 ms
2	Salesforce Commerce Cloud	2,623	342 ms
3	WordPress	92,595	404 ms
4	Dealer	8,639	433 ms
5	Ecwid	1,399	739 ms
6	Tilda	7,478	783 ms
7	Squarespace	39,492	911 ms
8	Adobe Business Catalyst	3,181	971 ms
9	Shopify	107,323	1058 ms
10	Weebly	13,271	1433 ms
11	CDK Dealer Management	3,429	1568 ms
12	Hatena Blog	13,206	1579 ms
13	Wix	54,909	5963 ms

Marketing

These scripts are from marketing tools that add popups/newsletters/etc.

Rank	Name	Usage	Average Impact
1	Beeketing	3,986	74 ms
2	RD Station	6,547	76 ms
3	iZooto	1,376	83 ms
4	Ve	1,848	149 ms
5	Listrak	1,073	154 ms
6	Hubspot	33,137	181 ms
7	Yotpo	13,632	202 ms
8	Mailchimp	23,376	204 ms
9	OptinMonster	7,494	260 ms
10	Bronto Software	1,057	261 ms
11	Pardot	1,435	404 ms
12	Albacross	1,920	490 ms
13	Sumo	18,438	686 ms
14	Bigcommerce	10,096	976 ms
15	Drift	6,565	1279 ms
16	Judge.me	8,305	1376 ms
17	PureCars	2,697	1901 ms
18	Tray Commerce	3,173	2328 ms

Customer Success

These scripts are from customer support/marketing providers that offer chat and contact solutions. These scripts are generally heavier in weight.

Rank	Name	Usage	Average Impact
1	SnapEngage	1,084	59 ms
2	Pure Chat	4,662	70 ms
3	Foursixty	1,411	81 ms
4	LivePerson	4,303	112 ms
5	Crisp	4,497	124 ms
6	iPerceptions	1,992	132 ms
7	Comm100	1,321	134 ms
8	Intercom	15,657	245 ms
9	Help Scout	2,183	258 ms
10	Tidio Live Chat	12,655	383 ms
11	Tawk.to	63,460	405 ms
12	LiveChat	19,469	411 ms
13	ContactAtOnce	3,247	491 ms
14	Jivochat	45,111	553 ms
15	Olark	6,904	656 ms
16	Smartsupp	14,643	794 ms
17	ZenDesk	69,488	892 ms

Content & Publishing

These scripts are from content providers or publishing-specific affiliate tracking.

Rank	Name	Usage	Average Impact
1	Accuweather	1,499	72 ms
2	Booking.com	1,657	157 ms
3	Tencent	2,257	163 ms
4	OpenTable	1,563	165 ms
5	SnapWidget	2,610	174 ms
6	Covert Pics	1,007	185 ms
7	AMP	74,588	308 ms
8	Medium	1,157	474 ms
9	Embedly	5,513	514 ms
10	Spotify	3,225	602 ms
11	issuu	1,938	669 ms
12	SoundCloud	4,464	987 ms
13	Dailymotion	1,838	1253 ms

CDNs

These are a mixture of publicly hosted open source libraries (e.g. jQuery) served over different public CDNs and private CDN usage. This category is unique in that the origin may have no responsibility for the performance of what's being served. Note that rank here does not imply one CDN is better than the other. It simply indicates that the scripts being served from that origin are lighter/heavier than the ones served by another.

Rank	Name	Usage	Average Impact
1	Bootstrap CDN	1,449	55 ms
2	Adobe TypeKit	18,280	94 ms
3	Yandex CDN	25,669	185 ms
4	JSDelivr CDN	97,342	193 ms
5	Microsoft Hosted Libs	4,943	203 ms
6	Monotype	4,073	212 ms
7	Azure Web Services	10,612	228 ms
8	jQuery CDN	167,644	269 ms
9	Unpkg	6,918	274 ms
10	Google CDN	947,798	288 ms
11	Akamai	7,364	335 ms
12	Cloudflare CDN	117,339	339 ms
13	FontAwesome CDN	47,298	413 ms
14	CreateJS CDN	3,518	2322 ms

Tag Management

These scripts tend to load lots of other scripts and initiate many tasks.

Rank	Name	Usage	Average Impact
1	Google Tag Manager	722,184	115 ms
2	Yahoo! Tag Manager	5,822	116 ms
3	TagCommander	1,072	145 ms
4	BrightTag / Signal	3,674	158 ms
5	Adobe Tag Manager	32,617	311 ms
6	Tealium	12,108	362 ms
7	Ensighten	3,768	443 ms

Consent Management Provider

IAB Consent Management Providers are the 'Cookie Consent' popups used by many publishers. They're invoked for every page and sit on the critical path between a page loading and adverts being displayed.

Rank	Name	Usage	Average Impact
1	Quantcast Choice	12,499	121 ms
2	Optanon	8,102	128 ms

Mixed / Other

These are miscellaneous scripts delivered via a shared origin with no precise category or attribution. Help us out by identifying more origins!

Rank	Name	Usage	Average Impact
1	ResponsiveVoice	1,243	70 ms
2	Amazon Web Services	39,066	160 ms
3	All Other 3rd Parties	1,393,280	320 ms
4	Parking Crew	5,147	326 ms
5	Heroku	2,007	609 ms
6	uLogin	2,316	1223 ms

Third Parties by Total Impact

This section highlights the entities responsible for the most script execution across the web. This helps inform which improvements would have the largest total impact.

Name	Popularity	Total Impact	Average Impact
Google/Doubleclick Ads	680,998	991,972 s	1457 ms
All Other 3rd Parties	1,393,280	445,644 s	320 ms
YouTube	408,343	346,701 s	849 ms
Facebook	1,461,908	335,212 s	229 ms
Wix	54,909	327,400 s	5963 ms
Google CDN	947,798	272,554 s	288 ms
Yandex Metrica	292,542	169,622 s	580 ms
Google Maps	267,444	139,019 s	520 ms
Shopify	107,323	113,512 s	1058 ms
Other Google APIs/SDKs	580,694	97,640 s	168 ms
Google Analytics	1,202,015	92,974 s	77 ms
Google Tag Manager	722,184	83,404 s	115 ms
ZenDesk	69,488	62,012 s	892 ms
AddThis	118,289	47,740 s	404 ms
jQuery CDN	167,644	45,069 s	269 ms
Cloudflare CDN	117,339	39,773 s	339 ms
WordPress	92,595	37,441 s	404 ms
Squarespace	39,492	35,967 s	911 ms
Twitter	214,251	29,810 s	139 ms
Yandex APIs	27,480	29,681 s	1080 ms
Tawk.to	63,460	25,679 s	405 ms
Jivochat	45,111	24,937 s	553 ms
Cloudflare	53,332	24,773 s	465 ms
PIXNET	15,434	23,272 s	1508 ms
AMP	74,588	22,996 s	308 ms
POWr	16,407	22,375 s	1364 ms
Hatena Blog	13,206	20,855 s	1579 ms
FontAwesome CDN	47,298	19,543 s	413 ms
Weebly	13,271	19,021 s	1433 ms
JSDelivr CDN	97,342	18,831 s	193 ms
Pubmatic	52,655	16,794 s	319 ms
Tumblr	7,974	16,328 s	2048 ms
Hotjar	177,526	14,882 s	84 ms
Criteo	62,765	14,736 s	235 ms
Sumo	18,438	12,648 s	686 ms
Wistia	13,083	12,148 s	929 ms
Smartsupp	14,643	11,630 s	794 ms
Judge.me	8,305	11,424 s	1376 ms
Bridgewell DSP	14,079	11,133 s	791 ms
Adobe Tag Manager	32,617	10,130 s	311 ms
Bigcommerce	10,096	9,855 s	976 ms
ShareThis	40,215	9,410 s	234 ms
Stripe	23,539	9,146 s	389 ms
Taboola	24,645	8,975 s	364 ms
Drift	6,565	8,400 s	1279 ms
Integral Ad Science	8,328	8,281 s	994 ms
CreateJS CDN	3,518	8,169 s	2322 ms
Yandex Ads	25,906	8,114 s	313 ms
Esri ArcGIS	1,692	8,036 s	4750 ms
LiveChat	19,469	8,004 s	411 ms
Blogger	23,690	7,744 s	327 ms
Tray Commerce	3,173	7,388 s	2328 ms
Sentry	9,062	6,480 s	715 ms
Amazon Web Services	39,066	6,239 s	160 ms
Hubspot	33,137	6,003 s	181 ms
Tilda	7,478	5,858 s	783 ms
Klaviyo	14,595	5,714 s	392 ms
Moat	6,917	5,609 s	811 ms
Snapchat	13,345	5,466 s	410 ms
CDK Dealer Management	3,429	5,376 s	1568 ms
Brightcove	6,617	5,352 s	809 ms
Rambler	7,714	5,319 s	690 ms
Adloox	5,759	5,306 s	921 ms
PayPal	15,436	5,287 s	343 ms
Salesforce	10,894	5,221 s	479 ms
PureCars	2,697	5,127 s	1901 ms
Media.net	14,562	4,976 s	342 ms
Bugsnag	8,688	4,945 s	569 ms
Histats	13,545	4,885 s	361 ms
LiveJournal	3,680	4,882 s	1327 ms
Tidio Live Chat	12,655	4,849 s	383 ms
AB Tasty	3,010	4,798 s	1594 ms
Mailchimp	23,376	4,767 s	204 ms
Yandex CDN	25,669	4,750 s	185 ms
MGID	8,214	4,721 s	575 ms
GoDaddy	6,687	4,678 s	700 ms
Mapbox	5,206	4,568 s	877 ms
Olark	6,904	4,528 s	656 ms
SoundCloud	4,464	4,404 s	987 ms
Tealium	12,108	4,387 s	362 ms
Optimizely	19,584	4,379 s	224 ms
Mediavine	6,490	4,252 s	655 ms
Privy	13,047	4,036 s	309 ms
fam	2,135	3,857 s	1807 ms
Intercom	15,657	3,829 s	245 ms
Dealer	8,639	3,741 s	433 ms
Lucky Orange	7,531	3,699 s	491 ms
Vimeo	10,406	3,691 s	355 ms
Rubicon Project	40,208	3,632 s	90 ms
WebsiteBuilder.com	738	3,441 s	4663 ms
mPulse	13,190	3,244 s	246 ms
Adobe Business Catalyst	3,181	3,089 s	971 ms
Cookiebot	20,839	3,062 s	147 ms
Kakao	18,109	2,853 s	158 ms
uLogin	2,316	2,832 s	1223 ms
Embedly	5,513	2,831 s	514 ms
Adroll	24,347	2,809 s	115 ms
Amazon Ads	26,048	2,771 s	106 ms
Yotpo	13,632	2,754 s	202 ms
WordAds	18,479	2,745 s	149 ms
Skimbit	15,827	2,648 s	167 ms
Trust Pilot	15,202	2,488 s	164 ms
Akamai	7,364	2,464 s	335 ms
Azure Web Services	10,612	2,417 s	228 ms
Permutive	9,558	2,327 s	243 ms
Dailymotion	1,838	2,303 s	1253 ms
VigLink	7,100	2,191 s	309 ms
Infolinks	3,699	2,157 s	583 ms
AddToAny	24,501	2,121 s	87 ms
Tynt	23,214	2,047 s	88 ms
Twitch	1,068	2,027 s	1898 ms
Inspectlet	5,605	2,022 s	361 ms
OptinMonster	7,494	1,950 s	260 ms
Spotify	3,225	1,943 s	602 ms
LKQD	1,337	1,936 s	1448 ms
Unpkg	6,918	1,896 s	274 ms
Bold Commerce	13,136	1,889 s	144 ms
Hotmart	811	1,878 s	2315 ms
Fastly	6,806	1,860 s	273 ms
Bing Ads	15,836	1,847 s	117 ms
Segment	9,541	1,825 s	191 ms
Instagram	9,444	1,735 s	184 ms
Adobe TypeKit	18,280	1,723 s	94 ms
DTSCOUT	12,711	1,682 s	132 ms
Parking Crew	5,147	1,677 s	326 ms
Ensighten	3,768	1,670 s	443 ms
VK	13,492	1,648 s	122 ms
ContactAtOnce	3,247	1,594 s	491 ms
iubenda	12,784	1,586 s	124 ms
LinkedIn	14,057	1,561 s	111 ms
Sizmek	1,487	1,555 s	1046 ms
Pinterest	17,278	1,540 s	89 ms
Quantcast Choice	12,499	1,509 s	121 ms
Signyfyd	1,691	1,466 s	867 ms
FullStory	7,655	1,408 s	184 ms
Gigya	2,262	1,313 s	580 ms
issuu	1,938	1,296 s	669 ms
Adform	6,868	1,283 s	187 ms
LongTail Ad Solutions	3,444	1,259 s	365 ms
Disqus	1,252	1,245 s	995 ms
New Relic	13,101	1,245 s	95 ms
Nielsen NetRatings SiteCensus	11,880	1,239 s	104 ms
Heroku	2,007	1,222 s	609 ms
Secomapp	2,151	1,175 s	546 ms
Baidu Analytics	10,740	1,142 s	106 ms
Okas Concepts	673	1,112 s	1652 ms
Dynamic Yield	964	1,095 s	1136 ms
LoyaltyLion	1,301	1,080 s	830 ms
Sekindo	376	1,055 s	2807 ms
Optanon	8,102	1,039 s	128 ms
Ecwid	1,399	1,033 s	739 ms
Microsoft Hosted Libs	4,943	1,006 s	203 ms
DoubleVerify	1,776	981 s	552 ms
Amazon Pay	5,751	974 s	169 ms
DigiTrust	559	973 s	1741 ms
Albacross	1,920	941 s	490 ms
Intent Media	8,019	931 s	116 ms
sovrn	4,758	902 s	190 ms
Salesforce Commerce Cloud	2,623	897 s	342 ms
Outbrain	8,216	875 s	107 ms
Monotype	4,073	862 s	212 ms
Kaltura Video Platform	506	852 s	1684 ms
Feefo.com	1,686	846 s	502 ms
Technorati	841	825 s	981 ms
Cxense	4,375	784 s	179 ms
Ezoic	1,331	773 s	581 ms
VWO	4,728	748 s	158 ms
Bazaarvoice	1,845	733 s	397 ms
33 Across	1,197	732 s	611 ms
AdMatic	793	719 s	906 ms
Vidible	1,148	717 s	624 ms
CallRail	7,019	710 s	101 ms
Revolver Maps	1,146	697 s	608 ms
Smart AdServer	4,272	692 s	162 ms
Yieldify	513	683 s	1332 ms
AppDynamics	1,283	675 s	526 ms
Yahoo! Tag Manager	5,822	674 s	116 ms
Index Exchange	5,527	672 s	122 ms
RevJet	1,211	670 s	553 ms
Admixer for Publishers	1,043	669 s	642 ms
StickyADS.tv	853	651 s	764 ms
Scorecard Research	7,851	649 s	83 ms
SocialShopWave	2,044	617 s	302 ms
Teads	3,464	614 s	177 ms
Yahoo! JAPAN Ads	4,367	597 s	137 ms
OpenX	3,701	594 s	160 ms
ZEDO	795	592 s	745 ms
Fraudlogix	2,245	590 s	263 ms
Vidyard	536	587 s	1094 ms
LINE Corporation	5,184	585 s	113 ms
BrightTag / Signal	3,674	579 s	158 ms
Pardot	1,435	579 s	404 ms
Mixpanel	7,528	568 s	75 ms
Help Scout	2,183	563 s	258 ms
Crisp	4,497	559 s	124 ms
Snowplow	5,845	555 s	95 ms
TINT	224	553 s	2470 ms
Crazy Egg	4,994	551 s	110 ms
Medium	1,157	549 s	474 ms
IBM Digital Analytics	806	546 s	677 ms

Future Work

1. Introduce URL-level data for more fine-grained analysis, i.e. which libraries from Cloudflare/Google CDNs are most expensive.
2. Expand the scope, i.e. include more third parties and have greater entity/category coverage.

FAQs

I don't see entity X in the list. What's up with that?

This can be for one of several reasons:

1. The entity does not have references to their origin on at least 50 pages in the dataset.
2. The entity's origins have not yet been identified. See How can I contribute?

What is "Total Occurences"?

Total Occurrences is the number of pages on which the entity is included.

How is the "Average Impact" determined?

The HTTP Archive dataset includes Lighthouse reports for each URL on mobile. Lighthouse has an audit called "bootup-time" that summarizes the amount of time that each script spent on the main thread. The "Average Impact" for an entity is the total execution time of scripts whose domain matches one of the entity's domains divided by the total number of pages that included the entity.

Average Impact = Total Execution Time / Total Occurrences

How does Lighthouse determine the execution time of each script?

Lighthouse's bootup time audit attempts to attribute all toplevel main-thread tasks to a URL. A main thread task is attributed to the first script URL found in the stack. If you're interested in helping us improve this logic, see Contributing for details.

The data for entity X seems wrong. How can it be corrected?

Verify that the origins in data/entities.js are correct. Most issues will simply be the result of mislabelling of shared origins. If everything checks out, there is likely no further action and the data is valid. If you still believe there's errors, file an issue to discuss futher.

How can I contribute?

Only about 90% of the third party script execution has been assigned to an entity. We could use your help identifying the rest! See Contributing for details.

Contributing

Thanks

A huge thanks to @simonhearne and @soulgalore for their assistance in classifying additional domains!

Updating the Entities

The domain->entity mapping can be found in data/entities.js. Adding a new entity is as simple as adding a new array item with the following form.

{
    "name": "Facebook",
    "homepage": "https://www.facebook.com",
    "categories": ["social"],
    "domains": [
        "*.facebook.com",
        "*.fbcdn.net"
    ],
    "examples": [
        "www.facebook.com",
        "connect.facebook.net",
        "staticxx.facebook.com",
        "static.xx.fbcdn.net",
        "m.facebook.com"
    ]
}

Updating Attribution Logic

The logic for attribution to individual script URLs can be found in the Lighthouse repo. File an issue over there to discuss further.

Updating the Data

You need to have a BigQuery project called lighthouse-infrastructure with a dataset called third_party_web The queries used to compute the data are in the sql/ directory.

1. Edit all-observed-domains-query.sql to query the correct month's HTTPArchive run.
2. Run all-observed-domains-query.sql in BigQuery.
3. Download the results and check them in at data/YYYY-MM-01-observed-domains.json.
4. Edit origin-query.sql to query the correct month's HTTPArchive run.
5. Run yarn build to regenerate the latest canonical domain mapping.
6. Create a new table in lighthouse-infrastructure.third_party_web BigQuery table of the format YYYY_MM_01 with the csv in dist/domain-map.csv with three columns domain, canonicalDomain, and category.
7. Run origin-query.sql in BigQuery.
8. Download the results and check them in at data/YYYY-MM-01-origin-scripting.json.
9. Edit origin-query.sql to join on the table you just created.
10. Run yarn build to regenerate the queries.
11. Run entity-per-page.sql in BigQuery.
12. Download the results and check them in at data/YYYY-MM-01-entity-scripting.json.
13. Run web-almanac-all-observed-domains-identification.sql in BigQuery.
14. Save the results to a BigQuery table YYYY_MM_01_all_observed_domains.

Updating this README

This README is auto-generated from the templates lib/ and the computed data. In order to update the charts, you'll need to make sure you have cairo installed locally in addition to yarn install.

# Install `cairo` and dependencies for node-canvas
brew install pkg-config cairo pango libpng jpeg giflib
# Build the requirements in this repo
yarn build
# Regenerate the README
yarn start

Updating the website

The web code is located in www/ directory of this repository. Open a PR to make changes.