swalker-888 / google-indexing-api-bulk

Submit URLs in bulk to Google's Indexing API using a batch request

google-indexing-api-bulk

Created by Steve at Journey Further SEO

Requires node.js - https://nodejs.org/en/download/

This script will help you index your website's pages in bulk, without having to manually request each URL for submission in the Search Console interface.

First off you will need to set up access to the Indexing API in Google Cloud Platform - follow the instructions below.

https://developers.google.com/search/apis/indexing-api/v3/prereqs

Once you have access to Indexing API you'll be able to download a public/private key pair JSON file, this contains all of your credentials and should be saved as "service_account.json".

Add the URLs to the urls.txt file that you need to be crawled/indexed.

Verify site ownership in Search Console to submit URLs for indexing

In this step, you'll verify that you have control over your web property.

To verify ownership of your site you'll need to add your service account email address (see service_account.json - client_email) and add it as an owner ('delegated') of the web property in Search Console.

You can find your service account email address in two places:

• The client_email field in the JSON private key that you downloaded when you created your project.
• The Service account ID column of the Service Accounts view in the Developer Console.
• The email address has a format similar to the following:

For example, "[e-mail]".

Then...

1. Go to Google Webmaster Central
2. Click your verified property
3. Scroll down and click 'Add an owner'.
4. Add your service account email address as an owner to the property.

Quotas

100 URLs per request batch

200 URLs per day

google-indexing-api-bulk's Issues

Error 400: "Missing attribute. 'url' attribute is required.",

Hi!

Could you push a working example of urls.txt file?

I tried to put this format i.e:

https://canonicaldomain.com
https://canonicaldomain.com/url1
https://canonicaldomain.com/url2
https://canonicaldomain.com/url3

but I got this error:

--batch_bxcXt4TswdrogJK9TDNfl1ZzywnLgbL-
Content-Type: application/http
Content-ID: response-

HTTP/1.1 400 Bad Request
Vary: Origin
Vary: X-Origin
Vary: Referer
Content-Type: application/json; charset=UTF-8

{
  "error": {
    "code": 400,
    "message": "Missing attribute. 'url' attribute is required.",
    "status": "INVALID_ARGUMENT"
  }
}

--batch_bxcXt4TswdrogJK9TDNfl1ZzywnLgbL---

Error: No key or keyFile set.

Via another application I have a working keyset and I am not able to figure out - when using the code - this error keeps popping up.

c:\google-indexing-api-bulk-master>node index.js
Error: No key or keyFile set.
at GoogleToken.getTokenAsync (c:\google-indexing-api-bulk-master\node_modules\gtoken\build\src\index.js:125:19)
at GoogleToken.getToken (c:\google-indexing-api-bulk-master\node_modules\gtoken\build\src\index.js:77:21)
at JWT.refreshTokenNoCache (c:\google-indexing-api-bulk-master\node_modules\google-auth-library\build\src\auth\jwtclient.js:134:36)
at JWT.refreshToken (c:\google-indexing-api-bulk-master\node_modules\google-auth-library\build\src\auth\oauth2client.js:142:25)
at JWT.authorizeAsync (c:\google-indexing-api-bulk-master\node_modules\google-auth-library\build\src\auth\jwtclient.js:117:35)
at JWT.authorize (c:\google-indexing-api-bulk-master\node_modules\google-auth-library\build\src\auth\jwtclient.js:110:18)
at Object. (c:\google-indexing-api-bulk-master\index.js:19:11)
at Module._compile (internal/modules/cjs/loader.js:1063:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
at Module.load (internal/modules/cjs/loader.js:928:32)

Not working

Hi

not work!

error below:

• C:\Users\Mr.Seo\Desktop\mygit\index.js
  ←[90m at Function.Module._resolveFilename (internal/modules/cjs/loader.js:880:15)←[39m
  ←[90m at Function.Module._load (internal/modules/cjs/loader.js:725:27)←[39m
  ←[90m at Module.require (internal/modules/cjs/loader.js:952:19)←[39m
  ←[90m at require (internal/modules/cjs/helpers.js:88:18)←[39m
  at Object. (C:\Users\Mr.Seo\Desktop\mygit\index.js:2:15)
  ←[90m at Module._compile (internal/modules/cjs/loader.js:1063:30)←[39m
  ←[90m at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10)←[39m
  ←[90m at Module.load (internal/modules/cjs/loader.js:928:32)←[39m
  ←[90m at Function.Module._load (internal/modules/cjs/loader.js:769:14)←[39m
  ←[90m at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:72:12)←[39m {
  code: ←[32m'MODULE_NOT_FOUND'←[39m,
  requireStack: [ ←[32m'C:\Users\Mr.Seo\Desktop\mygit\index.js'←[39m ]
  }

Problem after command node index.js

It seems to be correct, but the query gives out a huge errors

GaxiosError: invalid_grant: Invalid JWT: Token must be a short-lived token (60 minutes) and in a reasonable timeframe. Check your iat and exp values in the JWT claim. at Gaxios._request (C:\Users\trust\Desktop\google-indexing-api-bulk-master\node_modules\gaxios\build\src\gaxios.js:129:23) at processTicksAndRejections (node:internal/process/task_queues:96:5) at async GoogleToken.requestToken (C:\Users\trust\Desktop\google-indexing-api-bulk-master\node_modules\gtoken\build\src\index.js:231:23) at async GoogleToken.getTokenAsync (C:\Users\trust\Desktop\google-indexing-api-bulk-master\node_modules\gtoken\build\src\index.js:142:20) at async JWT.refreshTokenNoCache (C:\Users\trust\Desktop\google-indexing-api-bulk-master\node_modules\google-auth-library\build\src\auth\jwtclient.js:171:23) at async JWT.authorizeAsync (C:\Users\trust\Desktop\google-indexing-api-bulk-master\node_modules\google-auth-library\build\src\auth\jwtclient.js:152:24) { response: { config: { method: 'POST', url: 'https://www.googleapis.com/oauth2/v4/token', data: [Object], headers: [Object], responseType: 'json', paramsSerializer: [Function: paramsSerializer], body: 'grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Ajwt-bearer&assertion=eyJhbGciOiJSUzI1NiJ9.eyJpc3MiOiJ0YWtldG9uZXMtNDc0QHRha2V0b25lcy5pYW0uZ3NlcnZpY2VhY2NvdW50LmNvbSIsInNjb3BlIjoiaHR0cHM6Ly93d3cuZ29vZ2xlYXBpcy5jb20vYXV0aC9pbmRleGluZyIsImF1ZCI6Imh0dHBzOi8vd3d3Lmdvb2dsZWFwaXMuY29tL29hdXRoMi92NC90b2tlbiIsImV4cCI6MTY0ODgyMDc4MywiaWF0IjoxNjQ4ODE3MTgzLCJzdWIiOm51bGx9.MV7GFkye0QZnJb5GC8uWX_BrCpxQxIajH5vsLLseGsFwNJhL0LTtuGPykAASiVHFyLDRUUCsC-B7pdl_AvsQkXyokTP-OPYUCwW7FNaib-1wQvywBqXeQhV2WvJnj-YTvOlv7_TSzmpkd_6X1jd_L-CRQcGBaQmQNFjNiWfvcQ7dxNP0T_Xeq0Bfhy5goD121ndjn2yrrc_sCJRSu685GAXHE2ovIjDv42RRm657pCCD9RkseC2xAV0O6-4JGrqNPNt5hD7ACYaDWkULUeXYqP40wrNTg4ak1-gd--WoDbM24H88zWRcwxOXAzdL7yCIF8-eIN9goAlO_Cu4vhgj7w', validateStatus: [Function: validateStatus] }, data: { error: 'invalid_grant', error_description: 'Invalid JWT: Token must be a short-lived token (60 minutes) and in a reasonable timeframe. Check your iat and exp values in the JWT claim.' }, headers: { 'alt-svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"', 'cache-control': 'private', connection: 'close', 'content-encoding': 'gzip', 'content-type': 'application/json; charset=UTF-8', date: 'Fri, 01 Apr 2022 11:46:23 GMT', server: 'scaffolding on HTTPServer2', 'transfer-encoding': 'chunked', vary: 'Origin, X-Origin, Referer', 'x-content-type-options': 'nosniff', 'x-frame-options': 'SAMEORIGIN', 'x-xss-protection': '0' }, status: 400, statusText: 'Bad Request', request: { responseURL: 'https://www.googleapis.com/oauth2/v4/token' } }, config: { method: 'POST', url: 'https://www.googleapis.com/oauth2/v4/token', data: { grant_type: 'urn:ietf:params:oauth:grant-type:jwt-bearer', assertion: 'eyJhbGciOiJSUzI1NiJ9.eyJpc3MiOiJ0YWtldG9uZXMtNDc0QHRha2V0b25lcy5pYW0uZ3NlcnZpY2VhY2NvdW50LmNvbSIsInNjb3BlIjoiaHR0cHM6Ly93d3cuZ29vZ2xlYXBpcy5jb20vYXV0aC9pbmRleGluZyIsImF1ZCI6Imh0dHBzOi8vd3d3Lmdvb2dsZWFwaXMuY29tL29hdXRoMi92NC90b2tlbiIsImV4cCI6MTY0ODgyMDc4MywiaWF0IjoxNjQ4ODE3MTgzLCJzdWIiOm51bGx9.MV7GFkye0QZnJb5GC8uWX_BrCpxQxIajH5vsLLseGsFwNJhL0LTtuGPykAASiVHFyLDRUUCsC-B7pdl_AvsQkXyokTP-OPYUCwW7FNaib-1wQvywBqXeQhV2WvJnj-YTvOlv7_TSzmpkd_6X1jd_L-CRQcGBaQmQNFjNiWfvcQ7dxNP0T_Xeq0Bfhy5goD121ndjn2yrrc_sCJRSu685GAXHE2ovIjDv42RRm657pCCD9RkseC2xAV0O6-4JGrqNPNt5hD7ACYaDWkULUeXYqP40wrNTg4ak1-gd--WoDbM24H88zWRcwxOXAzdL7yCIF8-eIN9goAlO_Cu4vhgj7w' }, headers: { 'Content-Type': 'application/x-www-form-urlencoded', Accept: 'application/json' }, responseType: 'json', paramsSerializer: [Function: paramsSerializer], body: 'grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Ajwt-bearer&assertion=eyJhbGciOiJSUzI1NiJ9.eyJpc3MiOiJ0YWtldG9uZXMtNDc0QHRha2V0b25lcy5pYW0uZ3NlcnZpY2VhY2NvdW50LmNvbSIsInNjb3BlIjoiaHR0cHM6Ly93d3cuZ29vZ2xlYXBpcy5jb20vYXV0aC9pbmRleGluZyIsImF1ZCI6Imh0dHBzOi8vd3d3Lmdvb2dsZWFwaXMuY29tL29hdXRoMi92NC90b2tlbiIsImV4cCI6MTY0ODgyMDc4MywiaWF0IjoxNjQ4ODE3MTgzLCJzdWIiOm51bGx9.MV7GFkye0QZnJb5GC8uWX_BrCpxQxIajH5vsLLseGsFwNJhL0LTtuGPykAASiVHFyLDRUUCsC-B7pdl_AvsQkXyokTP-OPYUCwW7FNaib-1wQvywBqXeQhV2WvJnj-YTvOlv7_TSzmpkd_6X1jd_L-CRQcGBaQmQNFjNiWfvcQ7dxNP0T_Xeq0Bfhy5goD121ndjn2yrrc_sCJRSu685GAXHE2ovIjDv42RRm657pCCD9RkseC2xAV0O6-4JGrqNPNt5hD7ACYaDWkULUeXYqP40wrNTg4ak1-gd--WoDbM24H88zWRcwxOXAzdL7yCIF8-eIN9goAlO_Cu4vhgj7w', validateStatus: [Function: validateStatus] }, code: '400' }

module file missing?

Hi, when i tried to run de index.js i got this error.

c:\google-indexing-api-bulk-master>node index.js
internal/modules/cjs/loader.js:638
throw err;
^

Error: Cannot find module 'request'
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15)
at Function.Module._load (internal/modules/cjs/loader.js:562:25)
at Module.require (internal/modules/cjs/loader.js:690:17)
at require (internal/modules/cjs/helpers.js:25:18)
at Object. (c:\google-indexing-api-bulk-master\google-indexing-api-bulk-master\index.js:3:15)
at Module._compile (internal/modules/cjs/loader.js:776:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:787:10)
at Module.load (internal/modules/cjs/loader.js:653:32)
at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
at Function.Module._load (internal/modules/cjs/loader.js:585:3)

Add sponsor link 🙂

https://www.flashserp.com ?

Error: Cannot find module 'request'

HELP ME. I'M NOT GOOD AT PROGRAMMING BUT THERE'S AN ERROR IN THE CODE.

Terminal:

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.

C:\Users\user\Desktop\Bulk Google Indexer\google-indexing-api-bulk-master>node index.js
internal/modules/cjs/loader.js:968
throw err;
^

Error: Cannot find module 'request'
Require stack:

• C:\Users\user\Desktop\Bulk Google Indexer\google-indexing-api-bulk-master\inde
  x.js
  at Function.Module._resolveFilename (internal/modules/cjs/loader.js:965:15)
  at Function.Module._load (internal/modules/cjs/loader.js:841:27)
  at Module.require (internal/modules/cjs/loader.js:1025:19)
  at require (internal/modules/cjs/helpers.js:72:18)
  at Object. (C:\Users\user\Desktop\Bulk Google Indexer\google-inde
  xing-api-bulk-master\index.js:2:15)
  at Module._compile (internal/modules/cjs/loader.js:1137:30)
  at Object.Module._extensions..js (internal/modules/cjs/loader.js:1157:10)
  at Module.load (internal/modules/cjs/loader.js:985:32)
  at Function.Module._load (internal/modules/cjs/loader.js:878:14)
  at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js
  :71:12) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
  'C:\Users\user\Desktop\Bulk Google Indexer\google-indexing-api-bulk-mas
  ter\index.js'
  ]
  }

C:\Users\user\Desktop\Bulk Google Indexer\google-indexing-api-bulk-master>