Use multiple workers to generate the static files for your Nuxt.js project
Install the package
yarn add nuxt-generate-cluster
Add a generate script to your package.json
"scripts": {
"generate": "nuxt-generate -w 4"
}Configure the generate options in nuxt.config.js
generate: {
workers: 4,
workerConcurrency: 500,
concurrency: 500,
routes (callback, params) {
return axios.get('https://api.example.com/routes?since=' + params.lastFinished)
.then((res) => {
return res.data
})
},
done ({ duration, errors, workerInfo }) {
if (errors.length) {
axios.post('https://api.example.com/routes', { generate_errors: errors })
}
}
}- Default: number of processors
The number of workers that should be started. It probably has no use to start more workers then number of processors in your system.
- Default:
500
To even the load between workers they are sent batches of routes to generate, otherwise a worker with 'easy' routes might finish long before others. Workers will also still use the concurrency option from Nuxt.
The default Nuxt.js routes method has been extended so you can pass additional parameters to it, see params parameter in example config under Setup. By default it will list 3 timestamps:
-
lastStartedThe unix timestamp when the nuxt-generate command was last executed, should be just now -
lastBuiltThe unix timestamp when the nuxt project was last built by nuxt-generate -
lastFinishedThe unix timestamp when nuxt-generate last finished succesfully (eg not interrupted byctrl+c)
Timestamps are locally stored in
~/.data-store/nuxt-generate-cluster.json, see data-store
This method is called on the master just before the workers are started/forked. It receives the Nuxt options as first argument.
Use this method if you experience serialization issues or when your Nuxt config is too big. The Nuxt options are stringified and then passed as environment variable to the workers, on Windows there seems to be a maximum size of 32KB for env variables.
Properties which should be safe to remove are (not a complete list):
- buildModules (and their options)
- serverMiddleware
This method will be called when all workers are finished, it receives two arguments:
-
The first argument is an object with statistics:
-
durationThe total time in seconds that the command ran -
errorsAn array of all the errors that were encountered. Errors can have typehandledorunhandled, for the latter the error message will contain the stacktrace
[ { type: 'handled', route: '/non-existing-link', error: { statusCode: 404, message: 'The message from your 404 page' } } ]
workerInfoAn object with detailed information about each worker. Data passed is from the watchdog object that we use internally to monitor the worker status.
{{ '6707': // the process id of the worker { start: [ 1929158, 859524606 ], // process.hrtime the worker was started duration: 109567, // how long the worker was active signal: 0, // the signal by which the worker was killed (if any) code: 0, // the exit status of the worker routes: 73, // the number of routes generated by this worker errors: [] }, // the errors this worker encountered, errors of all workers // combined is the error argument above }
-
-
The second argument is the Nuxt instance
Please note that you need to explicitly indicate with
-bthat you want to (re-)build your project
$ ./node_modules/.bin/nuxt-generate -h
Multi-threaded generate for nuxt using cluster
Description
Generate a static web application (server-rendered)
Usage
$ nuxt-generate <dir>
Options
-b, --build Whether to (re-)build the nuxt project
-c, --config-file Path to Nuxt.js config file (default: nuxt.config.js)
-h, --help Displays this message
-p, --params Extra parameters which should be passed to routes method
(should be a JSON string or queryString)
-q, --quiet Decrease verbosity (repeat to decrease more)
-v, --verbose Increase verbosity (repeat to increase more)
--fail-on-page-error Immediately exit when a page throws an unhandled error
-w, --workers [NUM] How many workers should be started
(default: # cpus)
-wc [NUM], How many routes should be sent to
--worker-concurrency [NUM] a worker per iteration
If you need to have more control which routes should be generated, use the -p option to pass additional parameters to your routes method.
# nuxt.config.js
generate: {
routes (callback, params) {
console.log(params)
}
}
$ nuxt-generate -w 2 -p id=1&id=2
// will print =>
{ id: [ '1', '2' ],
lastStarted: 1508609323,
lastBuilt: 0,
lastFinished: 0 }
If you are using a npm script under bash use -- to pass the parameters to nuxt-generate instead of npm:
$ npm run generate -- -p '{ "id": [1,2,3] }'
// will print =>
{ id: [ 1, 2, 3 ],
lastStarted: 1508786574,
lastBuilt: 0,
lastFinished: 0 }
You can use multiple -v or -q on the command-line to increase or decrease verbosity.
We use consola for logging and set a default log level of 3 unless DEBUG is set, then its 5.
If you want to log debug messages without setting DEBUG you can use -vv on the command-line
The difference between log levels 2, 3 and 4 are:
-
Level 2(-q) Only print master messages like worker started / exited. No worker messages. -
Level 3Also print which routes the workers generated -
Level 4(-v) Also print how much time the route generation took and messages between master and workers
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent