Beautiful Open Api Template System
License: MIT License
An OpenAPI & AsyncAPI templating system with Nunjucks... write less YAML... do more.
Simple examples can be found here:
(Refer to the documentation for additional features and details.)
BOATS is nothing without the support of:
Given a definition:
post.yml
type: object
required:
- name
- coordinates
properties:
name:
type: string
coordinates:
type: array
items:
type: stringThe helper should allow me to include it another definition with all the properties made optional.
i.e the result of using the helper will be something like this:
patch.yml
type: object
properties:
name:
type: string
coordinates:
type: array
items:
type: stringCurrently in openapi you have to add a ref to common responses for all 404 500 etc etc.
Lots of typing and more stuff to manage. Would be nice to have an easier way to inject common responses. Especially handy for AWS x-* values.
Alternate title:
mapIndex produces empty object for AsyncAPI documentDescribe the bug
inject([...]) filter functionality (excludeChannel, includeOnlyChannels) doesn't work, scope unclear.
To Reproduce
I'm using a standard, valid asyncAPI document, I'm not sure if it works for other things.
Expected behavior
excludeChannel, includeOnlyChannels params are respected
Additional context
I'm debugging into Injector.ts, and the offending code seems to be the mapIndex function. With this particular example at least, the problem seems to be that the $ref field is selected twice, once by the Object.values(methods) call, and then a second time by the explicit methodToFileRef.$ref accessor.
You can see in the debugger that Object.values(methods) is already a string array:
The result is that methodToFileRef is a string and methodToFileRef.$ref is undefined.
Therefore nothing ever gets added to this.this.fileToRouteMap so nothing ever passes the filter whitelist or blacklists. They have no entries to filter against.
Bundling the files to a single file is most useful for lots of tools, but there are other tools that can parse multiple files without requiring bundling.
Allow a no-bundle flag that will leave multiple files as multiple files. Will not work for the json-refs bundler though as these files fail validation due to: https://www.npmjs.com/package/boats#history
Describe the bug
Adding 2 paths to 1 file results in only 1 path being resolved and not both
To Reproduce
tags:
- {{ autoTag() }}
summary: Report a channel
operationId: {{ uniqueOpId() }}
produces:
- application/json
parameters:
- $ref: @parameters/pathSlug.yml
responses:
'200':
description: Successful unlock
schema:
$ref: @definitions/channel/model.yml
Expected behavior
Both paths are resolved, but instead only the last 1 is but the 1st is left as a shorthand
For example, with a fragment of spec like so:
items:
discriminator:
mapping:
First: Specfile_ThingOne.yaml
Second: Specfile_ThingTwo.yaml
propertyName: thingType
oneOf:
- $ref: Specfile_ThingOne.yaml
- $ref: Specfile_ThingTwo.yamlIt generates a rendered file like so:
items:
discriminator:
mapping:
First: Specfile_ThingOne.yaml
Second: Specfile_ThingTwo.yaml
propertyName: thingType
oneOf:
- $ref: '#/components/schemas/FolderNameSpecFileThingOneYaml'
- $ref: '#/components/schemas/FolderNameSpecFileThingTwoYaml'As the mappings on first/second aren't templated the same way, the rendered spec doesn't work.
Not sure if this is a problem in the nunjunks templating or json-schema-ref-parser, or both.
We could guess the generated name for the indexed file, but that seems error prone.
npm run build:json && npm run build:yaml will always run build:yaml, because the json process never exits with code > 0
so if you have a script that runs npm run build:json && cp output to other place, this will erroneously copy the file every time
in cli.js:
}).catch(error => {
console.trace(error)
process.exit(1) // <- this is required
})when loading a helper from another project which has it's own js-yaml node module at v4+, safeLoad and safeDump throw errors when used:
Error: Function yaml.safeDump is removed in js-yaml 4. Use yaml.dump instead, which is now safe by default.
Describe the bug
If more than one typescript nunjucks helpers are added via the -f CLI option the ts-node module is registered twice and causes the second file to be compiled twice. This obviously doesn't work.
The offending piece of code is https://github.com/j-d-carmichael/boats/blob/main/src/Template.ts#L295
require('ts-node').register(); is called for each *.ts file, when it should not be called more than once.
To Reproduce
boats ... -f a.ts -f b.ts
b.ts will get double compiled and produces confusing errors
Expected behavior
No error
Additional context
This issue would be fixed if the following issue is fixed in ts-node:
TypeStrong/ts-node#1690
TypeStrong/ts-node#409
Version check takes 5 minutes or more to timeout if boats is run on a CI runner with no internet access.
There should be an option to disable the version check entirely, not just the user interaction.
In the meantime I'm going to use patch-package as a workaround to remove the version check.
Describe the bug
A clear and concise description of what the bug is.
When using the inject template, it is not possible to exclude root paths from injections.
To Reproduce
{{
inject([
{ toAllOperations: {
content: {
security: [{ jwtToken: [] }]
},
excludePaths: [
'/towns',
'/'
]
}
}
])
}}
Expected behavior
The bundled specifications must not inject security into root paths.
Additional context
Add any other context about the problem here.
Hello! I'm using Boats to assemble and build the OpenAPI spec for an application I'm developing.
Boats gets invoked every time the application is compiled, which is done automatically in a CI pipeline and when building the Docker image. Every now and then the build is broken by the following message produced by Boats:
Checking version with npm-tool-version-check from: https://raw.githubusercontent.com/johndcarmichael/boats/master/package.json
WARNING: The version of BOATS you are running, 1.20.4, is OUTDATED!
THERE IS A BETTER VERSION: 1.23.0
? Are you sure you want to continue with an outdated package? This will result in some serious technical dept in the future and prevent security updates arriving... (y/N)
Is there a way I can disable this check so that the build can proceed even if I'm using an older version of boats?
Upgrading a dependency is something that requires careful consideration and is always done manually to validate the new build, I cannot afford to have broken builds due to a tool that wants to update itself.
When running the command "npm run build" on Windows in the Visual Studio Code integrated Terminal (node v12.18.2; boats version 1.18.5), the following error appears:
Trace: Error: Path method found without an operationId: get:\invoice at UniqueOperationIds.checkOpIdsAreAllUnique (C:\Users\michael.intelmann\dev\mlc-ofp-backend-swagger\node_modules\boats\build\src\UniqueOperationIds.js:71:31) at Object.exports.default (C:\Users\michael.intelmann\dev\mlc-ofp-backend-swagger\node_modules\boats\build\src\bundlerSwaggerParse.js:29:38) at async parseCli (C:\Users\michael.intelmann\dev\mlc-ofp-backend-swagger\node_modules\boats\build\src\cli.js:39:31) at catchHandle (C:\Users\michael.intelmann\dev\mlc-ofp-backend-swagger\node_modules\boats\build\src\cli.js:44:13)
First assumption is the double-colon at the beginning of the output and missing "operationId": see attachment
On Ubuntu or on Ubuntu WSL sub-system the error is not thrown and everything works fine.
https://mozilla.github.io/nunjucks/templating.html#include
./src/components/jwt/access/model.yml.njk
Try to include this component from another project but it will throw an error.
{% include "../../../../ms-authentication/src/components/jwt/access/model.yml.njk" %}
It is possible to use the $ref from openapi/asyncapi - but this means juggling with other features eg allOf or nesting in another attribute. Would be cleaner to be able to just pull the code in.
There should be a warning when the .boatsrc file doesn't exist.
It could be added here: https://github.com/johndcarmichael/boats/blob/master/src/GetCheckCorrectBoatsRc.ts#L11
summary: get the list you can add to
description: |
get the list of lists
responses:
200:
description: An array of prospects
schema:
$ref: ../../../../definitions/list/models.yml
nice.. the tag is a helper and added to the inject helper.. as is the opid etc etc.
But summary is not and it is a pain to manually write this out when the rolder structure should basically be doing this for us.
eg
src/paths/user/{email}/lists/get.yml
Could equate to
Get lists for {email} for users
Describe the bug
it used to use npm version check npm package which was configured to pull the version from github - github on occasion would just hang.
To Reproduce
add it back in: 106ff89
Expected behavior
doesn't hang
There should be a more reliable option.
Note from @zewish idea might be to inject some sort of date based cache buster - but there might be a more official method.
By default currently boats uses the default "tags" which is double curly braces but can easily be changed with: https://mozilla.github.io/nunjucks/api.html#customizing-syntax
The double curly braces through nasty red lines in my ide.
Describe the bug
I followed the Getting Started and tried to run the demo but got the spawn npm enoent error.
To Reproduce
npm initnpm i boats on the same project as I ran the npm init command"scripts": {
"boats": "boats",
"build:json": "boats -i ./src/index.yml -o ./build/awesome-api-d.json",
"build:yaml": "boats -i ./src/index.yml -o ./build/awesome-api-d.yml",
"build": "npm run build:json && npm run build:yaml"
},
"dependencies": {
"boats": "latest"
}
npm run boats -- --initCompleted: Injected a .boatsrc file
Completed: Installed boats skeleton files to C:/Users/User/Desktop/dsl/boats_test2/src
Completed: Created a build output directory
Completed: BOATS build scripts added to your package.json
node:events:368
throw er; // Unhandled 'error' event
^
Error: spawn npm ENOENT
at Process.ChildProcess._handle.onexit (node:internal/child_process:282:19)
at onErrorNT (node:internal/child_process:477:16)
at processTicksAndRejections (node:internal/process/task_queues:83:21)
Emitted 'error' event on ChildProcess instance at:
at Process.ChildProcess._handle.onexit (node:internal/child_process:288:12)
at onErrorNT (node:internal/child_process:477:16)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
errno: -4058,
code: 'ENOENT',
syscall: 'spawn npm',
path: 'npm',
spawnargs: [ 'install' ]
}
Expected behavior
Run without any errors.
Additional context
Here's the full console log of npm run boats -- --init:
PS C:\Users\User\Desktop\openapi\dsl\boats_test2> npm run boats -- --init
> [email protected] boats
> boats "--init"
Checking version from npm api
(ꙨပꙨ) This local version looks fresh and shiny, nice!
? Enter the name of the api file, press enter to use the current package.json name attribute: boats_test2
? oaType: OpenAPI 3.0.0
? Press Y and enter to install. This will make a copy of the template files to ./src, an output directory ./build and a config file ./.boatsrc Yes
Completed: Injected a .boatsrc file
Completed: Installed boats skeleton files to C:/Users/User/Desktop/openapi/dsl/boats_test2/src
Completed: Created a build output directory
Completed: BOATS build scripts added to your package.json
node:events:368
throw er; // Unhandled 'error' event
^
Error: spawn npm ENOENT
at Process.ChildProcess._handle.onexit (node:internal/child_process:282:19)
at onErrorNT (node:internal/child_process:477:16)
at processTicksAndRejections (node:internal/process/task_queues:83:21)
Emitted 'error' event on ChildProcess instance at:
at Process.ChildProcess._handle.onexit (node:internal/child_process:288:12)
at onErrorNT (node:internal/child_process:477:16)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
errno: -4058,
code: 'ENOENT',
syscall: 'spawn npm',
path: 'npm',
spawnargs: [ 'install' ]
}
Is your feature request related to a problem? Please describe.
Boats currently doesn't support OpenAPI 3.1 specs. Tracking any work required to support the latest spec.
Describe the solution you'd like
We're blocked by swagger-parser, which currently doesn't support OpenAPI 3.1, but other blockers I'm not aware of may exist.
Describe alternatives you've considered
Replace swagger-parser altogether, which seems overly intrusive.
Enable a command line flag that skips the use of swagger-parser for 3.1 specs, this sacrifices value.
Additional context
Thought I'd start tracking the dependencies needed to make OpenAPI 3.1 work here, specifically the webhooks feature would be useful for our use-case.
Describe the bug
When trying to import boats programmatically, I get the following error:
Error: Cannot find module '<path-to-repo>/node_modules/boats/src/index.js'. Please verify that the package.json has a valid "main" entry
To Reproduce
include import boats from 'boats' (or the CJS equivalent) in a source file and run it.
Expected behavior
As per the docs
You can also use BOATS programmatically, just require (import if you are using a bundler) the lib into your project.
I would expect this to work and provide some programmatic API.
A colleague of mine mentioned that he prefers to not bundle multiple api files into 1 and instead to use many. Seems like a fairly trivial addition that could bring some nice benefits so will add at some point in the future when i find the time.
The solution I have in mind would be to pass an addition option -m / '-multifile_output'. This option would be picked up by the cli tool and then walk over the input directory, parse the nunjuks syntax and output directory and validate the parsed output with swagger-parser.
Currently, the -y stops the version check, but I will be nice if it will check and just display a new version message without stopping the build, and possibly a link to the release notes. In this case, I'll know there's a new version and I'll update when I see fit. @p-mcgowan
Currently the bin/cli.js links to an index.js which in turn requires the compiled es5 code.
This should be cleaned up and the bin/cli require a cli script and the index should merely expose the es5 code.
1.17 introduced root hash ref resolution (eg inject content $ref: "#/params/whateverModel")
This is not processed on the original file, only the content.
instead, we can modify the injector (see below) to do this for all tamplates
const deepmerge = require('deepmerge')
const jsYaml = require('js-yaml')
const nunjucks = require('nunjucks')
const path = require('path')
class Injector {
constructor () {
this.fileToRouteMap = {}
}
/**
* Render the base template and inject content if provided
*
* @param {string} inputPath Target template file
* @param {string} inputIndexYaml Root input YAML file
*
* @return {string} YAML string
*/
injectAndRender (inputPath, inputIndexYaml) {
const fullPath = path.join(process.cwd(), inputPath)
const relativePathToRoot = path.relative(path.dirname(inputPath), path.dirname(inputIndexYaml))
const yaml = nunjucks.render(fullPath).replace(/(\$ref[ '"]*:[ '"]*)#\/([^ '"$]*)/g, (_, ref, rootRef) => {
const newPath = `${path.dirname(rootRef)}/index.yml#/${path.basename(rootRef)}`
return `${ref}${relativePathToRoot}/${newPath}`
})
if (!global.boatsInject) {
return yaml
}
if (!/\/(paths|channels)\//.test(inputPath)) {
return yaml
}
if (/index\./.test(path.basename(inputPath))) {
this.mapIndex(yaml, inputPath)
return yaml
}
let jsonTemplate = jsYaml.safeLoad(yaml)
for (let { toAllOperations } of global.boatsInject) {
if (this.shouldInject(toAllOperations, inputPath)) {
jsonTemplate = this.mergeInjection(jsonTemplate, relativePathToRoot, toAllOperations.content)
}
}
return jsYaml.safeDump(jsonTemplate)
}
/**
* Merge the JSON from the YAML with the JSON injection content
*
* @param {object} jsonTemplate JSON representation of the YAML file
* @param {string} relativePathToRoot Path from current file to root index (../ repeated)
* @param {object} content Content to be injected
*
* @return {object} Merged JSON of the template
*/
mergeInjection (jsonTemplate, relativePathToRoot, content) {
if (!jsonTemplate || !content) {
return jsonTemplate
}
if (typeof content === 'object') {
content = JSON.stringify(content)
}
content = content.replace(/(\$ref[ '"]*:[ '"]*)#\/([^ '"$]*)/g, (_, ref, rootRef) => {
const newPath = `${path.dirname(rootRef)}/index.yml#/${path.basename(rootRef)}`
return `${ref}${relativePathToRoot}/${newPath}`
})
const injectionContent = jsYaml.safeLoad(nunjucks.renderString(content))
return deepmerge(jsonTemplate, injectionContent)
}
/**
* Checks if the content should be injected
*
* @param {object} injection Injection rule
* @param {string} inputPath Path to target file
*
* @return {boolean} True if the path satisfies the rule
*/
shouldInject (injection, inputPath) {
if (!injection) {
return false
}
const {
exclude,
excludePaths,
includeMethods,
} = {
exclude: [],
excludePaths: [],
includeMethods: [],
...injection,
}
const operationName = this.fileToRouteMap[inputPath]
const methodName = path.basename(inputPath).replace(/\..*/, '')
let shouldSkipMethod = () => false
if (includeMethods.length) {
const methodsRegex = new RegExp(`\\b(${includeMethods.join('|')})\\b`, 'i')
shouldSkipMethod = (method) => !methodsRegex.test(method)
}
if (/channels/.test(inputPath) && exclude.includes(operationName)) {
return false
}
if (/paths/.test(inputPath) && (excludePaths.includes(operationName) || shouldSkipMethod(methodName))) {
return false
}
return true
}
/**
* Map filenames to routes so that exclude paths can be
* calculated from the input filename
*
* @param {string} yaml The YAML of a path or channel index
* @param {string} inputPath Path to YAML index file
*/
mapIndex (yaml, inputPath) {
const indexRoute = path.dirname(inputPath)
const index = jsYaml.safeLoad(yaml)
Object.entries(index).forEach(([route, methods]) => {
Object.values(methods).forEach(methodToFileRef => {
if (methodToFileRef && methodToFileRef['$ref']) {
const fullPath = `${indexRoute}/${methodToFileRef['$ref'].replace('./', '')}`
this.fileToRouteMap[fullPath] = route
}
})
})
}
}
module.exports = new Injector()which would allow for templating like so:
src/paths/users/get.yml:
tags:
- <$ autoTag() $>
summary: List all users
operationId: <$ uniqueOpId() $>
description: |
Lists all users
parameters:
- $ref: ../../parameters/index.yml#/queryPage
- $ref: ../../parameters/index.yml#/queryPerPage
- $ref: ../../parameters/index.yml#/queryQNumber
- $ref: ../../parameters/index.yml#/queryCNumber
- $ref: ../../parameters/index.yml#/queryDealershipCode
- $ref: ../../parameters/index.yml#/queryName
- $ref: ../../parameters/index.yml#/queryFirstName
- $ref: "#/parameters/queryLastName"
responses:
'200':
description: Ok
schema:
$ref: ../../definitions/index.yml#/UserModels
'404':
description: Not foundthe typescript above was tested on node modules and appears to work correctly
main index might look like this:
{{
inject([
{
toAllOperations: {
excludePaths: [
'/health'
],
content: {
tags: [ '{{ autoTag() }}' ],
summary: '{{ autoSummary() }}',
operationId: '{{ uniqueOpId() }}',
'x-permission': '{{ routePermission() }}',
security: [{
jwtToken: []
}],
responses: {
'200': {
description: 'OK'
},
'400': {
description: 'Bad Request'
},
'401': {
description: 'Unauthorized'
},
'403': {
description: 'Forbidden'
},
'406': {
description: 'Not Acceptable'
}
}
}
}
},
{
toAllOperations: {
includeMethods: ['post', 'patch'],
content: {
responses: {
'422': {
description: 'Not Acceptable'
}
}
}
}
}
])
}}
A path file like this:
parameters:
- $ref: ../../../components/parameters/pathId.yml
responses:
'200':
content:
application/json:
schema:
$ref: ../../../components/schemas/user/model.yml
The final output results in the order being non-standard:
parameters:
- $ref: ../../../components/parameters/pathId.yml
responses:
'200':
content:
application/json:
schema:
$ref: ../../../components/schemas/user/model.yml
description: OK
'400':
description: Bad Request
'401':
description: Unauthorized
'403':
description: Forbidden
'406':
description: Not Acceptable
tags:
- Users
summary: Get users based on {id}
operationId: usersIdGet
x-permission: userRolePermissionsReadUsersIdGet
security:
- jwtToken: []
Openapi has a nice feature.. allOf.. but doesn't offer the ability to pick attributes from an existing model.
This allows you to create a set of base attributes to be used as say search results... lighter versions of the models. Then using allOf extend to create a fuller model for a detail view... but then when you need a 3rd version.. the whole thing becomes too complex to easily manage.
A function in boats to pick attribute from the full model to create a new model would be much easier to manage, all attributes held in the the main model, the lighters model cherry picking what they needed:
{{ pickForNewObject('../item/model.yml', ['name', 'description', 'createdAt']) }}
Would result in:
type: object
properties:
description:
type: string
name:
type: string
createdAt
type: string
format: date-time
Where the full model might look like:
type: object
properties:
comments:
type: array
items:
$ref: ./comment/model.yml
createdAt
type: string
format: date-time
updatedAt
type: string
format: date-time
deletedAt:
type: string
format: date-time
description:
type: string
name:
type: string
likes:
type: array
items:
$ref: ./like/model.yml
There will however be an issue when a picked attribute contains a $ref
Our use-case is similar to seen elsewhere.
Create a public / private documentation file out of a single set of shared specifications.
We could do this by manually indexing the "public" file, and using autoIndexer for the "private" one, but that seems laborious.
AutoIndexer doesn't seem to have a good extension point for this, so at the moment the best approach I can see would re-create a significant amount of AutoIndexer just to add the filter step in the middle.
The preferred approach at this point would be to enable users to create a "filteredPathsIndexer" in their own repo like so:
import AutoIndexer from '@/AutoIndexer';
export default function (): string {
return AutoIndexer.getIndexYaml(this.env.globals.currentFilePointer,
{
paths: true,
filterFunction: (path) => { return true; }
});}
optionalProps helper i just hit a very irritating bug.
{{ optionalProps('../../user/model.yml') }}
Hits reference issues when your optionalProp file includes a $ref
The result will be that the optionalProp helper tries to resolve the nested $ref from the location of the said helper. This just results in an ugly error as the JSON Refs parser rightly cannot find the file. To make it even harder to debug.. if you have a double nested ref. the pointer seems to get really lost.
Describe the bug
Following screenshot shows an incorrectly formatted array items. The preceding dash on line 8 should not be there, but it's easy to put it in by mistake. This isn't caught in boats and gets put into the overall yml file.
Additional context
When you make this mistake and then use generate-it, generate-it throws an error like this:
TypeError [ERR_INVALID_ARG_TYPE]: The "data" argument must be of type string or an instance of Buffer, TypedArray, or DataView. Received undefined
In our microservice arch, we often find we are copy pasting the same models from one place to another.
We should be able to cut down on a lot of error-prone duplication by specifying a url (either raw git url or something else) so that on rebuilding / re-boats-ing the project, it automatically updates the schemas.
This way, we can have 1 definition from some external service, and just have to rebuild to get the latest, updated, and accurate models from shared services.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.