luckymarmot / api-flow Goto Github PK
View Code? Open in Web Editor NEWUniversal data structure and converter for API formats (Swagger, RAML, Paw, Postman…)
License: MIT License
Universal data structure and converter for API formats (Swagger, RAML, Paw, Postman…)
License: MIT License
Swagger define a root level field named parameters
, which can be used to share parameters between requests, and the Swagger Importer could move these into environment variables.
Currently, the RAML Parser does not check for file references inside its content, despite having a shimming file reader that replaces missing files with a ::fileRef::
reference file tag.
We should be able to parse the content to check if there's a fileRef in it.
When importing to paw the text-body newlines are represented as \n
dynamic values rather than \n
chars.
There can be multiple auths of the same type, which are hard to tell apart from each other when trying to export them. This is an issue for swagger and raml, as they use a root folder containing the security definitions
./random.sh 2>1
is not supported
array
, as they behave very differently from other typesschema
field from Response
, since it has been moved into parameterstags
and operationId
?^x-.*
patterns ?_extractResponseExternals
and _extractExternals
_extractResponseBodies
and _extractBodies
-- behaviors are somewhat different, not mergedNote: This list will be updated as we progress in the migration
It would be nice to have a method to rapidly detect the format of a file, if it supported.
Hi,
I'm trying to import postman's data, I did as README told, clone the repo, run "make install" with exit code 0, but there's no "./node_modules/lib/api-flow.js" file, if I try to use "./bin/api-flow.js", error comes:
node bin/api-flow.js /tmp/v2.json -f postman-2 -t raml
/tmp/tmp.jDw4AygtOF/API-Flow/lib/parsers/swagger/Parser.js:105
throw new Error(m + _tv2.default.error);
^
Error: Invalid Swagger File (invalid schema / version < 2.0):
ValidationError: Invalid type: string (expected object)
at SwaggerParser.parse (/tmp/tmp.jDw4AygtOF/API-Flow/lib/parsers/swagger/Parser.js:105:23)
at FlowCLI.run (/tmp/tmp.jDw4AygtOF/API-Flow/lib/runners/flow-node.js:269:34)
at Object. (/tmp/tmp.jDw4AygtOF/API-Flow/bin/api-flow.js:4:5)
at Module._compile (module.js:541:32)
at Object.Module._extensions..js (module.js:550:10)
at Module.load (module.js:458:32)
at tryModuleLoad (module.js:417:12)
at Function.Module._load (module.js:409:3)
at Module.runMain (module.js:575:10)
at run (bootstrap_node.js:352:7)
We have been using a char-by-char string parsing which may lead to a performance hit. As per this comment on PR #29:
What's the goal of this char-by-char string parsing? Why not simply a regexp to match
{{env}}
?/\{\{[^}]+\}\}/
would work nicely…
If this is to be able to resolve nested env variables, I'm concerned that we are trading a "nice to have" feature that is likely to not even be officially supported by Postman – and may not be even used by anyone (why on Earth would you nest environment variable names?!) – for a massive performance hit. Most JavaScript regexp engines are probably written in plain C, and is super efficient, doing this by hand here, on every single string we import, may be really slow…Maybe should we keep this function around, as an alternative? And quickly make a simpler one (can be done in a few lines I think). It's also about code complexity, I wouldn't want to resolve bugs in this myself :)
Would be nice to look into this later, and see what can be done. I'm thinking of an option to resolve nested environment names, which would trigger this char-by-char stack based parsing instead of a simpler and more efficient regexp-based parser.
I just imported the Swagger file from Instagram (on APIs.guru) and it doesn't set the Authorization URL for me:
Clearly the Swagger has it:
securityDefinitions:
api_key:
in: query
name: access_token
type: apiKey
instagram_auth:
authorizationUrl: 'https://instagram.com/oauth/authorize/'
flow: implicit
scopes:
basic: to read a user's profile info and media (granted by default)
comments: to post and delete comments on a user's behalf
follower_list: to read the list of followers and followed-by users
likes: to like and unlike media on a user's behalf
public_content: to read any public profile info and media on a user’s behalf
relationships: to follow and unfollow accounts on a user's behalf
type: oauth2
API-Flow should support configuration files to give users a finer control on the behavior of the tool.
Here are some of the options which could be useful:
name: [swagger]
: the name of the parser (e.g. swagger
, raml
, postman
, curl
, etc.). Default is swagger
.instance: [null]
if name is custom
, an instance
field could be used to pass a custom parserno-x: [false]
. If this option is set to true
, the parser will not attempt to parse x-*
patterned fields that it recognizesversion: [2.0]
. Not supported in v0.1. In the future, changing this option will allow user to parse v1.2
swagger files. If set to auto
, the parser will attempt to guess which version is the most suited.warn-on-missing: [no-warn]
. The level at which missing files raise warnings for the user. It can be any of [group, request, parameter, schema, example, none]
flatten: [null]
. If flatten is set to a number, it will represent the maximum depth of nested references that are supported before resolution. e.g. if flatten: 1
the following reference user{{{{domain}}-userId}}
will be flattened to user{{production-userId}}
. set flatten to 0
to resolve immediately.bash: [inline]
. Not supported in v0.1. Search for bash environment variables.speech-marks: [false]
. If set to true
, the parser will try to parse unicode speech marks as regular speech marks potentially delimiting keywords.true
.name: [web]
: the name of the resolver (e.g. node
, web
, paw
, atom
, etc.)instance: [null]
. If name is custom
, an instance
field could be used to pass a custom resolvercwd: [.]
. Not supported in v0.1. the directory/url from which to resolve all the uris.resolve: [true]
. resolve accepts either a boolean
, an object
, or an array
, which define how the granularity of the resolution.[true/false]
. if set to false
, no resolution is done. if set to true
, the resolver will try to resolve as much as possible.[object]
: the object should be of the following form:{
remote: [true/false],
local: [true/false],
custom: [array]
}
[array]
, is equivalent to [object]
with remote and local set to true
, and custom set to [array]
. The array should respect the following form:[
{
relative: '#/some/relative/uri',
resolve: [true/false],
value: [null/*]
},
...
]
name: [web]
: the name of the serializer (e.g. postman
, raml
, swagger
, curl
, etc.)instance: [null]
. If name is custom
, an instance
field could be used to pass a custom serializerobfuscate: [false]
. if set to true
, the serializer will try to obfuscate commonly used authentication fields, such as user
, username
, password
, secret
, etc.no-x: [false]
if set to true
, drops all x-*
patterned fields that provide additional informations created by API-Flow.format: [json]
. alternative value is yaml
domain: [null]
, if domain is set, use that name instead of the ones from API-Flow as a domain name.environment: [API-Flow]
. if environment is set, use that name instead of the ones from API-Flow as an environment name.no-comments: [false]
. if set to true
, output will have no comments to help readability.no-vars: [false]
. if set to true
, output will have no environment variables.There are probably other options that could be added.
We are using Swagger in an admittedly unusual way by annotating our swagger.yaml
with some internal info, e.g. which controller class of our service will handle the request.
This looks something like this:
/somePath:
x-swagger-router-controller: somePathController
get:
description: Returns the some data
Now when using the Swagger Importer, it parses x-swagger-router-controller
as just another HTTP method which us understandable but doesn't make sense in our case.
Would you be open to maybe add either a filtering for e.g. everything that starts with x-
or maybe simply ignore anything that's not in a valid-HTTP-methods whitelist?
The current architecture and models could be strongly improved through a careful refactoring of the app.
Here are some of the current issues that we encounter:
RequestContext.schema
was added to solve schema references issues with the Swagger parserRequestContext.environments
was added to solve Environment variables issues with the Postman parserThese fields feel like ad-hoc solutions to every format we meet. This is bad design, and we should move away from it. Ultimately, environments variables and schemas are only references that can be resolved locally.
Here are cool things we could integrate:
Here is a rough draft of the modification we suggest to make:
KeyValue
to Parameter
model, that will be used for queryParams
, urlEncodedParams
and formDataParams
, and headers
.Constraint
model that defines under which constraints a Parameter
can be used. Each Parameter
would therefore have an external
list of Constraint
under which they can be used/included.Constraint
list as an internal
field for Parameter
, that would describe the constraint the the Parameter itself must respect when being sent.headers
, queryParams
, urlEncodedParams
, and formDataParams
as fields of a ParameterContainer
model.Body
model, that contains a Constraint
list. It should be possible to filter the ParameterContainer
based on this Constraint
list to generate the actual content of the request.bodies
field in Request
, that would contain a list of Body
.description
, name
and example
fields everywhere.flatten
method to Group
generate
method to Parameter
, that uses Faker.js
or Chance.js
to create realistic and compliant data. This would be useful for on the fly example generation.name
, version
, description
, and other descriptive fields directly to RequestContext
exporters
in exporters/paw
-- changed to serializers/paw
Importer
files as Exporter
-- changed to Serializer
Parser
files as Importer
-- kept as Parser
immutables
directory as models
RequestContext
to Core
Auth
module structure by regrouping all the authentication methods in a single Auth
object.Core
module structure by grouping the multiple classes intelligently (Didn't look into it yet)Context
by removing the schema
and environment
fields. Create a references
field instead that would contain all the references that appear in the model. Maybe use a ReferenceContainer
, similar to the ParameterContainer
?See Paw-SwaggerImporter issue#9 for more information
_loadSwaggerCollection should only try to parse in one format (JSON/YAML) if an extension parameter is provided.
null
authentication is slightly different from no authentication.
From a semantic point of view, it means that the request can be done without any authentication, while no authentication could also mean that the authentications were not provided in the file for various reason (e.g. the format does not support this auth format).
So we make sure our code is always nice.
currently OAuth1Auth is not defined using the correct parameters.
It should.
in
field in Parameter
Body
object to RequestEnvironment
or something like that, since that's what it actually is - it has nothing to do with the body contentSchema
use.default
field to Parameter
, for use case where the Parameter.value field
has to be overloaded with other definitions (e.g. when type is array
)name
field to Auth
methods../root.json#/definitions/User
should be supported even in files that are resolvedWith the addition of names to auth methods, it is possible to put auths methods that are shared in a reference environment.
Some additional work in Paw is still required to support variations on a shared object (e.g. scopes in OAuth2) before this can be implemented.
we should be able to see the name of requests via comments in cURL-exporter
API-Flow should support imports from cURL.
I think the package.json can be improved. I don't know well the standards in organizing a npm module, but probably having a main index.js
script would make sense? We point to src/CurlImporter.js
as a main, and it's not a valid path.
See Paw-SwaggerImporter issue#12 for more information.
API-Flow should raise an error on failure, instead of the silent logging behavior we currently have.
See Paw-SwaggerImporter issue#4 for more information
example of a file using parameters at the path level:
https://raw.githubusercontent.com/APIs-guru/api-models/master/APIs/googleapis.com/siteVerification/v1/swagger.yaml
note: this will be updated as we progress in the migration
It feels heavy to import only one request in a group (especially for curl
). Can we make it so if there's only one request to be imported it's not wrapped in a group?
requested by a user
cURL importer is an object with field "requests".
Should be "context" and "group"
when exporting the API's from POW to PostMan or swagger, groups get ignored,.
If the basePath is /
, it is not properly trimmed, given results such as https://api.github.com//feeds
instead of https://api.github.com/feeds
See Paw-SwaggerImporter issue#5 for more information
Swagger v3 will soon be stable. it would be nice to have the possibility to import swagger v3.
Currently the swagger importer does not extract scopes from the swagger file, although it could be done relatively easily.
Swagger and RAML require requests to have unique paths/method product.
If a user has multiple requests that share the same path and method, the export is not fully compatible with Swagger and RAML, and it would be a good idea to raise a warning or something.
Swagger v3 will soon be stable. it would be nice to have the possibility to migrate swagger v2 to swagger v3
Currently, our postman parser is a bit lacking on OAuth2.
With more documentation/samples, it should be possible to improve it.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.