aml-org / amf

AMF (AML Modeling Framework) is an open-source library capable of parsing and validating AML metadata documents.

AML Modeling Framework

AMF (AML Modeling Framework) is an open-source programming framework, capable of parsing, generating and validating metadata documents defined using AML. It can be used as a library in Scala, Java, or JavaScript projects. The modular design of AMF facilitates creating plugins capable of parsing other metadata syntaxes not defined by AML.

📃 Documentation

• The AML Project
• AMF Documentation website
• AMF model documentation
• GitHub repository of AMF code examples

📦 Artifacts

Gradle

// add mulesoft repository
repositories {
    maven {
        url 'https://repository-master.mulesoft.org/nexus/content/repositories/releases'
    }
}
dependencies {
    compile 'com.github.amlorg:amf-api-contract_2.12:x.y.z'
}

Maven

<dependency>
    <groupId>com.github.amlorg</groupId>
    <artifactId>amf-api-contract_2.12</artifactId>
    <version>x.y.z</version>
</dependency>

NOTE: you may use the -SNAPSHOT versions of the artifacts at your own risk since those snapshot versions may contain breaking changes.

JavaScript

$ npm install --save amf-client-js

Generate artifacts directly from cloned repository

To build into a JVM jar:

sbt package

To build into a JS bundle:

sh js-build.sh

More info on how to add AMF to your project here.

AMF Native support

AMF natively supports the following formats:

• YAML
• JSON

the following semantic models:

• WebApi (or "Web APIs" as in "APIs accessible over the network")
• AsyncApi

and the following syntactic models:

• JSON-LD "AMF model"
• RAML 0.8 / 1.0 (mapped to "WebApi")
• OpenAPI (OAS) 2.0 / 3.0 (mapped to "WebApi")
• AsyncAPI 2.0 (mapped to "AsyncApi")

The models above and any other models may be extended and supported via custom AML-defined models. Other formats and models that cannot be expressed with AML may also be supported via plugins.

Guaranteed output

The only guaranteed output of AMF is the JSON-LD "AMF model". Any other output such as any output provided natively by the models listed under the section above may change at any time. This means that while the semantic representation of those outputs may remain unchanged, the syntactical expression such as the order in which the outputted metadata is expressed and any other syntax-related constructs may change from one version of AMF to another. If this is an issue for your use-case, you may consider using a custom resolution/generation pipeline.

AMF ecosystem modules

The following image shows each module in the AMF ecosystem as a dependency graph.

For AMF adopters it is recommended to use the amf-api-contract module which contains transitive dependencies with every module in the ecosystem except the CLI. For AML adopters (with no Web API nor Custom validation features usage) it is recommended to adopt the amf-aml module which includes parsing, validation & resolution for AML documents only. For more details on AML visit the AML repository.

The amf-api-contract and amf-aml are the recommended modules for AMF and AML adopters respectively.

Contributing

If you are interested in contributing to this project, please make sure to read our contributing guidelines.

amf's Issues

Union type does not have examples in json/ld

RAML:

#%RAML 1.0
title: Headers example
version: v1
baseUri: http://domain.api.domain.com/{version}/

/endpoint:
  get:
    headers:
      x-union:
        type: string | integer
        displayName: Union type
        examples:
          stringExample: Hello union
          numberExample: 25

Output:

[
  {
    "@id": "file://raml-examples/raml-headers/api.raml",
    "@type": [
      "http://raml.org/vocabularies/document#Document",
      "http://raml.org/vocabularies/document#Fragment",
      "http://raml.org/vocabularies/document#Module",
      "http://raml.org/vocabularies/document#Unit"
    ],
    "http://raml.org/vocabularies/document#encodes": [
      {
        "@id": "file://raml-examples/raml-headers/api.raml#/web-api",
        "@type": [
          "http://schema.org/WebAPI",
          "http://raml.org/vocabularies/document#RootDomainElement",
          "http://raml.org/vocabularies/document#DomainElement"
        ],
        "http://schema.org/name": [
          {
            "@value": "Headers example"
          }
        ],
        "http://raml.org/vocabularies/http#host": [
          {
            "@value": "domain.api.domain.com"
          }
        ],
        "http://raml.org/vocabularies/http#scheme": [
          {
            "@value": "http"
          }
        ],
        "http://raml.org/vocabularies/http#basePath": [
          {
            "@value": "/{version}/"
          }
        ],
        "http://schema.org/version": [
          {
            "@value": "v1"
          }
        ],
        "http://raml.org/vocabularies/http#endpoint": [
          {
            "@id": "file://raml-examples/raml-headers/api.raml#/web-api/end-points/%2Fendpoint",
            "@type": [
              "http://raml.org/vocabularies/http#EndPoint",
              "http://raml.org/vocabularies/document#DomainElement"
            ],
            "http://raml.org/vocabularies/http#path": [
              {
                "@value": "/endpoint"
              }
            ],
            "http://www.w3.org/ns/hydra/core#supportedOperation": [
              {
                "@id": "file://raml-examples/raml-headers/api.raml#/web-api/end-points/%2Fendpoint/get",
                "@type": [
                  "http://www.w3.org/ns/hydra/core#Operation",
                  "http://raml.org/vocabularies/document#DomainElement"
                ],
                "http://www.w3.org/ns/hydra/core#method": [
                  {
                    "@value": "get"
                  }
                ],
                "http://www.w3.org/ns/hydra/core#expects": [
                  {
                    "@id": "file://raml-examples/raml-headers/api.raml#/web-api/end-points/%2Fendpoint/get/request",
                    "@type": [
                      "http://raml.org/vocabularies/http#Request",
                      "http://raml.org/vocabularies/document#DomainElement"
                    ],
                    "http://raml.org/vocabularies/http#header": [
                      {
                        "@id": "file://raml-examples/raml-headers/api.raml#/web-api/end-points/%2Fendpoint/get/request/parameter/x-union",
                        "@type": [
                          "http://raml.org/vocabularies/http#Parameter",
                          "http://raml.org/vocabularies/document#DomainElement"
                        ],
                        "http://schema.org/name": [
                          {
                            "@value": "x-union"
                          }
                        ],
                        "http://www.w3.org/ns/hydra/core#required": [
                          {
                            "@value": true
                          }
                        ],
                        "http://raml.org/vocabularies/http#binding": [
                          {
                            "@value": "header"
                          }
                        ],
                        "http://raml.org/vocabularies/http#schema": [
                          {
                            "@id": "file://raml-examples/raml-headers/api.raml#/web-api/end-points/%2Fendpoint/get/request/parameter/x-union/union/schema",
                            "@type": [
                              "http://raml.org/vocabularies/shapes#UnionShape",
                              "http://www.w3.org/ns/shacl#Shape",
                              "http://raml.org/vocabularies/shapes#Shape"
                            ],
                            "http://www.w3.org/ns/shacl#name": [
                              {
                                "@value": "schema"
                              }
                            ],
                            "http://schema.org/name": [
                              {
                                "@value": "Union type"
                              }
                            ],
                            "http://raml.org/vocabularies/shapes#inherits": [
                              {
                                "@id": "file://raml-examples/raml-headers/api.raml#/web-api/end-points/%2Fendpoint/get/request/parameter/x-union/union/schema/union/null",
                                "@type": [
                                  "http://raml.org/vocabularies/shapes#UnionShape",
                                  "http://www.w3.org/ns/shacl#Shape",
                                  "http://raml.org/vocabularies/shapes#Shape"
                                ],
                                "http://raml.org/vocabularies/shapes#anyOf": [
                                  {
                                    "@id": "null/union/null/scalar/null",
                                    "@type": [
                                      "http://www.w3.org/ns/shacl#ScalarShape",
                                      "http://www.w3.org/ns/shacl#Shape",
                                      "http://raml.org/vocabularies/shapes#Shape",
                                      "http://raml.org/vocabularies/document#DomainElement"
                                    ],
                                    "http://www.w3.org/ns/shacl#datatype": [
                                      {
                                        "@id": "http://www.w3.org/2001/XMLSchema#string"
                                      }
                                    ]
                                  },
                                  {
                                    "@id": "null/scalar/null",
                                    "@type": [
                                      "http://www.w3.org/ns/shacl#ScalarShape",
                                      "http://www.w3.org/ns/shacl#Shape",
                                      "http://raml.org/vocabularies/shapes#Shape",
                                      "http://raml.org/vocabularies/document#DomainElement"
                                    ],
                                    "http://www.w3.org/ns/shacl#datatype": [
                                      {
                                        "@id": "http://www.w3.org/2001/XMLSchema#integer"
                                      }
                                    ]
                                  }
                                ]
                              }
                            ]
                          }
                        ]
                      }
                    ]
                  }
                ]
              }
            ]
          }
        ]
      }
    ]
  }
]

The output is missing examples description.

JDK9: define a stable automatic module name with `Automatic-Module-Name` entry in MANIFEST

body of the issue copied from

JDK9 is out and introduces a module system.

AMF does not currently use modules. When we use it in a JDK9 modular application, Java turns the JAR into a so-called automatic module, whose name is derived from the JAR archive name. However, the default name does not follow the recommended module naming conventions (reverse-dns style, module name derived from the main exported package).

It is possible to specify a stable automatic module name through a Automatic-Module-Name manifest entry, while still targeting JDK8:

Automatic-Module-Name: io.reactivex

Selecting a stable module name is very important, because Java does not allow two modules to own the same package, and you may end up with a module hell. The naming issues are well-explained on Stephen Colebourne's blog:

http://blog.joda.org/2017/04/java-se-9-jpms-module-naming.html
http://blog.joda.org/2017/05/java-se-9-jpms-automatic-modules.html <- the explanation of module hell problem

And the comment of Mike Reinhold, chief Java architect:

Strongly recommend that all modules be named according to the reverse Internet domain-name convention. A module's name should correspond to the name of its principal exported API package, which should also follow that convention. If a module does not have such a package, or if for legacy reasons it must have a name that does not correspond to one of its exported packages, then its name should at least start with the reversed form of an Internet domain with which the author is associated.

Generated uris must be url encoded

When generating uris for a new entity id or property, the generated URL must be URL encoded.

Query parameters model very depends on location [json/ld]

In HTTP method model query parameters are defined under http://raml.org/vocabularies/http#parameter property as an array of a type of http://raml.org/vocabularies/http#Parameter.

Trait's query parameters are, however, defined differently. They comes as a http://raml.org/vocabularies/data#queryParameters property where parameters are keys of an object and the object is of type of http://raml.org/vocabularies/data#Object.

This makes you create two completely different implementation of extracting essentially the same data. So the question is if there is any practical reason for query parameters to have different models representing the same information?

Serialised JSON-LD documents from a dialect cannot be parsed again

Missing logic to generate the right model from generic domain nodes

Mule integration: Xerces conflicts

The library cannot be executed in Mule because of conflicts with Xerces introduced as a transitive dependency through SHACL.
We need to make sure that the library can be executed correctly in Mule

Parsing input streams

I can see:

parseString* and parseFile* which cover many use cases but it would be interesting to also have parseInputStream so files downloaded from internet could be passed without writting to the disk or consuming memory.

Remove unused dialect/vocabulary files

Remove unused dialect/vocabulary files

JS version doesnt load absolute file paths

Loading an absolute (///) path like:

file:///tmp/path/to/file.raml

fails in the JS version, works in the JVM version

commons-io used but not declared as dependency

https://github.com/mulesoft/amf/blob/1e0f7d9ad9889b7830edc850cc5282ee8ddba167/amf-validation/jvm/src/main/scala/amf/plugins/features/validation/SHACLValidator.scala#L7

Won't work in client applications unless commons-io is manually included.

Web client: loading files from web filesystsem

One of the requirements for the API console is to accept zip files as an input of RAML package.
The client unzips the zip file into virtual filesystem. It's a copy of deprecated web filesystem and contains Blobs for each file entry.

In current setup the API console asks application to resolve dependency so I can choose whether the app should download the dependency (Fetch, XHR) or to provide a content (Blob to string conversion and returning string from file). It allows the app to handle all situations while parsing RAML.

Is this possible with AMF? So far I can't find interface to do it.

References as links in dialects

When parsing references to declared domain entities (whether local to the document or remote to a library) a link to the element must be inserted instead of an in-lined version of the element when generating the JSON-LD of the element.

It should be possible to parse back the JSON-LD document into the model when parsing the generated output.

Any additional properties can be added to the link (besides the @id property) to facilitate the parsing task from JSON-LD back to the model

Can't load examples with amf-playground

Hi.

I cant load local multi-file examples in amf-playground and the error seems to come from amf.

Here's what I do:

1. Build the amf (with es5 output mode) and use it as a dependency for amf-playground;
2. Run gulp serve and open a playground.html page in a browser;
3. Click Load API Specification and select Banking API (RAML) example. Click Load;
4. Then I get an error saying amf.core.remote.PathResolutionError: Error resolving path: url: traits/content-cacheable.raml - Unsupported operativeSystem operation.

Note that it works fine when hosted on Github Pages. Also I believe amf-playground uses amf as an npm module - the code.

So far I've figured out it is caused by this line. I've tried changing this another line to always use new JsServerPlatform() and it started loading those examples.

PS. Is using amf in browser still a thing? Am I somehow using amf incorrectly with amf-playground?

Type example is a string instead of any structured data

Following declaration:

#%RAML 1.0
title: Query example
version: v1
baseUri: http://domain.api.domain.com/{version}/

types:
  Image:
    type: object
    properties:
      type:
        type: string
        enum: [thumb, full, hd, 4k]
      url: string
  Product:
    displayName: A product resource
    description: A single product representing an item in the store.
    type: object
    properties:
      id:
        description: |
          Product id. It is a UUID of the database record.
          __This property will be ignored when creating an object.__
          It will be available when the product is stored in the datastore.
        pattern: "[0-9a-zA-Z-]+"
      name:
        type: string
        description: Product name
        example: Acme product - mentol flavor, 500 ml.
        required: true
      quantity:
        type: number
        description: |
          The quantity of the product in the one unit of measurement.
          See `unit` property for more information.
        example: 500
        required: true
      unit:
        type: string
        description: The unit of measuremet for the quantity property.
        example: ml
        required: true
      upc:
        type: string
        pattern: "[0-9]{12,12}"
        description: |
          The Universal Produc Code. It consists of 12 numerical digits. However, because of the
          integer precision limitation in JavaScript it is represented as a string.
        example: "042100005264"
        required: true
      available:
        type: boolean
        description: |
          Product current availability in the store.
          Product may be not available but the users still can order it with later delivery date.
        required: true
        example: true
      images: Image[]
    example:
      id: we2322-4f4f4f-f4f4ff-f4f4ff4
      name: Acme Product
      quantity: 200
      unit: ml
      upc: "123456789101"
      available: true
      images:
        - type: thumb
          url: https://domain.com

Produces following example model:

"http://raml.org/vocabularies/document#examples": [
          {
            "@id": "file://raml-examples/raml-body-params/api.raml#/declarations/Product/example/null",
            "@type": [
              "http://raml.org/vocabularies/document#Example",
              "http://raml.org/vocabularies/document#DomainElement"
            ],
            "http://raml.org/vocabularies/document#value": [
              {
                "@value": "      id: we2322-4f4f4f-f4f4ff-f4f4ff4\n      name: Acme Product\n      quantity: 200\n      unit: ml\n      upc: \"123456789101\"\n      available: true\n      images:\n        - type: thumb\n          url: https://domain.com\n"
              }
            ],
            "http://raml.org/vocabularies/document#strict": [
              {
                "@value": true
              }
            ]
          }
        ]

This information is useless until I parse it as a YAML string and then transform it to JS object (for example). RAML JS parser results with example being a JS object (map of key - value pairs). INHO AMF should process structured type example as objects and not strings.

Validate scalar ranges in dialect definitions

Loading from classpath

JVM application would benefit from being able to load resources from the classpath, using a URL like jar:/path/to/resources.
This should be available at the platform level and integrated with the resource resolution logic

Upload CLI jar to repository

The CLI jar, meant to be a quick and handy tool has its whole point defeated by the fact that the only way to obtain it is to download the sources and compiling.

file scheme missing in URLs when fetching local files

When we are parsing local files, we are storing the URL without the file scheme.
This is making my REPL fail when trying to build java.net.URI objects.

I have tracked the error to the fetchFile https://github.com/mulesoft-labs/amf/blob/master/amf-jvm/src/main/scala/amf/remote/JvmPlatform.scala#L40 function in the platform implementation for the JVM and JS/server.

We are building the Content object only with the path there.

"Summary" / "Summaries" not supported in pluralize and singularize

InflectorBase contains a number of plural, singular, irregular and uncountable rules. The word "Summary" should pluralize to "Summaries" and vice versa.

My suggestion is to update InflectorBase to include all the rules to the popular pluralize library, the full list can be found here

Getting a weird : in the file path with the JS client

I'm trying to get the amf javascript client to work (I'm on windows) but it always inserts a : into the path

file://api.raml becomes :\api.raml

version: 1.6.0

code

const amf = require('amf-client-js');

amf.plugins.document.WebApi.register();
amf.plugins.document.Vocabularies.register();
amf.plugins.features.AMFValidation.register();

const ramlParser = amf.Core.parser('RAML 1.0', 'application/raml');

amf.Core.init().then(async () => {
  const res = await ramlParser.parseFileAsync('file://api.raml');
  res //?
}).catch(console.error.bind(console));

error

$c_Lamf_core_remote_FileNotFound {
  's$1':
   'java.io.IOException: ENOENT: no such file or directory, open \'C:\\Users\\harmony\\Code\\Web\\WebstormProjects\\amf-raml\\:\\api.raml\'',
  'e$1':
   $c_Ljava_io_IOException {
     's$1':
      'ENOENT: no such file or directory, open \'C:\\Users\\harmony\\Code\\Web\\WebstormProjects\\amf-raml\\:\\api.raml\'',
     'e$1': null,
     'stackTrace$1': null,
     stackdata: [Circular] },
  'stackTrace$1': null,
  'cause$3':
   $c_Ljava_io_IOException {
     's$1':
      'ENOENT: no such file or directory, open \'C:\\Users\\harmony\\Code\\Web\\WebstormProjects\\amf-raml\\:\\api.raml\'',
     'e$1': null,
     'stackTrace$1': null,
     stackdata: [Circular] },
  stackdata: [Circular] }

Domain model constructor methods produce error in JS

With version 3.1.1 installed from npm and Node v10.14.1 following code [1] generates proper RAML.

But when line .withServers([server]) is replaced with .withServer('http://someorg.bar/{version}') an error is produced [2] pointing to the .withServer. call.

This happens with all methods that receive basic types like string and produce instances of AMF classes. I tested it with WebApi, EndPoint, Operation, Response, Payload.

[1]

// demo-amf.js
const amf = require('amf-client-js')

function main () {
  const server = new amf.model.domain.Server()
    .withUrl('http://someorg.bar/{version}')

  const api = new amf.model.domain.WebApi()
    // .withServer('http://someorg.bar/{version}')
    .withServers([server])

  const model = new amf.model.document.Document()
    .withEncodes(api)

  return amf.AMF.init()
    .then(() => {
      return amf.AMF.raml10Generator().generateString(model)
    })
    .then((generated) => {
      console.log('Generated from constructed:\n', generated)
    })
    .catch((e) => {
      console.log(e)
    })
}

return main()

[2]

/home/someuser/projects/test-amf/node_modules/amf-client-js/amf.js:303
function iha(a,b){if(dc(b)){var c=(new Bs).Xm(a.uG).Ng.xr();a:{for(;c.hb();){var e=c.Va();if(e.R(b.mc())){c=(new A).c(e);break a}}c=w()}if(x(c))return a.uG.R(c.s).R(b);if(w()===c)throw ec(fc(),(new gc).f("Cannot find builder for object meta "+b.mc()));throw(new v).c(c);}if(cc(b)){c=(new Bs).Xm(a.uG).Ng.xr();a:{for(;c.hb();)if(e=c.Va(),e.R(b.mc())){c=(new A).c(e);break a}c=w()}if(x(c))return a.uG.R(c.s).R(b);if(w()===c)throw ec(fc(),(new gc).f("Cannot find builder for object meta "+b.mc()));throw(new v).c(c);
                                                                                                                                                                                   ^
Error
    at gc.d.pR (/home/someuser/projects/test-amf/node_modules/amf-client-js/amf.js:1714:195)
    at gc.d.ge (/home/someuser/projects/test-amf/node_modules/amf-client-js/amf.js:1724:187)
    at gc.f (/home/someuser/projects/test-amf/node_modules/amf-client-js/amf.js:2256:70)
    at iha (/home/someuser/projects/test-amf/node_modules/amf-client-js/amf.js:303:203)
    at Ds (/home/someuser/projects/test-amf/node_modules/amf-client-js/amf.js:304:233)
    at F_ (/home/someuser/projects/test-amf/node_modules/amf-client-js/amf.js:2373:444)
    at Tm.d.EU (/home/someuser/projects/test-amf/node_modules/amf-client-js/amf.js:5478:479)
    at Tm.d.AO (/home/someuser/projects/test-amf/node_modules/amf-client-js/amf.js:5479:30)
    at Tm.withServer (/home/someuser/projects/test-amf/node_modules/amf-client-js/amf.js:5483:159)
    at main (/home/someuser/projects/test-amf/demo-amf.js:8:6)

Being able to define an url of the thing

We need a mechanism to define url for the thing.

One option is a link header, but it does not always work.

empty value for any resource in RAML should be valid

Assume the following RAML:

#%RAML 1.0

version: 1.0
#mediaType: application/json

/customers:
  displayName: Customer Resource
  /{customer_id}:

Despite the title is missing, the following command ./amf.js validate -p RAML -in RAML api.raml fails with java.lang.Exception: Expected map but found: "".

There is no value for /{customer_id} which should be still valid in RAML.

Custom validation for a dialect

JS client: TypeError: Cannot read property 'length' of undefined

I am having following error in node version:

TypeError: Cannot read property 'length' of undefined
    at $c_sci_StringOps.length__I (/Users/pawelpsztyc/workspace/tests/amf/node_modules/amf-client-js/amf.js:167639:20)
    at $f_sc_IndexedSeqOptimized__isEmpty__Z (/Users/pawelpsztyc/workspace/tests/amf/node_modules/amf-client-js/amf.js:165765:17)
    at $c_sci_StringOps.isEmpty__Z (/Users/pawelpsztyc/workspace/tests/amf/node_modules/amf-client-js/amf.js:167550:12)
    at $f_sc_TraversableOnce__nonEmpty__Z (/Users/pawelpsztyc/workspace/tests/amf/node_modules/amf-client-js/amf.js:31144:18)
    at $c_Lamf_core_remote_Context$.apply__Lamf_core_remote_Platform__T__sci_Map__Lamf_core_remote_Context (/Users/pawelpsztyc/workspace/tests/amf/node_modules/amf-client-js/amf.js:6919:15)
    at $c_Lamf_core_remote_Context$.apply__Lamf_core_remote_Platform__T__Lamf_core_remote_Context (/Users/pawelpsztyc/workspace/tests/amf/node_modules/amf-client-js/amf.js:6952:43)
    at $c_Lamf_core_AMFCompiler.context$lzycompute__p1__Lamf_core_remote_Context (/Users/pawelpsztyc/workspace/tests/amf/node_modules/amf-client-js/amf.js:4833:78)
    at $c_Lamf_core_AMFCompiler.context__p1__Lamf_core_remote_Context (/Users/pawelpsztyc/workspace/tests/amf/node_modules/amf-client-js/amf.js:4820:50)
    at $c_Lamf_core_AMFCompiler.build__s_concurrent_Future (/Users/pawelpsztyc/workspace/tests/amf/node_modules/amf-client-js/amf.js:4651:14)
    at /Users/pawelpsztyc/workspace/tests/amf/node_modules/amf-client-js/amf.js:4862:239

I have installed nevest version of the library from npm. Any walkarounds?

Missing javadoc and sources

We should provide artifacts for the javadoc and the sources of the library to facilitate the development in the JVM

Fix validation concurrency issues

SHACL is not thread-safe we need to find a solution for concurrent invocations to the parser

Review dialect example fragment URLs

URLs for included fragments in a dialect example is not being generated correctly.
A new URL is used instead of pointing at the dialect link

no reporting after validation or parsing

I have the following RAML:

#%RAML 1.0

version: 1.0
mediaType: application/json

# uses:
#   shapes: ./dataTypes/shapes.raml

types:
  MonetaryAmountData:
    properties:
      value: number
      currency:

  AddressData:
    properties:
      address_country: 
      address_locality?: 
      address_region?: 
      postal_code:  
      street_address?: 

  CustomerData:
    discriminator: type
    properties:
      type: 
      lei: 
      tax_id: 
      email: 
      address: AddressData

  PersonData:
    discriminatorValue: Person
    type: CustomerData
    properties:
      id: 
      title?:
        type: string
#        enum: [mr, mrs, ms, dr]
      given_name: 
      family_name: 
      gender:
        type: string
#        enum: [female, male]
      vat_id?: 
      birth_date: date-only
      death_date?: date-only

  CustomerPatchData:
    properties:
      lei?: 
      tax_id?: 
      email?: 
      title?:
        type: string
#        enum: [mr, mrs, ms, dr]
      name?: 
      family_name?: 
      gender?:
        type: string
#        enum: [female, male]
      vat_id?: 
      birth_date?: date-only
      death_date?: date-only
      address_country?: 
      address_locality?: 
      address_region?: 
      postal_code?: 
      street_address?: 


  NewBankAccountRequestData:
    properties:
      accountType:
        type: string
#        enum: [ standard, saver ]

  BankAccountData:
    properties:
      id:
      account_number:
      accountType:
        type: string
#        enum: [ standard, saver ]
      amount: MonetaryAmountData
      lei:
      fees_and_comissions:
      review_state:
        type: string
#        enum: [ requested, cancelled, opened, closed ]
      interest_rate: number
      annual_interest_rate: number
      minimum_inflow: MonetaryAmountData
      overdraft_limit: MonetaryAmountData

/customers:
  displayName: Customer Resource
  /{customer_id}:
    uriParameters:
      customer_id:
        type: string
    get:
      description: Returns customer data
      responses:
        200:
          body:
            application/json:
              type: PersonData
              example: 
                type: Person
                id: "!23456"
                lei: 54930084UKLVMY22DS16
                tax_id: "999999999"
                email: [email protected]
                given_name: Dirk
                family_name: Fabian
                gender: male
                birth_date: 1987-09-30
                address:
                  address_country: US
                  address_locality: CA
                  postal_code: "90003"
    patch:
      description: Updates customer data
      responses:
        200:
          body:
            application/json:
              type: CustomerPatchData
    delete:
      description: Removes a customer from the system  
    /Accounts:
      get:
        description: Returns a collection of accounts.
        responses:
          200:
            body:
              application/json:
                type: array
                items: BankAccountData
      post:
        description: Requests the creation of a new account.
        body:
          application/json:
            type: NewBankAccountRequestData
      /{account_id}:
        uriParameters:
          account_id: 
            type: string
        get:
          description: Returns account data
          responses:
            200:
              body:
                application/json:
                  type: BankAccountData
                  example: 
                    id: my_account
                    account_number: "12345667"
                    accountType: standard
                    amount: 
                      value: 123.45
                      currency: Euro
                    lei: 54930084UKLVMY22DS16
                    fees_and_comissions: no fees
                    review_state: opened
                    interest_rate: 12
                    annual_interest_rate: 15
                    minimum_inflow: 
                      value: 1000
                      currency: Euro
                    overdraft_limit: 
                      value: 500
                      currency: Euro
        delete:
          description: Removes a account from the system

Executing ./amf.js parse -v true -p RAML -in RAML api.raml or ./amf.js validate -p RAML -in RAML api.raml, I would expect some reporting that the title is missing but I don't see anything except the JSON-LD of the model.

Exception while parsing RAML files from java

Hi guys, I am parsing RAML files using Java api and observing following exception:

Caused by: java.lang.Exception: Cannot find builder for object meta amf.core.metamodel.document.ExternalFragmentModel$@37ec39ef
	at amf.core.remote.Platform.wrapFn(Platform.scala:84)
	at amf.core.remote.Platform.wrapFn$(Platform.scala:75)
	at amf.core.remote.JvmPlatform.wrapFn(JvmPlatform.scala:12)

According to my understanding it seems that Java wrappers for fragments are missed

Thanks in advance,
Pavel

Non literal types are not honored

Defining a vocabulary term with:

createdAt:
    displayName: createdAt
    description: Date and time when the repository was created
    range: dateTime

In a dialect:

createdAt:
        propertyTerm: about-github.createdAt
        range: string

Ends up creating a triple like this:

"http://mulesoft.com/vocabularies/about-github#createdAt": [
          {
            "@value": "2012-12-31T23:57:00"
          }
        ],

Note the lack of ^^xsd:dateTime after the literal.

Ranges of propertyTerms are not validated properly

As you may see from the image, ranges of the property terms are not validated:

Regards,
Pavel

Fixing modular entities URIs and implement resolution in Vocabularies

Uris of the declared modular units and those units composed as references in a document must match.

It must be possible to resolve modular units and get a unified model without references or modules but preserving the information and URIs

Supporting libraries of nodeMappings for dialects

Dialect definitions should also support libraries

Loading file error in JS client

My setup:

Node JS server side code.
Package dependency:

"@mulesoft/amf-js": "0.0.6-142"

Application code:

const amf = require('@mulesoft/amf-js');
const parser = new amf.RamlParser();

const url = 'http://cdn.rawgit.com/advanced-rest-client/raml-example-api/42cad8c7/api.raml';
// const url = '../../advanced-rest-client/raml-example-api/api.raml';

parser.parseFile(url, {
  success: function(doc) {
    console.log(doc);
  },

  error: function(exception) {
    console.log(exception.toString());
  }
});

For url defined as a local path i've got the following error:

java.lang.Exception: Unsupported url: ../../advanced-rest-client/raml-example-api/api.raml

For url with https scheme:

_http_client.js:129
    throw new Error('Protocol "' + protocol + '" not supported. ' +
    ^

Error: Protocol "https:" not supported. Expected "http:"
    at new ClientRequest (_http_client.js:129:11)
    at request (http.js:39:10)
    at Object.get (http.js:43:13)
    at $c_Lamf_remote_server_JsServerPlatform.fetchHttp__T__s_concurrent_Future (/Users/pawelpsztyc/workspace/test/amf/node_modules/@mulesoft/amf-js/target/artifact/amf-module.js:16466:13)
    at $f_Lamf_remote_Platform__resolve__T__s_Option__s_concurrent_Future (/Users/pawelpsztyc/workspace/test/amf/node_modules/@mulesoft/amf-js/target/artifact/amf-module.js:1520:22)
    at $c_Lamf_remote_server_JsServerPlatform.resolve__T__s_Option__s_concurrent_Future (/Users/pawelpsztyc/workspace/test/amf/node_modules/@mulesoft/amf-js/target/artifact/amf-module.js:16421:12)
    at $c_Lamf_compiler_AMFCompiler.root__s_concurrent_Future (/Users/pawelpsztyc/workspace/test/amf/node_modules/@mulesoft/amf-js/target/artifact/amf-module.js:3294:26)
    at $c_Lamf_compiler_AMFCompiler.compile__p1__s_concurrent_Future (/Users/pawelpsztyc/workspace/test/amf/node_modules/@mulesoft/amf-js/target/artifact/amf-module.js:3190:17)
    at $c_Lamf_compiler_AMFCompiler.build__s_concurrent_Future (/Users/pawelpsztyc/workspace/test/amf/node_modules/@mulesoft/amf-js/target/artifact/amf-module.js:3180:32)
    at $c_Lamf_client_RamlParser.parseAsync__T__s_Option__s_concurrent_Future (/Users/pawelpsztyc/workspace/test/amf/node_modules/@mulesoft/amf-js/target/artifact/amf-module.js:14894:365)

For http scheme:

java.lang.Exception: Reference not found

The example RAML is the one I was using all the time working on the API console and RAML parser parses it without errors. Is there anything I can do to make it work?

Remove intermediate property in dialect modules

Right now when parsing a dialect library like:

#% RAML Library / My Dialect 0.1

syntaxToIntroduceDeclaration:
  myDeclaration: 
     props: here

We are generating a module definition in JSON-LD like:

mymodule.raml a raml-doc:Module ;
             raml-doc:declares mylibrary.raml#.
mylibrary.raml# a raml-doc:module ;
            raml-doc:syntaxToIntroduceDeclaration mylibrary#myDeclaration .

That intermediate node of type raml-doc:module and the raml-doc:syntaxToIntroduceDeclaration are an error.

The expected graph would be:

mymodule.raml a raml-doc:Module ;
            raml-doc:declares mylibrary#myDeclaration .

Additional annotations can be added to track the information about the property in the syntax that introduces the declaration (syntaxToIntroduceDeclaration)

Dynamic linking of dialects

Right now, in order to validate a a dialect instance, the association between dialect and document must be known in advance by the parser through the dialects registry.

With this change, we want to make sure that it is possible to work with dialects the parser (and dialect designer) don't know until parsing time.

The mechanism proposed is based in making the link between the dialect instance and the dialect definition explicit in the header of the RAML document.

For instance, provided a dialect published at "http://myorg.com/dialects/marketing.raml", named "Marketing" version 0.1, and introducing fragments "Customer", customer fragment must start with the header, for example the fragment at "http://myrog.com/marketing/customers/1":

#%Marketing / Customer 0.1
myfragment: here

In order to parse this fragment the parser needs to know in advance that the "Marketing 0.1" documents are describe by the dialect located at "http://myorg.com/dialects/dialect1.raml".

We want to make this association explicit so the dialect can be parsed at parsing time. For this, the dialect instance must include the link to the dialect between "<>" and separated by a "|" in the header of the document.

For example, for the previous example, the Fragment will become:

#%Marketing / Customer 0.1 | <http://myorg.com/dialects/dialect1.raml>
myfragment: here

With this information if the parser needs to process the document at "http://myrog.com/marketing/customers/1", it will discover the link, auto-register the dialect "http://myorg.com/dialects/marketing.raml" and finally validate and parse the dialect.

It will be encouraged that users provide the explicit link to the dialect definition to improve the reusability and inter-linking of dialect instances.

The parsed fragment graph must include the triple:

<http://myorg.com/marketing/customers/1> <http://raml.org/vocabularies/meta#describedBy>  <http://myorg.com/dialects/marketing.raml> .

To be able to reconstruct the dependencies in the document graph.
This link must disappear after resolution.

If the dialect is registered explicitly in the parser, linked dialects in the documents will not be parsed and the local information will be used.

targetNode in parser side validations for dialects should point to a domain element URI

Right now targetNode is reporting a value not a URI.

Compare:

Node client: Cannot find target for parametrized resource type

I am trying to use resolve function to produce canonical form from the API spec. I am experiencing the following error:

Cannot find target for parametrized resource type file://raml-example-api/api.raml#/web-api/end-points/%2Fpeople%2F%7BpersonId%7D/

The API I am trying to use in my example API that I am using to work with demo applications:
https://github.com/advanced-rest-client/raml-example-api

Let me know if you need more details.

Parser-side validation errors in dialects

Generate parser-side validations in dialects parsing instead of throwing exception with lexical information

Remove global state from validations

Supporting nodeMappings fragments for dialects

Fragments should be supported in dialect definition documents to encode a single nodeMapping

!extend as an specialised !include in vocabularies/dialects

!extend (better than !extends in order to match !include, not !includes) should behave as !include plus an optional map of specialisations

Let's implement it only for fragments leaving apart libraries for the time being.

Upload to repository the sources

JMV users expect the source to be published along side the artifact so tooling can download to help debugging:

apimf-411

Having the following Dialect Library

#%RAML Library / RAML 1.0 Dialect

uses:
  anypoint-vocabulary: ./anypoint-vocabulary.raml

nodeMappings:

  asset:
    classTerm: anypoint-vocabulary.Asset
    mapping:
      group:
        propertyTerm: anypoint-vocabulary.group
        mandatory: true
        range: string
      artifact:
        propertyTerm: anypoint-vocabulary.artifact
        mandatory: true
        range: string
      version:
        propertyTerm: anypoint-vocabulary.version
        mandatory: true
        range: string
      classifier:
        propertyTerm: anypoint-vocabulary.classifier
        mandatory: false
        range: string
      packaging:
        propertyTerm: anypoint-vocabulary.packaging
        mandatory: true
        range: string
      composedBy:
        propertyTerm: anypoint-vocabulary.asset
        mandatory: false
        range: asset

and then used it as a fragment in a library

#%RAML 1.0 Dialect

dialect: Mule Agent Configuration
version: 0.1

uses:
    mule-runtime: ./mule-runtime-vocabulary.raml
    anypoint-vocabulary: ./anypoint-vocabulary.raml
    anypoint-dialect: ./anypoint-dialect.raml

external:
  schema-org: http://schema.org/

raml:
  document:
    encodes: muleAgentConfiguration
    declares:
      assets: anypoint-dialect.asset

  fragments:
    encodes:
      Asset: anypoint-dialect.asset

nodeMappings:

  muleAgentConfiguration:
    classTerm: mule-runtime.Agent
    mapping:
      asset:
        propertyTerm: anypoint-vocabulary.asset
        mandatory: false
        range: anypoint-dialect.asset

throws the following exception:

Exception in thread "main" java.util.concurrent.ExecutionException: java.lang.Exception: Encoded domain element is not a dialect domain entity ExternalDomainElement(amf.core.parser.Fields@1b0076fe,amf.core.parser.Annotations@2ca00669)
	at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)

With the following requesT:

#%Mule Agent Configuration 0.1

asset: include ./salesforce-asset.raml

Archive.zip

Strange "enum" validation

RAML:

#%RAML 1.0

version: 1.0
title: API

types:
  PersonData:
    properties:
      title?:
        type: string
        enum: [ "mr", "mrs", "ms", "dr" ]

/customers:
  displayName: Customer Resource
  get:
    description: Returns customer data
    responses:
      200:
        body:
          application/json:
            type: PersonData

Command line:

./amf.js validate -p RAML -in RAML api.raml

Report:

{
  "@type": "http://www.w3.org/ns/shacl#ValidationReport",
  "http://www.w3.org/ns/shacl#conforms": false,
  "http://www.w3.org/ns/shacl#result": [
    {
      "@type": "http://www.w3.org/ns/shacl#ValidationResult",
      "http://www.w3.org/ns/shacl#resultSeverity": {
        "@id": "http://www.w3.org/ns/shacl#Violation"
      },
      "http://www.w3.org/ns/shacl#focusNode": {
        "@id": "file:/test.raml#/declarations/PersonData/property/title?/scalar/title?"
      },
      "http://www.w3.org/ns/shacl#resultPath": {
        "@id": "http://www.w3.org/ns/shacl#in"
      },
      "http://www.w3.org/ns/shacl#resultMessage": "Property 'enum'  must have a single list of values",
      "http://www.w3.org/ns/shacl#sourceShape": {
        "@id": "http://raml.org/vocabularies/amf/parser#Shape-in-maxCount"
      },
      "http://raml.org/vocabularies/amf/parser#lexicalPosition": {
        "@type": "http://raml.org/vocabularies/amf/parser#Position",
        "http://raml.org/vocabularies/amf/parser#start": {
          "@type": "http://raml.org/vocabularies/amf/parser#Location",
          "http://raml.org/vocabularies/amf/parser#line": 11,
          "http://raml.org/vocabularies/amf/parser#column": 14
        },
        "http://raml.org/vocabularies/amf/parser#end": {
          "@type": "http://raml.org/vocabularies/amf/parser#Location",
          "http://raml.org/vocabularies/amf/parser#line": 11,
          "http://raml.org/vocabularies/amf/parser#column": 41
        }
      }
    }
  ]
}

I am not sure why it is failing validation. Seems completely valid for me.

missing semantic sugar for URI parameters

Assume the following RAML:

#%RAML 1.0

title: ACME Banking HTTP API
version: 1.0
mediaType: application/json

/customers:
  displayName: Customer Resource
  /{customer_id}:
    uriParameters:
      customer_id: string

Executing ./amf.js validate -cp file://acme-profile.raml -in RAML example/api.raml, AMF validator returns java.lang.Exception: Expected map but found: "string".

Modifying customer_id: string to use the extended form with type validates correctly.

Rename .raml files in vocabularies/ folder

• Change extension .raml to .yaml to avoid confusion
• Update reference in docbook documentation: https://github.com/aml-org/docbook/blob/master/documentation/doc_model.md

Range of dialect definitions and !include / !extend

This issue reviews the semantics of range for object properties after the introduction of the !extend keyword.

The facet range can be used to set the range of a property to be literal (not reviewed in this issue) or a target node mapping reference.

The target node can be define:

• with class and properties

nodeMappings:
  
  A:
    classTerm: vocab.A
    mapping:
      a:
       propertyTerm: vocab.a
       range: string

  B:
    classTerm: vocab.B
    mapping:
      b:
        propertyTerm: vocab.b
        range: A # A node has class and property restrictions

• Only properties

nodeMappings:
  
  A:
    mapping:
      a:
       propertyTerm: vocab.a
       range: string

  B:
    classTerm: vocab.B
    mapping:
      b:
        propertyTerm: vocab.b
        range: A # A node has only properties

• Only class term:

nodeMappings:
  
  A:
    classTerm: vocab.A

  B:
    classTerm: vocab.B
    mapping:
      b:
        propertyTerm: vocab.b
        range: A # A node has only class term

• the special AnyNode reference:

nodeMappings:

  B:
    classTerm: vocab.B
    mapping:
      b:
        propertyTerm: vocab.b
        range: AnyNode

Each of this definitions have an impact in the triples that will be generated:

• if class and properties are defined, triples for rdf:type and property triples will be generated
• if only class is present, only the rdf:type triple will be generated
• if only properties are present only property tripels will be generated (not rdf:type)

This will also have an impact on the validation of the target node, because only type, properties or both of them will be parsed.

AnyNode has an special behaviour that can be used to link any external fragment using !extend.
In this case the linked fragment will be parsed (using the fragment header or explicit dialect link in the fragment header) and the parsed graph will be linked wihout further validation under the property in the range.

If the user is linking a fragment using !extend all the available information fro the constraints of the node will be used to validate the linked fragment.

• if AnyNode is used, no local validation will be performed.
• if only classTerm facet is used, the fact that the graph for the remoter fragment is an instance of the class will be validated
• if only properties are defined, constraints for each of the properties will also be validated over the remote fragment graph
• if both classTerm and properties are locally defined all those constraints will be validated additionally to the remote fragment validation.

A note about the constraint validation implementation. All this must be possible to validate using SHACL shapes after resolution. No local ad-hoc validation code must be added.

Validation silently ignores inexisting types references in params

Hi.
See below for a RAML and code example. Note that params in RAML point to in-existing types:

type: { collection: { targetType: XXX } }
type: { item: { itemType: VVV } }

When performing a validation though this is not reported. The only thing that gets output by the code below is:

Unresolved valid: true
Resolved valid: false
closed shapes cannot inherit from open shapes

Resolution ignores these type params as well and simply replaces application/json: <<targetType>>[] with type: any.

Tested with [email protected] installed from NPM.

Is this an expected behaviour?

The code:

var AMF = require('amf-client-js').AMF

var ramlStr = `
  #%RAML 1.0
  title: The BeerTrader api
  description: This is the API for the world famous Beer Trader API
  version: v1
  baseUri:
   value: http://www.beertrader.com/services/{version}
  protocols: [  HTTP, HTTPS ]
  mediaType: [ application/json ]
  types:
   UUID:
       type: string
       pattern: "[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}"

   Identified:
     additionalProperties: false
     properties:
       id: UUID

   UserProperties:
     additionalProperties: false
     type: Identified
     properties:
       name: string
       email: string
   User:
     additionalProperties: true
     type: UserProperties
  resourceTypes:
   collection:
     get:
       responses:
         200:
           body:
             application/json: <<targetType>>[]
     post:
       body:
         application/json: <<targetType>>
       responses:
         201:
           headers:
             Location: string
           body:
             application/json: <<targetType>>
   item:
     get:
       responses:
         200:
           body:
             application/json: <<itemType>>
         404:
     delete:
       responses:
         200:
         404:
     put:
       body:
         application/json: <<itemType>>
       responses:
         200:
         404:
  /users:
   type: { collection: { targetType: XXX } }
   /{userId}:
     type: { item: { itemType: VVV } }
`

var unit

AMF.init()
  .then(function () {
    return new AMF.raml10Parser().parseStringAsync(ramlStr)
  })
  .then(function (u) {
    unit = u
    return AMF.validate(unit, 'RAML 1.0', 'RAML 1.0')
  })
  .then(function (report) {
    console.log('Unresolved valid:', report.conforms)
  })
  .then(function () {
    return new AMF.resolveRaml10(unit)
  })
  .then(function (resolved) {
    return AMF.validate(resolved, 'RAML 1.0', 'RAML 1.0')
  })
  .then(function (report) {
    console.log('Resolved valid:', report.conforms)
    if (!report.conforms) {
      report.results.forEach(function (res) {
        console.log(res.message)
      })
    }
  })