Add support for multiple examples about raml-spec

[freezy]

Parsing the files with raml2obj, then feeding the result to a Jade template, running everything through a Metalsmith plugin via Grunt. Plus some home brew CSS. :)

[sichvoge]

sure you can:

400:
  body:
    examples:
      user_not_match:
        description: This is my description
        value: |
          {"error": "user_not_match","error_description": "The user id in the access token doesn’t match the user in the session or user_id parameter" }

Have a look at the section that describes what you can define here.

[dawidv]

👍
This will really improve the ability for us to flesh out the mocking side of things.

[rojoca]

It would also be useful when different scenarios require different schemas:

{
    "type": "a",
    "common_property": "value",
    "only_valid_for_type_a_property": "ABC"
}

vs

{
    "type": "b",
    "common_property": "value",
    "only_valid_for_type_b_property": "123"
}

[dmartinezg]

Hi @rojoca this is certainly in our radar for the next version of RAML.

Thanks for the feedback

[freezy]

Same use case here. We have resources that return different objects depending on the type, and being able to provide examples for any type is a must.

Do you already have a concrete idea how to go about that? Would you need any help?

[usarid]

Are you saying you need a separate example for each different response
media type, or multiple examples for the same response media type? The
former is already provided for in the current RAML version.
On Jun 3, 2014 6:50 AM, "freezy" [email protected] wrote:

Same use case here. We have resources that return different objects
depending on the type, and being able to provide examples for any type is a
must.

Do you already have a concrete idea how to go about that? Would you need
any help?

—
Reply to this email directly or view it on GitHub
#37 (comment).

[freezy]

No, by "type" I meant arbitrary object types for the same media type. They would have semantically different values or even structures, independently on how they are serialized. Sorry if that wasn't clear.

[usarid]

Got it. Would there be any programmatic way to know which responses you
should expect, or is that something humans would need to figure out based
on reading the docs? One reason for asking that is deciding how the mocking
service should react to multiple examples: return the first one, randomly
pick one to return, or look for some semantics to correlate the request
with one of the possible responses.
On Jun 3, 2014 7:23 AM, "freezy" [email protected] wrote:

No, by "type" I meant arbitrary object types for the same media type. They
would have semantically different values or even structures, independently
on how they are serialized. Sorry if that wasn't clear.

—
Reply to this email directly or view it on GitHub
#37 (comment).

[freezy]

Good question. For us, being able to display different examples (maybe along with a title or even description) in the generated API doc would already be very helpful.

About the mock server, reading a "magic" parameter that indicates which example to return would probably be best (and just fall back to the first example if not provided).

[usarid]

That magic parameter is ideally the Accept header, with different media
types used to distinguish the different options ;-) but of course many/most
people and APIs use the same media type and do something else to
distinguish what responses to return. We're looking to come up with a
simple way that API authors can express that, and then services like
mocking, testing, etc. can latch onto it. Any ideas would be greatly
appreciated.
On Jun 3, 2014 7:40 AM, "freezy" [email protected] wrote:

Good question. For us, being able to display different examples (maybe
along with a title or even description) in the generated API doc would
already be very helpful.

About the mock server, reading a "magic" parameter that indicates which
example to return would probably be best (and just fall back to the first
example if not provided).

—
Reply to this email directly or view it on GitHub
#37 (comment).

[bergeroy]

In my case, the different JSON Schemas / examples would be determined by the value of a query parameter (ex: details = true or false)

[usarid]

You might imagine that each item in a responses collection (at some level,
or maybe at multiple levels) would contain a condition property, which
would need to get evaluated to determine whether this is the one to show.
It could have access to the request, so it could base its logic on the
request, and maybe a few more things like a probabilistic selector (in 10%
of the cases, do this…). But its language, the DSL expressing the
condition, is ideally something that doesn’t assume the server/mock is
running node or Ruby or anything else specific. Any suggestions for what
that could be?

On Tue, Jun 3, 2014 at 11:16 AM, bergeroy [email protected] wrote:

In my case, the different JSON Schemas / examples would be determined by
the value of a query parameter (example details = true or false)

—
Reply to this email directly or view it on GitHub
#37 (comment).

[freezy]

How about something like this:

body:
  application/json:
    examples:
      simple:
        body: !include simple.json
        header-match: 
          X-Scope: simple
      detailed:
        body: !include detailed.json
        query-match:
          scope: detailed

So we'd have two ways of identifying the right example for the mock: via header-match or query-match, where the mock would check a given header or query parameter respectively. Both would be optional attributes and if not specified, the first (or undetermined) example would be returned by the mock.

[fpoulin]

Having the ability to provide a few examples of request bodies (with a description) would be great for the API doc. As long as this is kept on the request side of things, there is not much impact on the mocks, right?

As for the response side ... just a quick brainstorming (hopefully not complete nonsense): it would be quite easy to express the "mapping" between a request and a specific response when there is a request body node. This is often the case with post request... Could be something like this:

post:
  body:
    application/json:
      examples:
        - simple:
            body: !include simple.json
            description: A standard example
            responseRef: simpleRes
        - tricky:
            body: !include trickyFeature.json
            description: !include trickyFeatureDescription.md
            responseRef: trickyRes
        - wrong:
            body: !include wrong.json
            description: Missing required field `cvv`
            responseRef: validationError
  responses:
    201:
      body:
        application/json:
          examples:
            - simpleRes: !include simple-res.json
            - trickyRes: !include tricky-res.json
    422:
      body:
        application/json:
          examples:
            - validationError: !include validationError.json

And for requests without body, you could have some kind of balancer (sorry for the name) as an alternative to the example(s) node, which would both define the collection of examples and how to choose amongst them. Something along those lines:

get:
  responses:
    200:
      body:
        application/json:
          balancer: 
            type: randomPick
            examples: 
              - !include res1-json
              - !include res2-json
              - !include res3-json

or

get:
  responses:
    200:
      body:
        application/json:
          balancer: 
            type: fancy-matcher
            examples: 
              - res1: 
                body: !include res1-json
                header-match: 
                  X-Scope: scope1
              - res2:
                body: !include res2-json
                query-match: 
                  scope: scope2

Balancers would become an extension point, just like the security schemes types ...

[jharmn]

In a scenario where you have a specific GET in a resource collection, specifying which mocked ID example you want is really useful. Clearly this is a subset of the solution, but good for mocking/testing with pre-defined data sets.
The above example of randomized results is not really useful in real world testing. Random data produces random test results, sort of the antithesis of predictability.
Specify the identifier in a resource collection expressively to indicate which sample to produce:

/{widget_id}
  get:
    responses:
      200:
        body:
          application/json:
            examples: 
              /2313: !include: examples/widget-2313.json
              /2314: !include: examples/widget-2314.json

This could get weird with APIs which utilize query parameters to specify items, but might work.

      examples: 
        ?widget_id=2313: !include: examples/widget-2313.json
        ?widget_id=2314: !include: examples/widget-2314.json

Stretching this idea further, parameterizing file names could make this much more terse, but probably complicate the tooling factor of file location verification a bit (i.e. parameterized file locations would have to be ignored/trusted):

      example: !include: examples/widget-{widget_id}.json

This would probably require another attribute specifying the default identifier, to allow for docs to produce a default sample response:

/{widget_id}
  default: 3213  #The level of where to include this is debateable...perhaps multiple is best
  get:
    responses:
      200:
        body:
          application/json:
            example: !include: examples/widget-{widget_id}.json

[smadigan-good]

I don't think the mocking service should look at the examples at all when generating a response. Responses should be generated by varying within what the schema permits. This may require a richer mock backend, but it makes more sense from a testing perspective. Also, providing examples of invalid requests/responses is often valuable. Keep it simple! Otherwise, if you must respond with an example, just do something deterministic such as returning them in order, or only the first one.

There is a real danger here in turning the spec (raml) into a bunch of mock instructions. That should be purely a function of a specialized mock-server facility which would read the spec but also take additional instructions in some other DSL to specify responses based on inputs.

[olensmar]

this would be perfect for the SoapUI plugin - as it can generate Mocks using all examples and allow the user to select which dispatch-strategy to use for selecting responses (sequential, random, etc)

[triplepoint]

I agree with @smadigan-good; it's asking too much for a specification document like RAML to contain any of the response-generation logic of the web service itself. I think it would remove a lot of the value that RAML offers in the first place - namely a human-readable, human-writable high-level definition document. RAML's value is in describing the artifacts of a service, and I think it should remain ignorant of the behavior.

It seems to me that the original multiple-examples ask is a good idea, but trying to define the logic that leads to each of those examples is going too far. That functionality is best left to an additional tool, specialized for building mock services and/or validation tests.

[abraae]

+1 for multiple examples. Don't know about the mocking service (we don't use it) but without the ability to have multiple examples (ideally each having also with a simple label clarifying under which conditions the example is relevant) we are forced to wedge our examples into the description field.

[usarid]

Ok, coming in RAML 1.0: https://github.com/raml-org/raml-org/spec/issues/107

[spfonseca]

Hi,

it's been some time since this issue was discussed. Is multiple example support part of RAML 1.0? Is the 1.0 definition activity active and when will it complete? When would the corresponding tooling be available? A single example is a pretty significant limitation IMO and would certainly curtail the ambitions I have at Intuit to use RAML.

For the mocking service, I think the easiest thing to do is cycle through the examples where request/response content types and URL are the same. This is what I have done in the past with SOAP services and it worked nicely. Note that this is behavior associated with the mocking service and not part of the semantics of RAML, per others' reservations.

[ivolazy]

+1

Ivan

On Mar 18, 2015, at 19:34, spfonseca [email protected] wrote:

Hi,

it's been some time since this issue was discussed. Is multiple example support part of RAML 1.0? Is the 1.0 definition activity active and when will it complete? When would the corresponding tooling be available? A single example is a pretty significant limitation IMO and would certainly curtail the ambitions I have at Intuit to use RAML.

For the mocking service, I think the easiest thing to do is cycle through the examples where request/response content types and URL are the same. This is what I have done in the past with SOAP services and it worked nicely. Note that this is behavior associated with the mocking service and not part of the semantics of RAML, per others' reservations.

—
Reply to this email directly or view it on GitHub.

[freezy]

I've ended up abusing the content type in order to provide different examples (content-type in our API is always application/json anyway). You can see an example of the RAML here and the output here (click on Response to see the different examples).

Since I'm only using RAML to generate doc, this works well.

[abraae]

Sorry to hijack the thread but your doc looks amazing! How do you generate it from RAML?

[yogirackspace]

Any update on this thread. RAML is awesome and much appreciate the same. However not having the ability to provide multiple responses seriously cripples the RAML. As of now I am managing multiple samples by varying the mime type. Though it works the outcome is not very elegant. Any update on this request would be appreciated.

[clearly]

+1

[usarid]

Yes, multiple examples are a prime candidate of RAML 1.0.

And @freezy: brilliant workaround!

On Fri, Aug 21, 2015 at 11:05 AM, Aaron Bawcom [email protected]
wrote:

+1

—
Reply to this email directly or view it on GitHub
#37 (comment).

[greg-hornby-roam]

Is there anyway to switch which example to use based on a uri parameter, such as this example from earlier in the thread

/{widget_id}:
  get:
    responses:
      200:
        body:
          application/json:
            example: !include: examples/widget-{widget_id}.json

[sichvoge]

That's not possible. This is very much runtime behaviour and would make validation very difficult.

[blackumbrella]

@usarid , is it possible to add comments or descriptions for each example?

my example likes:

400:
        body:
          examples:
            user_not_match: |
              {"error": "user_not_match","error_description": "The user id in the access token doesn’t match the user in the session or user_id parameter" }
            invalid_iam_session: |
              {  "error":"invalid_iam_session","error_description":"The  session is invalid

We would like to add more detailed explanation for each error example.