Does not compare against specified schema an apiary.apib file about dredd

[Almad]

Thanks for the report, that should fail and we'll take a look.

[mgaut72]

Here is a very simple sample that demonstrates the problem that I am having:

https://github.com/mgaut72/dredd_incorrect_schema_validation

[ecordell]

@mgaut72 Can definitely reproduce this locally. Validation is handled by gavel which in turn calls amanda for schema validations.

Just a note for investigating this issue:

• Type validation is handled here
• And in particular the isNumber method (which should be called)

Haven't yet installed gavel and amanda locally to figure out exactly where this issue is happening, but the test project definitely helps.

[mgaut72]

I just did a quick test, and it also turns out that if you set the schema to

...
                           "token": {
                                "type":"string",
                                "required":flase
                            }
...

then omit the token part of the response in api.py, the dredd test still fails, saying that "token is required".

I know nothing about your codebase, nor about coffeescript, but this line, taken completely out of context, seems of interest?

expected['schema'] = response['schema'] if response['schema']

perhaps response['schema'] is false-y, and gavel is taking to determining what the schema is based on parsing the expected response body?

[ecordell]

I get this output when running with -d

expected:
headers:
    Content-Type: application/json

body:
{
  "data": {
    "token": "234khsdfs9d8fsdf",
    "expires": 1424641830
  }
}
status: 200
schema:
{
  "type": "object",
  "required": true,
  "properties": {
    "data": {
      "type": "object",
      "required": true,
      "properties": {
        "expires": {
          "type": "number",
          "required": true
        },
        "token": {
          "type": "string",
          "required": true
        }
      }
    }
  }
}

Which suggests that the expected['schema'] is getting set (it would be blank otherwise). Just to be safe, I added:

expected['schema'] = response['schema'] if response['schema']
if response['schema']
  console.log 'Schema is truthy'

and got this output:

info: Beginning Dredd testing...
Schema is truthy

So it was a good thought, but it doesn't appear that's the issue...I also tried printing transaction.expected right before it gets passed to gavel.isValid, and it's definitely the right schema there. That leads me to believe it's something in gavel, since Amanda looks like it's handling numbers properly at first glance.

[mgaut72]

@ecordell should I move this over to the gavel.js issues, or how should this proceed?

[fdragu]

I have the same problem and by looking at both Dredd and Gavel it seems there is a mismatch between the domain model for the expected request/response in Gavel (in http-response-test.coffee and sampleHttpResponseSchema in fixtures.coffee) with what Dredd (in execute-transaction.coffee) sends to Gavel.
It seems like the gavel model has an 'expected' object as part of the expected http response and the schema should be set on the inner expected object vs the top level object.
Any chance to get someone to try to match them?

[zdne]

@ecordell @mgaut72

Looking into this and @fdragu is indeed right! There is a domain model mismatch between what Gavel.js expects and what is Dredd feeding in:

Gavel's input from Dredd (the expected value in this call:

gavel.isValid real, expected, 'response', (error, isValid) ->`

)

is:

{ headers: { 'Content-Type': 'application/json' },
  body: '{\n    "data": {\n        "token": "234khsdfs9d8fsdf",\n        "expires": 1424641830\n    }\n}\n',
  status: '200',
  schema: '{\n    "type":"object",\n    "required":true,\n    "properties":{\n        "data": {\n            "type":"object",\n            "required":true,\n            "properties":{\n                "expires": {\n                    "type":"number",\n                    "required":true\n                },\n                "token": {\n                    "type":"string",\n                    "required":true\n                }\n            }\n        }\n    }\n}\n' }

However Gavel.js expects (and takes) this:

{ statusCode: undefined,
  statusMessage: undefined,
  headers: { 'Content-Type': 'application/json' },
  body: '{\n    "data": {\n        "token": "234khsdfs9d8fsdf",\n        "expires": 1424641830\n    }\n}\n',
  headersSchema: undefined,
  bodySchema: undefined }

I will fix it.