dareid / chakram Goto Github PK

addFormat should be exposed from the chakram module to customize the schema validator. This should be implemented in the same way as the currently exposed addSchema.
See #37.

The JSON schema assertion can currently check the length of a JSON array. This capability is not obvious or easy. For such a common requirement, this should be simple.

Introduce a JSON array length assertion which is able to assert the length of an array at the root of the return body or an array within the return body. Should call the JSON schema assertion to avoid reimplementation (refer to the compression assertion for an example of exploiting an existing assertion).

Add a new cookie test, demonstrating use of cookies across multiple requests. Demonstrates the capability for new users and ensure it works correctly.

Im not sure how to use this 2 methods.
I have 2 .js file, one testing the user authenticated scenario and the other one testing a non authenticated user.

in the 1st, i set a default header in the request like this :

              chakram.setRequestDefaults({
                headers :  {
                  'Content-Type': 'application/json',
                  'Authorization': 'Bearer ' + token
                }});

Then in my second test file, i try to reset it to with

      chakram.clearRequestDefaults();

but when i check the header in the further request, the header is still set with my authorization token.

What did i do wrongly ?

@m1r4ge raised the problem of incorrect highlighting in #6. This should be explored and a suitable issue/pull request raised in the chai subset library

i have testdata.js file with json test data as below:

testdata.js file:

module.exports = {

createHost:{

    name : "System-001",
           description: "Creating System",

},

}

In another test, I am trying to create a variable called testcase1 with the above test data. Changing the property value in testcase1, changes the value in the testdata.js file json object.

it("Create a host", function(){
var testcase1 = testdata.createHost;

testcase1.name="sys-002";

console.log(testcase1.name);
console.log(testdata.createHost.name);
});

Response:
sys-002
sys-002

My requirement is to create a copy of testdata instead of creating a reference. How can I do this?

Is there a way to get the API response time while testing it ?
would be nice addon

    it("should list all avilable players", function () {
        var responseStructure = {
            type: "array",
            properties:{
                enabled : {type:"string"},  //This is expected to be an integer, but test passes
                defaultLanguage :{type:"integer"},
                defaultCurrency :{type:"integer"},
                lastname :{type:"string"},
                firstname:{type:"string"},
                passwordSalt:{type:"string"},
                password:{type:"string"},
                email:{type:"string"},
                id:{type:"string"} //Also this, expected to be an integer, but it passes the test
            }
        }
        var response = chakram.get(endpoint);
        return expect(response).to.have.schema(playerDataStructure);

I observed that the properties are not checked even when the properties expected doesn't match the types in the reponse, the test still passes.

Should use tv4 to detect all schema errors instead of just the first. See #37.

The following test always passes:

it('always passes', function() {
    expect({body: {a: 1, b:2}}).to.include.json({a:1});
    expect({body: {a: 1, b:2}}).to.not.include.json({a:1});
});

Currently to apply custom assertions need to call initialise, which implies that you should do this only once. Also adds confusion whether you need to initialise without custom assertions.

Instead offer users an interface similar to addAssertion and removeAssertion. Behind the scenes chai would still need to be reinitialised but this could be hidden from the user.

Chakram shouldn't use the delete keyword for the delete HTTP method because 'delete' is a reserved keyword in JS
or at least offer an alias (proposing: 'del')
example where it can create problems:

import {delete} from 'chakram`;

How do I set global defaults for the request module used by chakram?

I'm getting this error TypeError: Cannot read property 'statusCode' of undefined when I use "mocha -w" command.

// 1st run
  Minimal example
    ✓ should provide a simple async testing frameworkssss (1217ms)


  1 passing (1s)

// 2nd, 3rd and so on...

  1) Minimal example should provide a simple async testing frameworkssss:
     TypeError: Cannot read property 'statusCode' of undefined




    ✓ should provide a simple async testing frameworkssss (606ms)


  1 passing (608ms)

I'm not sure if this is related to chakram or its an issue w/ mocha

this is the test:

var chakram = require('chakram'),
    expect = chakram.expect;

describe("Minimal example", function() {    
    it("should provide a simple async testing framework", function () {
        var response = chakram.get("http://httpbin.org/get");
        expect(response).to.have.status(200);
        expect(response).not.to.have.header('non-existing-header');
        return chakram.wait();
    });
});

post('/login', {"user" : "user1", "password": "password"}) This will return the user 'userId' and this id will be inserted in to next get request.

get(/user/{userId}/requests)

How to pass userId in above get request, dyanmically?

I don't know how to obtain property value assigned to "importId" from body to print it with console.log or use it for other test case.

This is my response:
{
state: 'fulfilled',
value:
{
error: null,
response:
IncomingMessage {
-readableState: [Object],
readable: false,
domain: null,
-events: [Object],
-eventsCount: 4,
-maxListeners: undefined,
socket: [Object],
connection: [Object],
httpVersionMajor: 1,
httpVersionMinor: 1,
httpVersion: '1.1',
complete: true,
headers: [Object],
rawHeaders: [Object],
trailers: {},
rawTrailers: [],
upgrade: false,
url: '',
method: null,
statusCode: 200,
statusMessage: 'OK',
client: [Object],
-consuming: true,
-dumped: false,
req: [Object],
request: [Object],
toJSON: [Function: responseToJSON],
caseless: [Object],
read: [Function],
body: [Object] },
body: { importId: 4591, status: 'OK' },
jar: RequestJar { _jar: [Object] },
url: 'http://myUrl'
}
}

@dareid this is an amazing framework!

As suggested by @BigstickCarpet - thanks. Presentation @ https://youtu.be/bhrg-7f2e8k?t=1800

We are trying to perform api test using local intranet rest service url with chakram npm module but we are getting issues like below.

Error: timeout of 5000ms exceeded. Ensure the done() callback is being called in this test.

we don't have any clue to resolve the issue and there is no specific errors in the console. Could you guide us on this?

I wonder how could I test a code the uses callbacks instead of promises like below:

describe('with valid params', function() {
  it('should create a ticket', function () {

    // ... removed code

    return chakram
      .post(API_URL + '/tickets/', payload, options)
      .then(function(response) {

        // check that the ticket has been saved to the db
        Ticket.find({where: {id: 1}}, function() {
          expect(ticket.subject).to.equal(payload.subject);
        });
      });
  });
});

The test above will always pass even if it doesn't meet this expectation expect(ticket.subject).to.equal(payload.subject); .

I tried changing it to: expect(ticket.subject).to.equal('notTheExpectedSubject'); and it still passes. If using the done() callback is possible, I can just add it below the expectation. I tried it but for some reason the test becomes flaky.

Using promises works and its not flaky:

return Ticket.find({
  where: {
    id: 1
  }
}).then(function(ticket) {
  expect(ticket.subject).to.equal(payload.subject);
});

I'm curious how to make it work using callbacks though.

I want to send a email with status code, response format.
how do i get them? ex. : var = response.statusCode

Currently to add custom assertions you need to understand chai and it requires a fair bit of code just for a small assertion.
Can we make this easier?

As an idea, maybe only request an assertion name and a call back which is called with the response object? Would limit the amount of exposure to chai.

Would be nice to see all tests of a spec and run specific tests if possible via a UI.
Instead of running the complete test/spec js users can select a test & run it via UI - think this would be very helpful.

I added a Typescript definition file request over on the DefinitelyTyped repo:

DefinitelyTyped/DefinitelyTyped#7975

It would be great if we could use Chakram.js in our current project, but Typescript won't be happy without a definition. If you have the bandwidth, please consider adding one.

expect(response).to.have.json("result", null);
always results in
Error: could not find path 'result' in object {"result": null}

I found that the path value with null is taken as an error.

lib/utils/objectPath.js

if(subObject === undefined || subObject === null) {
            throw new Error("could not find path '"+path+"' in object "+JSON.stringify(obj)); 
        }

I'm testing an API which accepts XML data on POST method.
Is it possible to post XML data by chakram?

I have provided the schema as follows:

var expectSchema = {
type: "object",
properties: {
Host : {type: 'string'},
origin : {type: 'string'},
link : {type: 'string'},
},
required: [
'Host',
'link',
'origin'
]
};

I get the below error:

1. Chakram Validate the returned JSON schema:
   AssertionError: expected body to match JSON schema {
   "type": "object",
   "properties": {
   "Host": {
   "type": "string"
   },
   "origin": {
   "type": "string"
   },
   "link": {
   "type": "string"
   }
   },
   "required": [
   "Host",
   "link",
   "origin"
   ]

.

error: Invalid type: undefined (expected object).
data path: .
schema path: /type.

........................pm ERR! Test failed. See above for more details.

Any thought about the error?

is there any way to set proxy in chakram module?

Trying to do JSON schema validation and it always returns true.

response contains the actual JSON response.
expectedSchema contains the expected schema structure

Below is the assertion code:
expect(response.body).to.have.schema(expectedSchema);

Executed the above code by providing the improper schema structure but still the assertion gets passed.

When I try to run the example test from the README.md using Chakram v0.0.3, I get an error on the valid email address test.

It appears there is an issue with a function being passed as the test condition for json().

how the hell i have to do a post request with link http://www.worthpoint.com/iosService/similarItems and parameter itemId = 14321 ?

I tried passing params like this to chakram.get() but it did not work:

var params = {
    agentOptions : {
        key: fs.readFileSync('key.pem'), 
        cert: fs.readFileSync('cert.pem')        
    }
}

I'm trying to use chakram to develop a small API. I found it useful that the tests are run with mocha so everything I did before in unit tests, can still be used with chakram. However, when running mocha with the --watch flag, the tests start failing after a few re-runs, for no apparent reason. If I then kill mocha and start it again, it works but only until 2-3 re-runs. After that, the tests fails.

My simple test

var chakram = require('chakram')
var expect = chakram.expect

function basicExpections(response) {
  expect(response).to.have.status(200)
  expect(response).to.have.header('Content-Type', 'application/json; charset=utf-8')
  expect(response).not.to.be.encoded.with.gzip
}

describe('Nodes', () => {
    it('Should be able to see empty list of nodes', () => {
      var response = chakram.get('http://localhost:3000/nodes')
      basicExpections(response)
      expect(response).to.comprise.of.json([])
      return chakram.wait()
    });
});

The error I receive after a few re-runs:

  0 passing (9ms)
  1 failing

  1) Nodes Should be able to see empty list of nodes:
     TypeError: Cannot read property 'statusCode' of undefined
      at null.<anonymous> (node_modules/chakram/lib/assertions/statuscode.js:15:44)
      at ctx.(anonymous function) (node_modules/chai/lib/chai/utils/addMethod.js:40:25)
      at node_modules/chai-as-promised/lib/chai-as-promised.js:304:26
      at _fulfilled (node_modules/chakram/node_modules/q/q.js:834:54)
      at self.promiseDispatch.done (node_modules/chakram/node_modules/q/q.js:863:30)
      at Promise.promise.promiseDispatch (node_modules/chakram/node_modules/q/q.js:796:13)
      at node_modules/chakram/node_modules/q/q.js:604:44
      at runSingle (node_modules/chakram/node_modules/q/q.js:137:13)
      at flush (node_modules/chakram/node_modules/q/q.js:125:13)

A new assertion should be added which allows the existance of an object within an array to be checked. This should work for arrays at the root of a response or within the response.

I was refactoring my JSONSchema and messed up the internal $ref's (like #/definitions/foo), but the tests still pass.

Turns out this is behaviour from TV4 that returns true for validation even if the schema cannot resolve some of it's references. The TV4 docs note:

If schemas are referenced ($ref) but not known, then validation will return true and the missing schema(s) will be listed in tv4.missing.

This seems like an odd choice, since it makes the test brittle and prone to break silently (unless you test your tests).

Maybe chakram should also check for this missing attribute?

Trying to use the example in the documentation doesn't work:

chakram.addSchemaFormat('decimal-digits', function (data, schema) {
    if (typeof data === 'string' && !/^[0-9]+$/.test(data)) {
        return null;
    }
    return "must be string of decimal digits";
});

// ...

it('should have the correct schema', function() {
    const expectedSchema = {
        type: 'object',
        properties: {
            foo: { type: 'decimal-digits' }
        }
    };

    return expect(response).to.have.schema(expectedSchema);
});

Results in a test failure:

 error: Invalid type: string (expected decimal-digits).

Can someone help out, i'm trying to add basic auth to a request

var chakram = require('chakram');
    expect = chakram.expect;

var requestHeader = {
      "Authorization": "Basic xxxx",
    };

describe("Chakram", function() {
    it("should provide HTTP specific assertions", function () {
        var response = chakram.get("<host>", requestHeader);
        return expect(response).to.have.status(200);
    });
});

Thanks

reference error when chakram called outside of mocha test.

Steps to reproduce:

1. Create test.js file with the next content
   var chakram = require('chakram');
return chakram.get("http://httpbin.org/get").then((response) => console.log(response.body));
2. execute node command
   node test.js

c:\dev\client\node_modules\chakram\lib\chakram.js:136
afterEach(function() {
^

ReferenceError: afterEach is not defined
at Object. (c:\dev\client\node_modules\chakram\lib\chakram.js:136:1)
at Module._compile (module.js:399:26)
at Object.Module._extensions..js (module.js:406:10)
at Module.load (module.js:345:32)
at Function.Module._load (module.js:302:12)
at Module.require (module.js:355:17)
at require (internal/module.js:13:17)
at Object. (c:\temp\dev\chkram\client.js:1:15)
at Module._compile (module.js:399:26)
at Object.Module._extensions..js (module.js:406:10)

Shakram supports validating JSON schema by doing something like this:

"expect(getRequest).to.have.schema(jsonData, jsonSchema);"

This works fine. But how do I add other JSON schemas besides the 'jsonSchema'? Let's say my 'jsonData' holds a reference to another jsonSchema, according to the tv4(underlying JSON schema validator) documentation this must be done before a call to the validation function. When using tv4 with Chai one can do the following to add a JSON schema to the validator before validating:

chai.tv4.addSchema('url',schema);

But how can I do this when using Chakram? I think it is a matter of getting the chai or tv4 instance extended by Chakram. Then i could do as above. I can't figure out how to get it. This should be documented somewhere.

The chakram instance has a chai property, but is has a null value. This seems odd to me.

I tried following the requestjs way of doing it, but it didn't work:

chakram.request('POST', '/path/to/my/url', {formData: {'file': fs.createReadStream('/path/to/file')}});

Cant seem to figure it out using the docs either. Could you let me know how I can do this?

Thanks!

Using Mocha 2.2.4, chai 2.3.0, chai-as-promised 5.0.0 - following the examples on the documentation, a failed test shows:

    ✓ should have a 200 response status
    1) should have a 200 response status


  1 passing (65ms)
  1 failing

  1) API Tests Create should have a 200 response status:
  Error: Some expectation promises were not fulfilled before the test finished. Ensure you are waiting for all the expectations to run

Running the pure demo code even fails:

it('should have a 200 response status', function() {
    var google = chakram.get("http://google.com");
    expect(google).to.have.status(200);
});

If the test is successful, everything works as expected. An error causes the test to show twice in the list, show as both failed and passed.

If we wrap the promise ourselves, it works in both cases.

it('should have a 200 response status', function(done) {
    response.then(function(result) {
        try {
            expect(result).to.have.status(200);
            done();
        } catch (e) {
            done(e);
        }
    });
});

Please provide more information about error given by tv4 lib. Current implementation gives only general information about the error. Please concat some major information like dataPath, schemaPath. This would be very useful.

this.assert(
    valid,
    'expected body to match JSON schema '+JSON.stringify(schema)+'. error: ' + tv4.error.message + ' ' + tv4.error.dataPath + ' ' + tv4.error.schemaPath,
    'expected body to not match JSON schema ' + JSON.stringify(schema)
);

This is related to #32

I'm unclear what the resolution here is for schemas with $ref in. I can see how this issue clears up the silent fail of the validation where the $ref isn't included, but can't see how to actually include the referenced parts properly.

An example, I am validating an errors response from a jsonapi-compatible endpoint. The spec is:

{
 type: 'object',
 properties: { 
    errors: 
    { 
      type: 'array', 
      items: { '$ref': '#/definitions/Error' } 
    } 
  } 
}

So that is what I am passing to .schema():

 expect(respObj).to.have.schema(swagger.definitions.Errors);  // whole json file is in var swagger, only passing Errors definition which contains $ref

Obviously this now fails, which is correct, because we are giving an incomplete schema to chakram:

     AssertionError: expected body to match JSON schema {
  "type": "object",
  "properties": {
    "errors": {
      "type": "array",
      "items": {
        "$ref": "#/definitions/Error"
      }
    }
  }
}.
 unresolved schemas: [""].

OK, so problem is how do we also pass the referenced elements. So let's try:

 expect(respObj).to.have.schema(swagger.definitions);  // add all the definitions

Passes. But I know this is erroneous, because it continues to pass even when I change the underlying definition to refer to an unknown definition, e.g. items: { '$ref': '#/definitions/E2rror' }. Would have expected a fail here.

Let's try:

 expect(respObj).to.have.schema([swagger.definitions.Errors,swagger.definitions.Error]);  // add all the definitions

Passes in all circumstances, again even with duff references

  chakram.addSchema(swagger.definitions); // add all definitions
  expect(respObj).to.have.schema(swagger.definitions.Errors) // validate against Errors

Fails. I thought add schema would help.

Any ideas how to correctly pass in the referenced schema elements?

Link to documentation from Getting Started section is broken.

Please provide a method to check whether API response time is within the expected time limit

Example call could be like -
expect(response).to.have.responsetime(500);

where 500 would be the expected time in milliseconds in which the response should be received.

Based on #6, we should warn the user if we detect the use of a chained method in any of the recorded expectations.

The feature would allow developers to import their server lib and run tests locally without having to fire up a "live" http server. For context, one can look at the feature as it is supported by supertest.

var request = require('supertest');
var server = require('../server/server.js');

describe('Unfiltered item search', function() {
    it('should return all configured items', function (done) {
        request(server)
            .get('/api/qc/items')
            .set('Accept', 'application/json')
            .expect(200)
            .expect(function(res){
                if(!res.body[0].itemNumber) {
                    console.log('it was true');
                    throw new Error('missing itemNumber');
                }
            })
            .end(done);
    });
});

For debugging would be useful to log the request and response received. Should support per request logging and default logging for all requests.

Does Chakram support for expressions in json response validation.

My requirement is the response object value can be provided as expressions.

For example data property value changes from creation time to time. Can I provide the format in the json to validate it.

Next the user can have any of the multiple values for example the created_by can be admin, superadmin or user..