postmanlabs / newman

Newman is a command-line collection runner for Postman

Manage all of your organization's APIs in Postman, with the industry's most complete API development environment.

newman _{the cli companion for postman}

Newman is a command-line collection runner for Postman. It allows you to effortlessly run and test a Postman collection directly from the command-line. It is built with extensibility in mind so that you can easily integrate it with your continuous integration servers and build systems.

Table of contents

1. Getting Started
2. Usage
   1. Using Newman CLI
   2. Using Newman as a Library
   3. Using Reporters with Newman
3. Command Line Options
   1. newman-options
   2. newman-run
   3. SSL
   4. Configuring Proxy
4. API Reference
   1. newman run
   2. Run summary object
   3. Events emitted during a collection run
5. Reporters
   1. Configuring Reporters
   2. CLI Reporter
   3. JSON Reporter
   4. JUnit Reporter
   5. HTML Reporter
6. External Reporters
   1. Using External Reporters
   2. Creating Your Own Reporter
7. File Uploads
8. Using Newman with the Postman API
9. Using Newman in Docker
10. Using Socks Proxy
11. Migration Guide
12. Compatibility
13. Contributing
14. Community Support
15. License

Getting started

To run Newman, ensure that you have Node.js >= v16. Install Node.js via package manager.

Installation

The easiest way to install Newman is using NPM. If you have Node.js installed, it is most likely that you have NPM installed as well.

$ npm install -g newman

This installs Newman globally on your system allowing you to run it from anywhere. If you want to install it locally, Just remove the -g flag.

Using Homebrew

Install Newman globally on your system using Homebrew.

$ brew install newman

back to top

Usage

Using Newman CLI

The newman run command allows you to specify a collection to be run. You can easily export your Postman Collection as a json file from the Postman App and run it using Newman.

$ newman run examples/sample-collection.json

If your collection file is available as an URL (such as from our Cloud API service), Newman can fetch your file and run it as well.

$ newman run https://www.getpostman.com/collections/631643-f695cab7-6878-eb55-7943-ad88e1ccfd65-JsLv

For the complete list of options, refer the Command Line Options section below.

Using Newman as a Library

Newman can be easily used within your JavaScript projects as a Node.js module. The entire set of Newman CLI functionality is available for programmatic use as well. The following example runs a collection by reading a JSON collection file stored on disk.

const newman = require('newman'); // require newman in your project

// call newman.run to pass `options` object and wait for callback
newman.run({
    collection: require('./sample-collection.json'),
    reporters: 'cli'
}, function (err) {
	if (err) { throw err; }
    console.log('collection run complete!');
});

For the complete list of options, refer the API Reference section below.

Using Reporters with Newman

Reporters provide information about the current collection run in a format that is easy to both: disseminate and assimilate. Reporters can be configured using the -r or --reporters options. Inbuilt reporters in newman are: cli, json, junit, progress and emojitrain.

CLI reporter is enabled by default when Newman is used as a CLI, you do not need to specifically provide the same as part of reporters option. However, enabling one or more of the other reporters will result in no CLI output. Explicitly enable the CLI option in such a scenario. Check the example given below using the CLI and JSON reporters:

$ newman run examples/sample-collection.json -r cli,json

For more details on Reporters and writing your own External Reporters refer to their corresponding sections below.

back to top

Command Line Options

newman [options]

• -h, --help
  Show command line help, including a list of options, and sample use cases.

• -v, --version
  Displays the current Newman version, taken from package.json

newman run <collection-file-source> [options]

• -e <source>, --environment <source>
  Specify an environment file path or URL. Environments provide a set of variables that one can use within collections. Read More

• -g <source>, --globals <source>
  Specify the file path or URL for global variables. Global variables are similar to environment variables but have a lower precedence and can be overridden by environment variables having the same name.

• -d <source>, --iteration-data <source>
  Specify a data source file (JSON or CSV) to be used for iteration as a path to a file or as a URL. Read More

• -n <number>, --iteration-count <number>
  Specifies the number of times the collection has to be run when used in conjunction with iteration data file.

• --folder <name>
  Run requests within a particular folder/folders or specific requests in a collection. Multiple folders or requests can be specified by using --folder multiple times, like so: --folder f1 --folder f2 --folder r1 --folder r2.

• --working-dir <path>
  Set the path of the working directory to use while reading files with relative paths. Default to current directory.

• --no-insecure-file-read
  Prevents reading of files situated outside of the working directory.

• --export-environment <path>
  The path to the file where Newman will output the final environment variables file before completing a run.

• --export-globals <path>
  The path to the file where Newman will output the final global variables file before completing a run.

• --export-collection <path>
  The path to the file where Newman will output the final collection file before completing a run.

• --timeout <ms>
  Specify the time (in milliseconds) to wait for the entire collection run to complete execution.

• --timeout-request <ms>
  Specify the time (in milliseconds) to wait for requests to return a response.

• --timeout-script <ms>
  Specify the time (in milliseconds) to wait for scripts to complete execution.

• -k, --insecure
  Disables SSL verification checks and allows self-signed SSL certificates.

• --ignore-redirects
  Prevents newman from automatically following 3XX redirect responses.

• --delay-request
  Specify the extent of delay between requests (milliseconds).

• --cookie-jar <path>
  Specify the file path for a JSON Cookie Jar. Uses tough-cookie to deserialize the file.

• --export-cookie-jar <path>
  The path to the file where Newman will output the final cookie jar file before completing a run. Uses tough-cookie's serialize method.

• --bail [optional modifiers]
  Specify whether or not to stop a collection run on encountering the first test script error.
  Can optionally accept modifiers, currently include folder and failure.
  folder allows you to skip the entire collection run in case an invalid folder was specified using the --folder option or an error was encountered in general.
  On the failure of a test, failure would gracefully stop a collection run after completing the current test script.

• -x, --suppress-exit-code
  Specify whether or not to override the default exit code for the current run.

• --color <value>
  Enable or Disable colored CLI output. The color value can be any of the three: on, off or auto(default).
  With auto, Newman attempts to automatically turn color on or off based on the color support in the terminal. This behaviour can be modified by using the on or off value accordingly.

• --disable-unicode
  Specify whether or not to force the unicode disable option. When supplied, all symbols in the output will be replaced by their plain text equivalents.

• --global-var "<global-variable-name>=<global-variable-value>"
  Allows the specification of global variables via the command line, in a key=value format. Multiple CLI global variables can be added by using --global-var multiple times, like so: --global-var "foo=bar" --global-var "alpha=beta".

• --env-var "<environment-variable-name>=<environment-variable-value>"
  Allows the specification of environment variables via the command line, in a key=value format. Multiple CLI environment variables can be added by using --env-var multiple times, like so: --env-var "foo=bar" --env-var "alpha=beta".

• --verbose
  Show detailed information of collection run and each request sent.

SSL

Client Certificates

Client certificates are an alternative to traditional authentication mechanisms. These allow their users to make authenticated requests to a server, using a public certificate, and an optional private key that verifies certificate ownership. In some cases, the private key may also be protected by a secret passphrase, providing an additional layer of authentication security.

Newman supports SSL client certificates, via the following CLI options:

Using a single SSL client certificate

• --ssl-client-cert
  The path to the public client certificate file.

• --ssl-client-key
  The path to the private client key (optional).

• --ssl-client-passphrase
  The secret passphrase used to protect the private client key (optional).

Using SSL client certificates configuration file (supports multiple certificates per run)

• --ssl-client-cert-list
  The path to the SSL client certificate list configuration file (JSON format). See examples/ssl-client-cert-list.json.

This option allows setting different SSL client certificate according to URL or hostname. This option takes precedence over --ssl-client-cert, --ssl-client-key and --ssl-client-passphrase options. If there is no match for the URL in the list, these options are used as fallback.

Trusted CA

When it is not wanted to use the --insecure option, additionally trusted CA certificates can be provided like this:

• --ssl-extra-ca-certs
  The path to the file, that holds one or more trusted CA certificates in PEM format

Configuring Proxy

Newman can also be configured to work with proxy settings via the following environment variables:

• HTTP_PROXY / http_proxy
• HTTPS_PROXY / https_proxy
• NO_PROXY / no_proxy

For more details on using these variables, refer here.

back to top

API Reference

newman.run(options: object , callback: function) => run: EventEmitter

The run function executes a collection and returns the run result to a callback function provided as parameter. The return of the newman.run function is a run instance, which emits run events that can be listened to.

Parameter	Description
options	This is a required argument and it contains all information pertaining to running a collection. Required Type: object
options.collection	The collection is a required property of the options argument. It accepts an object representation of a Postman Collection which should resemble the schema mentioned at https://schema.getpostman.com/. The value of this property could also be an instance of Collection Object from the Postman Collection SDK. As string, one can provide a URL where the Collection JSON can be found (e.g. Postman Cloud API service) or path to a local JSON file. Required Type: object|string PostmanCollection
options.environment	One can optionally pass an environment file path or URL as string to this property and that will be used to read Postman Environment Variables from. This property also accepts environment variables as an object. Environment files exported from Postman App can be directly used here. Optional Type: object|string
options.envVar	One can optionally pass environment variables as an array of key-value string object pairs. It will be used to read Postman Environment Variables as well as overwrite environment variables from options.environments. Optional Type: array|object
options.globals	Postman Global Variables can be optionally passed on to a collection run in form of path to a file or URL. It also accepts variables as an object. Optional Type: object|string
options.globalVar	One can optionally pass global environment variables as an array of key-value string object pairs. It will be used to read Postman Global Environment Variables as well as overwrite global environment variables from options.globals. Optional Type: array|object
options.iterationCount	Specify the number of iterations to run on the collection. This is usually accompanied by providing a data file reference as options.iterationData. Optional Type: number, Default value: 1
options.iterationData	Path to the JSON or CSV file or URL to be used as data source when running multiple iterations on a collection. Optional Type: string
options.folder	The name or ID of the folder/folders (ItemGroup) in the collection which would be run instead of the entire collection. Optional Type: string|array
options.workingDir	The path of the directory to be used as working directory. Optional Type: string, Default value: Current Directory
options.insecureFileRead	Allow reading files outside of working directory. Optional Type: boolean, Default value: true
options.timeout	Specify the time (in milliseconds) to wait for the entire collection run to complete execution. Optional Type: number, Default value: Infinity
options.timeoutRequest	Specify the time (in milliseconds) to wait for requests to return a response. Optional Type: number, Default value: Infinity
options.timeoutScript	Specify the time (in milliseconds) to wait for scripts to return a response. Optional Type: number, Default value: Infinity
options.delayRequest	Specify the time (in milliseconds) to wait for between subsequent requests. Optional Type: number, Default value: 0
options.ignoreRedirects	This specifies whether newman would automatically follow 3xx responses from servers. Optional Type: boolean, Default value: false
options.insecure	Disables SSL verification checks and allows self-signed SSL certificates. Optional Type: boolean, Default value: false
options.bail	A switch to specify whether or not to gracefully stop a collection run (after completing the current test script) on encountering the first error. Takes additional modifiers as arguments to specify whether to end the run with an error for invalid name or path. Available modifiers: folder and failure. eg. bail : ['folder'] Optional Type: boolean|object, Default value: false
options.suppressExitCode	If present, allows overriding the default exit code from the current collection run, useful for bypassing collection result failures. Takes no arguments. Optional Type: boolean, Default value: false
options.reporters	Specify one reporter name as string or provide more than one reporter name as an array. Available reporters: cli, json, junit, progress and emojitrain. Optional Type: string|array
options.reporter	Specify options for the reporter(s) declared in options.reporters. e.g. reporter : { junit : { export : './xmlResults.xml' } } e.g. reporter : { html : { export : './htmlResults.html', template: './customTemplate.hbs' } } Optional Type: object
options.color	Enable or Disable colored CLI output. Available options: on, off and auto Optional Type: string, Default value: auto
options.sslClientCert	The path to the public client certificate file. Optional Type: string
options.sslClientKey	The path to the private client key file. Optional Type: string
options.sslClientPassphrase	The secret client key passphrase. Optional Type: string
options.sslClientCertList	The path to the client certificate configuration list file. This option takes precedence over sslClientCert, sslClientKey and sslClientPassphrase. When there is no match in this configuration list, sslClientCert is used as fallback. Optional Type: string|array
options.sslExtraCaCerts	The path to the file, that holds one or more trusted CA certificates in PEM format. Optional Type: string
options.requestAgents	Specify the custom requesting agents to be used when performing HTTP and HTTPS requests respectively. Example: Using Socks Proxy Optional Type: object
options.cookieJar	One can optionally pass a CookieJar file path as string to this property and that will be deserialized using tough-cookie. This property also accepts a tough-cookie CookieJar instance. Optional Type: object|string
options.newmanVersion	The Newman version used for the collection run. This will be set by Newman
callback	Upon completion of the run, this callback is executed with the error, summary argument. Required Type: function

newman.run~callback(error: object , summary: object)

The callback parameter of the newman.run function receives two arguments: (1) error and (2) summary

Argument	Description
error	In case newman faces an error during the run, the error is passed on to this argument of callback. By default, only fatal errors, such as the ones caused by any fault inside Newman is passed on to this argument. However, setting abortOnError:true or abortOnFailure:true as part of run options will cause newman to treat collection script syntax errors and test failures as fatal errors and be passed down here while stopping the run abruptly at that point. Type: object
summary	The run summary will contain information pertaining to the run. Type: object
summary.error	An error object which if exists, contains an error message describing the message Type: object
summary.collection	This object contains information about the collection being run, it's requests, and their associated pre-request scripts and tests. Type: object
summary.environment	An object with environment variables used for the current run, and the usage status for each of those variables. Type: object
summary.globals	This object holds details about the globals used within the collection run namespace. Type: object
summary.run	A cumulative run summary object that provides information on . Type: object
summary.run.stats	An object which provides details about the total, failed, and pending counts for pre request scripts, tests, assertions, requests, and more. Type: object
summary.run.failures	An array of failure objects, with each element holding details, including the assertion that failed, and the request. Type: array.<object>
summary.run.executions	This object contains information about each request, along with it's associated activities within the scope of the current collection run. Type: array.<object>

newman.run~events

Newman triggers a whole bunch of events during the run.

newman.run({
    collection: require('./sample-collection.json'),
    iterationData: [{ "var": "data", "var_beta": "other_val" }],
    globals: {
        "id": "5bfde907-2a1e-8c5a-2246-4aff74b74236",
        "name": "test-env",
        "values": [
            {
                "key": "alpha",
                "value": "beta",
                "type": "text",
                "enabled": true
            }
        ],
        "timestamp": 1404119927461,
        "_postman_variable_scope": "globals",
        "_postman_exported_at": "2016-10-17T14:31:26.200Z",
        "_postman_exported_using": "Postman/4.8.0"
    },
    globalVar: [
        { "key":"glboalSecret", "value":"globalSecretValue" },
        { "key":"globalAnotherSecret", "value":`${process.env.GLOBAL_ANOTHER_SECRET}`}
    ],
    environment: {
        "id": "4454509f-00c3-fd32-d56c-ac1537f31415",
        "name": "test-env",
        "values": [
            {
                "key": "foo",
                "value": "bar",
                "type": "text",
                "enabled": true
            }
        ],
        "timestamp": 1404119927461,
        "_postman_variable_scope": "environment",
        "_postman_exported_at": "2016-10-17T14:26:34.940Z",
        "_postman_exported_using": "Postman/4.8.0"
    },
    envVar: [
        { "key":"secret", "value":"secretValue" },
        { "key":"anotherSecret", "value":`${process.env.ANOTHER_SECRET}`}
    ],
}).on('start', function (err, args) { // on start of run, log to console
    console.log('running a collection...');
}).on('done', function (err, summary) {
    if (err || summary.error) {
        console.error('collection run encountered an error.');
    }
    else {
        console.log('collection run completed.');
    }
});

All events receive two arguments (1) error and (2) args. The list below describes the properties of the second argument object. Learn more

Event	Description
start	The start of a collection run
beforeIteration	Before an iteration commences
beforeItem	Before an item execution begins (the set of prerequest->request->test)
beforePrerequest	Before prerequest script is execution starts
prerequest	After prerequest script execution completes
beforeRequest	Before an HTTP request is sent
request	After response of the request is received
beforeTest	Before test script is execution starts
test	After test script execution completes
beforeScript	Before any script (of type test or prerequest) is executed
script	After any script (of type test or prerequest) is executed
item	When an item (the whole set of prerequest->request->test) completes
iteration	After an iteration completes
assertion	This event is triggered for every test assertion done within test scripts
console	Every time a console function is called from within any script, this event is propagated
exception	When any asynchronous error happen in scripts this event is triggered
beforeDone	An event that is triggered prior to the completion of the run
done	This event is emitted when a collection run has completed, with or without errors

back to top

Reporters

Configuring Reporters

• -r <reporter-name>, --reporters <reporter-name>
  Specify one reporter name as string or provide more than one reporter name as a comma separated list of reporter names. Available reporters are: cli, json, junit, progress and emojitrain.
  
  Spaces should not be used between reporter names / commas whilst specifying a comma separated list of reporters. For instance:
  
  ✅ -r cli,json,junit
  ❌ -r cli , json,junit

• --reporter-{{reporter-name}}-{{reporter-option}}
  When multiple reporters are provided, if one needs to specifically override or provide an option to one reporter, this is achieved by prefixing the option with --reporter-{{reporter-name}}-.
  
  For example, ... --reporters cli,json --reporter-cli-silent would silence the CLI reporter only.

• --reporter-{{reporter-options}}
  If more than one reporter accepts the same option name, they can be provided using the common reporter option syntax.
  
  For example, ... --reporters cli,json --reporter-silent passes the silent: true option to both JSON and CLI reporter.

Note: Sample collection reports have been provided in examples/reports.

CLI Reporter

The built-in CLI reporter supports the following options, use them with appropriate argument switch prefix. For example, the option no-summary can be passed as --reporter-no-summary or --reporter-cli-no-summary.

CLI reporter is enabled by default when Newman is used as a CLI, you do not need to specifically provide the same as part of --reporters option. However, enabling one or more of the other reporters will result in no CLI output. Explicitly enable the CLI option in such a scenario.

CLI Option	Description
--reporter-cli-silent	The CLI reporter is internally disabled and you see no output to terminal.

| --reporter-cli-show-timestamps | This prints the local time for each request made. | | --reporter-cli-no-summary | The statistical summary table is not shown. | | --reporter-cli-no-failures | This prevents the run failures from being separately printed. | | --reporter-cli-no-assertions | This turns off the output for request-wise assertions as they happen. | | --reporter-cli-no-success-assertions | This turns off the output for successful assertions as they happen. | | --reporter-cli-no-console | This turns off the output of console.log (and other console calls) from collection's scripts. | | --reporter-cli-no-banner | This turns off the newman banner shown at the beginning of each collection run. |

JSON Reporter

The built-in JSON reporter is useful in producing a comprehensive output of the run summary. It takes the path to the file where to write the report. The content of this file is exactly the same as the summary parameter sent to the callback when Newman is used as a library.

To enable JSON reporter, provide --reporters json as a CLI option.

CLI Option	Description
--reporter-json-export <path>	Specify a path where the output JSON file will be written to disk. If not specified, the file will be written to newman/ in the current working directory. If the specified path does not exist, it will be created. However, if the specified path is a pre-existing directory, the report will be generated in that directory.

JUNIT/XML Reporter

The built-in JUnit reporter can output a summary of the collection run to a JUnit compatible XML file. To enable the JUNIT reporter, provide --reporters junit as a CLI option.

CLI Option	Description
--reporter-junit-export <path>	Specify a path where the output XML file will be written to disk. If not specified, the file will be written to newman/ in the current working directory. If the specified path does not exist, it will be created. However, if the specified path is a pre-existing directory, the report will be generated in that directory.

HTML Reporter

An external reporter, maintained by Postman, which can be installed via npm install -g newman-reporter-html. This reporter was part of the Newman project but was separated out into its own project in V4.

The complete installation and usage guide is available at newman-reporter-html. Once the HTML reporter is installed you can provide --reporters html as a CLI option.

back to top

External Reporters

Using External Reporters

Newman also supports external reporters, provided that the reporter works with Newman's event sequence. Working examples of how Newman reporters work can be found in lib/reporters.

For instance, to use the Newman HTML Reporter:

• Install the reporter package. Note that the name of the package is of the form newman-reporter-<name>. The installation should be global if Newman is installed globally, local otherwise. (Remove -g flag from the command below for a local installation.)

$ npm install -g newman-reporter-html

• Use the installed reporter, either via the CLI, or programmatic usage. Here, the newman-reporter prefix is not required while specifying the reporter name in the options.
  

$ newman run /path/to/collection.json -r cli,html

const newman = require('newman');

newman.run({
    collection: '/path/to/collection.json',
    reporters: ['cli', 'html']
}, process.exit);

Community Maintained Reporters

Several members of the Postman community have created custom reporters offering different option to output the data coming from Newman. Listed below is a selection of these but more can be found here on NPM.

Once the custom reporter NPM package has been installed either globally or locally, this can be then used with Newman in the following ways:

$ newman run /path/to/collection.json -r htmlextra,csv

const newman = require('newman');

newman.run({
    collection: '/path/to/collection.json',
    reporters: ['htmlextra', 'csv']
}, process.exit);

• allure - This reporter allow to create fully-featured allure reports that can allow you to have easy to understand HTML reports with features like historical data, link tests to the JIRA and all other benefits of using allure framework.
• htmlextra - This is an updated version of the standard HTML reporter containing a more in-depth data output and a few helpful extras
• csv - This reporter creates a csv file containing the high level summary of the Collection run
• json-summary - A Newman JSON Reporter that strips the results down to a minimum
• teamcity - A reporter built to be used with the Team City CI server
• testrail - A reporter built for Test Rail, the test case management tool
• statsd - This reporter can be used to send the Collection run data to statsd and used on time series analytic tools like Grafana
• confluence - Confluence reporter for Newman that uploads a Newman report on a Confluence page
• influxdb - This reporter sends the test results information to InfluxDB which can be used from Grafana to build dashboards

Creating Your Own Reporter

A custom reporter is a Node module with a name of the form newman-reporter-<name>. To create a custom reporter:

1. Navigate to a directory of your choice, and create a blank npm package with npm init.
2. Add an index.js file, that exports a function of the following form:

function CustomNewmanReporter (emitter, reporterOptions, collectionRunOptions) {
  // emitter is an event emitter that triggers the following events: https://github.com/postmanlabs/newman#newmanrunevents
  // reporterOptions is an object of the reporter specific options. See usage examples below for more details.
  // collectionRunOptions is an object of all the collection run options: https://github.com/postmanlabs/newman#newmanrunoptions-object--callback-function--run-eventemitter
}
module.exports = CustomNewmanReporter

3. To use your reporter locally, use the npm pack command to create a .tgz file. Once created, this can be installed using the npm i -g newman-reporter-<name>.<version>.tgz command.

Once you're happy with your reporter, it can be published to npm using npm publish. This will then be made available for other people to download.

Scoped reporter package names like @myorg/newman-reporter-<name> are also supported. Working reporter examples can be found in lib/reporters.

back to top

File uploads

Newman also supports file uploads for request form data. The files must be present in the current working directory. Your collection must also contain the filename in the "src" attribute of the request.

In this collection, sample-file.txt should be present in the current working directory.

{
    "info": {
        "name": "file-upload"
    },
    "item": [
        {
            "request": {
                "url": "https://postman-echo.com/post",
                "method": "POST",
                "body": {
                    "mode": "formdata",
                    "formdata": [
                        {
                            "key": "file",
                            "type": "file",
                            "enabled": true,
                            "src": "sample-file.txt"
                        }
                    ]
                }
            }
        }
    ]
}

$ ls
file-upload.postman_collection.json  sample-file.txt

$ newman run file-upload.postman_collection.json

back to top

Using Newman with the Postman API

1 Generate an API key
2 Fetch a list of your collections from: https://api.getpostman.com/collections?apikey=$apiKey
3 Get the collection link via it's uid: https://api.getpostman.com/collections/$uid?apikey=$apiKey
4 Obtain the environment URI from: https://api.getpostman.com/environments?apikey=$apiKey
5 Using the collection and environment URIs acquired in steps 3 and 4, run the collection as follows:

$ newman run "https://api.getpostman.com/collections/$uid?apikey=$apiKey" \
    --environment "https://api.getpostman.com/environments/$uid?apikey=$apiKey"

back to top

Using Newman in Docker

To use Newman in Docker check our docker documentation.

Using Socks Proxy

When using Newman as a library, you can pass a custom HTTP(S) agent which will be used for making the requests. Here's an example of how to setup socks proxy using a custom agent.

const newman = require('newman');
const SocksProxyAgent = require('socks-proxy-agent');
const requestAgent = new SocksProxyAgent({ host: 'localhost', port: '1080' });

newman.run({
    collection: require('./sample-collection.json'),
    requestAgents: {
        http: requestAgent, // agent used for HTTP requests
        https: requestAgent, // agent used for HTTPS requests
    }
}, function (err) {
    if (err) { throw err; }
    console.log('collection run complete!');
});

back to top

Migration Guide

• Newman v5 to v6 Migration Guide
• Newman v5.x Documentation

Compatibility

NodeJS

Newman	Node
v3.x	>= v4.x
v4.x	>= v6.x
v5.x	>= v10.x
v6.x	>= v16.x

The current Node version compatibility can also be seen from the engines.node property in package.json

File Encoding

Newman attempts to detect file encoding for files that are provided as command line input. However, it mostly relies on NodeJS and the underlying operating system to do the heavy lifting. Currently, ASCII, UTF-8, UTF-16LE and ISO-8859-1 are the only ones that are detection assisted.

back to top

Contributing

Please take a moment to read our contributing guide to learn about our development process. Open an issue first to discuss potential changes/additions.

Community Support

If you are interested in talking to the Postman team and fellow Newman users, you can find us on our Postman Community Forum. Feel free to drop by and say hello. You'll find us posting about upcoming features and beta releases, answering technical support questions, and contemplating world peace.

Sign in using your Postman account to participate in the discussions and don't forget to take advantage of the search bar - the answer to your question might already be waiting for you! Don’t want to log in? Then lurk on the sidelines and absorb all the knowledge.

License

This software is licensed under Apache-2.0. Copyright Postdot Technologies, Inc. See the LICENSE.md file for more information.

newman's Issues

No result when i try to use it

Hi, Newman

I just tried to use this tool in command, it always no infromation, i don't know what happened. 

The steps i used to do test:

• Upload collection to postman, and got https://www.getpostman.com/collections/910fd2e72c4ac2f3add9
• Down load one environment from PostMan, as role.json :

{"id":"f5732ad5-9225-aef6-adf5-0d9de3fed909","name":"Role-owner","values":[{"key":"role","value":"owner","type":"text"}],"timestamp":1399973640780}

• Use new man to test result newman -u https://www.getpostman.com/collections/910fd2e72c4ac2f3add9 -e role.json -s

And i got nothing output except Iteration 1 of 1. So i was confused on how to use this tool.
So if i want to run only one specific request, is it possible?

Thanks!

Validate cookie scenario

Newman doesn't prefix http:// if not present in URL

Running a collection using the usual
newman -c xyz.json.postman_collection -e abc.postman_environment
was giving me a request error.

Couldn't figure why it wasnt able to construct the request. Turned out that my environment variables didn't have the http:// prefixed to them. Postman would understand that and execute the requests anyway, but Newman wouldn't.

Environments/Globals Newman re-use

Trying to work out how best to structure my environments/globals in postman to make re-use in newman as easy as possible. I have a number of scenarios to test which require different input data which up till now I've used environments for. I also use globals for common variables across all scenarios. There's no explicit support in newman afaics to allow me to pass both.

One way around this I thought was to make each environment a data file and consider it an iteration and use the globals as an environment file to pass in however the environment file format is not directly usable as a set of variables in a data file iteration which means some pre-processing is required.

Another thought I had was to merge the globals with each environment file and just run newman for each environment but this would be a manual process (there's no ability to export an environment and merge the globals in a single step). Even better would be for newman to support multiple environment files and treat it as a flat variable list, that way I could pass the environment and the globals at the same time.

Has anyone else faced this issue and how did you deal with it? Any thoughts appreciated.

Unable to handle https requests

I am attempting to test a service that exposes its API over https.
Everything works fine hitting this URL in Postman but when executing this in Newman I receive an error.

EXCEPTION - Syntax Error: Unexpected token u
RequestError: dfaa2798-66f5-00df-89f277629098 terminated. Error: undefined.

Generic tests to HTTPS passing in Postman but not in Newman ?

I'm writing very simple tests to pass against some calls made to an https ASP.Net Web API. These tests run and pass as predicted in Postman, but I get different results when I attempt the same in the commandline. I've checked the Collection json file that I am using; all the appropriate header declarations are present and nothing is different.

for comparison:

and

The specific tests that I am running are also very basic:

tests["Response time is less than 500ms"] = responseTime < 500;
tests["Status code is 200"] = responseCode.code === 200;
tests["Content-Type is present"] = responseHeaders.hasOwnProperty("Content-Type");

(Edited to add additional screenshots. I've also tried using both Content-Type and content-type.)

Any ideas? Apologies in advance if I have just overlooked something completely obvious.

Validate JSON format?

@viig99 Should we validate the collection JSON format if that complies to a schema?

Adding support of accessing environment variables in tests

SugarJS methods do not appear to work when executing tests with newman

I've got a few testcases and make relatively heavy use of the SugarJS Date extensions. These work fine in Postman and in the collection runner, but when trying to execute them in Newman it bails out with messages like this:

EXCEPTION - TypeError: Object function Date() { [native code] } has no method 'create'

I assume this is due to SugarJS not being included? Or perhaps some kind of node.js compatibility issue.

Each iteration should have a fresh set of environment variables

Currently, the environment variables set by one iteration will be carried over to the next iteration. Each iteration should have clean environment variables or you will see some strange behavior.

Errors when running from Code?

This is more of a question but wanted to see if there was any insight.

Just installed release 1.07 and I am able to run my entire collection from the command line with no problems.

However, I am attempting to execute directly from code (included below)
I noticed a couple things.

1. I will run into an error with the Helper.js
   TypeError: Cannot read property 'ok' of undefined
   at Object.jsface.Class.testCaseSuccess c:\NewmanPOC\node_modules\newman\src\utilities\Logger.js:60:40)
   at jsface.Class._logTestResults (c:\NewmanPOC\node_modules\newman\src\responseHandlers\TestResponseHandler.js:195:9)

I worked around this by changing my the local code in the Helper.js
exports.symbols = {
err: (process.platform === "win32") ? "\u00D7 " : "✗ ",
ok: (process.platform === "win32") ? "\u221A " : "✔ "
};
To
Helpers.symbols = {
err: (process.platform === "win32") ? "\u00D7 " : "✗ ",
ok: (process.platform === "win32") ? "\u221A " : "✔ "
};

2. Past this temp workaround I am able to run and see console messages for successful tests. However, once I hit a failure (anything that is supposed to pass through ErrorHandler.js) I am getting the following error:

Iteration 1 of 25
200
TypeError: Object # has no method 'testCaseError'
at Object.jsface.Class.testCaseError (c:\NewmanPOC\node_modules\newman\src\utilities\ErrorHandler.js:24:13)
at c:\NewmanPOC\node_modules\newman\src\responseHandlers\TestResponseHandler.js:197:18

Or another example
Iteration 2 of 25
200 505ms Get Auth Token https://####.com/v1/requestToken
√ Status code is 200

TypeError: Object # has no method 'error'
at Object.jsface.Class.responseError (c:\NewmanPOC\node_modules\newman\src\utilities\ErrorHandler.js:16:7)
at Object.jsface.Class._printResponse (c:\NewmanPOC\node_modules\newman\src\responseHandlers\AbstractResponseHandler.js:39:17)
at Object.jsface.Class._onRequestExecuted (c:\NewmanPOC\node_modules\newman\src\responseHandlers\AbstractResponseHandler.js:30:9)
at Object.jsface.Class._onRequestExecuted (c:\NewmanPOC\node_modules\newman\src\responseHandlers\TestResponseHandler.js:27:46)

Code File
var Newman = require('newman');
var JSON5 = require('jju');
var fs = require('fs');
// read the collectionjson file
var collectionJson = JSON5.parse(fs.readFileSync("POSTGoalCollection.json", 'utf8'));
// define Newman options
newmanOptions = {
envJson: JSON5.parse(fs.readFileSync("HerokuSanityEnvironment.json", "utf-8")), // environment file (in parsed json format)
iterationCount: 10, // define the number of times the runner should run
dataFile: "CreateGoalTests.json", // data file if required
outputFile: "outfile.json", // the file to export to
responseHandler: "TestResponseHandler", // the response handler to use
stopOnError: false
}
Newman.execute(collectionJson, newmanOptions);

Run collection N times

Add iteration count

Reading Collection and env from folder

Automatically read collection and env from folder given it follows a format?

Validating HTTPBIN Collections

Executing results in SyntaxError: Unexpected token u

Tests perform correctly using collection runner however executing tests with Newman result in an error EXCEPTION - SyntaxError: Unexexpected token u
Test case failed: (test case name)

I am passing in a data file, an environment file and a collection.

Incorrect exit code when a test fails

Newman, by default exits with a status code of 0 if everything runs well
i.e. without any exceptions. Continuous integration tools respond to these exit
codes and correspondingly pass or fail a build.

Unfortunately - this is incorrect. If I run a collection and some tests fail inside, newman returns a return code of 0.

Halt runner with an error code if a test fails

This as discussed would help in signalling errors for integration with CI/build systems.
If it's present already then probably help needs to be updated.

junit reporter format?

Any chance of adding support for custom templates for rendering test reports? A Junit test reporter would be particularly usefull for example to allow jenkins and similar CI tools process and render test results effectively .

Newman vs. ASP.NET MVC Web API 2 "No HTTP resource was found" message

Just tried running Newman 1.0.2 with node.js v0.10.28 on a 64-bit Windows 7 machine and received a "TypeError" error on a fairly common ASP.NET MVC Web API 2 error message.

D:\code>newman -c MVCWebApp03_test.json

Iteration 1 of 1
200 20ms Basic value to root http://localhost:26069/values/5
404 21ms Basic value http://localhost:26069/api/values/5

TypeError: Cannot read property 'two' of undefined
at getKey (C:\Users\tehuser\AppData\Roaming\npm\node_modules\newman\src\uti
lities\VariableProcessor.js:62:23)
at String.replace (native)
at Object.jsface.Class._findReplace (C:\Users\tehuser\AppData\Roaming\npm\n
ode_modules\newman\src\utilities\VariableProcessor.js:64:31)
at Object.jsface.Class._processPathVariable (C:\Users\tehuser\AppData\Roami
ng\npm\node_modules\newman\src\utilities\VariableProcessor.js:33:23)
at Object.jsface.Class.processRequestVariables (C:\Users\tehuser\AppData\Ro
aming\npm\node_modules\newman\src\utilities\VariableProcessor.js:107:8)
at Object.jsface.Class._processUrlUsingEnvVariables (C:\Users\tehuser\AppDa
ta\Roaming\npm\node_modules\newman\src\runners\RequestRunner.js:128:21)
at Object.jsface.Class.execute (C:\Users\tehuser\AppData\Roaming\npm\node
modules\newman\src\runners\RequestRunner.js:46:9)
at Object.jsface.Class._onRequestExecuted (C:\Users\tehuser\AppData\Roaming
\npm\node_modules\newman\src\runners\RequestRunner.js:80:8)
at EventEmitter.emit (events.js:117:20)
at Object.jsface.Class.emit (C:\Users\tehuser\AppData\Roaming\npm\node_modu
les\newman\src\utilities\EventEmitter.js:34:16)

Where “Basic value to root” returns:
"value"
…and “Basic value” returns:
{
"Message": "No HTTP resource was found that matches the request URI 'http://localhost:26069/api/values/5'.",
"MessageDetail": "No type was found that matches the controller named 'api'."
}

More info:

[email protected] C:\Users\jonathal\AppData\Roaming\npm\node_modules\newman
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected] ([email protected], [email protected], [email protected])
├── [email protected] ([email protected], [email protected], [email protected], reque
[email protected], [email protected], [email protected])
├── [email protected] ([email protected], [email protected])
└── [email protected] ([email protected], [email protected], [email protected], [email protected])

Make Newman importable

Right now if you have newman is installed in /node_modules you can't import as it is.

var newman = require('newman'); // doesnt work
var newman = require('src/Newman'); // works

I think it will be good if we add a index.js file in root folder exposing the Newman module so that its more intuitive to import.

Index.js

module.exports = require('./src/Newman');

This allows a simple import

var newman = require('newman');

Let me know if this makes sense and I can quickly push a fix. Will help us in writing a grunt plugin as well.

Crontab with newman

I'm trying to schedule my collection to run with crontab but even with full paths I still get an error about "node".

jpumph0414mac:Documents jpumph$ crontab -l
30 11 * * * /usr/local/bin/newman -c /Users/JPUMPH/Documents/ATD-JetPacks.json.postman_collection -e /Users/JPUMPH/Documents/ATD-FT.postman_environment > /Users/JPUMPH/Documents/Postman.log 2>&1

jpumph0414mac:Documents jpumph$ cat Postman.log
env: node: No such file or directory

Max call stack size exceeded

Url of the form {{url}}/blog/users goes into infinite recursion when not provided with an environment file.
However, url of the form http://{{url}}/blog/users ends gracefully.

Set global/local environment variable for running tests

Multiple data passes per test

There is an idea to have each test loop over a dataset and record the results of each pass.

This would work something like this, and the tests would be executed against each row of data.

supply data to the test, one object per row

var data = {
    "rows": [
        {"username":"someone1", "password":"somepass1","shouldPass":true},
        {"username":"someone2", "password":"somepass2","shouldPass":fail}
    ]
}

The test runner would use those values in the url like so:
http://somesite/login/user={$username} etc

testing responses

Each row of values should be available to the tests in its iteration. Sp the tests for this could be something like: (sudo code for ideas)

var data = JSON.parse(responseBody);
tests["User logged in"]  = data.loggedIn === row.shouldPass; //current row value

getting data into the test

There is a data field in the test json object, the data could be passed into that with a "dataMode" of "rows" or objectArray etc.

Possible extensions

The ability to process the results of multiple API calls including AVG response times etc.

Personally, I would like this in newman but can see the advatages of using it in postman too. Feel free to add/extend/run away with the idea :)

postman.setEnvironmentVariable is not working

Need to carry out through testing if this is working. Note to self: Check mail for detailed issue.

callback on iterationRunnerOver?

Shouldn't we have a callback function as a third parameter on Newman that is run when the iteration gets over - so that users can hook in their functionality. Although we are emitting an event when this happens, I'm not sure how others can know when Newman gets over. This is also required for the Grunt plugin. Something like this -

Newman.execute(collectionJson, optionsDict, callback); 

Usage: https://github.com/prakhar1989/grunt-newman/blob/master/tasks/newman.js#L10-L19

Or is there a better way to handle this?

Tests related to Content-Type fail with Newman but pass in Postman

I have noticed a recurring pattern in my tests whereas specs related specifically to Content-Type seems to fail when run with Newman but pass in Postman. Please observe the failing in Newman:

But they (the same suite and same env) pass in Postman:

Specifically, the tests take the form of things like:

tests["Content-Type is  UTF-8 application/json"] = responseHeaders.hasOwnProperty("Content-Type") && 'application/json; charset=utf-8' === responseHeaders['Content-Type'];

tests["Content-Type is text/html"] = responseHeaders.hasOwnProperty("Content-Type") && 'text/html' === responseHeaders['Content-Type'];

Have I done something wrong? Can I provide more information or help diagnose this issue?

Some basics about my system:

OSX 10.9.3

Darwin Concurrent-Chickpea.local 13.2.0 Darwin Kernel Version 13.2.0: Thu Apr 17 23:03:13 PDT 2014; root:xnu-2422.100.13~1/RELEASE_X86_64 x86_64

node v0.10.28 installed from homebrew

newman 1.0.6

Quick edit: The collection and environment files ->

https://github.com/SexualHealthInnovations/PrivateResults/blob/master/spec/data/Private-Results-API.json.postman_collection

https://github.com/SexualHealthInnovations/PrivateResults/blob/master/spec/data/Private-Results-Local.postman_environment

Standardize error handling for filename checks

On running newman, filename checks for the postman collection files provide a well handled error message.

However, for the same check in the environment variable filename, I see what looks like a trace.

For the same type of check, the error handling should be uniform.

Run collection N times

Iteration count.

Refactor request.transformed.data

Refactor code based on comments here - #74

Importing test results from Newman does not show collection

I can see everything except the collection name:

Windows, Full path for collection and environment file causes parse error

In Windows. Below I ran the test without the path (sitting in the correct directory) and it ran fine, then I added the directory to each file and it failed. Works fine both ways on Mac.

1st run-----------
C:\Users\jpumph\Desktop\REST Automation\Postman>newman -c ATD-JetPacks.json.postman_collection -e ATD-FT.postman_environment > C:\Users\jpumph\Desktop\REST Automation\Postman\Postman.log

2nd run-------------
C:\Users\jpumph\Desktop\REST Automation\Postman>newman -c C:\Users\jpumph\Desktop\REST Automation\Postman\ATD-JetPacks.json.postman_collection -e C:\Users\jpumph\Desktop\REST Automation\Postman\ATD-FT
.postman_environment > C:\Users\jpumph\Desktop\REST Automation\Postman\Postman.log

C:\Users\jpumph\AppData\Roaming\npm\node_modules\newman\node_modules\json5\lib\json5.js:44
throw error;
^
SyntaxError: Unexpected ''
at JSON5.parse.error (C:\Users\jpumph\AppData\Roaming\npm\node_modules\newman\node_modules\json5\lib\json5.js:40:25)
at JSON5.parse.word (C:\Users\jpumph\AppData\Roaming\npm\node_modules\newman\node_modules\json5\lib\json5.js:340:13)
at JSON5.parse.value (C:\Users\jpumph\AppData\Roaming\npm\node_modules\newman\node_modules\json5\lib\json5.js:443:56)
at Object.JSON5.stringify as parse
at main (C:\Users\jpumph\AppData\Roaming\npm\node_modules\newman\bin\newman:68:33)
at Object. (C:\Users\jpumph\AppData\Roaming\npm\node_modules\newman\bin\newman:104:1)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)

C:\Users\jpumph\Desktop\REST Automation\Postman>

Feature request, group tag for tests, similar to phpunit @group

Group tags per test

This is a feature request for extending the current JSON format for a test used in both Postman and Newman to include group tags.

The idea is similar to phpunit's @group where you can tag any test in any number of groups and run tests against either an individual group or multiple groups.

JSON format

{
    ...
    "time": 1402331140943,
    "version": 3???,
    "responses": [],
    "tests": "",
    "groups": [],
    "collectionId": "",
    "synced": false
}

Example usage:

Only run the user tests in a collection

newman -c collection.json -g "user"

Run the user and admin tests in a collection

newman -c collection.json -g "user,admin"

Run everything apart from the admin tests

newman -c collection.json -g "-admin"

Run everything apart from the user and the admin tests

newman -c collection.json -g "-user,-admin"

caveats

The JSON format version might need to be upgraded to V3

Output file is not written when -s and failure halts collection run

I call newman with -s and -o outputfile.json, a test fails and the run halts.
No output file is ever written to allow me to diagnose the collection run failure.

If using this with Jenkins it will be very important to rely on the output file for determining failures.

Finalize Export Format for collection / test output.

@a85 Need your input here to finalize JSON export format.

No output produced - only 'Iteration 1 of 1', due to lack of order in collection

Hey - I was experimenting with Postman/Newman for the first time today.

I had a couple of requests in a collection. However when I ran them, the only output I saw was "Iteration 1 of 1". For a while I wondered why they weren't running correctly. Did some more digging and looking at the collections of others I noticed the JSON produced from Postman didn't contain an order array like the others had.

In Postman, if you drag and drop the requests, this will now produce an order array in the JSON for anything you upload / download. This is a workaround in Postman. If you like I'll file a bug report over there too.

But perhaps Newman should run the requests in the order of the requests array if there's an absence of an order array?

Thanks for Postman & Newman - they're great so far.

POST calls with form-data fields does not work with Newman, works with collection Runner

Noticed that POST calls with form-data fields does not work with Newman(command line), but the same works with collection Runner.
I have confirmed the behavior, if I move the form-fields to the URL-params section, it works with Newman(command line).

POST call data with variables do not get evaluated for each iteration

Note: I'm on tip of tree, including the offending check in:
2ed4724

If I have a variable inside of my data for a POST or PUT, and I run multiple iterations, the variables inside will only be expanded once. In the above checkin, you assign the transformed data to the ACTUAL request, meaning next time this request is evaluated, the variables are already expanded, but only with the values assigned for the first iteration.

Better error message for lack of env json file

Provide a better error message in case the user runs a collection without the provided env json file.

Invalid response throws an exception due to response exporter

See screenshot below. Need to handle a case when response is not defined (as done in RequestRunner). Have added a hack at the moment. Please fix this. [Above is present, below is ideal case]

Getting Syntax and 'TypeError' when running the collection from Newman

Suite of tests run fine from Postman. When I try and run them from Newman using: **newman -c testcollection1.json -e Staging.postman_environment

the following error is shown in the console:

Iteration 1 of 1
EXCEPTION - SyntaxError: Unexpected token u
RequestError: e9a710e9-486f-3fc6-2838-d70082368c68 terminated. Error: undefined
RequestError: 41713c16-f095-8c35-1629-eba1f52eea20 terminated. Error: undefined
x Successful POST request
EXCEPTION - TypeError: Cannot call method 'has' of undefined
RequestError: 92d62b99-1cea-e6d2-bba1-ff5c7a2cc5df terminated. Error: undefined
x Status code is 200
RequestError: 41026d39-43b9-168d-b94f-263836e3cbab terminated. Error:      undefined
RequestError: 743f8ce7-94f0-2639-7420-eac88e28369e terminated. Error: undefined

Image attached containing the Json

Looks like its not handling the access_token variable

Collections not running

I'm trying to run a collection as a file through Newman. Everything works fine on the app, and I've saved the collection file correctly onto my desktop. However, when I type in the command to run my file; "newman -c UCRT_Test.json" I get the message "Please specify a Postman Collection either as a file or a URL". I'm pretty confused by this. Any help?

Add a "method" parameter in request object

Add a "method" parameter in request object for parity with postman.

Unreadable text on a black terminal

Can not read the request name as the text is black. Screenshot:

Run output on Windows vs Mac

When you live run the newman collection runner on screen results look like this:
200 669ms Delete Account
√ Status code is 200
On a Mac, if you redirect that output (newman -c .... > Postman.log) to a file for later use (like you would using cron), the output looks the same when you "cat" the file. But in Widows if you redirect > then "type" the file contents you get a bunch of extra characters.
←[39m←[32m200←[39m←[36m 669ms←[39m Delete Account
←[24m←[39m←[32m ←[32mΓêÜ Status code is 200←[39m

Is there a way around this? I need to see the runtime output rather than the json you get with the -o option, but I need it to be readable in Windows like it is on the Mac.

Data variables

Add data variables in postman.

Iteration property not followed if using a data file.

Running from command line or code does not honor the the iterationCount if a data file is also used. It is always running every record of the datafile.

My data file has 25 records.

newman -c POSTGoalCollection.json -e HerokuSanityEnvironment.json -d CreateGoalTests.json -n 10

or in code:
newmanOptions = {
envJson: JSON5.parse(fs.readFileSync("HerokuSanityEnvironment.json", "utf-8")), // environment file (in parsed json format)
iterationCount: 10, // define the number of times the runner should run
dataFile: "CreateGoalTests.json", // data file if required
outputFile: "outfile.json", // the file to export to
responseHandler: "TestResponseHandler", // the response handler to use
stopOnError: false
}
When this run 25 iterations are always executed. Only 10 should have occurred.

This is what I would expect as test cases:
Iteration is set and a data file provided = runs the collection the number of iterations
Iteration is not set and data file provided = runs the collection the number of records in data file
Iteration is set and no data file provided = runs the collection the number of iterations (Currently works)
iteration is not set and no data file provided = runs the collection once. (Currently works)

Test results overview

After running the test, I think Newman should show some stats. At the very least, how many tests failed. It is annoying to have to scroll to know how many failed.

These tests results overview could show test success or failure by subfolder. I work with a long test collection, and I organise it with folders (like registration, permissions, ...). It would be great to see the test outcome by folder. Something like this:

Overview
62 passed, 2 failed

-> registration
20 passed, 0 failed
-> permissions
42 passed, 2 failed

Exception Handling in eval code.

Enable exception handling in vm.runInThisContext to avoid, poor user code from breaking the application.

Usage docs for Newman

Will need a dedicated page for Newman on getpostman.com/docs/