Hi all,

is possibile forward all request to ip instead of URL?

We want to use toxy behind a corporate proxy for resilience testing of our client for a REST service provided by another institution. We can only access the service through the proxy server of our institute. I could not find any configuration options to achieve that. Have I overlooked something?

I tried to use global-agent it fails because the http-proxy dependency used by toxy always creates a new instance of the agent for every request and misses the parameters required by global-agent (because it is and likely cannot be aware of it). Before I raise an issue for http-proxy, I wanted to make sure that I am not overlooking a mechanism of toxy to be run behind a corporate proxy.

Just as a remark the issue with http-proxy is that it does not pass the agent option correctly to the http client API of nodejs. It converts null to false which has a different semantic. Therefore, the following two configuration lines for toxy are passed as false to http.request by http-proxy.

proxy.opts['agent'] = null;
proxy.opts['agent'] = false;

A nice-to-have feature, probably as embedded feature in toxy.

Implement a built-in server listening on a different port exposing a simple HTTP API for admin purposes and dynamic configuration.

API

var toxy = require('toxy')

var proxy = toxy()
var admin = toxy.admin()

admin.manage(proxy)

proxy.listen(3000)
admin.listen(9000)

Tasks

• Read
• Update
• Create
• Remove
• Hypermedia (HAL)

HTTP spec

GET /servers

GET /servers/:id

GET /servers/:id/routes

GET /servers/:id/routes/:id

POST /servers/:id/routes/:id

DELETE /servers/:id/routes/:id

GET /servers/:id/routes/:id/poisons

GET /servers/:id/routes/:id/poisons/:id

POST /servers/:id/routes/:id/poison

DELETE /servers/:id/routes/:id/poisons/:id

• Hierarchical support (global, route and poison level)
• Embed or not to embed as part of the poison feature

Based on rocky-cli

Hi,

is it posible proxy requests to all hosts and only poison specific routes?

proxy
  .forward('*')

proxy
  .forward('http://myhost')
  .get('/test')
  .poison(poisons.abort())

I'm using toxy to inject problems talking to the backend APIs into my single page web application.

So far so good, I had a working toxy proxy.

But when trying to bootstrap the design team with the proxy, they downloaded the latest node, which then caused the proxy to hang. I know its getting to transformRequestBody, because I see the log messages, but the browser thinks all the requests are pending until they time out.

Here's my config:

var toxy = require('toxy');
var express = require('express');
var poisons = toxy.poisons;
var rules = toxy.rules;

// Create a new toxy proxy
var proxy = toxy();
//var Array = require('Array');

var host = process.env.HOST || 'uicommon-lab-sc.lab.skyportsystems.com'


var app = express(),
port = process.env.PORT || 8080;

var cspheader = "default-src 'self'; script-src 'self' www.google-analytics.com" +
                " ajax.googleapis.com ; style-src 'unsafe-inline' 'self' data:; " +
                "img-src 'self' data:; media-src 'none'; font-src 'self' data:; " +
                "connect-src 'self'; report-uri https://" +
                host+"/api/v0/csp";

var cspheadernames=["Content-Security-Policy", "X-Content-Security-Policy", "X-WebKit-CSP"];

function setCSPHeaders(res, path, stat)
{
    for (var ix=0; ix < cspheadernames.length; ix++)
    {
        res.append(cspheadernames[ix],cspheader);
    }

}

function processRequest(req,res,next) {
        console.log(">>>>"+req.url);

        next();
}

function processResponse(req,res,next) {
        console.log("<<<<<"+req.url);
    var keys = Object.keys(res._headers),
        len = keys.length,
        prop,
        value;
        for (i=0; i < len;i++) 
        {
            prop = keys[i];
            value = res._headers[prop];
            //console.log("replacing",prop,value);
            if (Array.isArray(value))
            {
                for (jx=0; jx < value.length; jx++)
                {
                    v2 = value[jx];
                    value[jx] = v2.replace(/Secure/g,'');
                }
            }
            else
            {
                value.replace(/Secure/g,'');
                res._headers[prop]=value;
            }
        }

        next()
}

// Default server to forward incoming traffic
proxy
  .forward('https://'+host)

var roots = ['/api','/templates','/cdn_images','/session'];

for (var ix=0;ix< roots.length;ix++)
{
var root = roots[ix]+'/*';
    console.log('proxying',root);
    proxy
      .all(root)
      .withRule(rules.method(['POST', 'PUT', 'DELETE','GET']))
      .options({ secure: false })
    .transformRequest(processRequest)
    .transformResponse(processResponse)
      .host(host)
}



app.use(express.static(__dirname + '/webroot', {setHeaders: setCSPHeaders}));
app.use(proxy.middleware())
app.listen(port);


console.log('Server listening on port:', port)
console.log('Test it:', 'http://localhost:'+port+'/html/home.html')

I would like to use single toxy instance to throttle many different APIs.
I am trying to set up following:

proxy.all("/papyrus")
    .forward("https://server1")
    .poison(poisons.latency({ min: 50, max: 2000 }))

proxy.all("/gql/*")
    .forward("https://server2")

Problem is, part of the url in all is also being forwarded

Any way to forward excluding the base url in all?

Can toxy be used as a web browser proxy (to test some page loading behaviour with low connection speed)? Or is this package not suitable for such use case?

Thank you!

Enable/disable poisoning based on a specific time threshold. Useful for testing failures in specific recurrent periods of time.

proxy.rule(toxy.rules.timeThreshold({ period: 1000 * 60 * 60, duration: 500 }))

Or using a random value within a given range:

proxy.rule(toxy.rules.timeThreshold({ period: 1000 * 60 * 60, duration: [100, 1000] }))

Rate limit

Hi,

I installed toxy and simply trying to run the example. However it is throwing me error for module midware:

Cannot find module 'midware'

any ideas to resolve this error

Hi guys, first of all thank you for building this cool tool!

I've been making a small, simple POC for myself using toxy to simulate bad network behaviour and I stumble upon a problem.

Using the poison slowRead like so:

proxy
    .post('/upload')
    .poison(poisons.slowRead({ chunk: 102400, threshold: 500 }))

The browser becomes unable to submit files larger then 30KB, 70KB files sometimes get submitted 100KB for instance, doesn't.

You can see for yourselves: https://github.com/goncalvesjoao/up-your-ass-ets
the POC is fairly simple and quick to look at, am I missing something or am I configuring toxy in a wrong way?

Thanks,
John

Hi,

I am using the proxy for video player testing, switching network conditions within the playback session. I do this every time I want to change the network:

if (poison) {
    proxy.remove(poison);
    poison = null;
}

poison = poisons.throttle({ chunk: chunkSize, threshold: chunkDelay});
proxy.poison(poison);

And then I get this warning:

(node:19805) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 close listeners added. Use emitter.setMaxListeners() to increase limit

I did a separate test program and the warning did not appear when just switching poisons but rather it appeared when the proxy was also requests on the same time.

Is there a better way to change the network bandwidth in the same session?

Thanks

Useful for remote configuration control via toxy clients

A nice to have feature, particularly useful in certain scenarios.

Maybe this rule should be provided as third-party package.

I'd like to implement a poison on a resource that will disrupt the connection after some time. It looks like the abort poison is what I want, but I was not getting the functionality I wanted with the following configuration:

const toxy = require('..')
const poisons = toxy.poisons

const proxy = toxy()
const serverhost = 'mysite.de'
const timeout = 1000

proxy
  .all('/*')
  .forward('http://' + serverhost)

proxy
  .all('/myresource/*')
  .poison(toxy.poisons.abort(1000))
  .withRule(rules.method('GET'))
  .outgoingPoison(toxy.poisons.abort(1000))
  .withRule(rules.method('GET'))
  .forward('http://' + serverhost)

const port = 4567
proxy.listen(port)
console.log('Server listening on port:', port)

My landscape is:

[client] => [nginx] => [toxy] => [webapp]

Is this the right approach, or am I misunderstanding something?

Ideally, supporting HTTPS on both sides of the connection, providing a certificate which must be trusted, similar to Fiddler/Charles Proxy.

Created a simple pass through toxy script to just forward traffic and I'm seeing large ~20 sec spikes fairly regularly. I'm sending 60 requests per second through toxy. I've got some client side metrics enabled and the majority of the time is spent receiving the response from toxy. Is there some default poison applied that could be causing this? Can toxy not handle that rate of requests (I assume it can per some benchmarks on the rocky README)? When I run the same code without toxy (directly hitting the endpoint) I'm not getting any spikes. Hitting toxy with http and toxy is forwarding to an https endpoint.

var toxy = require('toxy')
var fs = require('fs')

var poisons = toxy.poisons
var rules = toxy.rules
var proxy = toxy()

// Configure and enable proxy
proxy.all("*").forward("https://dynamodb.us-west-2.amazonaws.com")
proxy.listen(6241)

Hi @h2non I'm making an OPTIONS request and rocky returns a 502 to the client.

I'm just forwarding:

  proxy
    .forward('http://127.0.0.1:9990');

  proxy
    .all('/*')

localhost 9990 is not getting any request, rocky server is complaining and responding 502.

Any idea?

I'm experiencing that I'm unable to delete outgoing poisons after adding them via the HTTP API.

I added an outgoing latency poison to a route via the HTTP API, resulting in the call to GET /servers/882/routes/20e/poisons looking like this:

[
   {
      "name":"latency",
      "enabled":true,
      "phase":"outgoing",
      "rules":[

      ],
      "links":{
         "self":{
            "href":"/servers/882/routes/20e/poisons/latency:outgoing"
         },
         "parent":{
            "href":"/servers/882/routes/20e/poisons"
         }
      }
   }
]

I then attempt to delete the poison via DELETE /servers/882/routes/20e/poisons and get back a 204 No Content, but the poison is still present.

I also experience the same problem when I try to delete the specific poison, both via DELETE /servers/882/routes/20e/poisons/latency and DELETE /servers/882/routes/20e/poisons/latency:outgoing; I get a 204 No Content, but the poison is still there.

Note that I can delete any incoming poisons; the problem is exclusive to outgoing ones.

Greetings

I am very interested about using toxy but I ran into usage issues right from the start.

I cannot seems to get a correct response from a multiple servers, cause some direct IP access and many many sites are refusing to forward, I have tried it at multiple AP home/work/cafferooms etc

I run proxy like this

~$ node name.js

var toxy = require('toxy')
var poisons = toxy.poisons
var rules = toxy.rules

// Create a new toxy proxy
var proxy = toxy()

// Default server to forward incoming traffic
proxy
    .all('/*')
    .forward('http://4chan.org/')

proxy.listen(3000)

console.log('Server listening on port:', 3000)

And 4chan kindly responds with Direct "IP access not allowed"

I have no firewall set on my debian machine.

more demos: https://youtu.be/dvarUf868S8

Thank you for the time.

Just happened after using Toxy (installed via current npm) on a random website:

Server listening on port: 3000
/node_modules/toxy/lib/helpers/each-series.js:2
var stack = arr.slice()
^
TypeError: Cannot read property 'slice' of null
at Object.eachSeries (/node_modules/toxy/lib/helpers/each-series.js:2:18)
at ServerResponse.res.end (/node_modules/toxy/lib/poisons/throttle.js:33:15)
at end [as _onTimeout] (/node_modules/toxy/lib/poisons/slow-close.js:40:17)
at Timer.listOnTimeout (timers.js:119:15)

I have implemented a custom throttling solution with node, but I am interested in using toxy. One of the features I need is the ability to begin throttling a request mid stream. In other words, if I am streaming a large video, I need to be able to start throttling the bandwidth some amount of time (say 10 seconds) into the stream to simulate poor network conditions. I see that I could create a poison to throttle the request and use the toxy API to asynchronously activate the poison, but my question is: will the poison immediately take effect (mid request) or will it not take effect until future requests arrive?

Summary: Toxy's HTTP API doesn't accept well formed CORS preflight requests. It encounters an error unless both a Content-Type header and body are present, neither of which should be in a CORS preflight request. This essentially makes CORS support non-functional.

Additionally, if authorization is enabled, it attempts to validate the Authorization or API-Key headers on the preflight request.

I have a Toxy server set up as such:

var toxy = require('toxy');

var admin = toxy.admin({ cors: true, apiKey: 'secret' });
var proxy = toxy();

proxy.forward('http://api.preview.mydomain.com');
proxy.all('/*');
proxy.listen(80);

admin.manage(proxy);
admin.listen(3000);

When I try to send an XHR to the HTTP API from my browser, it sends this preflight:

OPTIONS http://toxy.preview.mydomain.com:3000/ HTTP/1.1
Host: toxy.preview.mydomain.com:3000
Connection: keep-alive
Access-Control-Request-Method: GET
Origin: http://localhost:27614
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36
Access-Control-Request-Headers: API-Key
Accept: */*
Accept-Encoding: gzip, deflate
Accept-Language: en-US,en;q=0.9

And I receive this response:

HTTP/1.1 415 Unsupported Media Type
Date: Wed, 29 Nov 2017 22:51:01 GMT
Server: toxy (admin)
Content-Length: 0
Connection: keep-alive

If I copy the preflight request and send it with an additional Content-Type: application/json header and receive this error:

HTTP/1.1 500 Internal Server Error
Date: Wed, 29 Nov 2017 22:52:32 GMT
Server: toxy (admin)
Content-Length: 35
Connection: keep-alive

{"error":"Unexpected end of input"}

I change the preflight request again, to contain a body of {}, and receive yet another error:

HTTP/1.1 401 Unauthorized
Date: Wed, 29 Nov 2017 22:52:51 GMT
Server: toxy (admin)
Content-Length: 0
Connection: keep-alive

Finally, I add the API-Key: secret header, and it finally accepts the preflight request (which, at this point, is very malformed):

HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: *
Access-Control-Allow-Origin: localhost
Date: Wed, 29 Nov 2017 22:53:42 GMT
Server: toxy (admin)
Content-Length: 0
Connection: keep-alive

This is related to my other question: #36

I need to be able to poison requests conditionally on the session (or similar identifier). For example I want to be able to poison user A's request to /some-end-point without poisoning user B's request to /some-end-point. Specifically, I want user A to be able to hit the troxy admin API and specify a new poison with a specific bandwidth and have the poison only applied to user A's requests. In my custom throttling implementation, I do this using sessions. I am sure this can be accomplished with toxy, but I want to know: what would be the most straight forward mechanism for implementing this with toxy?

Why write forward address every time for every website, how to enable it just like straight proxy without any forwarding address strictly typed for concrete website?
And when i connected to website, which uses websockets in background, proxy doesn't hold it, i've websocket connection error, because it has ip address which diffs from main website, another strict forward problem i think? Thanks.

Ok, I admit it, I'm totally pimping my PR: (#57)

Is there a way I can channel my entire network traffic via toxy, just like the way we do it with tools like charles? Currently, I'm able to forward traffic from certain URL's via toxy and apply poisons to it, but don't see the whole network traffic going through the proxy server that I've opened in port 3000 despite adding "proxy.all('/*')". If I have an android device connected (with manual proxy in the Wi-Fi settings) to the proxy server running in my computer on port 3000, the network connection is not established and no sites are accessible.

Poisons are currently executed in the incoming traffic phase. It would be great to attach poisons that will be executed only when the proxy receives the response from the target server, allowing the poisons to determine when should be executed or not based on the server response.

Actually this feature can be achieve currently via the useResponse() middleware, but a convenient wrapper around it, adding a DSL for toxy, would be better and simple for the API consumers.

The API may look like this:

proxy.outgoingPoison(poison)
proxy.useOutgoingPoison(poison)
  .withRule(rule)

Tasks

• Add featured test
• Support filter via HTTP API outgoing and incoming poisons
• Rename poisons ID in HTTP API to include the poison phase
• Remove rules by phase
• Add proper documentation
• Add poisons phases explanation with proper graph
• Add API docs
• Add examples

I followed the HTTPS sample with the SSL option. When I query the toxy proxy, I'm getting this odd JSON
{"message":"self signed certificate"}

Yes I am using a self signed cert. Is there a work around to rejectUnauthorized: false?

Hi Team,

Is it possible that I can throttle network on my iOS app or specific URL?
Please help me to understand this, what are the things required for this if possible.

Thanks

Right now, when using the inject poison, you need to inject all three status, body and headers. I'd love to be able to only inject one or two of them.

Furthermore, it would be very useful if the inject poison would merge injected headers with the original response headers. Right now, these headers are disregarded, resulting in e.g. missing CORS headers. Any change this could be added?

I'm trying to use toxy to always allow OPTIONS requests for CORS, and intermittently fail everything else. I've tried the below but it seems only one poison ever takes effect, hopefully I'm making an obvious mistake.

https://gist.github.com/jdart/28aefe58ebb2a58d80ccca11c3d699d3

Care to help me out?

In the rocky documentation, the use of a TOML file is presented.

What is the best way to install and manage a self-signed cert for a toxy-based HTTPS proxy?

I would like to use single toxy instance to throttle many different APIs.
I am trying to set up following:

proxy.all("/papyrus")
    .forward("https://server1")
    .poison(poisons.latency({ min: 50, max: 2000 }))

proxy.all("/gql/*")
    .forward("https://server2")

Problem is, part of the url in all is also being forwarded

Any way to forward excluding the base url in all?