santthosh / aws-es-kibana

AWS ElasticSearch Kibana Proxy

AWS ES/Kibana Proxy

AWS ElasticSearch/Kibana Proxy to access your AWS ES cluster.

This is the solution for accessing your cluster if you have configured access policies for your ES domain

Usage

Install the npm module

npm install -g aws-es-kibana

Set AWS credentials

export AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXXX
export AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXX

Run the proxy (do not include the http or https from your cluster-endpoint or the proxy won't function)

aws-es-kibana <cluster-endpoint>

Where cluster-endpoint can be either a URL (i.e. https://search-xxxxx.us-west-2.es.amazonaws.com) or a hostname (i.e. search-xxxxx.us-west-2.es.amazonaws.com). Alternatively, you can set the AWS_PROFILE environment variable

AWS_PROFILE=myprofile aws-es-kibana <cluster-endpoint>

Example with hostname as cluster-endpoint:

Run within docker container

If you are familiar with Docker, you can run aws-es-kibana within a Docker container

You can pull the official container for use

docker pull santthosh/aws-es-kibana:latest

(or) Build the image

docker build -t aws-es-kibana .

Run the container (do not forget to pass the required environment variables)

docker run -e AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY -p 127.0.0.1:9200:9200 aws-es-kibana -b 0.0.0.0 <cluster-endpoint>

Credits

Adopted from this gist. Thanks @nakedible-p

aws-es-kibana's Issues

Question: Could your provide example of setting AWS_PROFILE?

Could your provide example of AWS_PROFILE setting? I mean the full string.
I use docker compose on my AWS instance and aws-es-kibana docker image.
Always get: "The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method."

Cannot access url

npm install -g aws-es-kibana
aws-es-kibana XXXX.us-west-2.es.amazonaws.com

/usr/local/lib/node_modules/aws-es-kibana/index.js:160
app.use(async function (req, res) {
^^^^^
SyntaxError: missing ) after argument list
at createScript (vm.js:56:10)
at Object.runInThisContext (vm.js:97:10)
at Module._compile (module.js:542:28)
at Object.Module._extensions..js (module.js:579:10)
at Module.load (module.js:487:32)
at tryModuleLoad (module.js:446:12)
at Function.Module._load (module.js:438:3)
at Module.runMain (module.js:604:10)
at run (bootstrap_node.js:393:7)
at startup (bootstrap_node.js:150:9)

HTTPS support

Is there a way we can access kibana endpoint over HTTPS? And proxy forward request to ES endpoint over HTTPS?

Basic authentication CLI parameters are ignored

When --user xxx --password xxx, the arguments are ignored and values are read from the environment instead (AUTH_USER/AUTH_PASSWORD or USER/PASSWORD).

export module as library

it would be great to hack around it

Unable to manage mappings via proxy

POST/ PUT requests fail when using aws-es-kibana:

_http_outgoing.js:346
    throw new Error('Can\'t set headers after they are sent.');
    ^

Error: Can't set headers after they are sent.
    at ClientRequest.OutgoingMessage.setHeader (_http_outgoing.js:346:11)
    at ProxyServer.<anonymous> (/usr/local/lib/node_modules/aws-es-kibana/index.js:112:14)
    at ProxyServer.emit (/usr/local/lib/node_modules/aws-es-kibana/node_modules/eventemitter3/index.js:119:35)
    at ClientRequest.<anonymous> (/usr/local/lib/node_modules/aws-es-kibana/node_modules/http-proxy/lib/http-proxy/passes/web-incoming.js:115:27)
    at emitOne (events.js:101:20)
    at ClientRequest.emit (events.js:188:7)
    at tickOnSocket (_http_client.js:541:7)
    at onSocketNT (_http_client.js:553:5)
    at _combinedTickCallback (internal/process/next_tick.js:74:11)
    at process._tickDomainCallback (internal/process/next_tick.js:122:9)

triggered by PUTting a mapping with curl

TimeoutError: Connection timed out after 1000ms

I tried using this, and got the following error:

aws-es-kibana https://xxxxxxxxxx.us-east-1.es.amazonaws.com
__________       _________    _________________    ________                            ______
___    |_ |     / /_  ___/    ___  ____/_  ___/    ___  __ \________________  ______  ____  /
__  /| |_ | /| / /_____ \     __  __/  _____ \     __  /_/ /_  ___/  __ \_  |/_/_  / / /_  /
_  ___ |_ |/ |/ / ____/ /     _  /___  ____/ /     _  ____/_  /   / /_/ /_>  < _  /_/ / /_/
/_/  |_|___/|__/  /____/      /_____/  /____/      /_/     /_/    \____//_/|_| _\__, / (_)
                                                                               /____/
AWS ES cluster available at http://127.0.0.1:9200
Kibana available at http://127.0.0.1:9200/_plugin/kibana/
/usr/local/lib/node_modules/aws-es-kibana/index.js:29
    if (err) throw err;
             ^

TimeoutError: Connection timed out after 1000ms
    at ClientRequest.<anonymous> (/usr/local/lib/node_modules/aws-es-kibana/node_modules/aws-sdk/lib/http/node.js:56:34)
    at ClientRequest.g (events.js:260:16)
    at emitNone (events.js:67:13)
    at ClientRequest.emit (events.js:166:7)
    at Socket.emitTimeout (_http_client.js:542:10)
    at Socket.g (events.js:260:16)
    at emitNone (events.js:67:13)
    at Socket.emit (events.js:166:7)
    at Socket._onTimeout (net.js:318:8)
    at _runOnTimeout (timers.js:524:11)

Any ideas?

(I noticed that you just posted this today-- imagine the chance that I was searching for this very thing, and it appeared today!)

node v10 Error [ERR_HTTP_HEADERS_SENT]

On node version 10. I get the following error:

_http_outgoing.js:481
    throw new ERR_HTTP_HEADERS_SENT('set');
    ^

Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client
    at ClientRequest.setHeader (_http_outgoing.js:481:11)
    at ProxyServer.<anonymous> (/usr/local/lib/node_modules/aws-es-kibana/index.js:175:14)
    at ProxyServer.emit (/usr/local/lib/node_modules/aws-es-kibana/node_modules/eventemitter3/index.js:184:35)
    at ClientRequest.<anonymous> (/usr/local/lib/node_modules/aws-es-kibana/node_modules/http-proxy/lib/http-proxy/passes/web-incoming.js:132:27)
    at ClientRequest.emit (events.js:187:15)
    at ClientRequest.EventEmitter.emit (domain.js:442:20)
    at tickOnSocket (_http_client.js:639:7)
    at onSocketNT (_http_client.js:655:5)
    at process._tickCallback (internal/process/next_tick.js:174:19)

On node 9.11.1 it works.

> n 9
> node --version
v9.11.1

Endpoint is not correctly parsed when ending with 2 letter country code

$ aws-es-kibana my-elastic-cluster-in-china.es.amazonaws.com.cn
region cannot be parsed from endpoint address, either the endpoint must end in .<region>.es.amazonaws.com or --region should be provided as an argument
..... <truncated>.....

App crashed on Heroku

After configuring the access policy on aws,
I can access kibana via aws-es-kibana proxy.

But after deploying to heroku, I can see the log message like this:

2017-06-30T09:03:16.344356+00:00 app[web.1]: Kibana available at http://127.0.0.1:50970/_plugin/kibana/
2017-06-30T09:03:16.344258+00:00 app[web.1]: AWS ES cluster available at http://127.0.0.1:50970

But when I access to https://<MY_HEROKUAPP>.herokuapp.com or https://<MY_HEROKUAPP>.herokuapp.com/_plugin/kibana,
Errors suddenly show up:

2017-06-30T09:04:33.967079+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/" host=<MY_HEROKUAPP>.herokuapp.com request_id=7b787b96-d988-4cab-9845-adf89a2d5bf4 fwd="IP" dyno= connect= service= status=503 bytes= protocol=https
2017-06-30T09:04:34.778436+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/_plugin/kibana" host=<MY_HEROKUAPP>.herokuapp.com request_id=2cc7a8ee-e679-47a2-b599-766e9325579c fwd="IP" dyno= connect= service= status=503 bytes= protocol=http

The weird thing is I don't get any other error message,
and the local version went just fine.

Error running on Amazon Linux AMI with docker

Here is the command I'm trying to run on a brand new EC2 instance running Amazon Linux:

sudo docker run -d -e LIMIT=1000kb -e AWS_ACCESS_KEY_ID=someID -e AWS_SECRET_ACCESS_KEY=secret -p 80:9200 aws-es-kibana -b 0.0.0.0 myURL

It starts without a problem, but if I run wget localhost I get the following.

--2017-08-10 21:00:58--  http://localhost/
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:80... connected.
HTTP request sent, awaiting response... 403 Forbidden
2017-08-10 21:00:58 ERROR 403: Forbidden.

Because I'm running this command from localhost there shouldn't be any security issues getting in the way.

Any ideas?

Option to disable SSL certificate verification

Hello,

First of all, congrats for this great Kibana proxy, it helps a lot!

When trying to use aws-es-kibana on a laptop connecting to AWS Kibana through an SSH tunnel, we get an [ERR_TLS_CERT_ALTNAME_INVALID] error due to http-proxy.

However there is an option on line 129 of index.js that is hardcoded to secure: true that would allow the use case to work.

Would it be possible to have a command-line argument to put this option as false?

Not Working with AWS IAM Roles

Hey guys,

im not really sure if this is intended or not but aws-es-kibana doesnt seem to work with CLI profiles which includes roles like the following construct.

[xxx-logon]
aws_access_key_id = xxx
aws_secret_access_key = xxx

[es-test]
role_arn = arn:aws:iam::xxxxx:role/es_test
source_profile = xxx-logon

When i use AWS_PROFILE=es-test the webserver is giving me the following return:

{"message":"The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.\n\nThe Canonical String for this request should have been\n'GET\n/\n\nhost:https\nx-amz-date:20170725T142058Z\nx-amz-security-................

Is anyone one of you working with roles instead of local users?

Best regards,
Markus

Socket hang up error

Hello,

I am using aws-es-kibana plugin as proxy for AWS kibana. It works fine for some time but then suddenly I am getting socket hang up error. Any idea?

aws-es-kibana on Docker hub

Hi,

It would be usefull to push regulary aws-es-kibana on your own Docker hub.

Indeed, I use aws-es-kibana with Jenkins. I would like to have automaticaly fresh aws-es-kibana without manual action.

To day, I build docker aws-es-kibana on my computer and push it on my own docker hub. Jenkins use this image to push on kibana. It's not very clean.

Thanks for advance

Does not bind to correct IP on Heroku

When starting this up on heroku, aws-es-kibana binds to 127.0.0.1 (local loopback), which heroku obviously isn't looking at, so 60 seconds later, we get this:

Error R10 (Boot timeout) -> Web process failed to bind to $PORT within 60 seconds of launch

Workaround: set the BIND_ADDRESS config (env) to 0.0.0.0, which for sure will bind to the address heroku can see. Confirmed this fixed the issue for us.

Suggestion: Bind to 0.0.0.0 by default (less secure in general, but fixes the problem on heroku)

Large request bodies cannot be proxied

When sending a large (>100kb) body the proxy fails:

Assuming the proxy is running on localhost:

printf '==========%.0s' {1..100000} > dummy.json
curl -X POST http://localhost:8200/index1/typ1 -d @dummy.json

=> body parser error:

Error: request entity too large

However increasing the bodyparser limit causes other issues:

app.use(bodyParser.raw({
    limit: '1mb', 
    type: function() { return true; }}));

=> Error: Can't set headers after they are sent.
at ClientRequest.OutgoingMessage.setHeader (_http_outgoing.js:357:11)

Is there a workaround for this issue?

Error: request entity too large

would be nice if we could (easily) configure for larger payloads?

i'm not that familiar with kibana (yet), but maybe there is a logical value that should be synced?

maybe this helpful?? http://stackoverflow.com/questions/19917401/node-js-express-request-entity-too-large

Error: request entity too large
    at readStream (/Users/user/.nvm/versions/node/v6.2.2/lib/node_modules/aws-es-kibana/node_modules/raw-body/index.js:196:17)
    at getRawBody (/Users/user/.nvm/versions/node/v6.2.2/lib/node_modules/aws-es-kibana/node_modules/raw-body/index.js:106:12)
    at read (/Users/user/.nvm/versions/node/v6.2.2/lib/node_modules/aws-es-kibana/node_modules/body-parser/lib/read.js:76:3)
    at rawParser (/Users/user/.nvm/versions/node/v6.2.2/lib/node_modules/aws-es-kibana/node_modules/body-parser/lib/types/raw.js:81:5)
    at Layer.handle [as handle_request] (/Users/user/.nvm/versions/node/v6.2.2/lib/node_modules/aws-es-kibana/node_modules/express/lib/router/layer.js:95:5)
    at trim_prefix (/Users/user/.nvm/versions/node/v6.2.2/lib/node_modules/aws-es-kibana/node_modules/express/lib/router/index.js:312:13)
    at /Users/user/.nvm/versions/node/v6.2.2/lib/node_modules/aws-es-kibana/node_modules/express/lib/router/index.js:280:7
    at Function.process_params (/Users/user/.nvm/versions/node/v6.2.2/lib/node_modules/aws-es-kibana/node_modules/express/lib/router/index.js:330:12)
    at next (/Users/user/.nvm/versions/node/v6.2.2/lib/node_modules/aws-es-kibana/node_modules/express/lib/router/index.js:271:10)
    at expressInit (/Users/user/.nvm/versions/node/v6.2.2/lib/node_modules/aws-es-kibana/node_modules/express/lib/middleware/init.js:33:5)

Security Key Invalid Error

Hi ,

i am getting error as security key invalid when we try to access using AWS STS credentials. But using that profile credentials i am able to access all aws services. I am using that same role in other programs for signing it works fine. When i try to use the same credentials in aws-es-kibana getting security key invalid error. Kindly some one help me to solve this issue.

Mohan

Error: socket hang up

I was using thius library for a while, but suddenly it starts to throw this error:

AWS ES cluster available at http://0.0.0.0:443
Kibana available at http://0.0.0.0:443/_plugin/kibana/
/usr/lib/node_modules/aws-es-kibana/node_modules/http-proxy/lib/http-proxy/index.js:120
throw err;
^

Error: socket hang up
at connResetException (internal/errors.js:604:14)
at TLSSocket.socketCloseListener (_http_client.js:400:25)
at TLSSocket.emit (events.js:333:22)
at TLSSocket.EventEmitter.emit (domain.js:482:12)
at net.js:668:12
at TCP.done (_tls_wrap.js:556:7) {
code: 'ECONNRESET'

Is there a fix for this error?

sigv4 host causing failure in aws

looks like the Host var that you put the ENDPOINT in is now being parsed in ES.(on host it is https).
i fixed by:
npm install url -save
require('url') in index.js
add line to index.js
var TARGETHOST = url.parse(TARGET).hostname;
Use that for populating Host header.

413 Entity too large

I am leveraging a proxy solution upstream of this service. Unfortunately my proxy inserts some rather large cookies and headers. I believe express is not happy about the size of the request header and is returning a 413 entity too large. Is there any option we can use to tune the request body or headers maximum size? Sorry if this has been asked already, I am not too familiar with NodeJS.

Well known USER environment variable overrides AUTH_USER

USER is set in most shells, including in Cloud Foundry and Heroku and will override AUTH_USER if set which can be confusing.

$ USER=vcap AUTH_USER=kibana AUTH_PASSWORD=secret aws-es-kibana --region us-gov-west-1 https://localhost:51409

Try to login with kibana/secret and you'll get a 401.

Pull request #48 breaks authentication using environment variables

This only applies to the master branch.

Was investigating the feasibility of using the basic auth fix under #49 but ran into issues with building/running under docker.

Error log

AWS ES cluster available at http://0.0.0.0:9200
Kibana available at http://0.0.0.0:9200/_plugin/kibana/
fs.js:1384
throw error;
^
Error: watch /root/.aws/credentials ENOENT
at _errnoException (util.js:992:11)
at FSWatcher.start (fs.js:1382:19)
at Object.fs.watch (fs.js:1408:11)
at Object.<anonymous> (/app/index.js:219:4)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
at Function.Module.runMain (module.js:693:10)

PR: #48

Cause: 2f1f6db#diff-168726dbe96b3ce427e7fedce31bb0bcR205

basic http authentication disabled

I think this commit completely disabled http authantication.

This leaves ES completely open to the public, even when deployed on heroku.

Am I missing something?

Amir

Custom timeout

It would be nice to expose proxy server's timeout as an env var

kibana maps visualization don't load

Hi,

I have created a co-ordinate map visualization in Kibana. However, the map continues to show white/blank background. On the developer console in chrome I am seeing example.com/v1/default/4/3/8.png:1 Failed to load resource: the server responded with a status of 404 ()
The network tab shows that the call is being initiated by kibana.bundle.js

I was wondering whether this could be because aws-es-kibana is acting as a proxy which is preventing kibana from accessing whatever cdn it uses.

Is there any way to resolve this?

no such file or directory, watch '/Users/mehrajuddin/.aws/credentials'

Step#1: sudo npm install -g aws-es-kibana
Step#2: export the accessKey and secretKey

Step#3: aws-es-kibana cluster-address

internal/fs/watchers.js:226
throw error;
^

Error: ENOENT: no such file or directory, watch '/Users/mehrajuddin/.aws/credentials'
at FSWatcher. (internal/fs/watchers.js:218:26)
at Object.watch (fs.js:1582:34)
at Object. (/usr/local/lib/node_modules/aws-es-kibana/index.js:211:4)
at Module._compile (internal/modules/cjs/loader.js:1085:14)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
at Module.load (internal/modules/cjs/loader.js:950:32)
at Function.Module._load (internal/modules/cjs/loader.js:790:14)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:76:12)
at internal/main/run_main_module.js:17:47 {
errno: -2,
syscall: 'watch',
code: 'ENOENT',
path: '/Users/mehrajuddin/.aws/credentials',
filename: '/Users/mehrajuddin/.aws/credentials'

Error: request entity too large

I'm getting the following error. Is there a way to increase the request size?

Error: request entity too large
    at readStream (/app/node_modules/raw-body/index.js:196:17)
    at getRawBody (/app/node_modules/raw-body/index.js:106:12)
    at read (/app/node_modules/body-parser/lib/read.js:76:3)
    at rawParser (/app/node_modules/body-parser/lib/types/raw.js:81:5)
    at Layer.handle [as handle_request] (/app/node_modules/express/lib/router/layer.js:95:5)
    at trim_prefix (/app/node_modules/express/lib/router/index.js:317:13)
    at /app/node_modules/express/lib/router/index.js:284:7
    at Function.process_params (/app/node_modules/express/lib/router/index.js:335:12)
    at next (/app/node_modules/express/lib/router/index.js:275:10)
    at compression (/app/node_modules/compression/index.js:220:5)