moshen / logstash-http-input

An Http input for Logstash <= 1.4.x (using Jetty). Logstash 1.5.0 now includes a built in http input plugin: https://github.com/logstash-plugins/logstash-input-http

Currently working at Xero

We're hiring! https://jobs.lever.co/xero?lever-via=XwiHpkoGYx

logstash-http-input's Issues

Please add HTTPS

Please add HTTPS with certificate checking.
It's needed for systemd journal-upload.

build destination folder

change plugins/logstash/inputs/http.rb => lib/logstash/inputs/http.rb

multiline support

Hi there,

I've been trying to add multiline support to my http input without success. When I try the codec=>line
is working but I need to make it work with stack traces.

Is the multiline supported with http input? Thanks.

Support for jsonp

Is it supported jsonp?

I want to send ajax to the jetty server(logstash-http-input), but it cross domain.
So with jQuery, I choose to use jsonp pattern to send ajax request.
It need response content-type: application/x-javascript; charset=utf-8.

Can I set it to the logstash-http-input?

Add a servlet to the jetty server

Can I add a servlet to the jetty server?
Because I need some extension. I need write some logic in the servlet and return something.

add support for logstash 1.5

Latest beta version of logstash 1.5 changed how contrib plugins are managed. Would like to try this awesome http plugin with new version 1.5. Thank you.

Fails to parse json

I use http input and output to forward the logs from one system to another.

The output works fairly well (aside from an easy bug to fix). On the other hand, the input plugin doesn't seem to like the directive codec and doesn't parse the logs accordingly.

Client configuration:

        http {
               format => "json"
                http_method => "post"
                url => "http://10.0.0.1:8080/"

Server configuration:

        http {
                threads => 10
               codec => "json"
        }

I made a packet capture to see what's going wrong and I can see the json is encoded correctly (validated) and the request is correct but I get 400 Bad request every single time. When I look on the server, I can see messages like this:

2015-07-30 15:27:55 -0600: HTTP parse error, malformed request (): #<Puma::HttpParserError: Invalid HTTP format, parsing fails.>
2015-07-30 15:27:55 -0600: ENV: {"rack.version"=>[1, 3], "rack.errors"=>#<IO:fd 2>, "rack.multithread"=>true, "rack.multiprocess"=>false, "rack.run_once"=>false, "SCRIPT_NAME"=>"", "CONTENT_TYPE"=>"text/plain", "QUERY_STRING"=>"", "SERVER_PROTOCOL"=>"HTTP/1.1", "SERVER_SOFTWARE"=>"2.11.3", "GATEWAY_INTERFACE"=>"CGI/1.2", "REQUEST_METHOD"=>"POST"}

which seem to indicate that the input http doesn't take the codec into account when decoding.

Disabling json on both sides fixes the issue.

How to make HTTP input plugin available to logstash? Is gem missing?

Hi,

I've followed this guide to install a plugin in logstash with your http input. According to this elastic/logstash#3208 issue, it looks like your plugin is not implementing the new gem based architecture. Is that correct?

--pluginpath becomes moot with the new ruby-gem based plugin system. Not deprecated, but moot.
That's why it has been removed.

For more information about writing plugins, see the "How to write a Logstash input|output|filter|codec]
Plugin" pages at http://www.elastic.co/guide/en/logstash/current/index.html

This is how I've tried to make your plugin available to logstash:

docker run \
  -v "$PWD/logstash-http-input/dirToCopyTheContentsOfBuildTo:/logstash-plugins"  \
  -it \
  --rm \
  -v "$PWD/logstash/config-dir":/config-dir logstash \
    logstash \
      -f /config-dir/logstash.conf \
      --pluginpath /logstash-plugins

This should mount my local copy of your code (the build version) into the /logstash-plugins directory inside the container. But apparently the --pluginpath is deprecated. See for yourself:

Clamp::UsageError: Unrecognised option '--pluginpath'
  signal_usage_error at /opt/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:103
         find_option at /opt/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/option/parsing.rb:62
       parse_options at /opt/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/option/parsing.rb:28
               parse at /opt/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:52
                 run at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.2.2-java/lib/logstash/runner.rb:80
                call at org/jruby/RubyProc.java:271
                 run at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.2.2-java/lib/logstash/runner.rb:96
                call at org/jruby/RubyProc.java:271
          initialize at /opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.20/lib/stud/task.rb:12

Regards,
Konrad