Coder Social home page Coder Social logo

h21lab / tsharkvm Goto Github PK

View Code? Open in Web Editor NEW
66.0 5.0 17.0 1.29 MB

tshark + ELK analytics virtual machine

Home Page: https://www.h21lab.com/tools/tshark-elasticsearch

License: Apache License 2.0

Ruby 19.38% Shell 43.86% Go 36.76%
tshark wireshark vm elk kibana logstash elasticsearch virtual-machine tshark-vm ndjson

tsharkvm's People

Contributors

h21lab avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

tsharkvm's Issues

Kibana dashboard and indexes are not created

Hi there,
the VM is up and running, however when I log into the Kibana application there are no dashboard and no index patterns pre-configured.
I use the cat index API to list the indices:

health status index                           uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   .geoip_databases                ql1AQ2dpQkq7M3B5pfnMkw   1   0         41            0     40.1mb         40.1mb
green  open   .kibana-event-log-7.14.0-000001 8eOYYzqUQ4is0I3g0Y_RwA   1   0          1            0      5.6kb          5.6kb
green  open   .kibana_7.14.0_001              bqji6E5LSL2PeAj3iJPGVA   1   0         39           40      2.5mb          2.5mb
green  open   .apm-custom-link                vO7HVTsET2mmpeBwGwnDzQ   1   0          0            0       208b           208b
green  open   .apm-agent-configuration        Nv42wG6WRYq-yi48FxiUSA   1   0          0            0       208b           208b
green  open   .kibana_task_manager_7.14.0_001 oKEX8N2wR2uvVu876pChfg   1   0         14        11328      1.2mb          1.2mb
yellow open   packets-2004.09.30              GOrWBkezT5qe2bLVg_ZWTQ   1   1         76            0    635.3kb        635.3kb

I can see that the packets index is generated correctly and there are 76 documents in it possibly from the test pcap.

I then check whether templates are generated:

name                              index_patterns               order      version composed_of
.monitoring-alerts-7              [.monitoring-alerts-7]       0          7140099 
.monitoring-beats                 [.monitoring-beats-7-*]      0          7140099 
.transform-notifications-000002   [.transform-notifications-*] 0          7140099 
logstash                          [logstash-*]                 0          60001   
.kibana-event-log-7.14.0-template [.kibana-event-log-7.14.0-*] 0                  
.monitoring-kibana                [.monitoring-kibana-7-*]     0          7140099 
.monitoring-es                    [.monitoring-es-7-*]         0          7140099 
.monitoring-logstash              [.monitoring-logstash-7-*]   0          7140099 
.ml-state                         [.ml-state*]                 2147483647 7140099 []
ilm-history                       [ilm-history-5*]             2147483647 5       []
.slm-history                      [.slm-history-5*]            2147483647 5       []
synthetics                        [synthetics-*-*]             100        1       [synthetics-mappings, data-streams-mappings, synthetics-settings]
.ml-anomalies-                    [.ml-anomalies-*]            2147483647 7140099 []
metrics                           [metrics-*-*]                100        1       [metrics-mappings, data-streams-mappings, metrics-settings]
.ml-notifications-000002          [.ml-notifications-000002]   2147483647 7140099 []
.deprecation-indexing-template    [.logs-deprecation.*]        1000       1       [.deprecation-indexing-mappings, .deprecation-indexing-settings]
packets_template                  [packets-*]                  0                  []
.watch-history-13                 [.watcher-history-13*]       2147483647 13      []
logs                              [logs-*-*]                   100        1       [logs-mappings, data-streams-mappings, logs-settings]
.ml-stats                         [.ml-stats-*]                2147483647 7140099 []

Which they are.

So I then import manually the saved object from the Kibana folder:

image

My guess is that there is a fail during the vagrant script and it doesn't load the file correctly.
Let me know if I can be on any help to debug the issue.

PS
I am not a vagrant expert.

need a license to use this

I just tried this but ELK is using a commercial version. Do you have any plan to use oss verson? Thanks.

Suggestion file tags!

Let me explain first, I need to replay a lot of PCAPS from different systems and time periods. The problem is then to find all the packets related to a specific pcap file name.
It would be very helpful to have a way to add a field which contains the PCAP file name.
I have looked at the way the logstash pipeline works and I don't believe there is an easy way to achieve that because it is just receiving the pcap on the tcp socket.

I am wondering if there is an alternative way via logstash to capture the input file name maybe via http POST (which should contain that)?

Unknown configuration section 'disksize'

Hi there,
wonderful project, I am just running it for the first time on a fresh Ubuntu 20.04 OS:

robomotic@TsharkVM:~/tsharkVM$ bash ./build.sh
rm: cannot remove './Vagrantfile': No such file or directory
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Box 'debian/buster64' could not be found. Attempting to find and install...
    default: Box Provider: virtualbox
    default: Box Version: >= 0
==> default: Loading metadata for box 'debian/buster64'
    default: URL: https://vagrantcloud.com/debian/buster64
==> default: Adding box 'debian/buster64' (v10.20210409.1) for provider: virtualbox
    default: Downloading: https://vagrantcloud.com/debian/boxes/buster64/versions/10.20210409.1/providers/virtualbox.box
==> default: Successfully added box 'debian/buster64' (v10.20210409.1) for 'virtualbox'!
There are errors in the configuration of this machine. Please fix
the following errors and try again:

Vagrant:
* Unknown configuration section 'disksize'.


./upload_pcaps.sh: connect: Connection refused
./upload_pcaps.sh: line 20: /dev/tcp/localhost/17570: Connection refused

Vagrant version: Vagrant 2.2.18

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.