h21lab / tsharkvm Goto Github PK
View Code? Open in Web Editor NEWtshark + ELK analytics virtual machine
Home Page: https://www.h21lab.com/tools/tshark-elasticsearch
License: Apache License 2.0
tshark + ELK analytics virtual machine
Home Page: https://www.h21lab.com/tools/tshark-elasticsearch
License: Apache License 2.0
Hi there,
the VM is up and running, however when I log into the Kibana application there are no dashboard and no index patterns pre-configured.
I use the cat index API to list the indices:
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open .geoip_databases ql1AQ2dpQkq7M3B5pfnMkw 1 0 41 0 40.1mb 40.1mb
green open .kibana-event-log-7.14.0-000001 8eOYYzqUQ4is0I3g0Y_RwA 1 0 1 0 5.6kb 5.6kb
green open .kibana_7.14.0_001 bqji6E5LSL2PeAj3iJPGVA 1 0 39 40 2.5mb 2.5mb
green open .apm-custom-link vO7HVTsET2mmpeBwGwnDzQ 1 0 0 0 208b 208b
green open .apm-agent-configuration Nv42wG6WRYq-yi48FxiUSA 1 0 0 0 208b 208b
green open .kibana_task_manager_7.14.0_001 oKEX8N2wR2uvVu876pChfg 1 0 14 11328 1.2mb 1.2mb
yellow open packets-2004.09.30 GOrWBkezT5qe2bLVg_ZWTQ 1 1 76 0 635.3kb 635.3kb
I can see that the packets index is generated correctly and there are 76 documents in it possibly from the test pcap.
I then check whether templates are generated:
name index_patterns order version composed_of
.monitoring-alerts-7 [.monitoring-alerts-7] 0 7140099
.monitoring-beats [.monitoring-beats-7-*] 0 7140099
.transform-notifications-000002 [.transform-notifications-*] 0 7140099
logstash [logstash-*] 0 60001
.kibana-event-log-7.14.0-template [.kibana-event-log-7.14.0-*] 0
.monitoring-kibana [.monitoring-kibana-7-*] 0 7140099
.monitoring-es [.monitoring-es-7-*] 0 7140099
.monitoring-logstash [.monitoring-logstash-7-*] 0 7140099
.ml-state [.ml-state*] 2147483647 7140099 []
ilm-history [ilm-history-5*] 2147483647 5 []
.slm-history [.slm-history-5*] 2147483647 5 []
synthetics [synthetics-*-*] 100 1 [synthetics-mappings, data-streams-mappings, synthetics-settings]
.ml-anomalies- [.ml-anomalies-*] 2147483647 7140099 []
metrics [metrics-*-*] 100 1 [metrics-mappings, data-streams-mappings, metrics-settings]
.ml-notifications-000002 [.ml-notifications-000002] 2147483647 7140099 []
.deprecation-indexing-template [.logs-deprecation.*] 1000 1 [.deprecation-indexing-mappings, .deprecation-indexing-settings]
packets_template [packets-*] 0 []
.watch-history-13 [.watcher-history-13*] 2147483647 13 []
logs [logs-*-*] 100 1 [logs-mappings, data-streams-mappings, logs-settings]
.ml-stats [.ml-stats-*] 2147483647 7140099 []
Which they are.
So I then import manually the saved object from the Kibana folder:
My guess is that there is a fail during the vagrant script and it doesn't load the file correctly.
Let me know if I can be on any help to debug the issue.
PS
I am not a vagrant expert.
I just tried this but ELK is using a commercial version. Do you have any plan to use oss verson? Thanks.
Let me explain first, I need to replay a lot of PCAPS from different systems and time periods. The problem is then to find all the packets related to a specific pcap file name.
It would be very helpful to have a way to add a field which contains the PCAP file name.
I have looked at the way the logstash pipeline works and I don't believe there is an easy way to achieve that because it is just receiving the pcap on the tcp socket.
I am wondering if there is an alternative way via logstash to capture the input file name maybe via http POST (which should contain that)?
Hi there,
wonderful project, I am just running it for the first time on a fresh Ubuntu 20.04 OS:
robomotic@TsharkVM:~/tsharkVM$ bash ./build.sh
rm: cannot remove './Vagrantfile': No such file or directory
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Box 'debian/buster64' could not be found. Attempting to find and install...
default: Box Provider: virtualbox
default: Box Version: >= 0
==> default: Loading metadata for box 'debian/buster64'
default: URL: https://vagrantcloud.com/debian/buster64
==> default: Adding box 'debian/buster64' (v10.20210409.1) for provider: virtualbox
default: Downloading: https://vagrantcloud.com/debian/boxes/buster64/versions/10.20210409.1/providers/virtualbox.box
==> default: Successfully added box 'debian/buster64' (v10.20210409.1) for 'virtualbox'!
There are errors in the configuration of this machine. Please fix
the following errors and try again:
Vagrant:
* Unknown configuration section 'disksize'.
./upload_pcaps.sh: connect: Connection refused
./upload_pcaps.sh: line 20: /dev/tcp/localhost/17570: Connection refused
Vagrant version: Vagrant 2.2.18
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.