Coder Social home page Coder Social logo

fiware-cybercaptor / cybercaptor-server Goto Github PK

View Code? Open in Web Editor NEW
5.0 4.0 5.0 911 KB

FIWARE Cyber seCurity Attack graPh moniTORing - Server (Computing)

Home Page: https://fiware-cybercaptor.github.io/cybercaptor-server/

License: GNU General Public License v3.0

Shell 0.01% Java 96.86% HTML 0.15% OpenEdge ABL 0.71% API Blueprint 2.27%

cybercaptor-server's Introduction

CyberCAPTOR Server

FIWARE Cyber seCurity Attack graPh moniTORing - Server

This project is part of FIWARE. For more information, please consult FIWARE website.

CyberCAPTOR is an implementation of the Cyber Security Generic Enabler, the future developments of the Security Monitoring GE.

Build Status: Build Status

Table of Contents

Development Version Installation

Prerequisite

  • Ubuntu
  • Java 1.7
  • Apache Tomcat 7
  • Apache Maven 3
  • XSB
  • MulVAL

Build

  1. Get sources from Github
git clone https://github.com/fiware-cybercaptor/cybercaptor-server.git
cd cybercaptor-server
  1. Use Maven to download dependencies and build the web application archive (.war).
mvn clean
mvn package

Installation

  1. Deploy the .war into tomcat.

Using command line

cp ./target/cybercaptor-server*.war /var/lib/tomcat7/webapps/cybercaptor-server.war

This can also be done using the tomcat GUI manager, or with Maven's tomcat7 plugin.

  1. Link the configuration and scripts repertory and fix permissions
sudo ln -s `pwd`/configuration-files /usr/share/tomcat7/.remediation
chmod -R o+rw ./configuration-files/
sudo chown -R tomcat7:tomcat7 /usr/share/tomcat7/
cd .. #Go in the parent folder of cybercaptor-server
git clone https://github.com/fiware-cybercaptor/cybercaptor-data-extraction.git # Clone the cyber-data-extraction for the "mulval-input-script-folder" parameter.
  1. Copy and edit the configuration file
cp ./configuration-files/config.properties.sample ./configuration-files/config.properties
vim ./configuration-files/config.properties

For more details, read the documentation Installation And adminsitration Manual.

Docker Version Deployment

Build container (optional)

docker build -t cybercaptor-server .

Run container

If you want to run the server in foreground, launch the following command:

docker run --rm --name cybercaptor-server -p 8000:8080 fiwarecybercaptor/cybercaptor-server

If you want to run the server in background, launch the following command:

docker run -d --name cybercaptor-server -p 8000:8080 fiwarecybercaptor/cybercaptor-server

Then, the application can be accessed at http://localhost:8000/cybercaptor-server/.

More details about building and/or running the Docker container can be found in container/README.md

Test

Go on URL : http://localhost:8080/cybercaptor-server/rest/json/initialize

If the result is {"status":"Loaded"}, the application has been properly built and installed.

For more details, read the documentation Installation And adminsitration Manual.

Debugging

Main logs files

  • /var/log/tomcat7/catalina.out
  • `pwd`/configuration-files/tmp/xsb_log.txt
  • `pwd`/configuration-files/tmp/input-generation.log

API

API usage

Version API calls

To use the CyberCAPTOR server API, the first call to test that the server is available is

curl http://localhost:8080/cybercaptor-server/rest/version/detailed

which should returns something like

{"version":"4.4"}

Initialization calls

Before using the API to manipulate the attack graph, the attack paths, and the remediations, the first call that needs to be done is

curl -c /tmp/curl.cookie http://localhost:8080/cybercaptor-server/rest/json/initialize

which loads the topology, generates the attack graph with MulVAL and computes the attack paths.

Note the -c /tmp/curl.cookie option of curl, allowing to keep the session cookie, necessary to chain calls and keep the attack graph and attack paths in session.

It is also possible to load the topology from an XML file, or a XML string containing the XML network topology, using the POST method of the /rest/json/initialize call :

Using a XML String:

curl -c /tmp/curl.cookie -H "Content-Type: application/xml" -X POST -d '<topology><machine><name>linux-user-1</name><security_requirement>7</security_requirement><interfaces><interface><name>eth0</name><ipaddress>192.168.1.111</ipaddress><vlan><name>user-lan</name><label>user-lan</label></vlan></interface></interfaces><routes><route><destination>0.0.0.0</destination><mask>0.0.0.0</mask><gateway>192.168.1.111</gateway><interface>eth0</interface></route></routes></machine><machine><name>linux-user-2</name><security_requirement>30</security_requirement><interfaces><interface><name>eth0</name><ipaddress>192.168.1.112</ipaddress><vlan><name>user-lan</name><label>user-lan</label></vlan></interface></interfaces><services><service><name>mdns</name><ipaddress>192.168.1.112</ipaddress><protocol>udp</protocol><port>5353</port><vulnerabilities><vulnerability><type>remoteExploit</type><cve>CVE-2007-2446</cve><goal>privEscalation</goal><cvss>10.0</cvss></vulnerability></vulnerabilities></service></services><routes><route><destination>0.0.0.0</destination><mask>0.0.0.0</mask><gateway>192.168.1.111</gateway><interface>eth0</interface></route></routes></machine></topology>' http://localhost:8080/cybercaptor-server/rest/json/initialize

Using a XML file:

curl -c /tmp/curl.cookie -X POST  -H "Content-Type: multipart/form-data"  -F "file=@./topology.xml" http://localhost:8080/cybercaptor-server/rest/json/initialize

The exhaustive description of this file is XML topological file is provided in https://github.com/fiware-cybercaptor/cybercaptor-data-extraction/blob/master/doc/topology-file-specifications.md. This file can be generated automatically using CyberCAPTOR-Data-Extraction.

Attack graph, attack paths and remediation calls

Then, the calls to get the attack paths, attack graph or remediations can be used:

Get the number of attack paths:

curl -b /tmp/curl.cookie http://localhost:8080/cybercaptor-server/rest/json/attack_path/number

Note the -b /tmp/curl.cookie option of curl, to load the previously saved session cookie.

Get the attack path 0:

curl -b /tmp/curl.cookie http://localhost:8080/cybercaptor-server/rest/json/attack_path/0

Get the attack graph

curl -b /tmp/curl.cookie http://localhost:8080/cybercaptor-server/rest/json/attack_graph

Get the remediations for attack path 0:

curl -b /tmp/curl.cookie http://localhost:8080/cybercaptor-server/rest/json/attack_path/0/remediations

Get the XML network topology (useful for backups):

curl -b /tmp/curl.cookie http://localhost:8080/cybercaptor-server/rest/json/topology

The full list of API calls and specifications is stored in apiary.apib and can be visualized on Apiary.io using the Apiary Blueprint format.

For more details, please refer to User & Programmers manual.

Developers

If you want to participate to the development of CyberCAPTOR-Server, all contributions are welcome.

Javadoc

The Javadoc can be found on github pages

It can be updated with Maven using

 mvn site-deploy

Don't forget to configure GitHub OAuth token in ~/.m2/settings.xml. Tokens can be generated on https://github.com/settings/tokens, with repo and user:email authorized scopes.

<settings>
      <servers>
          <server>
                <id>github</id>
                <password>OAuth token</password>
          </server>
      </servers>
</settings>

API verification

The API specified using Blueprint can be checked with the dredd tool. In order to do that, first install bredd with NPM (you should have Node.js installed).

sudo npm install -g dredd

Go in the folder in which is the dredd configuration file tools/api/dredd.yml:

cd tools/api

Execute dredd

dredd

In addition to the console reports provided by dredd, a detailed report file can be found in tools/api/report.html.

For more details, refer to the User & Programmers manual.

cybercaptor-server's People

Contributors

fxaguessy avatar

Stargazers

eliva avatar Ayush Rai avatar  avatar Su Zhang avatar  avatar

Watchers

James Cloos avatar Frederic avatar  avatar Damian H. avatar

cybercaptor-server's Issues

Cookie expiring?

I've got the cybercapture-server running in a docker container and it works with the example topology.

But after some hours the information can't be accessed anymore.

I run the listing of remediations, and it works:
[root@host1 ~]# curl -b /tmp/curl.cookie http://localhost:8080/cybercaptor-server/rest/json/attack_path/0/remediations

{"remediations":{"remediation":[{"habit_index":0,"remediation_actions":{"deployable_remediation":{"action":{"rule":"iptables -I INPUT -s 192.168.1.111/32 -d 192.168.1.112/32 -p TCP  --dport 5353:5353  -j DROP","type":"firewall-rule"},"machine":"linux-user-2"}},"cost":0},{"habit_index":0,"remediation_actions":{"deployable_remediation":{"action":{"rule":"iptables -I OUTPUT -s 192.168.1.111/32 -d 192.168.1.112/32 -p TCP  --dport 5353:5353  -j DROP","type":"firewall-rule"},"machine":"linux-user-1"}},"cost":0},{"habit_index":0,"remediation_actions":{"deployable_remediation":{"action":{"type":"patch","patchs":{"patch":"http://www.securityfocus.com/archive/1/archive/1/468542/100/0/threaded"}},"machine":"linux-user-2"}},"cost":0}]}}[root@cybercapture1 ~]#

After a night of sleep, in the same machine, same console session:
[root@host1 ~]# curl -b /tmp/curl.cookie http://localhost:8080/cybercaptor-server/rest/json/attack_path/0/remediations

{"error":"The monitoring object is empty. Did you forget to initialize it ?"}

repushing the topology with:
curl -c /tmp/curl.cookie -X POST -H "Content-Type: multipart/form-data" -F "file=@./de/cybercaptor-client/doc/dataSet.xml" http://localhost:8080/cybercaptor-server/rest/json/initialize

makes the server work again, untill after some time the monitoring object is empty again.

Is it the cookie expiring or does the server throw away the data or what is the problem?

Error in installing cybercaptor-server 4.4.3

A problem has been encountered while installing cybercaptor-server 4.4.3 following the guide document: "pwd/configuration-files" and "pwd/src/main/python/" do not exist in the project package.

Remediation Automation

Take into account the history of remediations that have been applied by administrators.
Then, give an "index of habit" to each remediations.

json/initialize yields ImportError in input-generation.log

I set up my environment for testing cybercaptor from scratch without docker image. I use XSB 3.60 and Fiware's MulVal and tomcat7. All on ubuntu 15.10.

Problem appears just after hitting http://192.168.53.89:8080/cybercaptor-server/rest/json/initialize:

{"error":"the attack graph is empty"}

  • catalina file after hit:
lut 25, 2016 2:34:33 PM org.fiware.cybercaptor.server.rest.RestJsonAPI initialise
INFO: Generating topology and mulval inputs /usr/share/tomcat7/.remediation/inputs/topology-generated.xml
lut 25, 2016 2:34:33 PM org.fiware.cybercaptor.server.api.InformationSystemManagement prepareMulVALInputs
INFO: Genering MulVAL inputs
lut 25, 2016 2:34:33 PM org.fiware.cybercaptor.server.api.InformationSystemManagement prepareMulVALInputs
INFO: Launch generation of MulVAL inputs with command : 
/usr/bin/python /home/damian/cybercaptor-data-ex-WIP/cybercaptor-data-extraction/main.py --hosts-interfaces-file /usr/share/tomcat7/.remediation/inputs/hosts-interfaces.csv --vlans-file /usr/share/tomcat7/.remediation/inputs/vlans.csv --flow-matrix-file /usr/share/tomcat7/.remediation/inputs/flow-matrix.csv --vulnerability-scan /usr/share/tomcat7/.remediation/inputs/scan.nessus --routing-file /usr/share/tomcat7/.remediation/inputs/routing.csv --mulval-output-file /usr/share/tomcat7/.remediation/tmp/mulval-input-generated.P --to-fiware-xml-topology /usr/share/tomcat7/.remediation/inputs/topology-generated.xml 
lut 25, 2016 2:34:33 PM org.fiware.cybercaptor.server.api.InformationSystemManagement prepareMulVALInputs
WARNING: A problem happened in the generation of mulval inputs
lut 25, 2016 2:34:33 PM org.fiware.cybercaptor.server.rest.RestJsonAPI initialise
INFO: Loading topology /usr/share/tomcat7/.remediation/inputs/topology-generated.xml
lut 25, 2016 2:34:33 PM org.fiware.cybercaptor.server.api.InformationSystemManagement generateAttackGraphWithMulValUsingAlreadyGeneratedMulVALInputFile
INFO: Launching MulVAL
lut 25, 2016 2:34:33 PM org.fiware.cybercaptor.server.api.InformationSystemManagement generateAttackGraphWithMulValUsingAlreadyGeneratedMulVALInputFile
INFO: Empty attack graph!
  • ocalhost.log is empty
  • /usr/share/tomcat7/.remediation/tmp has only .gitignore and input-generation.log; the latter:
Traceback (most recent call last):
  File "/home/damian/cybercaptor-data-ex-WIP/cybercaptor-data-extraction/main.py", line 21, in <module>
    from topology import Topology, FlowMatrix, VLAN
  File "/home/damian/cybercaptor-data-ex-WIP/cybercaptor-data-extraction/topology.py", line 23, in <module>
    from database import db_session
  File "/home/damian/cybercaptor-data-ex-WIP/cybercaptor-data-extraction/database.py", line 21, in <module>
    from sqlalchemy import create_engine
ImportError: No module named sqlalchemy

Data-extraction's dependences were installed (pip3). What is wrong?

Move Dockerfile to container/

Easier to maintain for the automatic build of docker containers of docker-hub, and to use a specific README.md file.

Dynamic data/alerts from SIEM?

How is the data from a SIEM supposed to come to the server so that it can be included into the dynamic risk analysis?

Allow to upload the topology XML file within /initialize REST call

Currently, the inputs (CSV files transformed into an XML input file) are statically on the server.
To change the topology, the user has to replace the CSV inputs files, on the server.

This is to add an apli call POST /initialize, in which the client uploads the topology.xml file that will be taken has input for the global CyberCAPTOR analysis.

Inconsistencies in logical/topological attack graphs/paths

I'm facing a problem with understanding attack graphs, attack paths in both: logical and topological view - I conisder generated graphs are inconsistent.

Example 1

Firstly I tried to analyze this input file from examples: https://github.com/fiware-cybercaptor/cybercaptor-client/blob/master/doc/dataSet.xml

I uploaded the xml and generated graph with no errors or warnings. Graph has only one attack path.

Example 1a: Logical Attack Graph + Attack Path

As far as I understand Attack Path differs from Attack graph in that way the directed arcs/arrows are aimed at opposite direction. In a Graph theory a Path is a subgraph of a Graph. The problem is that both generated logical graphs (attack path and attack graph) are structurally different: Major differences are marked on the picture above. A subpath containing most important vertex (the attack indicator - rule execCode()) is marked green. Marked vertices (green and purple) are directly connected to that subpath on the Attack Graph, but not on the Attack Path. Why?

Example 1b: Topological Attack Graph + Attack Path

The structure of topological attack graph and attack path have different direction of edges. Why?

Example 2

Second example is from this appendix: https://github.com/fiware-cybercaptor/cybercaptor-data-extraction/blob/master/doc/topology-file-specifications.md [end of the page]

There are generated 2 attack paths - only first is considered.
Example 2 (three screenshots): Logical and topological view of attack path and topological attack graph

Logical attack path shows that the consequence of an attack is code execution on linux-user-2 execCode('linux-user-2', user) - which is show and marked with a rectangle. Topological attack path shows that the target is linux-user-1. This looks like contradiction of targets! Isn't it?

Bottom screenshot show topological view of attack graph where one of the targets is linux-user-2.

For me these observed differences are crucial and discredit whole attack graph analysis.
Please correct me if I'm wrong, maybe I understand these graphs in wrong way, so please explain me the process of reading the results - how and why these graphs are so different. I'm desirious to involve myself into this project, it's great and after 2 weeks of fiddling with it I've got with many ideas how to improve it. If these issues are real please guide me how to fix them.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.