yampelo / beagle

Beagle is an incident response and digital forensics tool which transforms security logs and data into graphs.

Beagle

1. About Beagle
2. Installation
   1. Docker
   2. Python Package
   3. Configuration
3. Web Interface
   1. Uploading Data
   2. Browsing Existing Graphs
   3. Graph Interface
      1. Inspecting Nodes and Edges
      2. Expanding Neighbours
      3. Hiding Nodes
      4. Running Mutators
      5. Toggling Node and Edge Types
      6. Undo/Redo Action and Reset
      7. Graph Perspectives
4. Python Library
   1. Controlling Edge Generation
   2. Loading Saved JSON
5. Documentation

About Beagle

Beagle is an incident response and digital forensics tool which transforms data sources and logs into graphs. Supported data sources include FireEye HX Triages, Windows EVTX files, SysMon logs and Raw Windows memory images. The resulting Graphs can be sent to graph databases such as Neo4J or DGraph, or they can be kept locally as Python NetworkX objects.

Beagle can be used directly as a python library, or through a provided web interface.

The library can be used either as a sequence of functional calls from a single datasource.

>>> from beagle.datasources import SysmonEVTX

>>> graph = SysmonEVTX("malicious.evtx").to_graph()
>>> graph
<networkx.classes.multidigraph.MultiDiGraph at 0x12700ee10>

As a graph generated from a set of multiple artifacts

>>> from beagle.datasources import SysmonEVTX, HXTriage, PCAP
>>> from beagle.backends import NetworkX

>>> nx = NetworkX.from_datasources(
    datasources=[
        SysmonEVTX("malicious.evtx"),
        HXTriage("alert.mans"),
        PCAP("traffic.pcap"),
    ]
)
>>> G = nx.graph()
<networkx.classes.multidigraph.MultiDiGraph at 0x12700ee10>

Or by strictly calling each intermediate step of the data source to graph process.

>>> from beagle.backends import NetworkX
>>> from beagle.datasources import SysmonEVTX
>>> from beagle.transformers import SysmonTransformer

>>> datasource = SysmonEVTX("malicious.evtx")

# Transformers take a datasource, and transform each event
# into a tuple of one or more nodes.
>>> transformer = SysmonTransformer(datasource=datasource)
>>> nodes = transformer.run()

# Transformers output an array of nodes.
[
    (<SysMonProc> process_guid="{0ad3e319-0c16-59c8-0000-0010d47d0000}"),
    (<File> host="DESKTOP-2C3IQHO" full_path="C:\Windows\System32\services.exe"),
    ...
]

# Backends take the nodes, and transform them into graphs
>>> backend = NetworkX(nodes=nodes)
>>> G = backend.graph()
<networkx.classes.multidigraph.MultiDiGraph at 0x126b887f0>

Graphs are centered around the activity of individual processes, and are meant primarily to help analysts investigate activity on hosts, not between them.

Installation

Docker

Beagle is available as a docker file:

docker pull yampelo/beagle
mkdir -p data/beagle
docker run -v "$PWD/data/beagle":"/data/beagle" -p 8000:8000 yampelo/beagle

Python Package

It is also available as library. Full API Documentation is available on https://beagle-graphs.readthedocs.io

pip install pybeagle

Note: Only Python 3.6+ is currently supported.

Rekall is not automatically installed. To install Rekall execute the following command instead:

pip install pybeagle[rekall]

Configuration

• Complete overview of each configuration entry

Any entry in the configuration file can be modified using environment variables that follow the following format: BEAGLE__{SECTION}__{KEY}. For example, in order to change the VirusTotal API Key used when using the docker image, you would use -e parameter and set the BEAGLE__VIRUSTOTAL__API_KEY variable:

docker run -v "data/beagle":"/data/beagle" -p 8000:8000 -e "BEAGLE__VIRUSTOTAL__API_KEY=$API_KEY" beagle

Environment variables and directories can be easily defined using docker compose

version: "3"

services:
    beagle:
        image: yampelo/beagle
        volumes:
            - /data/beagle:/data/beagle
        ports:
            - "8000:8000"
        environment:
            - BEAGLE__VIRUSTOTAL__API_KEY=$key$

Web Interface

Beagle's docker image comes with a web interface that wraps around the process of both transforming data into graphs, as well as using them to investigate data.

Uploading Data

The upload form wraps around the graph creation process, and automatically uses NetworkX as the backend. Depending on the parameters required by the data source, the form will either prompt for a file upload, or text input. For example:

• VT API Sandbox Report asks for the hash to graph.
• FireEye HX requires the HX triage.

Any graph created is stored locally in the folder defined under the dir key from the storage section in the configuration. This can be modified by setting the BEAGLE__STORAGE__DIR environment variable.

Optionally, a comment can be added to any graph to better help describe it.

Each data source will automatically extract metadata from the provided parameter. The metadata and comment are visible later on when viewing the existing graphs of the datasource.

Browsing Existing Graphs

Clicking on a datasource on the sidebar renders a table of all parsed graphs for that datasource.

Graph Interface

Viewing a graph in Beagle provides a web interface that allows analysts to quickly pivot around an incident.

The interface is split into two main parts, the left part which contains various perspectives of the graph (Graph, Tree, Table, etc), and the right part which allows you to filter nodes and edges by type, search for nodes, and expand a node's properties. It also allows you to undo and redo operations you perform on the graph.

Any element in the graph that has a divider above it is collapsible:

Inspecting Nodes and Edges

Nodes in the graph display the first 15 characters of a specific field. For example, for a process node, this will be the process name.

Edges simply show the edge type.

A single click on a node or edge will focus that node and display its information in the "Node Info" panel on the right sidebar.

Focusing on a Node

Focusing on an Edge

Expanding Neighbours

A double click on a node will pull in any neighbouring nodes. A neighbouring node is any node connected to the clicked-on node by an edge. If there are no neighbors to be pulled in, no change will be seen in the graph.

• This is regardless of direction. That means that a parent process or a child process could be pulled in when double clicking on a node.
• Beagle will only pull in 25 nodes at a time.

Hiding Nodes

A long single click on a node will hide it from the graph, as well as any edges that depend on it.

Running Mutators

Right clicking on a node exposes a context menu that allows you to run graph mutators. Mutators are functions which take the graph state, and return a new state.

Two extremely useful mutators are:

1. Backtracking a node: Find the sequence of nodes and edges that led to the creation of this node.
   • Backtracking a process node will show its process tree.
2. Expanding all descendants: From the current node, show every node that has this node as an ancestor.
   • Expanding a process node will show every child process node it spawned, any file it may have touched, and pretty much every activity that happened as a result of this node.

Backtracking a node

Backtracking a node is extremely useful and is similar to doing a root cause infection in log files.

Expanding Node Descendants

Expanding a node's descendants allows you to immediately view everything that happened because of this node. This action reveals the subgraph rooted at the selected node.

Toggling Node and Edge Types

Sometimes, a Node or Edge might not be relevant to the current incident, you can toggle edge and node types on and off. As soon as the type is toggled, the nodes or edges of that type are removed from the visible graph.

Toggling a node type off prevents that node type to be used when using mutators, or when pulling in neighbours.

Undo/Redo Action and Reset

Any action in the graph is immediately reversible! Using the undo/redo buttons you can revert any action you perform. The reset button sets the graph state to when it loaded, saving you a refresh.

Graph Perspectives

As you change the graph's current state using the above action, you might also want to view the current set of visible node and edges in a different perspective. The tabs at the top of the graph screen allow you to transform the data into a variety of views:

• Graph (Default perspective)
• Tree
• Table
• Timeline
• Markdown

Each of the perspectives supports focusing on nodes by clicking on them.

Python Library

The graph generation process can be performed programmatically using the python library. The graph generation process is made up of three steps:

1. DataSource classes parse and yield events one by one.
2. Transformer classes take those inputs and transform them into various Node classes such as Process.
3. Backend classes take the array of nodes, place them into a graph structure, and send them to a desired location.

The Python package can be installed via pip:

pip install pybeagle

Creating a graph requires chaining these together. This can be done for you using the to_graph() function.

from beagle.datasources import HXTriage

# By default, using the to_graph() class uses NetworkX and the first transformer.
G = HXTriage('test.mans').to_graph()
<networkx.classes.multidigraph.MultiDiGraph at 0x12700ee10>

It can also be done explicitly at each step. Using the functional calls, you can also define which Backend you wish to use for example, to send data to DGraph

from beagle.datasources import HXTriage
from beagle.backends import DGraph
from beagle.transformers import FireEyeHXTransformer

# The data will be sent to the DGraph instance configured in the
# configuration file
backend = HXTriage('test.mans').to_graph(backend=DGraph)

# Can also specify the transformer
backend = HXTriage('test.mans').to_transformer(transformer=FireEyeHXTransformer).to_graph(backend=DGraph)

When calling the to_graph or to_transformer methods, you can pass in any arguments to those classes:

from beagle.datasources import HXTriage
from beagle.backends import Graphistry

# Send the graphistry, anonymize the data first, and return the URL
graphistry_url = HXTriage('test.mans').to_graph(backend=Graphistry, anonymize=True, render=False)

You can also manually invoke each step in the above process, accessing the intermediary outputs

>>> from beagle.backends import NetworkX
>>> from beagle.datasources import HXTriage
>>> from beagle.transformers import FireEyeHXTransformer

>>> datasource = HXTriage("test.mans")
>>> transformer = FireEyeHXTransformer(datasource=datasource)
>>> nodes = transformer.run()
>>> backend = NetworkX(nodes=nodes)
>>> G = backend.graph()

If you want to manually call each step, you will need to ensure that the Transformer class instance is compatible with the output of the provided DataSource class.

• All Backends are compatible with all Transformers.

Each data source defines the list of transformers it is compatible with, and this can be accessed via the .transformers attribute:

>>> from beagle.datasources import HXTriage
>>> HXTriage.transformers
[beagle.transformers.fireeye_hx_transformer.FireEyeHXTransformer]

Controlling Edge Generation

By default, edges are not condensed, that means that if a process node u writes to a file node v 5000 times, you will have 5000 edges between those nodes. Sometimes, especially when trying to visualize the data, this may overwhelm an analyst.

You can condense all 5000 edges into a single edge for that type of action (wrote in this case), by passing the backend class the consolidate_edges=True parameter, for example:

SysmonEVTX("data/sysmon/autoruns-sysmon.evtx").to_graph(NetworkX, consolidate_edges=False)
# Graph contains 826 nodes and 2469 edges.

SysmonEVTX("data/sysmon/autoruns-sysmon.evtx").to_graph(NetworkX, consolidate_edges=True)
# Graph contains 826 nodes and 1396 edges

By default, the web interface will consolidate the edges.

Loading Saved JSON

Using the NetworkX backend allows you to save the resulting graph as a JSON object, saving it for later. You can use the NetworkX.from_json static method to load it back in.

import json
from beagle.datasources import SysmonEVTX
from beagle.backends import NetworkX

# Get back a nx.MultiDiGraph object
graph = SysmonEVTX("malicious.evtx").to_graph()

# Convert the object to JSON
graph_json = NetworkX.graph_to_json(graph)

with open("my_graph.json", "w") as f:
    json.dump(graph_json, f)

# Later on:
# G is the same nx.MultiDiGraph object generated by `.to_graph()`
G = NetworkX.from_json("my_graph.json")

Documentation

• REST API Overview
• Configuration
• Development
• Design Logic

beagle's Issues

Can't import results from Splunk

Hi,

First of all thanks a lot for the awesome tool,

I just have a problem with the format of fields sent from splunk to beagle, I tried wineventlog and xmleventlog, both of them are not working

here is the log snippet

2020-10-17T17:14:03 | beagle.web.api.views.new:240 | INFO | Recieved upload request for datasource=, transformer=, backend=
2020-10-17T17:14:03 | beagle.web.api.views.new:243 | INFO | Transforming data to a graph.
2020-10-17T17:14:03 | beagle.web.api.views._setup_params:441 | DEBUG | Setting up parameters
2020-10-17T17:14:03 | beagle.web.api.views._setup_params:452 | INFO | ExternalDataSource params received {'spl': 'index=botsv1 sourcetype="win*" sourcetype=wineventlog src_user="LOCAL SERVICE"', 'earliest': '0'}
2020-10-17T17:14:03 | beagle.web.api.views._setup_params:464 | DEBUG | Set up parameters
2020-10-17T17:14:03 | beagle.datasources.splunk_spl.setup_session:101 | INFO | Creating Splunk client for host=172.17.0.1
2020-10-17T17:14:03 | beagle.transformers.generic_transformer.init:20 | INFO | Created Generic Transformer.
2020-10-17T17:14:03 | beagle.transformers.base_transformer.run:77 | DEBUG | Launching transformer
2020-10-17T17:14:03 | beagle.transformers.base_transformer.run:86 | DEBUG | Started producer thread
2020-10-17T17:14:03 | beagle.transformers.base_transformer.run:98 | DEBUG | Started 3 consumer threads
2020-10-17T17:14:03 | beagle.datasources.splunk_spl.events:119 | INFO | Creating splunk search with sid=1602954843.214, waiting for job=Done
2020-10-17T17:14:03 | beagle.datasources.splunk_spl.events:122 | DEBUG | Job not done, sleeping
2020-10-17T17:14:08 | beagle.datasources.splunk_spl.events:125 | INFO | Job is done, getting results
2020-10-17T17:14:08 | beagle.datasources.splunk_spl.events:130 | INFO | Processed 3 splunk results
2020-10-17T17:14:08 | beagle.transformers.base_transformer._producer_thread:125 | DEBUG | Producer Thread Thread-1 finished after 3 events
2020-10-17T17:14:08 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-2 finished after processing 4 events
2020-10-17T17:14:08 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-4 finished after processing 1 events
2020-10-17T17:14:08 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-3 finished after processing 1 events
2020-10-17T17:14:08 | beagle.transformers.base_transformer.run:111 | INFO | Finished processing of events, created 0 nodes.
2020-10-17T17:14:08 | beagle.backends.networkx.init:56 | INFO | Initialized NetworkX Backend
2020-10-17T17:14:08 | beagle.backends.networkx.graph:73 | INFO | Beginning graph generation.
2020-10-17T17:14:08 | beagle.common._merge_batch:87 | DEBUG | Merging batch of size 0
2020-10-17T17:14:08 | beagle.common._merge_batch:105 | DEBUG | Merged down to size 0
2020-10-17T17:14:08 | beagle.backends.networkx.graph:83 | INFO | Completed graph generation.
2020-10-17T17:14:08 | beagle.backends.networkx.graph:84 | INFO | Graph contains 0 nodes and 0 edges.
2020-10-17T17:14:08 | beagle.web.api.views._create_graph:516 | INFO | Cleaning up tempfiles
2020-10-17T17:14:08 | beagle.web.api.views._create_graph:523 | INFO | Finished generating graph

Add Alert edges to generic transformer

Generic transformer is lacking the ability to transform alerts.

Calculate and show Number of occurrences for edges

Hello, can you please point me at code(or maybe this functionality already exists), where it is possible to add and show a number of occurrences for edges. I think it somewhere during of deduplication of nodes?

Crash when 'File Of' edges are selected (Windows EVTX)

I am experiencing an issue where if the 'file of' Edge Type is selected, and a 'File' is linked to a process node (in a Windows EVTX file), when I click on a 'timeline' or any of the edges do display their info, the GUI for beagle will kick me out onto a completely blank page, and my progress on the chart gets completely wiped:

The CLI which is running the docker run command is not showing any errors.

ERROR: WARNING: Failed to import pybindgen.

While building an image with the provided dockerfile:

ERROR: Complete output from command /usr/local/bin/python -u -c 'import setuptools, tokenize;file='"'"'/tmp/pip-install-k659x2dw/fastchunking/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-_qadqzwb --python-tag cp36:
ERROR: WARNING: Failed to import pybindgen. If you called setup.py egg_info,this is probably acceptable; otherwise, build will fail.You can resolve this problem by installing pybindgen beforehand.
running bdist_wheel
running build
running build_py
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/fastchunking
copying fastchunking/benchmark.py -> build/lib.linux-x86_64-3.6/fastchunking
copying fastchunking/test.py -> build/lib.linux-x86_64-3.6/fastchunking
copying fastchunking/init.py -> build/lib.linux-x86_64-3.6/fastchunking
creating build/lib.linux-x86_64-3.6/lib
copying lib/rabinkarpgen.py -> build/lib.linux-x86_64-3.6/lib
copying lib/rabinkarp_gen.py -> build/lib.linux-x86_64-3.6/lib
copying lib/init.py -> build/lib.linux-x86_64-3.6/lib
running build_ext
building 'fastchunking._rabinkarprh' extension
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/tmp
creating build/temp.linux-x86_64-3.6/tmp/pip-install-k659x2dw
creating build/temp.linux-x86_64-3.6/tmp/pip-install-k659x2dw/fastchunking
creating build/temp.linux-x86_64-3.6/tmp/pip-install-k659x2dw/fastchunking/build
creating build/temp.linux-x86_64-3.6/lib
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -Ilib -I/usr/local/include/python3.6m -c /tmp/pip-install-k659x2dw/fastchunking/build/rabinkarprh.cpp -o build/temp.linux-x86_64-3.6/tmp/pip-install-k659x2dw/fastchunking/build/rabinkarprh.o
gcc: error: /tmp/pip-install-k659x2dw/fastchunking/build/rabinkarprh.cpp: No such file or directory
gcc: fatal error: no input files
compilation terminated.
error: command 'gcc' failed with exit status 1
ERROR: Failed building wheel for fastchunking

License clarification

setup.py indicates that beagle is released under the MIT license. Does that cover the entirety of this repository? If so, recommend/request inserting that fact more prominently.

Thanks for sharing this - it's an awesome project.

duplication position

Hello,

I'm running beagle 4.1 and I'm getting this:

WARNING: Imputation of ungenotyped markers will not be performed.
Imputation requires the "gt=" argument and called genotypes.
Exception in thread "main" java.lang.IllegalArgumentException: duplication position: 1 568322 0 0
at main.PlinkGeneticMap.fillMapPositions(PlinkGeneticMap.java:77)
at main.PlinkGeneticMap.(PlinkGeneticMap.java:54)
at main.PlinkGeneticMap.fromPlinkMapFile(PlinkGeneticMap.java:154)
at main.Main.geneticMap(Main.java:360)
at main.Main.main(Main.java:109)

I checked the input file, lines themselves are not duplicated.
Can I have some advice how to deal with it?

Thanks

PCAP Display Object of type 'bytes' is not JSON serializable

I supplied Beagle a pcap from Malware-Traffic-Analysis and received the below error. Anyone else experiencing issues with importing PCAPs?

2019-10-30T15:13:34 | beagle.web.api.views.new:357 | INFO | Finished generating graph
[2019-10-30 15:13:34,373] ERROR in app: Exception on /api/new [POST]
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
raise value
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
return self.view_functionsrule.endpoint
File "/opt/beagle/beagle/web/api/views.py", line 370, in new
json.dumps(graph.to_json(), sort_keys=True).encode("utf-8")
File "/usr/local/lib/python3.6/json/init.py", line 238, in dumps
**kw).encode(obj)
File "/usr/local/lib/python3.6/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/local/lib/python3.6/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/usr/local/lib/python3.6/json/encoder.py", line 180, in default
o.class.name)
TypeError: Object of type 'bytes' is not JSON serializable

File Upload Problem

Hello,

I ran the application using the Docker file, but I'm getting this error when I want to upload a file. Where can I be making a mistake? Can you help me?

Error;

HX Triage hangs after Transformer creation

Running latest version of Docker version on MacOS and latest pull of Beagle container. I have tried to no avail to graph a HX Triage file using the web gui. There seems to be an issue with the transformer hanging. I did note the HX Triage issue reported earlier in the week, so I started the container with 4 CPU and enabling the DEBUG feature. After the transformer starts, it hangs and eventually the worker process times out and is restarted. I have also tried renaming the .mans file to "triage.mans", just as a last attempt.

....have also tried multiple triage files from various other malicious files executed in a virtual environment.

I realize that this is a finite amount of info and may be able to get you the .mans file. Data sensitivity could be an issue, possibly drop to you in email.

Thanks in advance.

CLI Output:

docker run --cpus="4" -v "$PWD/data/beagle":"/data/beagle" -p 8000:8000 -e BEAGLE_GENERAL_LOG_LEVEL=DEBUG yampelo/beagle:latest

[2019-04-11 02:46:14 +0000] [1] [INFO] Starting gunicorn 19.9.0
[2019-04-11 02:46:14 +0000] [1] [INFO] Listening at: http://0.0.0.0:8000 (1)
[2019-04-11 02:46:14 +0000] [1] [INFO] Using worker: sync
[2019-04-11 02:46:14 +0000] [9] [INFO] Booting worker with pid: 9
[2019-04-11 02:46:14 +0000] [10] [INFO] Booting worker with pid: 10
[2019-04-11 02:46:14 +0000] [11] [INFO] Booting worker with pid: 11
[2019-04-11 02:46:14 +0000] [13] [INFO] Booting worker with pid: 13
[2019-04-11 02:46:14 +0000] [12] [INFO] Booting worker with pid: 12
[2019-04-11 02:46:14 +0000] [15] [INFO] Booting worker with pid: 15
[2019-04-11 02:46:14 +0000] [14] [INFO] Booting worker with pid: 14
[2019-04-11 02:46:14 +0000] [16] [INFO] Booting worker with pid: 16
[2019-04-11 02:46:14 +0000] [17] [INFO] Booting worker with pid: 17
[2019-04-11 02:46:14 +0000] [18] [INFO] Booting worker with pid: 18
[2019-04-11 02:46:14 +0000] [19] [INFO] Booting worker with pid: 19
[2019-04-11 02:46:14 +0000] [20] [INFO] Booting worker with pid: 20
2019-04-11T02:46:39 | beagle.web.api.views.new:208 | INFO | Recieved upload request for datasource=, transformer=
2019-04-11T02:46:39 | beagle.web.api.views.new:245 | INFO | Transforming data to a graph.
2019-04-11T02:46:39 | beagle.web.api.views.new:274 | INFO | Saved uploaded files {'triage': <tempfile._TemporaryFileWrapper object at 0x7f37f66264a8>}
2019-04-11T02:46:39 | beagle.datasources.hx_triage.init:53 | INFO | Setting up HXTriage for /tmp/tmpo5cbvy_0
2019-04-11T02:46:42 | beagle.transformers.fireeye_hx_transformer.init:17 | INFO | Created FireEyeHX Transformer.

Exporting to MISP objects with relationships

Beagle is really cool. Looking at it, it could make sense to export the result of the analysis and especially a graph in MISP objects format with relationships (it's a kind of graph) into MISP. This would allow users to share investigations and discoveries.

Sysmon dns events

Error on parse: time data , does not match format '%Y-%m-%d %H:%M:%S'

I've an issue while upload windows event logs to beagle it always gives this error. and these logs are exported from the windows machine and parsed already before.
So how to fix this issue?

thanks a lot.

Lazy importing to speed up import times

OSQuery DataSource + Transformer

TC E3 Event Timestamp convert

The timestamp in TC E3 can not be converted correctly. The edge information always shows 'Invalid Date'.

Timesketch Backend

Export the graph to a timesketch server:

references:
https://github.com/google/timesketch/blob/master/docs/CreateTimelineFromJSONorCSV.md
google/timesketch#601
https://github.com/deralexxx/redline2timesketch

Readline package error per Python install

pip install pybeagle fails at readline package
Collecting readline==6.2.4.1 (from pybeagle)
Downloading https://files.pythonhosted.org/packages/f4/01/2cf081af8d880b44939a5f1b446551a7f8d59eae414277fd0c303757ff1b/readline-6.2.4.1.tar.gz
Complete output from command python setup.py egg_info:
error: this module is not meant to work on Windows

Suggest a call for pyreadline for Windows installs.

Getting [Error] in gunicorn/arbiter.py - gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>

Hello, I am trying to use beagle on windows using Docker Tools. So far I was able to run the code were graph table is created. However, afterwards I am witnessing the above same error all the time, even-though I have remove the db and containers but still i am unable to get rid of this issue.

Please find the below error logs:

$ docker run -v "$PWD/data/beagle":"/data/beagle" -p 8000:8000 yampelo/beagle:latest
[2019-04-18 12:53:33 +0000] [1] [INFO] Starting gunicorn 19.9.0
[2019-04-18 12:53:33 +0000] [1] [INFO] Listening at: http://0.0.0.0:8000 (1)
[2019-04-18 12:53:33 +0000] [1] [INFO] Using worker: sync
[2019-04-18 12:53:33 +0000] [8] [INFO] Booting worker with pid: 8
[2019-04-18 12:53:33 +0000] [9] [INFO] Booting worker with pid: 9
[2019-04-18 12:53:33 +0000] [10] [INFO] Booting worker with pid: 10

[2019-04-18 12:53:33 +0000] [11] [INFO] Booting worker with pid: 11
[2019-04-18 12:53:33 +0000] [12] [INFO] Booting worker with pid: 12
[2019-04-18 12:53:33 +0000] [13] [INFO] Booting worker with pid: 13
[2019-04-18 12:53:33 +0000] [14] [INFO] Booting worker with pid: 14
[2019-04-18 12:53:33 +0000] [15] [INFO] Booting worker with pid: 15
[2019-04-18 12:53:33 +0000] [16] [INFO] Booting worker with pid: 16
[2019-04-18 12:53:33 +0000] [17] [INFO] Booting worker with pid: 17
[2019-04-18 12:53:33 +0000] [18] [INFO] Booting worker with pid: 18
[2019-04-18 12:53:33 +0000] [19] [INFO] Booting worker with pid: 19
[2019-04-18 12:53:52 +0000] [9] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: table graph already exists

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
import(module)
File "/opt/beagle/beagle/web/wsgi.py", line 3, in
app = create_app()
File "/opt/beagle/beagle/web/server.py", line 48, in create_app
db.create_all()
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 963, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 955, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4287, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2033, in _run_visitor
conn._run_visitor(visitorcallable, element, **kwargs)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1607, in _run_visitor
visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
return meth(obj, **kw)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 781, in visit_metadata
_is_metadata_operation=True,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
return meth(obj, **kw)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 826, in visit_table
include_foreign_key_constraints,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
return connection._execute_ddl(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1050, in _execute_ddl
compiled,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
e, statement, parameters, cursor, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table graph already exists
[SQL:
CREATE TABLE graph (
id INTEGER NOT NULL,
sha256 VARCHAR(255) NOT NULL,
meta VARCHAR NOT NULL,
category VARCHAR(255) NOT NULL,
comment VARCHAR(255),
file_path VARCHAR(255) NOT NULL,
PRIMARY KEY (id),
UNIQUE (sha256),
UNIQUE (file_path)
)

]
(Background on this error at: http://sqlalche.me/e/e3q8)
[2019-04-18 12:53:52 +0000] [15] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: table graph already exists

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
import(module)
File "/opt/beagle/beagle/web/wsgi.py", line 3, in
app = create_app()
File "/opt/beagle/beagle/web/server.py", line 48, in create_app
db.create_all()
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 963, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 955, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4287, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2033, in _run_visitor
conn._run_visitor(visitorcallable, element, **kwargs)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1607, in _run_visitor
visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
return meth(obj, **kw)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 781, in visit_metadata
_is_metadata_operation=True,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
return meth(obj, **kw)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 826, in visit_table
include_foreign_key_constraints,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
return connection._execute_ddl(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1050, in _execute_ddl
compiled,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
e, statement, parameters, cursor, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table graph already exists
[SQL:
CREATE TABLE graph (
id INTEGER NOT NULL,
sha256 VARCHAR(255) NOT NULL,
meta VARCHAR NOT NULL,
category VARCHAR(255) NOT NULL,
comment VARCHAR(255),
file_path VARCHAR(255) NOT NULL,
PRIMARY KEY (id),
UNIQUE (sha256),
UNIQUE (file_path)
)

]
(Background on this error at: http://sqlalche.me/e/e3q8)
[2019-04-18 12:53:52 +0000] [18] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: table graph already exists

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
import(module)
File "/opt/beagle/beagle/web/wsgi.py", line 3, in
app = create_app()
File "/opt/beagle/beagle/web/server.py", line 48, in create_app
db.create_all()
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 963, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 955, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4287, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2033, in _run_visitor
conn._run_visitor(visitorcallable, element, **kwargs)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1607, in _run_visitor
visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
return meth(obj, **kw)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 781, in visit_metadata
_is_metadata_operation=True,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
return meth(obj, **kw)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 826, in visit_table
include_foreign_key_constraints,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
return connection._execute_ddl(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1050, in _execute_ddl
compiled,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
e, statement, parameters, cursor, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table graph already exists
[SQL:
CREATE TABLE graph (
id INTEGER NOT NULL,
sha256 VARCHAR(255) NOT NULL,
meta VARCHAR NOT NULL,
category VARCHAR(255) NOT NULL,
comment VARCHAR(255),
file_path VARCHAR(255) NOT NULL,
PRIMARY KEY (id),
UNIQUE (sha256),
UNIQUE (file_path)
)

]
(Background on this error at: http://sqlalche.me/e/e3q8)
[2019-04-18 12:53:52 +0000] [12] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: table graph already exists

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
import(module)
File "/opt/beagle/beagle/web/wsgi.py", line 3, in
app = create_app()
File "/opt/beagle/beagle/web/server.py", line 48, in create_app
db.create_all()
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 963, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 955, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4287, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2033, in _run_visitor
conn._run_visitor(visitorcallable, element, **kwargs)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1607, in _run_visitor
visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
return meth(obj, **kw)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 781, in visit_metadata
_is_metadata_operation=True,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
return meth(obj, **kw)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 826, in visit_table
include_foreign_key_constraints,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
return connection._execute_ddl(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1050, in _execute_ddl
compiled,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
e, statement, parameters, cursor, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table graph already exists
[SQL:
CREATE TABLE graph (
id INTEGER NOT NULL,
sha256 VARCHAR(255) NOT NULL,
meta VARCHAR NOT NULL,
category VARCHAR(255) NOT NULL,
comment VARCHAR(255),
file_path VARCHAR(255) NOT NULL,
PRIMARY KEY (id),
UNIQUE (sha256),
UNIQUE (file_path)
)

]
(Background on this error at: http://sqlalche.me/e/e3q8)
[2019-04-18 12:53:52 +0000] [11] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: table graph already exists

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
import(module)
File "/opt/beagle/beagle/web/wsgi.py", line 3, in
app = create_app()
File "/opt/beagle/beagle/web/server.py", line 48, in create_app
db.create_all()
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 963, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 955, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4287, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2033, in _run_visitor
conn._run_visitor(visitorcallable, element, **kwargs)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1607, in _run_visitor
visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
return meth(obj, **kw)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 781, in visit_metadata
_is_metadata_operation=True,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
return meth(obj, **kw)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 826, in visit_table
include_foreign_key_constraints,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
return connection._execute_ddl(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1050, in _execute_ddl
compiled,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
e, statement, parameters, cursor, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table graph already exists
[SQL:
CREATE TABLE graph (
id INTEGER NOT NULL,
sha256 VARCHAR(255) NOT NULL,
meta VARCHAR NOT NULL,
category VARCHAR(255) NOT NULL,
comment VARCHAR(255),
file_path VARCHAR(255) NOT NULL,
PRIMARY KEY (id),
UNIQUE (sha256),
UNIQUE (file_path)
)

]
(Background on this error at: http://sqlalche.me/e/e3q8)
[2019-04-18 12:53:52 +0000] [19] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: table graph already exists

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
import(module)
File "/opt/beagle/beagle/web/wsgi.py", line 3, in
app = create_app()
File "/opt/beagle/beagle/web/server.py", line 48, in create_app
db.create_all()
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 963, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 955, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4287, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2033, in _run_visitor
conn._run_visitor(visitorcallable, element, **kwargs)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1607, in _run_visitor
visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
return meth(obj, **kw)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 781, in visit_metadata
_is_metadata_operation=True,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
return meth(obj, **kw)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 826, in visit_table
include_foreign_key_constraints,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
return connection._execute_ddl(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1050, in _execute_ddl
compiled,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
e, statement, parameters, cursor, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table graph already exists
[SQL:
CREATE TABLE graph (
id INTEGER NOT NULL,
sha256 VARCHAR(255) NOT NULL,
meta VARCHAR NOT NULL,
category VARCHAR(255) NOT NULL,
comment VARCHAR(255),
file_path VARCHAR(255) NOT NULL,
PRIMARY KEY (id),
UNIQUE (sha256),
UNIQUE (file_path)
)

]
(Background on this error at: http://sqlalche.me/e/e3q8)
[2019-04-18 12:53:52 +0000] [8] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: table graph already exists

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
import(module)
File "/opt/beagle/beagle/web/wsgi.py", line 3, in
app = create_app()
File "/opt/beagle/beagle/web/server.py", line 48, in create_app
db.create_all()
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 963, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 955, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4287, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2033, in _run_visitor
conn._run_visitor(visitorcallable, element, **kwargs)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1607, in _run_visitor
visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
return meth(obj, **kw)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 781, in visit_metadata
_is_metadata_operation=True,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
return meth(obj, **kw)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 826, in visit_table
include_foreign_key_constraints,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
return connection._execute_ddl(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1050, in _execute_ddl
compiled,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
e, statement, parameters, cursor, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table graph already exists
[SQL:
CREATE TABLE graph (
id INTEGER NOT NULL,
sha256 VARCHAR(255) NOT NULL,
meta VARCHAR NOT NULL,
category VARCHAR(255) NOT NULL,
comment VARCHAR(255),
file_path VARCHAR(255) NOT NULL,
PRIMARY KEY (id),
UNIQUE (sha256),
UNIQUE (file_path)
)

]
(Background on this error at: http://sqlalche.me/e/e3q8)
[2019-04-18 12:53:52 +0000] [17] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: table graph already exists

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
import(module)
File "/opt/beagle/beagle/web/wsgi.py", line 3, in
app = create_app()
File "/opt/beagle/beagle/web/server.py", line 48, in create_app
db.create_all()
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 963, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 955, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4287, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2033, in _run_visitor
conn._run_visitor(visitorcallable, element, **kwargs)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1607, in _run_visitor
visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
return meth(obj, **kw)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 781, in visit_metadata
_is_metadata_operation=True,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
return meth(obj, **kw)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 826, in visit_table
include_foreign_key_constraints,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
return connection._execute_ddl(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1050, in _execute_ddl
compiled,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
e, statement, parameters, cursor, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table graph already exists
[SQL:
CREATE TABLE graph (
id INTEGER NOT NULL,
sha256 VARCHAR(255) NOT NULL,
meta VARCHAR NOT NULL,
category VARCHAR(255) NOT NULL,
comment VARCHAR(255),
file_path VARCHAR(255) NOT NULL,
PRIMARY KEY (id),
UNIQUE (sha256),
UNIQUE (file_path)
)

]
(Background on this error at: http://sqlalche.me/e/e3q8)
[2019-04-18 12:53:52 +0000] [13] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: table graph already exists

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
import(module)
File "/opt/beagle/beagle/web/wsgi.py", line 3, in
app = create_app()
File "/opt/beagle/beagle/web/server.py", line 48, in create_app
db.create_all()
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 963, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 955, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4287, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2033, in _run_visitor
conn._run_visitor(visitorcallable, element, **kwargs)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1607, in _run_visitor
visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
return meth(obj, **kw)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 781, in visit_metadata
_is_metadata_operation=True,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
return meth(obj, **kw)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 826, in visit_table
include_foreign_key_constraints,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
return connection._execute_ddl(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1050, in _execute_ddl
compiled,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
e, statement, parameters, cursor, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table graph already exists
[SQL:
CREATE TABLE graph (
id INTEGER NOT NULL,
sha256 VARCHAR(255) NOT NULL,
meta VARCHAR NOT NULL,
category VARCHAR(255) NOT NULL,
comment VARCHAR(255),
file_path VARCHAR(255) NOT NULL,
PRIMARY KEY (id),
UNIQUE (sha256),
UNIQUE (file_path)
)

]
(Background on this error at: http://sqlalche.me/e/e3q8)
[2019-04-18 12:53:52 +0000] [10] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: table graph already exists

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
import(module)
File "/opt/beagle/beagle/web/wsgi.py", line 3, in
app = create_app()
File "/opt/beagle/beagle/web/server.py", line 48, in create_app
db.create_all()
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 963, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 955, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4287, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2033, in _run_visitor
conn._run_visitor(visitorcallable, element, **kwargs)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1607, in _run_visitor
visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
return meth(obj, **kw)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 781, in visit_metadata
_is_metadata_operation=True,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
return meth(obj, **kw)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 826, in visit_table
include_foreign_key_constraints,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
return connection._execute_ddl(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1050, in _execute_ddl
compiled,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
e, statement, parameters, cursor, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table graph already exists
[SQL:
CREATE TABLE graph (
id INTEGER NOT NULL,
sha256 VARCHAR(255) NOT NULL,
meta VARCHAR NOT NULL,
category VARCHAR(255) NOT NULL,
comment VARCHAR(255),
file_path VARCHAR(255) NOT NULL,
PRIMARY KEY (id),
UNIQUE (sha256),
UNIQUE (file_path)
)

]
(Background on this error at: http://sqlalche.me/e/e3q8)
[2019-04-18 12:53:52 +0000] [14] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: table graph already exists

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
import(module)
File "/opt/beagle/beagle/web/wsgi.py", line 3, in
app = create_app()
File "/opt/beagle/beagle/web/server.py", line 48, in create_app
db.create_all()
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 963, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 955, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4287, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2033, in _run_visitor
conn._run_visitor(visitorcallable, element, **kwargs)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1607, in _run_visitor
visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
return meth(obj, **kw)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 781, in visit_metadata
_is_metadata_operation=True,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
return meth(obj, **kw)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 826, in visit_table
include_foreign_key_constraints,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
return connection._execute_ddl(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1050, in _execute_ddl
compiled,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
e, statement, parameters, cursor, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table graph already exists
[SQL:
CREATE TABLE graph (
id INTEGER NOT NULL,
sha256 VARCHAR(255) NOT NULL,
meta VARCHAR NOT NULL,
category VARCHAR(255) NOT NULL,
comment VARCHAR(255),
file_path VARCHAR(255) NOT NULL,
PRIMARY KEY (id),
UNIQUE (sha256),
UNIQUE (file_path)
)

]
(Background on this error at: http://sqlalche.me/e/e3q8)
[2019-04-18 12:53:52 +0000] [9] [INFO] Worker exiting (pid: 9)
[2019-04-18 12:53:52 +0000] [19] [INFO] Worker exiting (pid: 19)
[2019-04-18 12:53:52 +0000] [8] [INFO] Worker exiting (pid: 8)
[2019-04-18 12:53:52 +0000] [12] [INFO] Worker exiting (pid: 12)
[2019-04-18 12:53:52 +0000] [15] [INFO] Worker exiting (pid: 15)
[2019-04-18 12:53:52 +0000] [17] [INFO] Worker exiting (pid: 17)
[2019-04-18 12:53:52 +0000] [11] [INFO] Worker exiting (pid: 11)
[2019-04-18 12:53:52 +0000] [14] [INFO] Worker exiting (pid: 14)
[2019-04-18 12:53:52 +0000] [18] [INFO] Worker exiting (pid: 18)
[2019-04-18 12:53:52 +0000] [13] [INFO] Worker exiting (pid: 13)
[2019-04-18 12:53:52 +0000] [10] [INFO] Worker exiting (pid: 10)
[2019-04-18 12:53:54 +0000] [16] [INFO] Worker exiting (pid: 16)
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 210, in run
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 360, in sleep
ready = select.select([self.PIPE[0]], [], [], 1.0)
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 245, in handle_chld
self.reap_workers()
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 525, in reap_workers
raise HaltServer(reason, self.WORKER_BOOT_ERROR)
gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/bin/gunicorn", line 10, in
sys.exit(run())
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 61, in run
WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 223, in run
super(Application, self).run()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 72, in run
Arbiter(self).run()
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 232, in run
self.halt(reason=inst.reason, exit_status=inst.exit_status)
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 345, in halt
self.stop()
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 393, in stop
time.sleep(0.1)
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 245, in handle_chld
self.reap_workers()
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 525, in reap_workers
raise HaltServer(reason, self.WORKER_BOOT_ERROR)
gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>

Update to installation guide

While going through the recent PR and issue conversation, I realized that it might be helpful to update the installation guidelines with the following:

1. Only Python 3.6 is currently supported: ref #5 #14
2. Rekall can be installed as an extra using pip install pybeagle[rekall] ref: #19

Let me know if this is alright. If so, I can make a PR for the issue.

Thanks!

Speed up EVTX Parsing

Move over to https://github.com/omerbenamram/pyevtx-rs

Double Byte in HX Triage file support?

Hi,
HX triages files from double byte windows (Chinese Windows).
Beagle show me the following errors while I upload the .mans file.

Error on parse: 'utf-8' codec can't decode byte 0x87 in position 16: invalid start byte

Regards,
Nicholas

HX Triage parsing timeout in Beagle web gui

Hello,

when uploading HX triage files over the web gui, apparently there is some initial parsing being done (see "MATCHED UP" log entries below). However, the worker times out after a couple of minutes, and no results are found under beagle:8000/fireeye_hx ("No Entries!").

2019-04-02T08:10:33 | beagle.web.api.views.new:208 | INFO | Recieved upload request for datasource=<HXTriage>, transformer=<FireEyeHXTransformer>
2019-04-02T08:10:33 | beagle.web.api.views.new:245 | INFO | Transforming data to a graph.
2019-04-02T08:10:33 | beagle.web.api.views.new:274 | INFO | Saved uploaded files {'triage': <tempfile._TemporaryFileWrapper object at 0x7f003153a240>}
2019-04-02T08:10:33 | beagle.datasources.hx_triage.__init__:53 | INFO | Setting up HXTriage for /tmp/tmpljshp3um
2019-04-02T08:10:33 | beagle.transformers.fireeye_hx_transformer.__init__:17 | INFO | Created FireEyeHX Transformer.
2019-04-02T08:10:38 | beagle.datasources.hx_triage.parse_alert_files:293 | INFO | Matched up 8f9a35e04670 to powershell from office programs
2019-04-02T08:10:38 | beagle.datasources.hx_triage.parse_alert_files:293 | INFO | Matched up b405089e5bfb to POWERSHELL DOWNLOADER C (METHODOLOGY)
2019-04-02T08:10:38 | beagle.datasources.hx_triage.parse_alert_files:293 | INFO | Matched up 5f795540aa46 to POWERSHELL DOWNLOADER (METHODOLOGY)
2019-04-02T08:10:38 | beagle.datasources.hx_triage.parse_alert_files:293 | INFO | Matched up af94a0db70fd to POWERSHELL DOWNLOADER D (METHODOLOGY)
2019-04-02T08:10:38 | beagle.datasources.hx_triage.parse_alert_files:293 | INFO | Matched up 8f9a35e04670 to powershell from office programs
[2019-04-02 08:18:54 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:13)
[2019-04-02 08:18:54 +0000] [13] [INFO] Worker exiting (pid: 13)
[2019-04-02 08:18:54 +0000] [42] [INFO] Booting worker with pid: 42

Ability to add alerts from upload page

It's possible to inject alerts using the bindings:

from beagle.nodes import Alert, Process
nodes = transformer.run(....)
alert = Alert(...)
process = Process(...)
alert.alerted_on[process].append(...)
nodes += [alert, process]
backend.graph()

but this isn't possible when only adding via the web interface.

Docker container failed to run

I have seen the other issues related to this one, however after downloading the latest image several times, (there is no beagle.db in $PWD/data/beagle to delete) i continue to get the same error

[2020-01-13 18:10:32 +0000] [1] [INFO] Starting gunicorn 19.9.0
[2020-01-13 18:10:32 +0000] [1] [INFO] Listening at: http://0.0.0.0:8000 (1)
[2020-01-13 18:10:32 +0000] [1] [INFO] Using worker: sync
[2020-01-13 18:10:32 +0000] [8] [INFO] Booting worker with pid: 8
[2020-01-13 18:10:32 +0000] [9] [INFO] Booting worker with pid: 9
[2020-01-13 18:10:32 +0000] [10] [INFO] Booting worker with pid: 10
[2020-01-13 18:10:32 +0000] [11] [INFO] Booting worker with pid: 11
[2020-01-13 18:10:32 +0000] [12] [INFO] Booting worker with pid: 12
[2020-01-13 18:10:32 +0000] [13] [INFO] Booting worker with pid: 13
[2020-01-13 18:10:32 +0000] [14] [INFO] Booting worker with pid: 14
[2020-01-13 18:10:32 +0000] [15] [INFO] Booting worker with pid: 15
[2020-01-13 18:10:32 +0000] [16] [INFO] Booting worker with pid: 16
[2020-01-13 18:10:32 +0000] [17] [INFO] Booting worker with pid: 17
[2020-01-13 18:10:32 +0000] [18] [INFO] Booting worker with pid: 18
[2020-01-13 18:10:32 +0000] [19] [INFO] Booting worker with pid: 19
[2020-01-13 18:10:51 +0000] [12] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 354, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 237, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 428, in init
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 453, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlite3.OperationalError: unable to open database file

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
import(module)
File "/opt/beagle/beagle/web/wsgi.py", line 3, in
app = create_app()
File "/opt/beagle/beagle/web/server.py", line 51, in create_app
db.create_all()
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 963, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 955, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4287, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2032, in _run_visitor
with self._optional_conn_ctx_manager(connection) as conn:
File "/usr/local/lib/python3.6/contextlib.py", line 81, in enter
return next(self.gen)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2024, in _optional_conn_ctx_manager
with self._contextual_connect() as conn:
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2226, in _contextual_connect
self._wrap_pool_connect(self.pool.connect, None),
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2266, in _wrap_pool_connect
e, dialect, self
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1536, in _handle_dbapi_exception_noconnection
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 354, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 237, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 428, in init
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 453, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file
(Background on this error at: http://sqlalche.me/e/e3q8)
[2020-01-13 18:10:51 +0000] [10] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 354, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 237, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 428, in init
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 453, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlite3.OperationalError: unable to open database file

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
import(module)
File "/opt/beagle/beagle/web/wsgi.py", line 3, in
app = create_app()
File "/opt/beagle/beagle/web/server.py", line 51, in create_app
db.create_all()
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 963, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 955, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4287, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2032, in _run_visitor
with self._optional_conn_ctx_manager(connection) as conn:
File "/usr/local/lib/python3.6/contextlib.py", line 81, in enter
return next(self.gen)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2024, in _optional_conn_ctx_manager
with self._contextual_connect() as conn:
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2226, in _contextual_connect
self._wrap_pool_connect(self.pool.connect, None),
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2266, in _wrap_pool_connect
e, dialect, self
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1536, in _handle_dbapi_exception_noconnection
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 354, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 237, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 428, in init
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 453, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file
(Background on this error at: http://sqlalche.me/e/e3q8)
[2020-01-13 18:10:51 +0000] [14] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 354, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 237, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 428, in init
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 453, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlite3.OperationalError: unable to open database file

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
import(module)
File "/opt/beagle/beagle/web/wsgi.py", line 3, in
app = create_app()
File "/opt/beagle/beagle/web/server.py", line 51, in create_app
db.create_all()
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 963, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 955, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4287, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2032, in _run_visitor
with self._optional_conn_ctx_manager(connection) as conn:
File "/usr/local/lib/python3.6/contextlib.py", line 81, in enter
return next(self.gen)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2024, in _optional_conn_ctx_manager
with self._contextual_connect() as conn:
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2226, in _contextual_connect
self._wrap_pool_connect(self.pool.connect, None),
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2266, in _wrap_pool_connect
e, dialect, self
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1536, in _handle_dbapi_exception_noconnection
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 354, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 237, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 428, in init
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 453, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file
(Background on this error at: http://sqlalche.me/e/e3q8)
[2020-01-13 18:10:51 +0000] [14] [INFO] Worker exiting (pid: 14)
[2020-01-13 18:10:51 +0000] [15] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 354, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 237, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 428, in init
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 453, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlite3.OperationalError: unable to open database file

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
import(module)
File "/opt/beagle/beagle/web/wsgi.py", line 3, in
app = create_app()
File "/opt/beagle/beagle/web/server.py", line 51, in create_app
db.create_all()
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 963, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 955, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4287, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2032, in _run_visitor
with self._optional_conn_ctx_manager(connection) as conn:
File "/usr/local/lib/python3.6/contextlib.py", line 81, in enter
return next(self.gen)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2024, in _optional_conn_ctx_manager
with self._contextual_connect() as conn:
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2226, in _contextual_connect
self._wrap_pool_connect(self.pool.connect, None),
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2266, in _wrap_pool_connect
e, dialect, self
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1536, in _handle_dbapi_exception_noconnection
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 354, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 237, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 428, in init
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 453, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file
(Background on this error at: http://sqlalche.me/e/e3q8)
[2020-01-13 18:10:51 +0000] [19] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 354, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 237, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 428, in init
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 453, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlite3.OperationalError: unable to open database file

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
import(module)
File "/opt/beagle/beagle/web/wsgi.py", line 3, in
app = create_app()
File "/opt/beagle/beagle/web/server.py", line 51, in create_app
db.create_all()
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 963, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 955, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4287, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2032, in _run_visitor
with self._optional_conn_ctx_manager(connection) as conn:
File "/usr/local/lib/python3.6/contextlib.py", line 81, in enter
return next(self.gen)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2024, in _optional_conn_ctx_manager
with self._contextual_connect() as conn:
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2226, in _contextual_connect
self._wrap_pool_connect(self.pool.connect, None),
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2266, in _wrap_pool_connect
e, dialect, self
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1536, in _handle_dbapi_exception_noconnection
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 354, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 237, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 428, in init
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 453, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file
(Background on this error at: http://sqlalche.me/e/e3q8)
[2020-01-13 18:10:51 +0000] [8] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 354, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 237, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 428, in init
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 453, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlite3.OperationalError: unable to open database file

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
import(module)
File "/opt/beagle/beagle/web/wsgi.py", line 3, in
app = create_app()
File "/opt/beagle/beagle/web/server.py", line 51, in create_app
db.create_all()
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 963, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 955, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4287, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2032, in _run_visitor
with self._optional_conn_ctx_manager(connection) as conn:
File "/usr/local/lib/python3.6/contextlib.py", line 81, in enter
return next(self.gen)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2024, in _optional_conn_ctx_manager
with self._contextual_connect() as conn:
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2226, in _contextual_connect
self._wrap_pool_connect(self.pool.connect, None),
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2266, in _wrap_pool_connect
e, dialect, self
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1536, in _handle_dbapi_exception_noconnection
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 354, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 237, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 428, in init
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 453, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file
(Background on this error at: http://sqlalche.me/e/e3q8)
[2020-01-13 18:10:51 +0000] [18] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 354, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 237, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 428, in init
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 453, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlite3.OperationalError: unable to open database file

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
import(module)
File "/opt/beagle/beagle/web/wsgi.py", line 3, in
app = create_app()
File "/opt/beagle/beagle/web/server.py", line 51, in create_app
db.create_all()
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 963, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 955, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4287, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2032, in _run_visitor
with self._optional_conn_ctx_manager(connection) as conn:
File "/usr/local/lib/python3.6/contextlib.py", line 81, in enter
return next(self.gen)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2024, in _optional_conn_ctx_manager
with self._contextual_connect() as conn:
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2226, in _contextual_connect
self._wrap_pool_connect(self.pool.connect, None),
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2266, in _wrap_pool_connect
e, dialect, self
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1536, in _handle_dbapi_exception_noconnection
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 354, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 237, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 428, in init
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 453, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file
(Background on this error at: http://sqlalche.me/e/e3q8)
[2020-01-13 18:10:51 +0000] [11] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 354, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 237, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 428, in init
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 453, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlite3.OperationalError: unable to open database file

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
import(module)
File "/opt/beagle/beagle/web/wsgi.py", line 3, in
app = create_app()
File "/opt/beagle/beagle/web/server.py", line 51, in create_app
db.create_all()
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 963, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 955, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4287, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2032, in _run_visitor
with self._optional_conn_ctx_manager(connection) as conn:
File "/usr/local/lib/python3.6/contextlib.py", line 81, in enter
return next(self.gen)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2024, in _optional_conn_ctx_manager
with self._contextual_connect() as conn:
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2226, in _contextual_connect
self._wrap_pool_connect(self.pool.connect, None),
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2266, in _wrap_pool_connect
e, dialect, self
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1536, in _handle_dbapi_exception_noconnection
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 354, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 237, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 428, in init
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 453, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file
(Background on this error at: http://sqlalche.me/e/e3q8)
[2020-01-13 18:10:51 +0000] [11] [INFO] Worker exiting (pid: 11)
[2020-01-13 18:10:51 +0000] [9] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 354, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 237, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 428, in init
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 453, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlite3.OperationalError: unable to open database file

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
import(module)
File "/opt/beagle/beagle/web/wsgi.py", line 3, in
app = create_app()
File "/opt/beagle/beagle/web/server.py", line 51, in create_app
db.create_all()
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 963, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 955, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4287, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2032, in _run_visitor
with self._optional_conn_ctx_manager(connection) as conn:
File "/usr/local/lib/python3.6/contextlib.py", line 81, in enter
return next(self.gen)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2024, in _optional_conn_ctx_manager
with self._contextual_connect() as conn:
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2226, in _contextual_connect
self._wrap_pool_connect(self.pool.connect, None),
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2266, in _wrap_pool_connect
e, dialect, self
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1536, in _handle_dbapi_exception_noconnection
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 354, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 237, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 428, in init
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 453, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file
(Background on this error at: http://sqlalche.me/e/e3q8)
[2020-01-13 18:10:51 +0000] [13] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 354, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 237, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 428, in init
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 453, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlite3.OperationalError: unable to open database file

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
import(module)
File "/opt/beagle/beagle/web/wsgi.py", line 3, in
app = create_app()
File "/opt/beagle/beagle/web/server.py", line 51, in create_app
db.create_all()
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 963, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 955, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4287, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2032, in _run_visitor
with self._optional_conn_ctx_manager(connection) as conn:
File "/usr/local/lib/python3.6/contextlib.py", line 81, in enter
return next(self.gen)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2024, in _optional_conn_ctx_manager
with self._contextual_connect() as conn:
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2226, in _contextual_connect
self._wrap_pool_connect(self.pool.connect, None),
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2266, in _wrap_pool_connect
e, dialect, self
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1536, in _handle_dbapi_exception_noconnection
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 354, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 237, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 428, in init
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 453, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file
(Background on this error at: http://sqlalche.me/e/e3q8)
[2020-01-13 18:10:51 +0000] [15] [INFO] Worker exiting (pid: 15)
[2020-01-13 18:10:51 +0000] [16] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 354, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 237, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 428, in init
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 453, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlite3.OperationalError: unable to open database file

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
import(module)
File "/opt/beagle/beagle/web/wsgi.py", line 3, in
app = create_app()
File "/opt/beagle/beagle/web/server.py", line 51, in create_app
db.create_all()
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 963, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 955, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4287, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2032, in _run_visitor
with self._optional_conn_ctx_manager(connection) as conn:
File "/usr/local/lib/python3.6/contextlib.py", line 81, in enter
return next(self.gen)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2024, in _optional_conn_ctx_manager
with self._contextual_connect() as conn:
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2226, in _contextual_connect
self._wrap_pool_connect(self.pool.connect, None),
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2266, in _wrap_pool_connect
e, dialect, self
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1536, in _handle_dbapi_exception_noconnection
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 354, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 237, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 428, in init
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 453, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file
(Background on this error at: http://sqlalche.me/e/e3q8)
[2020-01-13 18:10:51 +0000] [17] [ERROR] Exception in worker process
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 354, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 237, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 428, in init
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 453, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlite3.OperationalError: unable to open database file

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker.init_process()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
self.load_wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
self.wsgi = self.app.wsgi()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
return self.load_wsgiapp()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
return util.import_app(self.app_uri)
File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
import(module)
File "/opt/beagle/beagle/web/wsgi.py", line 3, in
app = create_app()
File "/opt/beagle/beagle/web/server.py", line 51, in create_app
db.create_all()
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 963, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/init.py", line 955, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4287, in create_all
ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2032, in _run_visitor
with self._optional_conn_ctx_manager(connection) as conn:
File "/usr/local/lib/python3.6/contextlib.py", line 81, in enter
return next(self.gen)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2024, in _optional_conn_ctx_manager
with self._contextual_connect() as conn:
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2226, in _contextual_connect
self._wrap_pool_connect(self.pool.connect, None),
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2266, in _wrap_pool_connect
e, dialect, self
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1536, in _handle_dbapi_exception_noconnection
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2262, in _wrap_pool_connect
return fn()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 354, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 751, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 483, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/impl.py", line 237, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 299, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 428, in init
self.__connect(first_connect_check=True)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 630, in __connect
connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/strategies.py", line 114, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 453, in connect
return self.dbapi.connect(*cargs, **cparams)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file
(Background on this error at: http://sqlalche.me/e/e3q8)
[2020-01-13 18:10:51 +0000] [10] [INFO] Worker exiting (pid: 10)
[2020-01-13 18:10:51 +0000] [8] [INFO] Worker exiting (pid: 8)
[2020-01-13 18:10:51 +0000] [18] [INFO] Worker exiting (pid: 18)
[2020-01-13 18:10:51 +0000] [9] [INFO] Worker exiting (pid: 9)
[2020-01-13 18:10:51 +0000] [12] [INFO] Worker exiting (pid: 12)
[2020-01-13 18:10:51 +0000] [19] [INFO] Worker exiting (pid: 19)
[2020-01-13 18:10:51 +0000] [16] [INFO] Worker exiting (pid: 16)
[2020-01-13 18:10:51 +0000] [17] [INFO] Worker exiting (pid: 17)
[2020-01-13 18:10:51 +0000] [13] [INFO] Worker exiting (pid: 13)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/bin/gunicorn", line 10, in
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 61, in run
WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 223, in run
super(Application, self).run()
File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 72, in run
Arbiter(self).run()
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 227, in run
except StopIteration:
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 245, in handle_chld
self.reap_workers()
File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 525, in reap_workers
raise HaltServer(reason, self.WORKER_BOOT_ERROR)
gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>

Ability to select backend from GUI

As requested in #25

FireEye HX Triage Copmprehensive: looping graphing data

Trying to load a mans coming from a comprehensive triage (~300MB) and after pressing submit, the gui shows graphing data and no ends.

Load graph from JSON

Users should be able to load a graph generated through beagle, then saved to json using the to_json function back into a networkx object.

OSSEM Transformer

Add support to transform data in the format defined in https://github.com/Cyb3rWard0g/OSSEM/tree/master/common_information_model
by @Cyb3rWard0g

0 Nodes Created

Hi I am getting the following errors using Docker Image. It fails to create nodes for the graph. I have tried Winevent and Procmon. Ay advise?. Than You.

2019-03-27T17:04:38 | beagle.web.api.views.new:208 | INFO | Recieved upload request for datasource=, transformer=
2019-03-27T17:04:38 | beagle.web.api.views.new:245 | INFO | Transforming data to a graph.
2019-03-27T17:04:39 | beagle.web.api.views.new:274 | INFO | Saved uploaded files {'procmon_csv': <tempfile._TemporaryFileWrapper object at 0x7ff841598898>}
2019-03-27T17:04:40 | beagle.datasources.procmon_csv.init:25 | INFO | Set up ProcmonCSVs
2019-03-27T17:04:40 | beagle.transformers.base_transformer.run:107 | INFO | Finished processing of events, created 0 nodes.
2019-03-27T17:04:40 | beagle.backends.networkx.init:51 | INFO | Initialized NetworkX Backend
2019-03-27T17:04:40 | beagle.backends.networkx.graph:65 | INFO | Beginning graph generation.
2019-03-27T17:04:40 | beagle.backends.networkx.graph:71 | INFO | Completed graph generation.
2019-03-27T17:04:40 | beagle.backends.networkx.graph:72 | INFO | Graph contains 0 nodes and 0 edges.
2019-03-27T17:04:40 | beagle.web.api.views.new:305 | INFO | Finished generating graph

Feature Request: Expand support for .evtx files

Currently there is only support for process creation events, we'd like to add support for additional event ids.

We'd like to work on this issue. Can you assign it to us? @julieces @carlyabraham @alishasonawalla

Thank you!!!

Fails to start

Heh...now we know why I created the other ticket ;) Wow lot's of errors with a fresh install of docker on Ubuntu 18 via install

[2019-03-31 13:34:37 +0000] [6] [INFO] Starting gunicorn 19.9.0
[2019-03-31 13:34:37 +0000] [6] [INFO] Listening at: http://0.0.0.0:8000 (6)
[2019-03-31 13:34:37 +0000] [6] [INFO] Using worker: sync
[2019-03-31 13:34:37 +0000] [9] [INFO] Booting worker with pid: 9
[2019-03-31 13:34:37 +0000] [10] [INFO] Booting worker with pid: 10
[2019-03-31 13:34:37 +0000] [11] [INFO] Booting worker with pid: 11
[2019-03-31 13:34:37 +0000] [12] [INFO] Booting worker with pid: 12
[2019-03-31 13:34:37 +0000] [13] [INFO] Booting worker with pid: 13
[2019-03-31 13:34:37 +0000] [14] [INFO] Booting worker with pid: 14
[2019-03-31 13:34:37 +0000] [15] [INFO] Booting worker with pid: 15
[2019-03-31 13:34:37 +0000] [16] [INFO] Booting worker with pid: 16
[2019-03-31 13:34:37 +0000] [20] [INFO] Booting worker with pid: 20
[2019-03-31 13:34:37 +0000] [21] [INFO] Booting worker with pid: 21
[2019-03-31 13:34:37 +0000] [22] [INFO] Booting worker with pid: 22
[2019-03-31 13:34:37 +0000] [23] [INFO] Booting worker with pid: 23
[2019-03-31 13:34:49 +0000] [10] [ERROR] Exception in worker process

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
    cursor, statement, parameters, context
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
    cursor.execute(statement, parameters)
sqlite3.OperationalError: table graph already exists

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
    __import__(module)
  File "/opt/beagle/beagle/web/wsgi.py", line 3, in <module>
    app = create_app()
  File "/opt/beagle/beagle/web/server.py", line 48, in create_app
    db.create_all()
  File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/__init__.py", line 963, in create_all
    self._execute_for_all_tables(app, bind, 'create_all')
  File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/__init__.py", line 955, in _execute_for_all_tables
    op(bind=self.get_engine(app, bind), **extra)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4287, in create_all
    ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2033, in _run_visitor
    conn._run_visitor(visitorcallable, element, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1607, in _run_visitor
    visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
    return meth(obj, **kw)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 781, in visit_metadata
    _is_metadata_operation=True,
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
    return meth(obj, **kw)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 826, in visit_table
    include_foreign_key_constraints,
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1050, in _execute_ddl
    compiled,
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
    e, statement, parameters, cursor, context
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
    util.raise_from_cause(sqlalchemy_exception, exc_info)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
    cursor, statement, parameters, context
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table graph already exists
[SQL: 
CREATE TABLE graph (
	id INTEGER NOT NULL, 
	sha256 VARCHAR(255) NOT NULL, 
	meta VARCHAR NOT NULL, 
	category VARCHAR(255) NOT NULL, 
	comment VARCHAR(255), 
	file_path VARCHAR(255) NOT NULL, 
	PRIMARY KEY (id), 
	UNIQUE (sha256), 
	UNIQUE (file_path)
)

]
(Background on this error at: http://sqlalche.me/e/e3q8)
[2019-03-31 13:34:49 +0000] [10] [INFO] Worker exiting (pid: 10)
[2019-03-31 13:34:49 +0000] [13] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
    cursor, statement, parameters, context
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
    cursor.execute(statement, parameters)
sqlite3.OperationalError: table graph already exists

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
    __import__(module)
  File "/opt/beagle/beagle/web/wsgi.py", line 3, in <module>
    app = create_app()
  File "/opt/beagle/beagle/web/server.py", line 48, in create_app
    db.create_all()
  File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/__init__.py", line 963, in create_all
    self._execute_for_all_tables(app, bind, 'create_all')
  File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/__init__.py", line 955, in _execute_for_all_tables
    op(bind=self.get_engine(app, bind), **extra)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4287, in create_all
    ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2033, in _run_visitor
    conn._run_visitor(visitorcallable, element, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1607, in _run_visitor
    visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
    return meth(obj, **kw)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 781, in visit_metadata
    _is_metadata_operation=True,
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
    return meth(obj, **kw)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 826, in visit_table
    include_foreign_key_constraints,
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1050, in _execute_ddl
    compiled,
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
    e, statement, parameters, cursor, context
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
    util.raise_from_cause(sqlalchemy_exception, exc_info)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
    cursor, statement, parameters, context
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table graph already exists
[SQL: 
CREATE TABLE graph (
	id INTEGER NOT NULL, 
	sha256 VARCHAR(255) NOT NULL, 
	meta VARCHAR NOT NULL, 
	category VARCHAR(255) NOT NULL, 
	comment VARCHAR(255), 
	file_path VARCHAR(255) NOT NULL, 
	PRIMARY KEY (id), 
	UNIQUE (sha256), 
	UNIQUE (file_path)
)

]
(Background on this error at: http://sqlalche.me/e/e3q8)
[2019-03-31 13:34:49 +0000] [11] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
    cursor, statement, parameters, context
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
    cursor.execute(statement, parameters)
sqlite3.OperationalError: table graph already exists

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
    __import__(module)
  File "/opt/beagle/beagle/web/wsgi.py", line 3, in <module>
    app = create_app()
  File "/opt/beagle/beagle/web/server.py", line 48, in create_app
    db.create_all()
  File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/__init__.py", line 963, in create_all
    self._execute_for_all_tables(app, bind, 'create_all')
  File "/usr/local/lib/python3.6/site-packages/flask_sqlalchemy/__init__.py", line 955, in _execute_for_all_tables
    op(bind=self.get_engine(app, bind), **extra)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/schema.py", line 4287, in create_all
    ddl.SchemaGenerator, self, checkfirst=checkfirst, tables=tables
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2033, in _run_visitor
    conn._run_visitor(visitorcallable, element, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1607, in _run_visitor
    visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
    return meth(obj, **kw)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 781, in visit_metadata
    _is_metadata_operation=True,
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/visitors.py", line 131, in traverse_single
    return meth(obj, **kw)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 826, in visit_table
    include_foreign_key_constraints,
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1050, in _execute_ddl
    compiled,
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
    e, statement, parameters, cursor, context
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
    util.raise_from_cause(sqlalchemy_exception, exc_info)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
    cursor, statement, parameters, context
  File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table graph already exists
[SQL: 
CREATE TABLE graph (
	id INTEGER NOT NULL, 
	sha256 VARCHAR(255) NOT NULL, 
	meta VARCHAR NOT NULL, 
	category VARCHAR(255) NOT NULL, 
	comment VARCHAR(255), 
	file_path VARCHAR(255) NOT NULL, 
	PRIMARY KEY (id), 
	UNIQUE (sha256), 
	UNIQUE (file_path)
)

]
(Background on this error at: http://sqlalche.me/e/e3q8)
[2019-03-31 13:34:49 +0000] [11] [INFO] Worker exiting (pid: 11)
[2019-03-31 13:34:49 +0000] [13] [INFO] Worker exiting (pid: 13)
[2019-03-31 13:34:49 +0000] [9] [INFO] Worker exiting (pid: 9)
[2019-03-31 13:34:49 +0000] [15] [INFO] Worker exiting (pid: 15)
[2019-03-31 13:34:49 +0000] [20] [INFO] Worker exiting (pid: 20)
[2019-03-31 13:34:49 +0000] [23] [INFO] Worker exiting (pid: 23)
[2019-03-31 13:34:49 +0000] [12] [INFO] Worker exiting (pid: 12)
[2019-03-31 13:34:49 +0000] [21] [INFO] Worker exiting (pid: 21)
[2019-03-31 13:34:49 +0000] [14] [INFO] Worker exiting (pid: 14)
[2019-03-31 13:34:49 +0000] [16] [INFO] Worker exiting (pid: 16)
[2019-03-31 13:34:49 +0000] [22] [INFO] Worker exiting (pid: 22)
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 210, in run
    self.sleep()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 360, in sleep
    ready = select.select([self.PIPE[0]], [], [], 1.0)
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 245, in handle_chld
    self.reap_workers()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 525, in reap_workers
    raise HaltServer(reason, self.WORKER_BOOT_ERROR)
gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/gunicorn", line 11, in <module>
    load_entry_point('gunicorn==19.9.0', 'console_scripts', 'gunicorn')()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 61, in run
    WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 223, in run
    super(Application, self).run()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 72, in run
    Arbiter(self).run()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 232, in run
    self.halt(reason=inst.reason, exit_status=inst.exit_status)
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 345, in halt
    self.stop()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 393, in stop
    time.sleep(0.1)
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 245, in handle_chld
    self.reap_workers()
  File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 525, in reap_workers
    raise HaltServer(reason, self.WORKER_BOOT_ERROR)
gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>

Error to parsh UtcTime

I am uploading Sysmon EVTX file for analysis, it is in UTC time format.

It is showing:

Error on parse: 'EventData_UtcTime'.

Right report format for FireEye AX?

Hi,

Couldn't find a way to upload a correct AX report. I generated all kinds of report from FireEye AX, but none seems to be correct. FireEye AX reports may be concise/normal/extended in detail and xml, json, text and csv in format. Which is the correct one to upload to beagle?

And thank you very much for this great work!

Best regards,

sending data to neo4j

Following the guide in order to use Neo4J I eported these variables (passing through -e in docker run...) :
BEAGLE__GENERAL__LOG_LEVEL=DEBUG
BEAGLE__NEO4J__HOST=bolt://my_neo4j_ip:7687
BEAGLE__NEO4J__USERNAME=myuser
BEAGLE__NEO4J__PASSWORD=mypwd

I am able to see the debug messages, but there is no communication with Neo4j.
I see
INFO | Initialized NetworkX Backend
Am I missing something?
I tried also to use docker-compose with same variables under "environment" but result is the same.
Thanks

empty comments after some upload

Is anyone experiencing the disappear of comments after uploading some date?
In this case after 6 mans:

Data inside is present, just the table with comments has problem, probably some js not correctly loaded.
Issue persists also restarting docker.

Broken Link in README.md

There is a broken link in the README.md file, in the sentence that reads:

Right clicking on a node exposes a context menu that allows you to run graph mutators.

It appears that the word graph mutators in the above sentence is intended to link to a mutators.md file under docs. But it seems that no such file exists. Is there another document it should be pointing to? Let me know if I can help to fix this in any way :)

Feature Request: searchable fields on multi reports and correlate different reports

It would be useful to have the possibility to have node/edge search working over all reports as well the possibility to order/group/filter the reports by alert/domain/platform etc
And (probably the hardest) also the possibility to graph through different reports, so if I have same alert on different machines and I want to graph something like all machines with alert wannacry, to see if are using same files etc..

Crash when 'File Of' edges are selected (Windows EVTX).

This is a continuation of this issue: #65

Hi @yampelo
It seems like the issue was fixed when i Double click the edge node, however when I switch views from 'Graph' to Timeline (any other tab works) while I have the 'File of' selected, I get redirected to a blank page and the following error occurs in chrome:

Failed to load resource: the server responded with a status of 404 (NOT FOUND)
3:8000/api/new:1 Failed to load resource: the server responded with a status of 500 (INTERNAL SERVER ERROR)
react-dom.production.min.js:4408 TypeError: Cannot use 'in' operator to search for 'timestamp' in null
at EventTimeline.tsx:29
at Array.filter ()
at EventTimeline.tsx:29
at Array.map ()
at t.getDerivedStateFromProps (EventTimeline.tsx:44)
at or (react-dom.production.min.js:2769)
at cr (react-dom.production.min.js:2852)
at Mo (react-dom.production.min.js:3759)
at Ao (react-dom.production.min.js:3960)
at Va (react-dom.production.min.js:5514)
da @ react-dom.production.min.js:4408
react-dom.production.min.js:5940 Uncaught TypeError: Cannot use 'in' operator to search for 'timestamp' in null
at EventTimeline.tsx:29
at Array.filter ()
at EventTimeline.tsx:29
at Array.map ()
at t.getDerivedStateFromProps (EventTimeline.tsx:44)
at or (react-dom.production.min.js:2769)
at cr (react-dom.production.min.js:2852)
at Mo (react-dom.production.min.js:3759)
at Ao (react-dom.production.min.js:3960)
at Va (react-dom.production.min.js:5514)

ax report bad format

trying to upload AX report i receive

Error
Error on parse: 'malwareDetected'

Feature Request: pcap file support

Would LOVE to see the ability to import a pcap and have it see the same type of graph...maybe even a dns request which then correlates to a tcp connection. Thank you.

TC3 DataSet Support

Support for converting this dataset: https://github.com/darpa-i2o/Transparent-Computing into networkx objects

pip install pybeagle command error

When I tried to run the command pip install pybeagle to install pybeagle as a library, I got the following error:

Could not find a version that satisfies the requirement pybeagle (from versions: ) No matching distribution found for pybeagle

I'm currently trying to install pybeagle on a MacOS Mojave, with Python 3.7.1. I also tried to run the command on another Mac OS Mojave machine with Python 2.7.15 and with Ubuntu WSL on Windows 10 with Python 2.7.15 and encountered the same issue.

Any idea why that might be the case? - Thanks!

HX Triage timeout with graph generation

I am running Dockers on Mac with the latest pull of Beagle today.

I had a HX Triage MANS file that has created 870223 nodes. The graph creation times out after 50 minutes. I ran Beagle with a specification of 12 CPU cores but it did not help.

root# docker run --cpus=12.000 -v "/data/beagle":"/data/beagle" -p 8000:8000 -e "BEAGLE__GENERAL__LOG_LEVEL=DEBUG" yampelo/beagle
[2019-04-17 04:59:40 +0000] [1] [INFO] Starting gunicorn 19.9.0
[2019-04-17 04:59:40 +0000] [1] [INFO] Listening at: http://0.0.0.0:8000 (1)
[2019-04-17 04:59:40 +0000] [1] [INFO] Using worker: sync
[2019-04-17 04:59:40 +0000] [9] [INFO] Booting worker with pid: 9
[2019-04-17 04:59:40 +0000] [10] [INFO] Booting worker with pid: 10
[2019-04-17 04:59:40 +0000] [11] [INFO] Booting worker with pid: 11
[2019-04-17 04:59:40 +0000] [15] [INFO] Booting worker with pid: 15
[2019-04-17 04:59:40 +0000] [16] [INFO] Booting worker with pid: 16
[2019-04-17 04:59:40 +0000] [18] [INFO] Booting worker with pid: 18
[2019-04-17 04:59:41 +0000] [54] [INFO] Booting worker with pid: 54
[2019-04-17 04:59:41 +0000] [55] [INFO] Booting worker with pid: 55
[2019-04-17 04:59:41 +0000] [56] [INFO] Booting worker with pid: 56
[2019-04-17 04:59:41 +0000] [68] [INFO] Booting worker with pid: 68
[2019-04-17 04:59:41 +0000] [71] [INFO] Booting worker with pid: 71
[2019-04-17 04:59:41 +0000] [84] [INFO] Booting worker with pid: 84
2019-04-17T05:00:03 | beagle.web.api.views.new:208 | INFO | Recieved upload request for datasource=<HXTriage>, transformer=<FireEyeHXTransformer>
2019-04-17T05:00:03 | beagle.web.api.views.new:245 | INFO | Transforming data to a graph.
2019-04-17T05:00:03 | beagle.web.api.views.new:274 | INFO | Saved uploaded files {'triage': <tempfile._TemporaryFileWrapper object at 0x7f228edf3e48>}
2019-04-17T05:00:03 | beagle.datasources.hx_triage.__init__:53 | INFO | Setting up HXTriage for /tmp/tmp0xd99xy_
2019-04-17T05:00:03 | beagle.datasources.hx_triage.__init__:57 | DEBUG | Generated temporary directory /tmp/tmpr0jkjmx3_beagle
2019-04-17T05:00:04 | beagle.datasources.hx_triage.__init__:93 | DEBUG | Mapped 5pq8DVSJOP993LATmGmm3c to stateagentinspector
2019-04-17T05:00:04 | beagle.transformers.fireeye_hx_transformer.__init__:17 | INFO | Created FireEyeHX Transformer.
2019-04-17T05:00:04 | beagle.transformers.base_transformer.run:77 | DEBUG | Launching transformer
2019-04-17T05:00:04 | beagle.transformers.base_transformer.run:86 | DEBUG | Started producer thread
2019-04-17T05:00:04 | beagle.transformers.base_transformer.run:98 | DEBUG | Started 11 consumer threads
2019-04-17T05:01:10 | beagle.transformers.base_transformer._producer_thread:125 | DEBUG | Producer Thread Thread-1 finished after 301402 events
2019-04-17T05:01:10 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-4 finished after processing 27957 events
2019-04-17T05:01:10 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-2 finished after processing 26870 events
2019-04-17T05:01:10 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-8 finished after processing 27376 events
2019-04-17T05:01:10 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-7 finished after processing 29033 events
2019-04-17T05:01:10 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-12 finished after processing 26999 events
2019-04-17T05:01:10 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-5 finished after processing 26087 events
2019-04-17T05:01:10 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-10 finished after processing 28742 events
2019-04-17T05:01:10 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-3 finished after processing 27132 events
2019-04-17T05:01:10 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-9 finished after processing 28139 events
2019-04-17T05:01:10 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-11 finished after processing 27934 events
2019-04-17T05:01:10 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-6 finished after processing 25144 events
2019-04-17T05:01:10 | beagle.transformers.base_transformer.run:111 | INFO | Finished processing of events, created 870223 nodes.
2019-04-17T05:01:10 | beagle.backends.networkx.__init__:51 | INFO | Initialized NetworkX Backend
2019-04-17T05:01:10 | beagle.backends.networkx.graph:65 | INFO | Beginning graph generation.
[2019-04-17 05:50:03 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:11)
[2019-04-17 05:50:03 +0000] [11] [INFO] Worker exiting (pid: 11)
[2019-04-17 05:50:04 +0000] [177] [INFO] Booting worker with pid: 177

I used this CURL command to submit the MANS file. The CURL timed out after 50 minutes.

# curl -F '[email protected]' -F 'datasource=HXTriage' -F 'transformer=FireEyeHXTransformer' -F 'comment=SP Triage' http://localhost:8000/api/new

curl: (52) Empty reply from server

I ran another MANS file that created only 308316 nodes. That graph was successfully generated.

2019-04-17T06:31:24 | beagle.web.api.views.new:208 | INFO | Recieved upload request for datasource=<HXTriage>, transformer=<FireEyeHXTransformer>
2019-04-17T06:31:24 | beagle.web.api.views.new:245 | INFO | Transforming data to a graph.
2019-04-17T06:31:24 | beagle.web.api.views.new:274 | INFO | Saved uploaded files {'triage': <tempfile._TemporaryFileWrapper object at 0x7f228edb1390>}
2019-04-17T06:31:24 | beagle.datasources.hx_triage.__init__:53 | INFO | Setting up HXTriage for /tmp/tmpy2mka_nf
2019-04-17T06:31:24 | beagle.datasources.hx_triage.__init__:57 | DEBUG | Generated temporary directory /tmp/tmpi3x4vzrw_beagle
2019-04-17T06:31:24 | beagle.datasources.hx_triage.__init__:93 | DEBUG | Mapped 95ndX1K8xy403iqPyLqAd3 to stateagentinspector
2019-04-17T06:31:24 | beagle.transformers.fireeye_hx_transformer.__init__:17 | INFO | Created FireEyeHX Transformer.
2019-04-17T06:31:24 | beagle.transformers.base_transformer.run:77 | DEBUG | Launching transformer
2019-04-17T06:31:24 | beagle.transformers.base_transformer.run:86 | DEBUG | Started producer thread
2019-04-17T06:31:24 | beagle.transformers.base_transformer.run:98 | DEBUG | Started 11 consumer threads
2019-04-17T06:31:47 | beagle.transformers.base_transformer._producer_thread:125 | DEBUG | Producer Thread Thread-1 finished after 103165 events
2019-04-17T06:31:47 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-7 finished after processing 9035 events
2019-04-17T06:31:47 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-3 finished after processing 10368 events
2019-04-17T06:31:47 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-10 finished after processing 8101 events
2019-04-17T06:31:47 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-6 finished after processing 8517 events
2019-04-17T06:31:47 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-8 finished after processing 9343 events
2019-04-17T06:31:47 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-12 finished after processing 8620 events
2019-04-17T06:31:47 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-2 finished after processing 9708 events
2019-04-17T06:31:47 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-4 finished after processing 9562 events
2019-04-17T06:31:47 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-11 finished after processing 11418 events
2019-04-17T06:31:47 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-9 finished after processing 9422 events
2019-04-17T06:31:47 | beagle.transformers.base_transformer._consumer_thread:136 | DEBUG | Consumer Thread Thread-5 finished after processing 9082 events
2019-04-17T06:31:47 | beagle.transformers.base_transformer.run:111 | INFO | Finished processing of events, created 308316 nodes.
2019-04-17T06:31:47 | beagle.backends.networkx.__init__:51 | INFO | Initialized NetworkX Backend
2019-04-17T06:31:47 | beagle.backends.networkx.graph:65 | INFO | Beginning graph generation.
2019-04-17T06:31:55 | beagle.backends.networkx.graph:71 | INFO | Completed graph generation.
2019-04-17T06:31:55 | beagle.backends.networkx.graph:72 | INFO | Graph contains 6332 nodes and 39520 edges.
2019-04-17T06:31:55 | beagle.web.api.views.new:305 | INFO | Finished generating graph
2019-04-17T06:31:56 | beagle.web.api.views.new:340 | INFO | Added graph to database with id=1
2019-04-17T06:31:58 | beagle.web.api.views.new:344 | INFO | Saved graph to /data/beagle/fireeye_hx/8203c3e57725e9671bb016fa1f47d1759e92dac0f915113880327009ac3dab62.json

Thanks for the great work so far with Beagle!

Regards

Feature Request: add multiple files from GUI

Implement possibility to add multiple files from GUI

Generic Schema Definition endpoint

The REST Api should provide the schemas required by the various events.

Fixed failure to display the names of parent process nodes for evtx

Hi,

When providing an evtx to the software, the nodes of the parent processes do not necessarily have a name because it is not specified in the transformer. This patch is used to display it.

Dockerless install instructions

Subject :) Would love to try and get this to run without docker..thank you.

Splunk / SPL Search Data Source

Documentation/Installing: docker run issue?

For reference:

• Operating System: Ubuntu WSL on Windows 10
• Docker Installation: Ubuntu docker installation but running daemon from Docker for Windows, as per this guide

I've been going through the installation, and seem to have come across an issue with the docker run command. The command from the docs is:
docker run -v "data/beagle":"/data/beagle" -p 8000:8000 beagle

The error I received was:

Unable to find image 'beagle:latest' locally
docker: Error response from daemon: pull access denied for beagle, repository does not exist or 
may require 'docker login'.
See 'docker run --help'.

Looking at my images list, it seems that beagle is listed under "yampelo/beagle":

Without running docker pull again, I was able to get docker run to work by making the path the same for the volume mount, and changing the image name:
docker run -v "/data/beagle":"/data/beagle" -p 8000:8000 yampelo/beagle

I'm not sure if this indicates a general installation issue, or if my using WSL creates this problem, though I'd bet the latter. Any insight on this would be great, thanks!