Request Type

Question

Work Environment

Cortex 1.0.2
Ubuntu 16.04

Problem Description

We are attempting to setup the Joe Sandbox analyser to connect our Online Joe Sandbox cloud account. Can you please confirm that the analyser supports a Joe Sandbox Cloud instance?

We are seeing numerous JSON decoding / structure errors.

Problem Description

have a unified way display short report, its content and its color

Possible Solutions

When possible, use a taxonomy that helps to show :

• Analyzer name or tool name or a short name that identify it (ex PT for PassiveTotal, VT for VirusTotal, JSB for JoeSandbox...)
• A service used, a criticity, a format, or whatever that identify what kind of result it is,
• the result, the score, the information or whatever that helps the analyst identify what to do with the observable

ex: VT:Scan=<SCORE>,

Regarding the color, we decided to use in 4 colors. Red means danger (label-danger in bootstrap). Orange means suspicious (label-warning). Green means innocuous(label-success). And blue is informational (label-info).

Cortexutils crashes on version check if python < 2.7

Request Type

Bug

Work Environment

Problem Description

When using Python 2.6.6, analysers crash with the following error:
"errorMessage": "Error: Invalid output\nTraceback (most recent call last):\n File "./virustotal.py", line 135, in \n VirusTotalAnalyzer().run()\n File "./virustotal.py", line 16, in init\n Analyzer.init(self)\n File "/usr/lib/python2.6/site-packages/cortexutils-1.0.0-py2.6.egg/cortexutils/analyzer.py", line 16, in init\n self.setEncoding()\n File "/usr/lib/python2.6/site-packages/cortexutils-1.0.0-py2.6.egg/cortexutils/analyzer.py", line 38, in setEncoding\n if sys.version_info.major == 3:\nAttributeError: 'tuple' object has no attribute 'major'\n"

Steps to Reproduce

1. Install/configure thehive
2. Install/configure cortex
3. Connect thehive to MISP
4. Run analyzers against an observable

Possible Solutions

In cortexutils/analyzer.py lines 38 and 44
Replace:
if sys.version_info.major == 3:

with
if sys.version_info[0] == 3:

Analyzer Caching

Request Type

Feature Request

Work Environment

NA

Problem Description

For analyzers, it would be nice to be able to identify a caching mechanism (say the Elastic Search cluster used for the hive) in such a way, that we could tune analyzers to check cache, and if there are hits within the specified timeline, that it returns the result from cache rather than requerying the endpoint in the analyzer. This would allow more efficient use of analyzer apis especially those that charge per request.

Problem Description

OTX Query can't get report for a file when running in Cortex. Get error "Hash is missing".

Steps to Reproduce

From Cortex UI, choose OTXQuery analyzer, and run it against a file.

Possible Solutions

Compute the file hash if not received in input.

Request Type

Analyzer Request

Work Environment

NA

Analyzer Description

Create an analyzer that can query one or several MISP instances.

The analyzer must be able to ingest observables, query MISP instances and provide the number of events where the observable can be found as a short report. The long analyzer report should contain useful metadata about the associated events if any and hyperlinks to access them.

Additional Information

TheHive should be able to preview and import the events of interest once this analyzer is executed.

Request Type

Analyzer Request

Work Environment

N/A

Problem Description

Create an analyzer that will submit files to a local FireEye AX sandbox instance and retrieve the report and indicators that are generated

There were no newlines on the GoogleSafebrowsing/requirements.txt and Virusshare/requirements.txt

This caused values to concatenate and error out if you wanted to do pip install $(cat /opt/Cortex-Analyzers/analyzers/*/requirements.txt | sort -u) or some similar mass install of the required libraries.

Pull request coming shortly.

Request Type

Analyzer Request

Work Environment

NA

Analyzer Description

Create an analyzer that can query an IntelMQ instance.

Request Type

Feature Request

Feature Summary

Would it be possible to develop an analyzer (or analyzers) for CarbonBlack (ER/EP) using the cbapi-python? https://github.com/carbonblack/cbapi-python

It should have the ability to lookup the following Data Types in each instance

CB EP: {file(via hash), filename, hash}
CB ER: {file(via hash), filename, fqdn, hash, ip, registry, url}

Thanks

Request Type

Analyzer Request

Work Environment

N/A

Problem Description

Create an analyzer that will submit files to a Joe Sandbox instance and retrieve the report that is generated.

Request Type

Feature Request

Description

A basic analyser for whois lookup using the WhoisXMLAPI service (https://www.whoisxmlapi.com/whois-api-doc.php)

Make analyzer's configuration file similar with all global options (especially regarging TLP) even if they do not use them.

Analyzer that checks if an ip can be found on https://iplists.firehol.org/.

Provided with the other analyzers.

Request Type

Analyzer Request

Work Environment

NA

Analyzer Description

Create an analyzer to tap into CIRCL.lu PassiveSSL service.

Problem description

Number of records indicated in short report does not match results given in long report

Possible Solutions

change

result['events'] = len(raw)

to

result['events'] = len(raw['events'])

in the file misp_analyzer.py, line 26

Request Type

Analyzer Request

Work Environment

NA

Analyzer Description

Create an analyzer which can submit a File, File via URL, or URL to Hybrid Total (https://www.hybrid-analysis.com/) for profiling and report retrieval. Analyser should support submission of hash or complete binary dependant on TLP attributes.

Request Type

Question

Question

Team, just wondering if a report template been created for JoeSandbox_Url_Analysis analyser?

Request Type

Bug

Work Environment

Problem Description

After upgrading from cortexutils 1.0 to 1.1.0 some analyzers stop working with observables that previously worked.

MaxMind GeoIP:
Error: Invalid output {"artifacts": [{"type": "ip", "value": "8.8.8.8"}], "full": {"city": {"geoname_id": 5375480, "confidence": null, "name": "Mountain View", "names": {"ru": {"input": {"dataType": "ip", "config": {"max_tlp": 3, "check_tlp": false, "service": ""}, "tlp": 0, "data": "8.8.8.8"}, "errorMessage": "Invalid IP address", "success": false}

Msgparser:
"errorMessage": "Unexpected Error: 'ascii' codec can't encode character u'\xed' in position 115: ordinal not in range(128)", "success": false}

Trying to identify the problem I executed the analyzer directly:
./geo.py <<< '{"tlp":0,"dataType":"ip","data":"8.8.8.8","config":{"check_tlp":false,"max_tlp":3,"service":""}}'
And it worked.

Possible Solutions

The error message in Msgparser is related with encoding. Checking analyzers.py I saw a line commented #self.__set_encoding() that in cortexutils 1.0 was not commented. After removing the comment the analyzers worked again.

API Keys to be submitted through Cortex for Analyzers

Request Type

Feature Request

Work Environment

NA

Problem Description

For some systems, having one key for all users doesn't match the terms of service of the endpoint APIs. Thus, it would be nice to have users in say thehive have the ability to specify keys for specific endpoints (Say Passive DNS or Virustotal) in their accounts, and when they request analyzer action, those keys are passed to the cortex, which then uses those keys as the key to the backend.

This would allow better usage tracking. There could be "Default" keys if a user doesn't specify a key to say virus total, it would use the shared key. But if we do that, we should get the ability to enforce per user limits on shared keys for specific services. This would force almost all requests coming from the thehive to cortext to include a user identifier.

Request Type

Feature Request

I would like to know if worth to make an analyzer that identify IPs from a list of Tor Exit Nodes (https://check.torproject.org/exit-addresses and/or http://torstatus.blutmagie.de/ip_list_exit.php/Tor_ip_list_EXIT.csv) to use with Cortex.

Request Type

Feature Request

I would like to know if worth to make an analyzer that gets IPs from GPF DNS Block List (https://www.gpf-comics.com/dnsbl/) to use with Cortex.

Request Type

Feature Request

Content

Hello guys,

I just skimmed through analyzer.py to see, if it's possible to use the Analyzer-class with python 3. The problematic dependency in this case is pdfminer which is not in use, basically, because the parser is always called with txt and json parameters.

Would if be possible to replace it with an self-implemented function (more or less) easily?

Lots of modules I depend on use python 3 and I just thought it would be great to directly start with python 3 here.

Have a nice day
3c7

Request Type

Feature Request

Problem Description

Based on what has been done by @3c7 on https://github.com/BSI-CERT-Bund/cortex-analyzers, the JSON files that describe analyzers should include the following additional fields:

• author (Optional): the author of the analyzer
• url (Optional): the URL to the analyzer's source code
• licence

These details will be displayed in Cortex and TheHive

Request Type

Feature Request

Details

IRMA Analyzer for offline assessment of files against various virus scanners.
http://irma.quarkslab.com/

Request Type

Analyzer Request

Work Environment

NA

Analyzer Description

Create an analyzer which can instantiate YARA and check a file against its rules.

The analyzer should be provide short and long reports.

Complementary information

It should be possible to update the YARA rules from Cortex interface (edit/update).

Request Type

Enhancement

Problem Description

As stated on the issue's title, the goal here is to make cortexutils compatible with python2 and python3 to make analyzer writers use the same cortexutils library

Request Type

Analyzer Request

Work Environment

NA

Analyzer Description

Create an analyzer than can submit files to VMRay Analyzer and retrieve the results back.

The analyzer should provide short and long reports.

Setting TLP:AMBER or up is not avoiding sending files to VT for scan.
there is no max_tlp in the Virustotal scan analyser configuration, and tlp is not checked in the main program

Just finished a Virusshare analyzer that searches through the Virusshare.com offline lists (md5 hashes of all samples available) obtainable through their site. Automatic downloading samples etc. is prohibited, so I won't implement that.

Will upload it with all the others.

Contribute

Problem Description

I have Analyzers for:

• https://www.threatcrowd.org/
• https://www.ipvoid.com / blacklist
• https://cymon.io

Steps to Contribute

1. How?

Thanks !

Request Type

Analyzer Request

Work Environment

N/A

Problem Description

Create an analyzer that will submit files to a local cuckoo sandbox instance and retrieve the report that is generated

Request Type

Enhancement

Problem Description

The VT Scan analyzer is asynchronous and need to poll VirusTotal to fetch the scan report of the submitted file or URL.
Currently the polling interval is set to 10 seconds which might consume a lot of API call quota.
The idea is to update that interval to 60 seconds, and make it customizable.

Request Type

Analyzer Request

Work Environment

N/A

Problem Description

Create an analyzer that will submit files to a local McAfee ATD sandbox instance and retrieve the report and indicators that are generated

Analyzer Rate Limiting

Request Type

Feature Request

Work Environment

NA

Problem Description

It would be nice if we could have a way to rate limit outgoing requests in such a way that informs the user of high volumes, thus delayed response. Since we are hitting external systems, we don't want to be banned for abuse reasons. (Even when there are keys involved, some systems hold back, or charge per request

Request Type

Analyzer Request

Work Environment

Irrelevant

Analyzer Description

Create an analyzer that takes as input an IP address or a FQDN and launches a scan using Nessus by leveraging its API. That will allow the analyst to quickly assess the attack surface of the asset, the services that it is exposing on the network, their vulnerabilities, banners and so on.

Additional Details

The analyzer must not allow the analyst to launch a scan against assets that do not belong to their constituency. So it must be configured prior to use with the IP addresses, ranges, CIDR, domain names of the constituency. When an observable is submitted, it must check it against its configuration and refuses to run if it is not among or within the configured IP addresses, ranges, CIDR or domain names.

The analyzer should not allow authenticated scans as the current Cortex has no authentication and we'd risk leaking the credentials Nessus would use to authenticate. Moreover, if the asset have been compromised, an authenticated scan would tip off the attacker that something is going on. They could also capture the Nessus credentials as a result and launch lateral movements through the network.

The analyzer must not retrieve the full-fedged Nessus report by default. It should limit the information to what an analyst really needs like services, banners, critical and high severity vulnerabilities.

The analyzer must use a safe scanning policy. Instructions (in the documentation for ex.) should be provided on how to set up the policy on Nessus.

Request Type

Bug

Work Environment

TheHive v. 2.10.1
Cortex v. 1.0.1

Problem Description

Report contains following error:

{
  "errorMessage": "Error: Invalid output\nFailed processing /tmp/cortex-8307665741774789030-datafile\nFailed processing /tmp/cortex-8307665741774789030-datafile\n{\"errorMessage\": \"Unexpected Error: file instance has no attribute 'pedict'\", \"success\": false}\n"
}

Steps to Reproduce

1. Upload PE file
2. Start File_Info analyzer
3. Errors thrown

Request Type

Analyzer Request

Work Environment

NA

Analyzer Description

Create an analyzer to query Google Safe Browsing.

The analyzer should provide short and long reports.

Request Type

Analyzer Request

Work Environment

NA

Analyzer Description

Create an analyzer to tap into CERT.at PassiveDNS service.

Request Type

Analyzer Request

Work Environment

N/A

Problem Description

Produce an analyzer to leverage FAME using its API.

Abuse_Finder analyzer analyzes "email" instead of "mail"

Request Type

Bug

Request Type

Analyzer

Work Environment

NA

Analyzer Description

Create an analyzer which can submit a URL to URLQuery for profiling.

Problem Description

Steps to Reproduce

The same message is given for any error problem encountered by the program.

Complementary information

{"errorMessage": "API Error! Please verify data type is correct.", "success": false}%

Have a standard naming convention for some common config properties in analyzers and produce a documentation.

Request Type

Bug

Problem Description

Fortiguard have change their online free API meaning that the analyser is not working. It looks like the new version provide less info so the webscraping code will need to be updated as well. The new URL is: http://www.fortiguard.com/webfilter?q=www.google.com.

From Cortex, running VirusTotal_GetReport returns an error message :

{
  "errorMessage": "Hash is missing",
  "success": false
}

Request Type

Bug

Problem Description

The cortexutils library provide a error that formats the result that have to be returned when an analyzer fails. This method includes the analysis input and removes any sensitive data like passwords, api keys etc...

When the analyzer don't have a configuration object (example of MaxMind analyzer) cortexutils hits a NoneType error when trying to play with the config object.

Review all analyzers to permit TheHive to read all relevant info in summary() to display short reports on the Observables tab -- TheHive-Project/TheHive#131

Hello guys,

great project so far! As @MichaelDwucet already mentioned, I'm working on some analyzers right now. This is just fyi, will provide details for each analyzer later on.

Request Type

Analyzer Request

Work Environment

NA

Analyzer Description

Create an analyzer to tap into CIRCL.lu PassiveDNS service.