otrf / attack-python-client Goto Github PK

View Code? Open in Web Editor NEW

553.0 553.0 115.0 19.79 MB

Python Script to access ATT&CK content available in STIX via a public TAXII server

License: BSD 3-Clause "New" or "Revised" License

Python 90.34% Dockerfile 2.12% Shell 7.53%

attack-python-client's People

Contributors

Stargazers

Watchers

Forkers

sctnightcore yanyangithub abotsis gavz yeyingtomorrow p3t3rp4rk3r aayush420 5cr1p70633k h4554nalmusajjen jreisdorffer 2xyo pilgrim-atraidez soprasteria hasantemel thewover ixxxxxx gmxp harveys3191 awesome-attack modulexcite shftr harmzway annathan barbaracoding josecsoto binaryflesh totoroha hitesh50 tobey123 benhe119 narayanr7 rcfontana jorson-chen charly837 hrbrmstr vanquishsecurity2 taosecurity ishell rahmiy rubinatorz ababook jonz-secops riparino eastierp achen0276 jingqiangliu smolige001 0xy4q x0341 azrara mamarilr bigbrobro canghailongheng isamalrikabi 5l1v3r1 sydbarrett46 freecamel morebettersecurity rivetgeek neo23x0 ammasajan tools-env cx01n scrutinize-systems alainw68 dekoder ihab000 carloanez jbarscheski-zz crackercat skiddybargins cyb3rpandah ag4github beermt double-q1015 gh0st0ne ccdev-labs magnologan ekmixon thegautamkumarjaiswal eslimasec johann-smith egbordzor drive-slow-homie cbresponse gh0stlykn1ght darksidesfear zikyhd re-expoc yakshavingcatherder ageekinside luckydragon333 mabidm-tsinghua mazitheseries jvossler jrandomsage dadokkio aaronfderybel hzami vanshjaiswal11

attack-python-client's Issues

Bug: enrich_data_sources is not working

Following this manual: https://attackcti.com/playground/1-Collect_All_Functions.html#enrich-techniques-data-sources

The following lines are not working ...

lift = attack_client()

def get_dataframe():
    enterprise = lift.get_techniques(enrich_data_sources=True)
    return None

... as the following error is generated:

[...]
enterprise = lift.get_techniques(enrich_data_sources=True)
  File "/home/user/.local/lib/python3.10/site-packages/attackcti/attack_api.py", line 943, in get_techniques
all_techniques = self.enrich_techniques_data_sources(all_techniques)
  File "/home/user/.local/lib/python3.10/site-packages/attackcti/attack_api.py", line 1760, in enrich_techniques_data_sources
    dc = dc_lookup[rl['source_ref']]
KeyError: 'x-mitre-data-component--4c12c1c8-bcef-4daf-8e5b-fca235f71d9e'

get_all_enterprise() fails

from attackcti import attack_client
lift = attack_client()
all_enterprise = lift.get_all_enterprise()
Traceback (most recent call last):
File "", line 1, in
File "\attack_api.py", line 203, in get_all_enterprise
enterprise_stix_objects[key] = self.parse_stix_objects(enterprise_stix_objects[key], key)
File "\attack_api.py", line 100, in parse_stix_objects
'created_by_ref': software['created_by_ref'],
File "*********\base.py", line 178, in getitem
return self._inner[key]
KeyError: 'created_by_ref'

Updated SANS CTI Summit 2022 Notebook

Renamed notebook prepared by @Cyb3rPandaH and set logging to CRITICAL to avoid default WARNINGS about pagination while querying ATT&CK TAXII Server.

Update Jupyterbook config and toc file

Project is running an old version of the main config and toc file format. Project needs to be updated.

Retrieving tactics from URL returns invalid JSON error

As title suggests, running:

!pip install attackcti
from attackcti import attack_client

lift = attack_client()
objs = lift.get_stix_objects()['enterprise']["tactics"]

returns:

JSONDecodeError: Unterminated string starting at: line 1 column 32259 (char 32258)

The above exception was the direct cause of the following exception:

InvalidJSONError                          Traceback (most recent call last)
/usr/local/lib/python3.7/dist-packages/six.py in raise_from(value, from_value)

InvalidJSONError: Invalid JSON was received from https://cti-taxii.mitre.org/stix/collections/95ecc380-afe9-11e4-9b6c-751b66dd541e/objects/?match%5Btype%5D=attack-pattern

This is a new error as the same code was working a few days ago. May be endpoint related and not the fault of this repo - issue nonetheless.

Rename enrich_data_source function to enrich_techniques_data_sources in get_enterprise_techniques

Current get_enterprise_techniques()

 def get_enterprise_techniques(self, skip_revoked_deprecated=True, include_subtechniques=True, enrich_data_sources = False, stix_format=True):
        """ Extracts all the available techniques STIX objects in the Enterprise ATT&CK matrix
        Args:
            skip_revoked_deprecated (bool): default True. Skip revoked and deprecated STIX objects. 
            include_subtechniques (bool): default True. Include techniques and sub-techniques STIX objects.
            enrich_data_sources (bool): default False. Adds data component and data source context to each technqiue.
            stix_format (bool):  Returns results in original STIX format or friendly syntax (e.g. 'attack-pattern' or 'technique')
        
        Returns:
            List of STIX objects
        """
        
        if include_subtechniques:
            enterprise_techniques = self.TC_ENTERPRISE_SOURCE.query(Filter("type", "=", "attack-pattern"))
        else:
            enterprise_techniques = self.TC_ENTERPRISE_SOURCE.query([
                Filter("type", "=", "attack-pattern"),
                Filter('x_mitre_is_subtechnique', '=', False)
            ])

        if skip_revoked_deprecated:
            enterprise_techniques = self.remove_revoked_deprecated(enterprise_techniques)

        if enrich_data_sources:
            enterprise_techniques = self.enrich_data_sources(enterprise_techniques)
        
        if not stix_format:
            enterprise_techniques = self.translate_stix_objects(enterprise_techniques)
        return enterprise_techniques

The new parameter to enrich data sources is pointing to a function that does not exist by the name enrich_data_sources

 if enrich_data_sources:
            enterprise_techniques = self.enrich_data_sources(enterprise_techniques)

That function should be ``enrich_techniques_data_sources`

Missing groups in all_groups() when group is "living" in multiple matrices

hi Roberto!

Enterprise, ICS and Mobile matrices are all included in the STIX objects. Some threat actor groups are active in multiple matrices, for example Windshift is active in both Enterprise and Mobile. Within the STIX objects this group is included with ID "intrusion-set--afec6dc3-a18e-4b62-b1a4-5510e1a498d1" in Enterprise as also in Mobile. But what happens when using get_groups() of attackcti, you will only get 1. The one from Mobile ("x_mitre_domains": ["mobile-attack"]). The following code will demonstrate this:

from attackcti import attack_client

client = attack_client(local_path="../cti/")
groups = client.get_groups()

windshift = "intrusion-set--afec6dc3-a18e-4b62-b1a4-5510e1a498d1"

for group in groups:
    if group['id'] == windshift:
        print(group)

OUTPUT (compressed):
{"type": "intrusion-set", "id": "intrusion-set--afec6dc3-a18e-4b62-b1a4-5510e1a498d1", 
"modified": "2021-04-26T14:37:33.234Z", "name": "Windshift", ...................., 
"x_mitre_domains": ["mobile-attack"], ....................}

I digged into this problem, because when getting all groups I'm now missing the one from Enterprise. I also have seen other items where groups are Enterprise and ICS for example and then only the ICS one is returned. And also ones where a PRE group is returned (see also my issue from earlier today #59).

It turned out that the query function of CompositeDataSource deduplicate items:

# remove exact duplicates (where duplicates are STIX 2.0
# objects with the same 'id' and 'modified' values)
if len(all_data) > 0:
    all_data = deduplicate(all_data)

It deduplicates based on the ID and modified date. The modified date is in many cases also the same, so I'm missing groups from other matrices now. I solved this by using the query function for each matrix separately and merge the results:

groups_enterprise = mitre.TC_ENTERPRISE_SOURCE.query(Filter("type", "=", "intrusion-set"))
groups_ics = mitre.TC_ICS_SOURCE.query(Filter("type", "=", "intrusion-set"))
groups_mobile = mitre.TC_MOBILE_SOURCE.query(Filter("type", "=", "intrusion-set"))
all_groups = groups_enterprise + groups_ics + groups_mobile

I think it's good to take this into account for attackcti. Especially for the all_groups function. And maybe it also applies to other data objects.

Remove function 'remove_revoked()' from available functions

the deprecated function remove_revoked() is still in two functions in the main script.

attack_client not workning (Err_connection)

Hello, upon running the examples from the site, I keep getting:

ConnectionError: HTTPSConnectionPool(host='cti-taxii.mitre.org', port=443): Max retries exceeded with url: /stix/collections/95ecc380-afe9-11e4-9b6c-751b66dd541e/ (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f6d3c7c7350>: Failed to establish a new connection: [Errno 110] Connection timed out'

Accessing the above website via browser: cti-taxii.mitre.org took too long to respond.,

I suppose the server is down.

Anyone else facing this issue?

SSL certificate problem

Hey Roberto,

I'm currently facing an SSL certificate error while I'm trying to communicate with the TAXII server. Knowing that I encounter this problem when I am behind my corporate proxy, I think this problem could be solved by offering the user the possibility not to block the communication when an insecure certificate is encountered.

For example, I find the following error when I launch a cURL command line on the TAXII server:
curl -H "Accept: application/vnd.oasis.taxii+json" https://cti-taxii.mitre.org/taxii/
curl: (60) SSL certificate problem: unable to get local issuer certificate More details here: https://curl.haxx.se/docs/sslcerts.html curl failed to verify the legitimacy of the server and therefore could not establish a secure connection to it. To learn more about this situation and how to fix it, please visit the web page mentioned above.

If I just add a -k parameter to the same cURL command line:
curl -H "Accept: application/vnd.oasis.taxii+json" https://cti-taxii.mitre.org/taxii/ -k
{"title":"CTI TAXII server","description":"This TAXII server contains a listing of ATT&CK domain collections expressed as STIX, including PRE-ATT&CK, ATT&CK for Enterprise, and ATT&CK Mobile.","contact":"[email protected]","default":"https://cti-taxii.mitre.org/stix/","api_roots":["https://cti-taxii.mitre.org/stix/"]}

I hope that my problem is clear enough, and I hope that a solution can be found.
Best regards,

Jupyter notebooks Python 2 compatibility

Requirements in the readme are "Python 3+ or 2.7", however, jupyterlab needed to play with notebooks requires Python 3.5+.

Maybe it should be mentioned in the Readme.

Client Timeout Error

Hello friends!

Since a few days ago, about a week, when invoking the client it returns an error when trying to establish the connection with cti-taxii.mitre.org.
I leave here the code and the error message received, as well as the configuration used. Thank you very much in advance.

Configuration:

Python - 3.11.9
attackcti - 0.4.2
pandas - 2.2.2
stix2 - 3.0.1
taxii2-client - 2.3.0
six - 1.16.0
pydantic - 2.7.1

Code:

from attackcti import attack_client
lift = attack_client()

Error:

TimeoutError Traceback (most recent call last)
File [...]\anaconda3\envs\py311\Lib\site-packages\urllib3\connection.py:198, in HTTPConnection._new_conn(self)
197 try:
--> 198 sock = connection.create_connection(
199 (self._dns_host, self.port),
200 self.timeout,
201 source_address=self.source_address,
202 socket_options=self.socket_options,
203 )
204 except socket.gaierror as e:

File [...]\anaconda3\envs\py311\Lib\site-packages\urllib3\util\connection.py:85, in create_connection(address, timeout, source_address, socket_options)
84 try:
---> 85 raise err
86 finally:
87 # Break explicitly a reference cycle

File [...]\anaconda3\envs\py311\Lib\site-packages\urllib3\util\connection.py:73, in create_connection(address, timeout, source_address, socket_options)
72 sock.bind(source_address)
---> 73 sock.connect(sa)
74 # Break explicitly a reference cycle

TimeoutError: [WinError 10060] Se produjo un error durante el intento de conexión ya que la parte conectada no respondió adecuadamente tras un periodo de tiempo, o bien se produjo un error en la conexión establecida ya que el host conectado no ha podido responder
...
--> 507 raise ConnectTimeout(e, request=request)
509 if isinstance(e.reason, ResponseError):
510 raise RetryError(e, request=request)

ConnectTimeout: HTTPSConnectionPool(host='cti-taxii.mitre.org', port=443): Max retries exceeded with url: /stix/collections/95ecc380-afe9-11e4-9b6c-751b66dd541e/ (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000002813341C750>, 'Connection to cti-taxii.mitre.org timed out. (connect timeout=None)'))

get_software_used_by_group returns all tools for groups with no actual tools/ software

Hello Roberto,

When getting all the software/ tools used by a group, there is an issue with groups that have no software listed.
The result is getting all 605 tools listed for those groups that have no actual software listed on the site.

Example:

groups = lift.get_groups()
one_group = groups[40] # FIN4/ G0085
group_software = lift.get_software_used_by_group(one_group) # returns all 605 tools

MITRE's site does not list any software for the group.
This is also the case with groups such as "APT34" (67) which is just an alias to "OilRig" (73/ G0049) which has the actual tools listed. Groups with the same issue include [2, 8, 20, 40, 41, 61, 67].

Possible reference: all_software_list

Apologies in advance if this was done by design which I know can be omitted if we choose to. I tried MITRE's live cti server to confirm this and indeed they did not list the tools or the group either if they had no software/ tools.

using dict() on a stix2 object will not correctly serialize datetime properties

This is apparently is a bug in Pandas and not in STIX2 according to this mitre/cti#17 . The suggestion is to apply a workaround:

Don't put them in dictionaries and instead serialize them and load the strings back to have a JSON object that you can pass to the pandas

Remove 'Pre' from get_stix_objects() function

'PRE' matrix is already deprecated.
Function get_stix_objects() runs get_pre() function. It needs to be removed.

Add API for ICS domain

As a user of ATT&CK Python Client, I would like to be able to access STIX data for the ATT&CK for ICS domain using this library. Ideally, functions similar to those implement for Enterprise, Pre-ATT&CK and Mobile would be developed for accessing ICS data.

ATT&CK v8 added of STIX data for the ICS domain. This can be accessed via the MITRE/CTI GitHub repository in the ics-attack folder, or using the official ATT&CK TAXII server via collection ID 02c3ef24-9cd4-48f3-a99f-b74ce24f1d34.

The ATT&CK for ICS domain has roughly the same data formatting as the Enterprise domain. The MITRE/CTI USAGE document explains the data model in more detail.

Connection Timeout Issue When Using 'attackcti' Library

I am encountering a connection timeout issue while using the 'attackcti' library to interact with the Mitre ATT&CK data. The specific error is a TimeoutError that occurs when attempting to connect to the URL 'https://cti-taxii.mitre.org/stix/collections/95ecc380-afe9-11e4-9b6c-751b66dd541e/'. The connection timeout seems to be affecting the library's ability to retrieve the necessary data.

How to Access the Cloud ATT&CK Matrix

Hi,

Is there any way to access and interact with the Cloud ATT&CK matrix using the ATTACK-Python-Client?

Thanks

group_references missing

Hello,

Thanks for creating the ATTACK-Python-Client as this is very useful. I have observed the following:

If I create a group variable

groups = all_attack['groups']

and then a Pandas Dataframe from the groups data

df = json_normalize(groups)

I can then see the group_references associated with the "Lazarus Group"

df[df.group == "Lazarus Group"].group_references.values[0]

['https://attack.mitre.org/wiki/Group/G0032',
 'https://www.us-cert.gov/ncas/alerts/TA17-164A',
 'https://www.operationblockbuster.com/wp-content/uploads/2016/02/Operation-Blockbuster-Report.pdf']

However, when I go to the page https://attack.mitre.org/wiki/Group/G0032 I see there are 11 references and not the 3 returned by ATTACK-Python-Client.

Is this a bug?

Thanks

Integrating examples from MITRE CTI - USAGE docs in GitHub

There are some examples in this document and will be great to review them, improve them (if possible) and add them to this library 🍻

https://github.com/mitre/cti/blob/master/USAGE.md

get_techniques_used_by_all_groups is broken by the new subtechniques change

Hi,

I'm getting the following error when running the "get_techniques_used_by_all_groups" function.
File "/Users/yallon/.virtualenvs/test_function/lib/python3.7/site-packages/attackcti/attack_api.py", line 660, in get_techniques_used_by_all_groups for phase in t['kill_chain_phases']: File "/Users/yallon/.virtualenvs/test_function/lib/python3.7/site-packages/stix2/base.py", line 216, in __getitem__ return self._inner[key] KeyError: 'kill_chain_phases'

Update SIX to six-1.15.0: No module named 'six.moves.collections_abc'

I wanted to use from attackcti import attack_client in a sample code and got the following:

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-6-9d1571bab569> in <module>
----> 1 from attackcti import attack_client
      2 from pandas.io.json import json_normalize
/opt/conda/lib/python3.7/site-packages/attackcti/__init__.py in <module>
      1 #!/usr/bin/env python
      2 
----> 3 from .attack_api import attack_client
/opt/conda/lib/python3.7/site-packages/attackcti/attack_api.py in <module>
     10 # https://stackoverflow.com/a/4406521
     11 
---> 12 from stix2 import TAXIICollectionSource, Filter, CompositeDataSource, FileSystemSource
     13 from stix2.utils import get_type_from_id
     14 from taxii2client import Collection
/opt/conda/lib/python3.7/site-packages/stix2/__init__.py in <module>
     27 from .confidence import scales
     28 from .datastore import CompositeDataSource
---> 29 from .datastore.filesystem import (
     30     FileSystemSink, FileSystemSource, FileSystemStore,
     31 )
/opt/conda/lib/python3.7/site-packages/stix2/datastore/filesystem.py in <module>
      9 import six
     10 
---> 11 from stix2 import v20, v21
     12 from stix2.base import _STIXBase
     13 from stix2.datastore import (
/opt/conda/lib/python3.7/site-packages/stix2/v20/__init__.py in <module>
     15 # flake8: noqa
     16 
---> 17 from .base import (
     18     _DomainObject, _Extension, _Observable, _RelationshipObject, _STIXBase20,
     19 )
/opt/conda/lib/python3.7/site-packages/stix2/v20/base.py in <module>
      1 """Base classes for STIX 2.0 type definitions."""
      2 
----> 3 from ..base import (
      4     _DomainObject, _Extension, _Observable, _RelationshipObject, _STIXBase,
      5 )
/opt/conda/lib/python3.7/site-packages/stix2/base.py in <module>
     17     MissingPropertiesError, MutuallyExclusivePropertiesError,
     18 )
---> 19 from .markings import _MarkingsMixin
     20 from .markings.utils import validate
     21 from .utils import (
/opt/conda/lib/python3.7/site-packages/stix2/markings/__init__.py in <module>
     20 """
     21 
---> 22 from stix2.markings import granular_markings, object_markings
     23 
     24 
/opt/conda/lib/python3.7/site-packages/stix2/markings/granular_markings.py in <module>
      4 from stix2.markings import utils
      5 from stix2.utils import is_marking
----> 6 from stix2.versioning import new_version
      7 
      8 
/opt/conda/lib/python3.7/site-packages/stix2/versioning.py in <module>
      7 
      8 import six
----> 9 from six.moves.collections_abc import Mapping
     10 
     11 import stix2.base

ModuleNotFoundError: No module named 'six.moves.collections_abc'

AttributeError: 'function' object has no attribute 'query' when using get_techniques_used_by_group_software

I'm unable to use get_techniques_used_by_group_software. Latest version of library is installed.

When using

group_name = lift.get_group_by_alias('APT12')
techniques_group_software = lift.get_techniques_used_by_group_software(group_name[0])

this error is raised

File ~/Documents/playbooks/lib/python3.10/site-packages/attackcti/attack_api.py:1787, in attack_client.get_techniques_used_by_group_software(self, stix_object, stix_format)
   1781 # Get all used by the software that is used by group
   1782 filter_objects = [
   1783     Filter('type', '=', 'relationship'),
   1784     Filter('relationship_type', '=', 'uses'),
   1785     Filter('source_ref', 'in', [r.target_ref for r in software_relationships])
   1786 ]
-> 1787 software_uses = self.COMPOSITE_DS.query.query(filter_objects)
   1788 # Get all techniques used by the software that is used by group
   1789 filter_techniques = [
   1790     Filter('type', '=', 'attack-pattern'),
   1791     Filter('id', 'in', [s.target_ref for s in software_uses])
   1792 ]

AttributeError: 'function' object has no attribute 'query'

KeyError: 'created_by_ref'

from attackcti import attack_client
from pandas import *
from pandas.io.json import json_normalize

lift = attack_client()
lift.get_all_techniques()
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-6-ef2a7b4f74a0> in <module>()
----> 1 lift.get_all_techniques()

/usr/local/lib/python3.6/site-packages/attackcti/attack_api.py in get_all_techniques(self)
    233         enterprise_techniques = self.get_all_enterprise_techniques()
    234         pre_techniques = self.get_all_pre_techniques()
--> 235         mobile_techniques = self.get_all_mobile_techniques()
    236         all_techniques = enterprise_techniques + pre_techniques + mobile_techniques
    237         return all_techniques

/usr/local/lib/python3.6/site-packages/attackcti/attack_api.py in get_all_mobile_techniques(self)
    389     def get_all_mobile_techniques(self):
    390         mobile_techniques = self.TC_MOBILE_SOURCE.query(Filter("type", "=", "attack-pattern"))
--> 391         mobile_techniques = self.parse_stix_objects(mobile_techniques, 'techniques')
    392         return mobile_techniques
    393 

/usr/local/lib/python3.6/site-packages/attackcti/attack_api.py in parse_stix_objects(self, stix_objects, stix_object_type)
     32                     'type': technique['type'],
     33                     'id': technique['id'],
---> 34                     'created_by_ref': technique['created_by_ref'],
     35                     'created': str(technique['created']),
     36                     'modified': str(technique['modified']),

~/Library/Python/3.6/lib/python/site-packages/stix2/base.py in __getitem__(self, key)
    176 
    177     def __getitem__(self, key):
--> 178         return self._inner[key]
    179 
    180     def __iter__(self):

KeyError: 'created_by_ref'

Data sources enrichment function removes data sources metadata from techniques that do not have 'detects` relationships

In the following function, if the technique ID does not match the target of any detects relationships (data component relationship), it removes the x_mitre_data_sources attribute. This happens to Mobile and ICS matrices. This is because ATT&CK has not added relationships for Mobile and ICS yet.

def enrich_techniques_data_sources(self, stix_object):
        # Get 'detects' relationships
        relationships = self.get_relationships(relationship_type='detects')

        # Get all data component objects
        data_components = self.get_data_components()

        # Get all data source objects without data components objects
        data_sources = self.get_data_sources()

        # Create Data Sources and Data Components lookup tables
        ds_lookup = {ds['id']:ds for ds in data_sources}
        dc_lookup = {dc['id']:dc for dc in data_components}

        # https://stix2.readthedocs.io/en/latest/guide/versioning.html
        for i in range(len(stix_object)):
            if 'x_mitre_data_sources' in stix_object[i].keys():
                technique_ds = dict()
                for rl in relationships:
                    if stix_object[i]['id'] == rl['target_ref']:
                        dc = dc_lookup[rl['source_ref']]
                        dc_ds_ref = dc['x_mitre_data_source_ref']
                        if dc_ds_ref not in technique_ds.keys():
                            technique_ds[dc_ds_ref] = ds_lookup[dc_ds_ref].copy()
                            technique_ds[dc_ds_ref]['data_components'] = list()
                        if dc not in technique_ds[dc_ds_ref]['data_components']:
                            technique_ds[dc_ds_ref]['data_components'].append(dc)
                new_data_sources = [ v for v in technique_ds.values()]
                stix_object[i] = stix_object[i].new_version(x_mitre_data_sources = new_data_sources)
        return stix_object

attack_client().get_techniques() leading to Invalid JSON error

Running the following:

from attackcti import attack_client
techniques = attack_client().get_techniques()

but getting this error... full trace here...

---------------------------------------------------------------------------
JSONDecodeError                           Traceback (most recent call last)
File ~/Library/Caches/pypoetry/virtualenvs/XXX-qNGBkjnK-py3.8/lib/python3.8/site-packages/requests/models.py:910, in Response.json(self, **kwargs)
    909 try:
--> 910     return complexjson.loads(self.text, **kwargs)
    911 except JSONDecodeError as e:
    912     # Catch JSON-related errors and raise as requests.JSONDecodeError
    913     # This aliases json.JSONDecodeError and simplejson.JSONDecodeError

File ~/Library/Caches/pypoetry/virtualenvs/XXX-qNGBkjnK-py3.8/lib/python3.8/site-packages/simplejson/__init__.py:525, in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, use_decimal, **kw)
    521 if (cls is None and encoding is None and object_hook is None and
    522         parse_int is None and parse_float is None and
    523         parse_constant is None and object_pairs_hook is None
    524         and not use_decimal and not kw):
--> 525     return _default_decoder.decode(s)
    526 if cls is None:

File ~/Library/Caches/pypoetry/virtualenvs/XXX-qNGBkjnK-py3.8/lib/python3.8/site-packages/simplejson/decoder.py:370, in JSONDecoder.decode(self, s, _w, _PY3)
    369     s = str(s, self.encoding)
--> 370 obj, end = self.raw_decode(s)
    371 end = _w(s, end).end()

File ~/Library/Caches/pypoetry/virtualenvs/XXX-qNGBkjnK-py3.8/lib/python3.8/site-packages/simplejson/decoder.py:400, in JSONDecoder.raw_decode(self, s, idx, _w, _PY3)
    399         idx += 3
--> 400 return self.scan_once(s, idx=_w(s, idx).end())

JSONDecodeError: Unterminated string starting at: line 1 column 64468 (char 64467)

During handling of the above exception, another exception occurred:

JSONDecodeError                           Traceback (most recent call last)
File ~/Library/Caches/pypoetry/virtualenvs/XXX-qNGBkjnK-py3.8/lib/python3.8/site-packages/taxii2client/common.py:124, in _to_json(resp)
    123 try:
--> 124     return resp.json()
    125 except ValueError as e:
    126     # Maybe better to report the original request URL?

File ~/Library/Caches/pypoetry/virtualenvs/XXX-qNGBkjnK-py3.8/lib/python3.8/site-packages/requests/models.py:917, in Response.json(self, **kwargs)
    916 else:
--> 917     raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)

JSONDecodeError: [Errno Unterminated string starting at] {"type":"bundle","id":"bundle--5c1e0454-6314-4d5b-af6d-8a50e518786a","spec_version":"2.0","objects":[{"object_marking_refs":["marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168"],"type":"attack-pattern","name":"Resource Forking","modified":"2021-10-16T01:50:40.276Z","created":"2021-10-12T20:02:31.866Z","kill_chain_phases":[{"kill_chain_name":"mitre-attack","phase_name":"defense-evasion"}]

leaving out part of string here...

...they may collect sensitive information such as proprietary source code or credentials contained within software's source code.  Having access to software's source code may allow adversaries to develop [Exploits](https://attack.mitre.: 64467

The above exception was the direct cause of the following exception:

InvalidJSONError                          Traceback (most recent call last)
Input In [3], in <module>
----> 1 techniques = attack_client().get_techniques()

File ~/Library/Caches/pypoetry/virtualenvs/XXX-qNGBkjnK-py3.8/lib/python3.8/site-packages/attackcti/attack_api.py:932, in attack_client.get_techniques(self, include_subtechniques, skip_revoked_deprecated, enrich_data_sources, stix_format)
    919 """ Extracts all the available techniques STIX objects across all ATT&CK matrices
    920 
    921 Args: 
   (...)
    928     List of STIX objects
    929 """
    931 if include_subtechniques:
--> 932     all_techniques = self.COMPOSITE_DS.query(Filter("type", "=", "attack-pattern"))
    933 else:
    934     all_techniques = self.COMPOSITE_DS.query([
    935         Filter("type", "=", "attack-pattern"),
    936         Filter('x_mitre_is_subtechnique', '=', False)
    937     ])

File ~/Library/Caches/pypoetry/virtualenvs/XXX-qNGBkjnK-py3.8/lib/python3.8/site-packages/stix2/datastore/__init__.py:570, in CompositeDataSource.query(self, query, _composite_filters)
    567 # federate query to all attached data sources,
    568 # pass composite filters to id
    569 for ds in self.data_sources:
--> 570     data = ds.query(query=query, _composite_filters=all_filters)
    571     all_data.extend(data)
    573 # remove exact duplicates (where duplicates are STIX 2.0
    574 # objects with the same 'id' and 'modified' values)

File ~/Library/Caches/pypoetry/virtualenvs/XXX-qNGBkjnK-py3.8/lib/python3.8/site-packages/stix2/datastore/taxii.py:301, in TAXIICollectionSource.query(self, query, version, _composite_filters)
    299 paged_request = tcv21.as_pages if isinstance(self.collection, tcv21.Collection) else tcv20.as_pages
    300 try:
--> 301     for resource in paged_request(self.collection.get_objects, per_request=self.items_per_page, **taxii_filters_dict):
    302         all_data.extend(resource.get("objects", []))
    303 except HTTPError as e:
    304     # if resources not found or access is denied from TAXII server, return empty list

File ~/Library/Caches/pypoetry/virtualenvs/XXX-qNGBkjnK-py3.8/lib/python3.8/site-packages/taxii2client/v20/__init__.py:36, in as_pages(func, start, per_request, *args, **kwargs)
     25 """Creates a generator for TAXII 2.0 endpoints that support pagination.
     26 
     27 Args:
   (...)
     33 Use args or kwargs to pass filter information or other arguments required to make the call.
     34 """
     35 resp = func(start=start, per_request=per_request, *args, **kwargs)
---> 36 yield _to_json(resp)
     37 total_obtained, total_available = _grab_total_items(resp)
     39 if total_available > per_request and total_obtained != per_request and total_obtained != float("inf"):

File ~/Library/Caches/pypoetry/virtualenvs/XXX-qNGBkjnK-py3.8/lib/python3.8/site-packages/taxii2client/common.py:127, in _to_json(resp)
    124     return resp.json()
    125 except ValueError as e:
    126     # Maybe better to report the original request URL?
--> 127     six.raise_from(InvalidJSONError(
    128         "Invalid JSON was received from " + resp.request.url
    129     ), e)

File <string>:3, in raise_from(value, from_value)

InvalidJSONError: Invalid JSON was received from https://cti-taxii.mitre.org/stix/collections/95ecc380-afe9-11e4-9b6c-751b66dd541e/objects/?match%5Btype%5D=attack-pattern

Thanks!

specify and update README.md file and requirements section

[TO-DO] Add case insensitive features to some of the search functions

One example: #23

The system cannot find the path specified: 'C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\PlatformSDK\\lib'

I had this error message when I tried to install the library in my windows 10 computer

[TO-DO] Process additional references found on relationships objects

503 Error when client = attack_client()

TAXII service returns incomplete data when run in Github actions

In the test used in our Sigma project, we've noticed that results returned from the TAXII service seem incomplete when data is retrieved in a Github workflow.

https://github.com/SigmaHQ/sigma/runs/4868994244?check_suite_focus=true#step:5:17

I've added a line that outputs the length of different lists and it differs when run locally and in a Github workflow. This seems to be the reason why our tests fail. Many MITRE ATT&CK techniques cannot be found in these incomplete lists.

from attackcti import attack_client

def get_mitre_data():
    """
    Generate tags from live TAXI service to get up-to-date data
    """
    # Get ATT&CK information
    lift = attack_client()
    # Techniques
    MITRE_TECHNIQUES = []
    MITRE_TECHNIQUE_NAMES = []
    MITRE_PHASE_NAMES = set()
    MITRE_TOOLS = []
    MITRE_GROUPS = []
    # Techniques 
    enterprise_techniques = lift.get_enterprise_techniques()
    for t in enterprise_techniques:
        MITRE_TECHNIQUE_NAMES.append(t['name'].lower().replace(' ', '_').replace('-', '_'))
        for r in t.external_references:
            if 'external_id' in r:
                MITRE_TECHNIQUES.append(r['external_id'].lower())
        if 'kill_chain_phases' in t:
            for kc in t['kill_chain_phases']:
                if 'phase_name' in kc:
                    MITRE_PHASE_NAMES.add(kc['phase_name'].replace('-','_'))
    # Tools / Malware
    enterprise_tools = lift.get_enterprise_tools()
    for t in enterprise_tools:
        for r in t.external_references:
            if 'external_id' in r:
                MITRE_TOOLS.append(r['external_id'].lower())
    enterprise_malware = lift.get_enterprise_malware()
    for m in enterprise_malware:
        for r in m.external_references:
            if 'external_id' in r:
                MITRE_TOOLS.append(r['external_id'].lower())
    # Groups
    enterprise_groups = lift.get_enterprise_groups()
    for g in enterprise_groups:
        for r in g.external_references:
            if 'external_id' in r:
                MITRE_GROUPS.append(r['external_id'].lower())

    # Debugging 
    print("MITRE ATT&CK LIST LENGTHS: %d %d %d %d %d" % (len(MITRE_TECHNIQUES), len(MITRE_TECHNIQUE_NAMES), len(list(MITRE_PHASE_NAMES)), len(MITRE_GROUPS), len(MITRE_TOOLS)))

You can find the full test script here: https://github.com/SigmaHQ/sigma/blob/rule-devel/tests/test_rules.py

Do you have any idea what could cause this error?

some external references are not available in technique data

Looping on techniques happens that some external references that are available on website are not in returned json data.
For example https://attack.mitre.org/techniques/T1016/001/ shows 3 reference on website so I was expecting 4 external references on json but only the mitre-attack one is available:

There is a reason why some external link are missing or is just because the server is not in sync with latest update on website?
Thanks

MITRE TAXII server doesn't support 2.1, but 2.0.

Yesterday a new version of taxii2-client (oasis-open / cti-taxii-client) was released (version 2.0.0) and this results in an error in attackcti:

Traceback (most recent call last):
File "/Users/ruben/vscode-projects/vscode-attackcti/testit.py", line 4, in
mitre = attack_client()
File "/Users/ruben/vscode-projects/venv-attackcti/lib/python3.8/site-packages/attackcti-0.3.1-py3.8.egg/attackcti/attack_api.py", line 46, in init
File "/Users/ruben/vscode-projects/venv-attackcti/lib/python3.8/site-packages/stix2-1.3.1-py3.8.egg/stix2/datastore/taxii.py", line 164, in init
raise DataSourceError(
stix2.datastore.DataSourceError: The underlying TAXII Collection resource defined in the supplied
Ruben Bouman:
TAXII Collection object provided could not be reached. Recieved error: "406 Client Error: Not Acceptable for url: https://cti-taxii.mitre.org/stix/collections/95ecc380-afe9-11e4-9b6c-751b66dd541e/"

I've investigated the issue and it's because taxii2-client now load TAXII 2.1 classes by default. And the MITRE TAXII server doesn't support 2.1, but 2.0.
When having taxtii2client already installed, there's no problem. But when you install attackcti in a fresh clean environment, taxii2-client==2.0.0 will be installed as dependency.
In my opinion there are two solutions:

Import the 2.0 Collection class in attack_api.py: from taxii2client.v20 import Collection (line 14). This is also mentioned in the release page of taxii2-client: https://github.com/oasis-open/cti-taxii-client/releases
Somehow make sure when attackcti is installed on a system, that taxii2-client==1.0.1 is installed instead of 2.0.0. I'm not sure how this works and if it's possible.
I've tested the first bullet and that's working. Please let me know how what your opinion is and how we can try to tackle this the best way.

Fail to convert "all_techniques" to json file

I want to convert "all_techniques" to JSON file by "json.dump" function.howerver, it reports errors "TypeError: Object of type AttackPattern is not JSON serializable"

lift = attack_client() all_techniques = lift.get_techniques() all_techniques = lift.remove_revoked(all_techniques)

Dynamic Interaction with stix2.v20.sdo object types

Rather than creating our own classes to handle STIX objects, we will use the library object types ;)

Create a function template for functions in attack_client class

A lot of the functions in attack_client class repeat code. We can create a function template and dynamically create those functions 🚀

KeyError: 'v21'

When building a python executable and trying to run a script that fetches Enterprise data through get_enterprise, I'm getting the following error:

  File "attackcti\attack_api.py", line 266, in get_enterprise
  File "stix2\datastore\taxii.py", line 309, in query
  File "stix2\datastore\taxii.py", line 309, in <listcomp>
  File "stix2\parsing.py", line 47, in parse
  File "stix2\parsing.py", line 129, in dict_to_stix2
  File "stix2\parsing.py", line 82, in _detect_spec_version

Is there some way to specify the stix-version when calling the taxii datastore?

Update Docs: Jupyter Notebooks explaining most of the functions available in the library

Several of the current notebooks are not up to date. A few functions need to be run again, some need to be removed and some new ones need to be added to the notebooks.

AttributeError: module 'pandas' has no attribute 'json_normalize'

When running the df.reindex output, using your python commands as a script, I received the error above.

the fix was to add this line to the imports at the top of the script:

from pandas.io.json import json_normalize

ref:
https://stackoverflow.com/questions/61336238/getting-attributeerror-module-pandas-has-no-attribute-json-normalize-while

Ability to retreive CAPEC IDs

Hello Cyb3rWard0g,

Was referred to check this repo out as I was using the MITRE ATT&CK Wiki and I've found your tool to be awesome thus far!

Quick question, I know this is in Beta, but are there any plans to add functionality to be able to grab CAPEC IDs of specific softwares? Currently I can grab all their various Technique IDs, but I'm not sure where or what I need to call, to grab CAPEC ID of any software I query about. As it stands, the MITRE Wiki provides CAPEC IDs for each software in their Software List, so I'm looking into how to get that via your python module.

Thanks!

Expired certificate causes library crash

Hi all!
I was playing around with ATT&CK Python Client when I noticed an error due to the cti-taxii[.]mitre[.]org expired certificate.
I tried to contact MITRE ATT&CK but I did not receive any answer yet.
The library throw an exception since it can not verify the certificate validity
In case this is not the right channel for the notification, please tell me the proper way.
Kind regards,
Giulio

Failed to pass a STIX object (to another function) that was retrieved by get_object_by_attack_id() and get_group_by_alias()

Hey Roberto,
I found few small things that seems like issues (it's also possible I don't yet know how to use this lib properly tho).
my scenario is that I wanted to retrieve a singular STIX object and pass it to another function to do its magic.
desired flow:
fin7_obj = client.get_object_by_attack_id('intrusion-set', 'G0046')
client.get_techniques_used_by_group(fin7_obj)

However I received the following error:
if stix_object['type'] == 'course-of-action': TypeError: list indices must be integers or slices, not str

Looking at the type of object I received from both get_object_by_attack_id() and get_group_by_alias(), it seems that they are retrieved as a list, where the client.get_techniques_used_by_group() expects a 'dict'

And another small issue is that get_group_by_alias() is case sensitive, meaning:
get_group_by_alias('fin7') - Wouldn't work
get_group_by_alias('FIN7') - Would work

Let me know if you need more info, or if i'm just not using this correctly please :D
Have a great day, Sahar

Removed Try Except features and set module to directly use CompositeDataSource queries

In the past, when using CompositeDataSource queries, the functions would fail when a STIX object type would not exist. Now it simply returns empty lists which is better rather than run try/excepts functions to process enterprise, mobile and ICS.

Tactic for T1506 is not present when calling `get_enterprise(stix_format=False)`

Currently using the latest version of the library, when iterating over the techniques returned by get_enterprise the tactic field does not appear present. Base on MITRE it should be get_enterprise

Current example implementation: https://github.com/splunk/security-content/blob/develop/bin/generate.py#L484

The is the print output:

{
  "external_references": [
    {
      "source_name": "mitre-attack",
      "external_id": "T1506",
      "url": "https://attack.mitre.org/techniques/T1506"
    },
    {
      "description": "Rehberger, J. (2018, December). Pivot to the Cloud using Pass the Cookie. Retrieved April 5, 2019.",
      "url": "https://wunderwuzzi23.github.io/blog/passthecookie.html",
      "source_name": "Pass The Cookie"
    },
    {
      "source_name": "Unit 42 Mac Crypto Cookies January 2019",
      "url": "https://unit42.paloaltonetworks.com/mac-malware-steals-cryptocurrency-exchanges-cookies/",
      "description": "Chen, Y., Hu, W., Xu, Z., et. al.. (2019, January 31). Mac Malware Steals Cryptocurrency Exchanges\u2019 Cookies. Retrieved October 14, 2019."
    }
  ],
  "revoked": true,
  "url": "https://attack.mitre.org/techniques/T1506",
  "matrix": "mitre-attack",
  "technique_id": "T1506",
  "technique": "Web Session Cookie",
  "id": "attack-pattern--c5e31fb5-fcbd-48a4-af8c-5a6ed5b932e5",
  "type": "attack-pattern",
  "modified": "2020-01-30T19:59:18.617Z",
  "created": "2019-10-08T20:08:56.205Z"
}
Traceback (most recent call last):
  File "bin/generate.py", line 534, in <module>
    generate_mitre_lookup()
  File "bin/generate.py", line 500, in generate_mitre_lookup
    csv_mitre_rows.append([technique['technique_id'], technique['technique'], '|'.join(technique['tactic']).replace('-',' ').title(), '|'.join(apt_groups)])
KeyError: 'tactic'

Current version:

┌──[jhernandez@jhernandez-mbp-cb9a8]─[~/splunk/security-content on  develop!]
└── # pip show attackcti
Name: attackcti
Version: 0.3.2
Summary: ATTACK CTI Libary
Home-page: https://github.com/hunters-forge/ATTACK-Python-Client
Author: Roberto Rodriguez
Author-email: None
License: BSD
Location: /Users/jhernandez/splunk/security-content/venv/lib/python3.7/site-packages
Requires: taxii2-client, stix2
Required-by:

Should PRE-attack be removed?

hi Roberto!

Because pre-attack is retired/deprecated, I think it should be removed from attackcti as well. What do you think? The thing is that this pre-attack data is not updated anymore in the STIX objects. Functions as get_groups are using the full CompositeDataSource with enterprise+ics+mobile+pre-attack. In this get_groups case, you will also have the pre-attack groups while those groups do not have all fields that enterprise/ics/mobile do have (like x_mitre_domains).

I can imagine that you would like to keep it because of backwardscompatability. But we then can maybe think of a solution that when you create an instance of the attack_client, you can pass an optional parameter to exclude pre-attack. If you want, I can propose a PR for that.

Regards,
Ruben

Remove pre-ATT&CK or mark it as deprecated in the documentation

ATT&CK version 8 migrated the pre-ATT&CK domain into a set of new tactics under the Enterprise domain. While the ATT&CK team will be keeping the STIX data for pre-ATT&CK available in order to avoid breaking any potential user automation, all data in the pre-ATT&CK domain has been marked with the x_mitre_deprecated tag, and all object description fields have been modified to note the migration. See the documentation on MITRE/CTI for more details.

It would be desirable for this tool to also reflect this change to encourage users to discontinue use of the old pre-ATT&CK dataset. However, removing the pre-ATT&CK API could potentially cause problems for users with automation relying on those functions. Several alternative ways to document the removal might be:

marking those functions as deprecated on readthedocs
printing warnings to stdout when the relevant functions are used
Removing the functions from the documentation but keeping them in the codebase
some combination of the above

Proxy Issue?

Hey Roberto,

I am attempting to leverage the attack python client via anaconda 4.6.14 on Windows. I was able to get the module installed but when I attempt to import it (from attackcti import attack_client), I receive an error.

I'm sitting behind a corporate proxy and assuming that is the issue. I've configured the proxy settings in the .condarc file as well as set the HTTP_PROXY and HTTPS_PROXY environment variables.

Is it possible to leverage the attack python client behind a proxy? Any assistance you can provide would be greatly appreciated. The error message is as follows:

---------------------------------------------------------------------------
TimeoutError                              Traceback (most recent call last)
~\AppData\Local\Continuum\anaconda3\envs\attack\lib\site-packages\urllib3\connection.py in _new_conn(self)
    159             conn = connection.create_connection(
--> 160                 (self._dns_host, self.port), self.timeout, **extra_kw)
    161 

~\AppData\Local\Continuum\anaconda3\envs\attack\lib\site-packages\urllib3\util\connection.py in create_connection(address, timeout, source_address, socket_options)
     79     if err is not None:
---> 80         raise err
     81 

~\AppData\Local\Continuum\anaconda3\envs\attack\lib\site-packages\urllib3\util\connection.py in create_connection(address, timeout, source_address, socket_options)
     69                 sock.bind(source_address)
---> 70             sock.connect(sa)
     71             return sock

TimeoutError: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

During handling of the above exception, another exception occurred:

NewConnectionError                        Traceback (most recent call last)
~\AppData\Local\Continuum\anaconda3\envs\attack\lib\site-packages\urllib3\connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    602                                                   body=body, headers=headers,
--> 603                                                   chunked=chunked)
    604 

~\AppData\Local\Continuum\anaconda3\envs\attack\lib\site-packages\urllib3\connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
    343         try:
--> 344             self._validate_conn(conn)
    345         except (SocketTimeout, BaseSSLError) as e:

~\AppData\Local\Continuum\anaconda3\envs\attack\lib\site-packages\urllib3\connectionpool.py in _validate_conn(self, conn)
    842         if not getattr(conn, 'sock', None):  # AppEngine might not have  `.sock`
--> 843             conn.connect()
    844 

~\AppData\Local\Continuum\anaconda3\envs\attack\lib\site-packages\urllib3\connection.py in connect(self)
    315         # Add certificate verification
--> 316         conn = self._new_conn()
    317         hostname = self.host

~\AppData\Local\Continuum\anaconda3\envs\attack\lib\site-packages\urllib3\connection.py in _new_conn(self)
    168             raise NewConnectionError(
--> 169                 self, "Failed to establish a new connection: %s" % e)
    170 

NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x0000000008756320>: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond

During handling of the above exception, another exception occurred:

MaxRetryError                             Traceback (most recent call last)
~\AppData\Local\Continuum\anaconda3\envs\attack\lib\site-packages\requests\adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    448                     retries=self.max_retries,
--> 449                     timeout=timeout
    450                 )

~\AppData\Local\Continuum\anaconda3\envs\attack\lib\site-packages\urllib3\connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    640             retries = retries.increment(method, url, error=e, _pool=self,
--> 641                                         _stacktrace=sys.exc_info()[2])
    642             retries.sleep()

~\AppData\Local\Continuum\anaconda3\envs\attack\lib\site-packages\urllib3\util\retry.py in increment(self, method, url, response, error, _pool, _stacktrace)
    398         if new_retry.is_exhausted():
--> 399             raise MaxRetryError(_pool, url, error or ResponseError(cause))
    400 

MaxRetryError: HTTPSConnectionPool(host='cti-taxii.mitre.org', port=443): Max retries exceeded with url: /stix/collections/95ecc380-afe9-11e4-9b6c-751b66dd541e/ (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x0000000008756320>: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond'))

During handling of the above exception, another exception occurred:

ConnectionError                           Traceback (most recent call last)
<ipython-input-2-8ab5632e9f1f> in <module>
----> 1 from attackcti import attack_client

~\AppData\Local\Continuum\anaconda3\envs\attack\lib\site-packages\attackcti\__init__.py in <module>
      1 #!/usr/bin/env python
      2 
----> 3 from .attack_api import attack_client

~\AppData\Local\Continuum\anaconda3\envs\attack\lib\site-packages\attackcti\attack_api.py in <module>
     20 MOBILE_ATTCK = "2f669986-b40b-4423-b720-4396ca6a462b"
     21 
---> 22 class attack_client(object):
     23     ENTERPRISE_COLLECTION = Collection(ATTCK_STIX_COLLECTIONS + ENTERPRISE_ATTCK + "/")
     24     PRE_COLLECTION = Collection(ATTCK_STIX_COLLECTIONS + PRE_ATTCK + "/")

~\AppData\Local\Continuum\anaconda3\envs\attack\lib\site-packages\attackcti\attack_api.py in attack_client()
     25     MOBILE_COLLECTION = Collection(ATTCK_STIX_COLLECTIONS + MOBILE_ATTCK + "/")
     26 
---> 27     TC_ENTERPRISE_SOURCE = TAXIICollectionSource(ENTERPRISE_COLLECTION)
     28     TC_PRE_SOURCE = TAXIICollectionSource(PRE_COLLECTION)
     29     TC_MOBILE_SOURCE = TAXIICollectionSource(MOBILE_COLLECTION)

~\AppData\Local\Continuum\anaconda3\envs\attack\lib\site-packages\stix2\datastore\taxii.py in __init__(self, collection, allow_custom)
    153 
    154         try:
--> 155             if collection.can_read:
    156                 self.collection = collection
    157             else:

~\AppData\Local\Continuum\anaconda3\envs\attack\lib\site-packages\taxii2client\__init__.py in can_read(self)
    410     @property
    411     def can_read(self):
--> 412         self._ensure_loaded()
    413         return self._can_read
    414 

~\AppData\Local\Continuum\anaconda3\envs\attack\lib\site-packages\taxii2client\__init__.py in _ensure_loaded(self)
    478     def _ensure_loaded(self):
    479         if not self._loaded:
--> 480             self.refresh()
    481 
    482     def _verify_can_read(self):

~\AppData\Local\Continuum\anaconda3\envs\attack\lib\site-packages\taxii2client\__init__.py in refresh(self, accept)
    493         """Update Collection information"""
    494         response = self.__raw = self._conn.get(self.url,
--> 495                                                headers={"Accept": accept})
    496         self._populate_fields(**response)
    497         self._loaded = True

~\AppData\Local\Continuum\anaconda3\envs\attack\lib\site-packages\taxii2client\__init__.py in get(self, url, headers, params)
    937         accept = merged_headers["Accept"]
    938 
--> 939         resp = self.session.get(url, headers=merged_headers, params=params)
    940 
    941         resp.raise_for_status()

~\AppData\Local\Continuum\anaconda3\envs\attack\lib\site-packages\requests\sessions.py in get(self, url, **kwargs)
    544 
    545         kwargs.setdefault('allow_redirects', True)
--> 546         return self.request('GET', url, **kwargs)
    547 
    548     def options(self, url, **kwargs):

~\AppData\Local\Continuum\anaconda3\envs\attack\lib\site-packages\requests\sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
    531         }
    532         send_kwargs.update(settings)
--> 533         resp = self.send(prep, **send_kwargs)
    534 
    535         return resp

~\AppData\Local\Continuum\anaconda3\envs\attack\lib\site-packages\requests\sessions.py in send(self, request, **kwargs)
    644 
    645         # Send the request
--> 646         r = adapter.send(request, **kwargs)
    647 
    648         # Total elapsed time of the request (approximately)

~\AppData\Local\Continuum\anaconda3\envs\attack\lib\site-packages\requests\adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    514                 raise SSLError(e, request=request)
    515 
--> 516             raise ConnectionError(e, request=request)
    517 
    518         except ClosedPoolError as e:

ConnectionError: HTTPSConnectionPool(host='cti-taxii.mitre.org', port=443): Max retries exceeded with url: /stix/collections/95ecc380-afe9-11e4-9b6c-751b66dd541e/ (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x0000000008756320>: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond'))

Error: AttributeError: 'attack_client' object has no attribute 'remove_revoked'

When running code in the provided ATT&CK-Data-Sources notebook, I receive an AttributeError - it appears that there is no remove_revoked attribute.

attack = get_attack_dataframe()
attack.head()

AttributeError                            Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_31480\3093517341.py in <module>
      1 import os
      2 os.environ["REQUESTS_CA_BUNDLE"] = r"c:\root.cer"
----> 3 attack = get_attack_dataframe()
      4 attack.head()

c:\mitre\repos\attack-datasources\docs\scripts\notebook_functions.py in get_attack_dataframe(matrix)
     49     if  (matrix.lower() == 'enterprise'):
     50         # Getting ATT&CK techniques
---> 51         attck = get_attck_from_stix(matrix = matrix)
     52         # Generating a dataframe with information collected
     53         attck = json_normalize(attck)

c:\mitre\repos\attack-datasources\docs\scripts\notebook_functions.py in get_attck_from_stix(matrix)
     38         attck = lift.get_enterprise_techniques(stix_format = False)
     39         # Removing revoked techniques
---> 40         attck = lift.remove_revoked(attck)
     41         return attck
     42     else:

AttributeError: 'attack_client' object has no attribute 'remove_revoked'

Update Navigator version in export_groups_navigator_layers() function to 4.5.5

Current template is defined as version 3.0

We need to update the version value to

"versions": {
  "attack": "10",
  "navigator": "4.5.5",
  "layer": "4.3"
},