Coder Social home page Coder Social logo

oasis-open / cti-stix-slider Goto Github PK

View Code? Open in Web Editor NEW
21.0 10.0 15.0 843 KB

OASIS TC Open Repository: The repository cti-stix-slider supports development of a Python application to convert STIX 2.0 content to STIX 1.x content

Home Page: https://cti-stix-slider.readthedocs.io/en/latest/

License: BSD 3-Clause "New" or "Revised" License

Python 100.00%
python python-script stix stix2 json xml cti oasis converter cyber-threat-intelligence

cti-stix-slider's Introduction

Build_Status Coverage Version

cti-stix-slider

NOTE: This is an OASIS TC Open Repository. See the Governance section for more information.

The stix-slider is a software tool for 'sliding' STIX 2.x JSON to STIX 1.x XML. Due to the differences between STIX 1.x and STIX 2.x, this conversion is best-effort only. During the conversion, stix-slider provides information on the assumptions it needs to make to produce valid STIX 1.x XML, and what information was not able to be converted.

The stix-slider is a work-in-progress. It should be used to explore how STIX 2.x content could potentially be represented in STIX 1.x. Using the current version of the slider will provide insight to issues that might need to be mitigated to convert your STIX 2.x content for use in application that accept only STIX 1.x content.

It should not be used in a production environment, and should not be considered final.

Please enter any comments on how to improve it into the issue tracker.

For more information, see the documentation on ReadTheDocs.

Requirements

  • Python 3.8+

  • python-stix and its dependencies

    Note

    Make sure to use either the latest version of python-stix 1.1.1.x or 1.2.0.x, depending on whether you want to support STIX 1.1.1 or STIX 1.2.

  • python-stix2 >= 3.0.0

  • stixmarx >= 1.0.8

  • stix-validator >= 3.0.0

Installation

Install with pip:

$ pip install stix2-slider

This will install all necessary dependencies, including the latest version of python-stix.

If you need to support older STIX 1.1.1 content, install python-stix 1.1.1.x first:

$ pip install 'stix<1.2'
$ pip install stix2-slider

You can also install the stix-slider from GitHub to get the latest (unstable) version:

$ pip install git+https://github.com/oasis-open/cti-stix-slider.git

Usage

It is recommended that you ensure that the input STIX 2.x file is valid before submitting it to the slider. Use the stix2-validator.

As A Script

The slider comes with a bundled script which you can use to convert STIX 2.x content to STIX 1.x content:

usage: stix2_slider [-h] [--no-squirrel-gaps] [--validator-args VALIDATOR_ARGS]
                    [-e ENABLE] [-d DISABLE] [-s]
                    [--message-log-directory MESSAGE_LOG_DIRECTORY]
                    [--log-level {DEBUG,INFO,WARN,ERROR,CRITICAL}]
                    [--use-namespace USE_NAMESPACE]
                    file

stix2-slider v4.0.1

The stix2-slider is a work-in-progress. It should be used to explore how
existing STIX 2.x would potentially be represented in STIX 1.x. Using the
current version of the stix2-slider will provide insight to issues that might need
to be mitigated so you can use an application that supports only STIX 1.x content.

positional arguments:   The input STIX 2.x document to be 'slid' to STIX 1.x..

optional arguments:
  -h, --help            show this help message and exit

  --no-squirrel-gaps    Do not include STIX 2.x content that cannot be
                        represented directly in STIX 1.x using the description
                        property.

  --validator-args VALIDATOR_ARGS
                        Arguments to pass to stix-validator. Example:
                        stix2_slider <file> --validator-args="--best-
                        practices"

  -e ENABLE, --enable ENABLE
                        A comma-separated list of the stix2-slider messages to
                        enable. If the --disable option is not used, no other
                        messages will be shown. Example: stix2_slider <file>
                        --enable 250

  -d DISABLE, --disable DISABLE
                        A comma-separated list of the stix2-slider messages to
                        disable. Example: stix2_slider <file> --disable
                        212,220

  -s, --silent          If this flag is set. All stix2-slider messages will be
                        disabled.

  --message-log-directory MESSAGE_LOG_DIRECTORY
                        If this flag is set, all stix2-slider messages will be
                        saved to file. The name of the file will be the input
                        file with extension .log in the specified directory.
                        Note, make sure the directory already exists. Example:
                        stix2_slider <file> --message-log-directory "../logs"

  --log-level {DEBUG,INFO,WARN,ERROR,CRITICAL}
                        The logging output level.

  --use-namespace USE_NAMESPACE
                        Override the 'example' namespace with the provided one.
                        The format is the prefix, namespace uri and optionally
                        the schema location separated by a space. Example:
                        stix2_slider <file> --use-namespace="example
                        http://example.com"

Refer to slider_log_messages.rst for all stix2-slider messages. Use the associated code number
to --enable or --disable a message. By default, the stix2-slider displays all
messages. Note: disabling the message does not disable the functionality.

As A Library

You can also use this library to integrate STIX 'sliding' into your own tools. You can slide a STIX 2.x file:

from stix2slider import slide_file
from stix2slider.options import initialize_options

initialize_options()
results = slide_file("stix_file.json")
print(results)

Additionally, a similar method exists to accept a string as an argument:

from stix2slider import slide_string
from stix2slider.options import initialize_options

initialize_options()
results = slide_string("...")
print(results)

To set options, use set_option_value, found in options.py.

Governance

This GitHub public repository ( https://github.com/oasis-open/cti-stix-slider ) was was created at the request of the the OASIS Cyber Threat Intelligence (CTI) TC as an OASIS TC Open Repository to support development of open source resources related to Technical Committee work.

While this TC Open Repository remains associated with the sponsor TC, its development priorities, leadership, intellectual property terms, participation rules, and other matters of governance are separate and distinct from the OASIS TC Process and related policies.

All contributions made to this TC Open Repository are subject to open source license terms expressed in the BSD-3-Clause License. That license was selected as the declared "Applicable License" when the TC Open Repository was created.

As documented in "Public Participation Invited", contributions to this OASIS TC Open Repository are invited from all parties, whether affiliated with OASIS or not. Participants must have a GitHub account, but no fees or OASIS membership obligations are required. Participation is expected to be consistent with the OASIS TC Open Repository Guidelines and Procedures, the open source LICENSE designated for this particular repository, and the requirement for an Individual Contributor License Agreement that governs intellectual property.

Statement of Purpose

Statement of Purpose for this OASIS TC Open Repository (cti-stix- slider) as proposed and approved [bis] by the TC:

This GitHub public repository is provided to support version- controlled development of a Python "slider" application which will convert STIX 2.x content to STIX 1.x content.

Maintainers

TC Open Repository Maintainers are responsible for oversight of this project's community development activities, including evaluation of GitHub pull requests and preserving open source principles of openness and fairness. Maintainers are recognized and trusted experts who serve to implement community goals and consensus design preferences.

Initially, the associated TC members have designated one or more persons to serve as Maintainer(s); subsequently, participating community members may select additional or substitute Maintainers, per consensus agreements.

Current Maintainers of this TC Open Repository

About OASIS TC Open Repositories

Feedback

Questions or comments about this TC Open Repository's activities should be composed as GitHub issues or comments. If use of an issue/comment is not possible or appropriate, questions may be directed by email to the Maintainer(s) listed above. Please send general questions about TC Open Repository participation to OASIS Staff at [email protected] and any specific CLA-related questions to [email protected].

cti-stix-slider's People

Contributors

ajnovice avatar aryabharat avatar clenk avatar ejratl avatar emmanvg avatar gtback avatar jweissm avatar nhova avatar robincover avatar rpiazza avatar yuriyvolkov avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cti-stix-slider's Issues

Stix1.x conversion: missing type="URL" for indicator

For given a stix2.1 bundle, upon converting it to stix1.x the final xml is missing type="URL" in cybox:Properties"> property. This is observed in case of indicator with URL.

Example:

{ "type": "bundle", "id": "bundle--a9d9a0c5-8e15-42b8-9795-45f32a003161", "objects": [ { "type": "marking-definition", "spec_version": "2.1", "id": "marking-definition--f88d31f6-486f-44da-b317-01333bde0b82", "created": "2017-01-20T00:00:00.000Z", "definition_type": "tlp", "name": "TLP:AMBER", "definition": { "tlp": "amber" } }, { "type": "indicator", "spec_version": "2.1", "id": "indicator--f9ca508e-3b75-4d25-955a-e3f150974691", "created": "2024-01-10T22:08:13.811Z", "modified": "2024-01-10T22:08:14.811Z", "name": "aklab3.com/favicon.ico", "indicator_types": [ "anomalous-activity" ], "pattern": "[url:value = 'aklab3.com/favicon.ico']", "pattern_type": "stix", "pattern_version": "2.1", "valid_from": "2024-01-10T22:08:10.07Z", "valid_until": "2024-01-17T22:08:10.07Z", "labels": [ "Recently Reported Spam or Unwanted Content", "Recently Detected Malware Distribution", "Historically Reported Spam or Unwanted Content", "Historically Detected Malware Distribution" ], "confidence": 65, "object_marking_refs": [ "marking-definition--f88d31f6-486f-44da-b317-01333bde0b82" ], "x_ctix_confidence_score": 25 } ] }

After convert it using stix2slider the output is:

`<stix:STIX_Package \n\txmlns:URIObj="http://cybox.mitre.org/objects#URIObject-2"\n\txmlns:tlpMarking="http://data-marking.mitre.org/extensions/MarkingStructure#TLP-1"\n\txmlns:marking="http://data-marking.mitre.org/Marking-1"\n\txmlns:cybox="http://cybox.mitre.org/cybox-2"\n\txmlns:indicator="http://stix.mitre.org/Indicator-2"\n\txmlns:stix="http://stix.mitre.org/stix-1"\n\txmlns:cyboxCommon="http://cybox.mitre.org/common-2"\n\txmlns:stixCommon="http://stix.mitre.org/common-1"\n\txmlns:example="http://example.com"\n\txmlns:xlink="http://www.w3.org/1999/xlink"\n\txmlns:ds="http://www.w3.org/2000/09/xmldsig#"\n\txmlns:xs="http://www.w3.org/2001/XMLSchema"\n\txmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"\n\t id="bundle--a9d9a0c5-8e15-42b8-9795-45f32a003161" version="1.2">\n    
<stix:STIX_Header>\n        
    <stix:Handling>\n            
        <marking:Marking>\n                
            <marking:Marking_Structure id="example:markingstructure-f88d31f6-486f-44da-b317-01333bde0b82" xsi:type=\'tlpMarking:TLPMarkingStructureType\' color="AMBER"/>\n
        </marking:Marking>\n
    </stix:Handling>\n
</stix:STIX_Header>\n
<stix:Indicators>\n        
    <stix:Indicator id="example:indicator-f9ca508e-3b75-4d25-955a-e3f150974691" timestamp="2024-01-10T22:08:14.811000+00:00" xsi:type=\'indicator:IndicatorType\' version="2.2">\n            
        <indicator:Title>aklab3.com/favicon.ico</indicator:Title>\n
        <indicator:Type>anomalous-activity</indicator:Type>\n
        <indicator:Valid_Time_Position>\n                
            <indicator:Start_Time precision="second">2024-01-10T22:08:10.070000+00:00</indicator:Start_Time>\n
            <indicator:End_Time precision="second">2024-01-17T22:08:10.070000+00:00</indicator:End_Time>\n
        </indicator:Valid_Time_Position>\n
        <indicator:Observable id="example:Observable-b6adb005-791f-4107-b2bc-caaf5a6fbc59">\n                
            <cybox:Keywords>\n                    
                <cybox:Keyword>Recently Reported Spam or Unwanted Content</cybox:Keyword>\n
                <cybox:Keyword>Recently Detected Malware Distribution</cybox:Keyword>\n
                <cybox:Keyword>Historically Reported Spam or Unwanted Content</cybox:Keyword>\n
                <cybox:Keyword>Historically Detected Malware Distribution</cybox:Keyword>\n
            </cybox:Keywords>\n
            <cybox:Object id="example:URI-22e7239a-31a7-4d29-8b1a-8c54cc7f51ca">\n                    
                <cybox:Properties xsi:type="URIObj:URIObjectType">\n                        
                    <cyboxCommon:Custom_Properties>\n                            
                        <cyboxCommon:Property name="x_ctix_confidence_score">25</cyboxCommon:Property>\n
                    </cyboxCommon:Custom_Properties>\n
                    <URIObj:Value condition="Equals">aklab3.com/favicon.ico</URIObj:Value>\n
                </cybox:Properties>\n
            </cybox:Object>\n
        </indicator:Observable>\n
        <indicator:Handling>\n                
            <marking:Marking>\n                    
                <marking:Controlled_Structure>../../../descendant-or-self::node() | ../../../descendant-or-self::node()/@*</marking:Controlled_Structure>\n
                <marking:Marking_Structure xsi:type=\'tlpMarking:TLPMarkingStructureType\' color="AMBER"/>\n
            </marking:Marking>\n
        </indicator:Handling>\n
        <indicator:Confidence timestamp="2024-01-12T13:23:28.957386+00:00">\n                
            <stixCommon:Value>Medium</stixCommon:Value>\n
        </indicator:Confidence>\n
    </stix:Indicator>\n
</stix:Indicators>\n

</stix:STIX_Package>\n`

python-cybox's win_user_object.py was renamed

A module which uses stix2slider got following error:

  File "/home/user/venv2/lib/python3.7/site-packages/stix2slider/convert_cyber_observables.py", line 55, in <module>
    from cybox.objects.win_user_object import UserAccount, WinUser                                                                                                                                                
ModuleNotFoundError: No module named 'cybox.objects.win_user_object' 

I checked python-cybox and found that win_user_object.py was renamed to win_user_account_object.py recently.

Namespace is hardcoded as "example"

In convert_stix.py, the Stix namespace is hardcoded as a global variable _ID_NAMESPACE to "example". This is then bound to several string building functions, and cannot be overridden as they refer to the variable directly, not through a package/module.

It would be good to change this value dynamically at runtime, which is not currently possible.

I would suggest defining the variable in stix2slider/init.py so functions can refer to stix2slider._ID_NAMESPACE to both change and retrieve the value.

Introduce a callback API for the slider library

I could be called for objects the Slider does not know how to convert. In this way, users of the slider as a library could introduce some custom code for the content they possess. Also a good idea for the elevator.

TypeError: 'TypedField' object is not callable

In [3]: from stix2slider import slide_file
...: from stix2slider.options import initialize_options as slider_options
...: from stix2validator import validate_file
...: slider_options()
...: result=validate_file(file_name)
...: print("Valid :",result.is_valid)
...: if result.is_valid:
...: results = slide_file(file_name)
...: print(results)
...:
Valid : True
[stixmarx.fields] [INFO ] [2018-05-18 12:02:18,036] No compatible cybox 2.1.0.17 mappings found. Loaded latest unchanged 2.1.0.16 field mappings.
[stix2slider.options] [501] [WARNING] [2018-05-18 12:02:18,260] The 'uses' relationship of relationship--26c5311c-9d9b-4b9b-b3b5-bac10e16a7a3 between campaign and attack-pattern is not supported in STIX 1.x
[stix2slider.options] [501] [WARNING] [2018-05-18 12:02:18,261] The 'uses' relationship of relationship--e794befc-3270-4050-b560-b6b080ab0418 between campaign and attack-pattern is not supported in STIX 1.x
[stix2slider.options] [501] [WARNING] [2018-05-18 12:02:18,261] The 'uses' relationship of relationship--77a4c40e-3c33-43dc-8c78-04992ebcabf2 between campaign and attack-pattern is not supported in STIX 1.x
[stix2slider.options] [501] [WARNING] [2018-05-18 12:02:18,261] The 'mitigates' relationship of relationship--134c393e-cbe0-433c-9a7a-95263ed8578f between course-of-action and malware is not supported in STIX 1.x


TypeError Traceback (most recent call last)
in ()
6 print("Valid :",result.is_valid)
7 if result.is_valid:
----> 8 results = slide_file(file_name)
9 print(results)
10

~/Cyware/source-code/CTIX-S/ctixserver/stix2slider/init.py in slide_file(fn, encoding)
17 obj = stix2.parse(json_content)
18 # TODO: validate STIX 2.0 content - what to do if it is invalid??
---> 19 stix_package = convert_bundle(obj)
20
21 if stix_package:

~/Cyware/source-code/CTIX-S/ctixserver/stix2slider/convert_stix.py in convert_bundle(bundle_obj)
956 for o in bundle_obj["objects"]:
957 if o["type"] == "relationship":
--> 958 process_relationships(o)
959 for o in bundle_obj["objects"]:
960 if "created_by_ref" in o:

~/Cyware/source-code/CTIX-S/ctixserver/stix2slider/convert_stix.py in process_relationships(rel)
711 add_method_info["method"](source_obj, rel["target_ref"], target_obj_idref_1x)
712 else:
--> 713 add_method_info["method"](source_obj, None, target_obj_idref_1x)
714
715

TypeError: 'TypedField' object is not callable

Handle Granular Markings in SDO/SROs

SDO/SRO object_marking_refs are supported by the slider. The granular_markings are not supported, but messages will be logged when encountered.

SerializerFieldNotFoundError: Could not generate an XPath for <stix.extensions.identity.ciq_identity_3_0.CIQIdentity3_0Instance object at 0x7fd6906b8400>

In [2]: from stix2slider import slide_file
...: from stix2slider.options import initialize_options as slider_options
...: from stix2validator import validate_file
...: slider_options()
...: result=validate_file(file_name)
...: print("Valid :",result.is_valid)
...: if result.is_valid:
...: results = slide_file(file_name)
...: print(results)
...:
Valid : True
[stixmarx.fields] [INFO ] [2018-05-18 14:47:43,608] No compatible cybox 2.1.0.17 mappings found. Loaded latest unchanged 2.1.0.16 field mappings.
[stix2slider.options] [502] [WARNING] [2018-05-18 14:47:43,829] labels not representable in a STIX 1.x ToolInformation. Found in tool--ce45f721-af14-4fc0-938c-000c16186418
[stix2slider.options] [502] [WARNING] [2018-05-18 14:47:43,831] labels not representable in a STIX 1.x ToolInformation. Found in tool--e9778c42-bc2f-4eda-9fb4-6a931834f68c
[stix2slider.options] [502] [WARNING] [2018-05-18 14:47:43,832] labels not representable in a STIX 1.x ToolInformation. Found in tool--1cf6a3b8-be43-4c1a-b042-546a890c31b2
[stix2slider.options] [502] [WARNING] [2018-05-18 14:47:43,833] labels not representable in a STIX 1.x ToolInformation. Found in tool--4d82bd3e-24a3-4f9d-b8f3-b57267fe06a9
[stix2slider.options] [502] [WARNING] [2018-05-18 14:47:43,834] labels not representable in a STIX 1.x ToolInformation. Found in tool--7de5dfcc-6809-4772-9f11-cf26c2be53aa
[stix2slider.options] [502] [WARNING] [2018-05-18 14:47:43,835] labels not representable in a STIX 1.x ToolInformation. Found in tool--266b12f2-aa16-4607-809e-f2d33eebb52e
[stix2slider.options] [502] [WARNING] [2018-05-18 14:47:43,836] labels not representable in a STIX 1.x ToolInformation. Found in tool--98fd8dc1-6cc7-4908-899f-07473f55149a
[stix2slider.options] [502] [WARNING] [2018-05-18 14:47:43,838] labels not representable in a STIX 1.x ToolInformation. Found in tool--4215b0e5-928e-4b2a-9b5f-64819f287f48
[stix2slider.options] [502] [WARNING] [2018-05-18 14:47:43,839] labels not representable in a STIX 1.x ToolInformation. Found in tool--a6dd62d0-9683-48bf-a9cd-61e7eceae57e
[stix2slider.options] [502] [WARNING] [2018-05-18 14:47:43,840] labels not representable in a STIX 1.x ToolInformation. Found in tool--806a8f83-4913-4216-bb19-02b48ae25da5


SerializerFieldNotFoundError Traceback (most recent call last)
in ()
6 print("Valid :",result.is_valid)
7 if result.is_valid:
----> 8 results = slide_file(file_name)
9 print(results)
10

~/Cyware/source-code/CTIX-S/ctixserver/stix2slider/init.py in slide_file(fn, encoding)
17 obj = stix2.parse(json_content)
18 # TODO: validate STIX 2.0 content - what to do if it is invalid??
---> 19 stix_package = convert_bundle(obj)
20
21 if stix_package:

~/Cyware/source-code/CTIX-S/ctixserver/stix2slider/convert_stix.py in convert_bundle(bundle_obj)
960 for k, v in _KILL_CHAINS.items():
961 pkg.ttps.kill_chains.append(v["kill_chain"])
--> 962 CONTAINER.flush()
963 CONTAINER = None
964 return pkg

~/Cyware/source-code/CTIX-S/venv/lib/python3.5/site-packages/stixmarx/container.py in flush(self)
319 """
320 writer = serializer.MarkingSerializer(marking_container=self)
--> 321 writer._apply_markings()
322
323 # Reset the collections so we don't return duplicates

~/Cyware/source-code/CTIX-S/venv/lib/python3.5/site-packages/stixmarx/serializer.py in _apply_markings(self)
96 def _apply_markings(self):
97 self._apply_global_markings()
---> 98 self._apply_field_markings()
99 self._apply_null_markings()
100

~/Cyware/source-code/CTIX-S/venv/lib/python3.5/site-packages/stixmarx/serializer.py in _apply_field_markings(self)
79
80 for field, markings_info in iteritems(field_markings):
---> 81 self._apply_markings_to_field(field, markings_info)
82
83 def _apply_null_markings(self):

~/Cyware/source-code/CTIX-S/venv/lib/python3.5/site-packages/stixmarx/serializer.py in _apply_markings_to_field(self, field, marking_info)
68 marking = copy.deepcopy(marking)
69 marking.controlled_structure, handling =
---> 70 self._find_path_and_handling(field, descendants)
71
72 handling.add_marking(marking)

~/Cyware/source-code/CTIX-S/venv/lib/python3.5/site-packages/stixmarx/serializer.py in _find_path_and_handling(self, field, descendants)
210 error = "Could not generate an XPath for {0}".format(field)
211 raise errors.SerializerFieldNotFoundError(entity=field,
--> 212 message=error)
213
214 xpath = xml.XPATH_SELECT_OPERATOR.join(xpath)

SerializerFieldNotFoundError: Could not generate an XPath for <stix.extensions.identity.ciq_identity_3_0.CIQIdentity3_0Instance object at 0x7fd6906b8400>

AttributeError: 'NoneType' object has no attribute 'append'

Hi, All,

When I tried to convert from a six 2.1 content with some external_reference properties, an AttributeError has occurred.

At a create_references function in convert_stix.py, this library tried to info_source.add_reference(rt).

In add_reference function, it tried to append an argument value into a self.references property, however self.references is None.

I will recommend to insert a None check in add_reference, if self.references is None then self.references = [] and append a value.

Fix convert_tool to use fix #310 from python_cybox

 if "labels" in tool2x:
       warn("labels not representable in a STIX 1.x ToolInformation.  Found in %s", 502, tool2x["id"])
       # bug in python_stix prevents using next line of code
       # tool1x.type_ = convert_open_vocabs_to_controlled_vocabs(tool2x["labels"], TOOL_LABELS_MAP)

AttributeError: 'Indicator' object has no attribute 'information_source'

I'm working with some Abuse.ch OSINT data which is published as valid STIX2. Trying to downgrade it to STIX1 but getting an error I don't grok.

trey@birdland:~/.../osint_feeds/abuse.ch:$ stix2_slider urlhaus.json 
Traceback (most recent call last):
  File "/home/trey/osint_feeds/abuse.ch/venv/bin/stix2_slider", line 11, in <module>
    sys.exit(main())
  File "/home/trey/osint_feeds/abuse.ch/venv/local/lib/python2.7/site-packages/stix2slider/cli.py", line 132, in main
    result = slide_file(slider_args.file_)
  File "/home/trey/osint_feeds/abuse.ch/venv/local/lib/python2.7/site-packages/stix2slider/__init__.py", line 28, in slide_file
    stix_package = convert_bundle(obj)
  File "/home/trey/osint_feeds/abuse.ch/venv/local/lib/python2.7/site-packages/stix2slider/convert_stix.py", line 1078, in convert_bundle
    create_references(o)
  File "/home/trey/osint_feeds/abuse.ch/venv/local/lib/python2.7/site-packages/stix2slider/convert_stix.py", line 875, in create_references
    info_source = get_info_source(ob1x, obj)
  File "/home/trey/osint_feeds/abuse.ch/venv/local/lib/python2.7/site-packages/stix2slider/convert_stix.py", line 847, in get_info_source
    if ob1x.information_source:
AttributeError: 'Indicator' object has no attribute 'information_source'

I've tried both the PyPi 1.0.1 and the master branch off GitHub. Makes no difference.

My STIX2 sample is attached.

STIX 2.1 observed-data not supported

When sliding STIX 2.1 observed-data, a KeyError is thrown. The "objects" field in STIX 2.1 is optional and deprecated. This sample observed-data comes from the STIX 2.1 spec (http://docs.oasis-open.org/cti/stix/v2.1/csprd01/stix-v2.1-csprd01.html#_Toc16070679). It also looks like there is not support for the "object_refs" field.

To Reproduce:

>>> from stix2 import v21
>>> from stix2slider import slide_bundle
>>> from stix2slider.options import initialize_options
>>> initialize_options()
>>> objects = [{'type': 'sighting', 'spec_version': '2.1', 'id': 'sighting--ee20065d-2555-424f-ad9e-0f8428623c75', 'created_by_ref': 'identity--f431f809-377b-45e0-aa1c-6a4751cae5ff', 'created': '2016-04-06T20:08:31.000Z', 'modified': '2016-04-06T20:08:31.000Z', 'first_seen': '2015-12-21T19:00:00Z', 'last_seen': '2015-12-21T19:00:01Z', 'count': 50, 'sighting_of_ref': 'indicator--8e2e2d2b-17d4-4cbf-938f-98ee46b3cd3f', 'observed_data_refs': ['observed-data--b67d30ff-02ac-498a-92f9-32f845f448cf'], 'where_sighted_refs': ['identity--b67d30ff-02ac-498a-92f9-32f845f448ff']}, {'type': 'observed-data', 'spec_version': '2.1', 'id': 'observed-data--b67d30ff-02ac-498a-92f9-32f845f448cf', 'created_by_ref': 'identity--f431f809-377b-45e0-aa1c-6a4751cae5ff', 'created': '2016-04-06T19:58:16.000Z', 'modified': '2016-04-06T19:58:16.000Z', 'first_observed': '2015-12-21T19:00:00Z', 'last_observed': '2016-04-06T19:58:16Z', 'number_observed': 50, 'object_refs': ['file--30038539-3eb6-44bc-a59e-d0d3fe84695a']}]
>>> bundle = v21.Bundle(objects)
>>> slide_bundle(bundle)
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File ".../cti-stix-slider/stix2slider/__init__.py", line 98, in slide_bundle
    stix_package = convert_bundle(bundle)
  File ".../cti-stix-slider/stix2slider/convert_stix.py", line 1221, in convert_bundle
    pkg.add_observable(convert_observed_data(o))
  File ".../cti-stix-slider/stix2slider/convert_stix.py", line 679, in convert_observed_data
    o1x.object_ = convert_cyber_observables(od2x["objects"], od2x["id"])
  File ".../venv/lib/python3.7/site-packages/stix2/base.py", line 200, in __getitem__
    return self._inner[key]
KeyError: 'objects'
>>> 

Convert object of type "identity"

It seems that object of type "identity" are not converted to Stix1.

It may be because the STIXPackage class from the base Stix1 library does not appear to have any ability to add identity information.

stix2.exceptions.InvalidValueError

Traceback (most recent call last):
  File "/usr/local/bin/stix2_slider", line 11, in <module>
    load_entry_point('stix2-slider==1.0.1', 'console_scripts', 'stix2_slider')()
  File "/usr/local/lib/python2.7/dist-packages/stix2slider/cli.py", line 132, in main
    result = slide_file(slider_args.file_)
  File "/usr/local/lib/python2.7/dist-packages/stix2slider/__init__.py", line 27, in slide_file
    obj = stix2.parse(json_content)
  File "/usr/local/lib/python2.7/dist-packages/stix2/core.py", line 100, in parse
    obj = dict_to_stix2(obj, allow_custom, version)
  File "/usr/local/lib/python2.7/dist-packages/stix2/core.py", line 147, in dict_to_stix2
    return obj_class(allow_custom=allow_custom, **stix_dict)
  File "/usr/local/lib/python2.7/dist-packages/stix2/core.py", line 68, in __init__
    super(Bundle, self).__init__(**kwargs)
  File "/usr/local/lib/python2.7/dist-packages/stix2/base.py", line 160, in __init__
    self._check_property(prop_name, prop_metadata, setting_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/stix2/base.py", line 93, in _check_property
    raise InvalidValueError(self.__class__, prop_name, reason=str(exc))
stix2.exceptions.InvalidValueError: Invalid value for Bundle 'objects': Invalid value for Indicator 'pattern': FAIL: Error found at line 1:1. extraneous input '"' expecting {IdentifierWithoutHyphen, IdentifierWithHyphen, '('}

Patterns with AND of the same property not handled correctly

in idioms-json/issue-62.json there is a pattern that checks for the existence of many different keys at the same time. The slider knows how to create ANDs on the same object, but not when the property is repeated.

To do this correctly, the AND expression must be processed so in this case each term in the pattern is tested within its own Observable.

"[(windows-registry-key:key = 'HKEY_CURRENT_USER\\Software\\Microsoft\\Windows\\CurrentVersion\\Explorer\\User Shell Folders' AND windows-registry-key:key = 'HKEY_CURRENT_USER\\Software\\Microsoft\\Windows\\CurrentVersion\\Explorer\\Shell Folders' AND windows-registry-key:key = 'HKEY_LOCAL_MACHINE\\Software\\Microsoft\\Tracing' AND windows-registry-key:key = 'HKEY_LOCAL_MACHINE\\Software\\Microsoft\\Windows\\CurrentVersion\\Explorer\\User Shell Folders' AND windows-registry-key:key = 'Software\\Microsoft\\Windows NT\\CurrentVersion\\Winlogon' AND windows-registry-key:key = 'HKEY_LOCAL_MACHINE\\Software\\Microsoft\\Windows\\CurrentVersion\\Explorer\\Shell Folders' AND windows-registry-key:key = 'Software\\Microsoft\\Windows\\CurrentVersion\\Explorer\\User Shell Folders' AND windows-registry-key:key = 'Software\\Microsoft\\Windows\\CurrentVersion\\Explorer\\Shell Folders' AND windows-registry-key:key = 'Software\\Microsoft\\windows\\CurrentVersion\\Internet Settings' AND windows-registry-key:key = 'Software\\Microsoft\\windows\\CurrentVersion\\Internet Settings\\Connections' AND windows-registry-key:key = 'HKEY_CURRENT_CONFIG\\Software\\Microsoft\\windows\\CurrentVersion\\Internet Settings' AND windows-registry-key:key = 'Software\\Microsoft\\SystemCertificates\\MY' AND windows-registry-key:key = 'HKEY_LOCAL_MACHINE\\System\\CurrentControlSet\\Services\\Tcpip\\Parameters')]",

stixCommon:Reference value must be an anyURI format.

I tried to slide a STIX 2.1 which contains vulnerability to STIX 1.2 by using stix-slider.
The vulnerability has an external_references property, and therefore the converted STIX 1.2 has a value like below.

<et:References>
    <stixCommon:Reference>SOURCE: cve - "some url value"</stixCommon:Refernce>
</et:References>

However, the value of stixCommon:Reference must be an anyURI format.

https://stix.mitre.org/language/version1.1/xsddocs/XMLSchema/exploit_target/1.1/stix_common_xsd.html#ReferencesType

I would like to recommend to fix a create_references_for_vulnerability in convert_stix.py like below.

        if er["source_name"] == 'cve' and "url" in er:
            v.add_reference(er["url"])

Issue with an idicator which contains an ipv4-addr and email-addr property.

I found that an indicator which contains "ipv4-addr" pattern strings was skipped to slide.

            "pattern": "[ipv4-addr:value = '8.8.8.8']",

I checked sources, and it seems that there is something wrong in stix2slider/convert_pattrn.py

Line 143:

    def toSTIX1x(self, id2x, existing_obj=None):
        if not existing_obj:
            cyboxClassConstructor = map_extensions_to_cybox_class(self.determine_expression_type())
            existing_obj = cyboxClassConstructor()

map_extensions_to_cybox_class returns cybox.objects.address_object.Address.
However, cyboxClassConstror() returns None.
I think that an Address Constructor shuld call with an indicator value like this.

# return None
obj_ = Address()
# return some instance
obj_ = Address('8.8.8.8')

I have also failed to slide if a pattern string contains email-addr:value.

"pattern": "[email-addr:value = '[email protected]']",

I guess that we should append a "email-addr" : EmailAddress element in _CYBOX_OBJECT_MAP.

Thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.