Coder Social home page Coder Social logo

mitre-atlas / arsenal Goto Github PK

View Code? Open in Web Editor NEW
78.0 2.0 8.0 17 MB

CALDERA plugin for adversary emulation of AI-enabled systems

Home Page: https://mitre-atlas.github.io/arsenal/

License: Apache License 2.0

Python 79.10% Shell 13.80% HTML 7.10%
caldera-plugin cybersecurity machine-learning

arsenal's Issues

venv dependency not found when running via caldera-atlas

Describe the bug
Arsenal encounters an error about not having python3.10-venv available during start within caldera-atlas usage, i.e. CALDERA in a Docker container.

Not sure where to address - is there a way to define additional dependencies for CALDERA to install from a plugin? Or is this something caldera-atlas should do.

To Reproduce
Steps to reproduce the behavior:

Set up caldera-atlas

  1. git clone --recursive https://github.com/mitre-atlas/caldera-atlas
    If one has SSH keys uploaded, this should check out all submodules.
    Otherwise, there may be failures because some submodules have SSH URLs. To resolve:
    a. git checkout https to get the branch with HTTPS URLs
    b. git submodule sync --recursive && git submodule update --init

  2. docker-compose build
    If one gets an OpenSSL UnsafeLegacyRenegotiation error, resolve it as one sees fit but can add this to caldera-atlas/caldera/Dockerfile ~L18 after the RUN apt-get ...

# OpenSSL 3.0 disables UnsafeLegacyRenegotiation by default, must re-enable it for some endpoints (see https://github.com/dotnet/runtime/issues/80641)
RUN sed -i 's/providers = provider_sect/providers = provider_sect\n\
        ssl_conf = ssl_sect\n\
        \n\
        [ssl_sect]\n\
        system_default = system_default_sect\n\
        \n\
        [system_default_sect]\n\
        Options = UnsafeLegacyRenegotiation/' /etc/ssl/openssl.cnf
  1. docker-compose up -d

The container exits shortly after this - see the trace below

Expected behavior
Container stays up

Trace
From the Docker container log of CALDERA (the caldera-atlas service):

2023-03-15 16:37:41 - INFO  (app_svc.py:116 load) Enabled plugin: access
2023-03-15 16:37:41 - DEBUG (arsenal_svc.py:122 _create_venv)
Building cf_venv...
The virtual environment was not created successfully because ensurepip is not
available.  On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.

    apt install python3.10-venv

You may need to use sudo with that command.  After installing the python3-venv
package, recreate your virtual environment.

Failing command: ['/root/venv/cf_venv/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']

2023-03-15 16:37:41 - ERROR (base_events.py:1744 default_exception_handler) Task exception was never retrieved
future: <Task finished name='Task-33' coro=<AppService.load_plugins.<locals>.load() done, defined at /usr/src/app/app/service/app_svc.py:108> exception=SystemExit(1)>
Traceback (most recent call last):
  File "/usr/lib/python3.10/venv/__init__.py", line 318, in _setup_pip
    subprocess.check_output(
  File "/usr/lib/python3.10/subprocess.py", line 420, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.10/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/root/venv/cf_venv/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.10/asyncio/base_events.py", line 633, in run_until_complete
    self.run_forever()
  File "/usr/lib/python3.10/asyncio/base_events.py", line 600, in run_forever
    self._run_once()
  File "/usr/lib/python3.10/asyncio/base_events.py", line 1896, in _run_once
    handle._run()
  File "/usr/lib/python3.10/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/src/app/app/service/app_svc.py", line 115, in load
    await plugin.enable(self.get_services())
  File "/usr/src/app/app/objects/c_plugin.py", line 67, in enable
    await plugin(services)
  File "/usr/src/app/plugins/arsenal/hook.py", line 15, in enable
    await arsenal_svc.populate_venv(env_name='cf_venv', with_counterfit=True)
  File "/usr/src/app/plugins/arsenal/app/arsenal_svc.py", line 62, in populate_venv
    await self._create_venv(env_name=env_name)
  File "/usr/src/app/plugins/arsenal/app/arsenal_svc.py", line 136, in _create_venv
    builder.create(env_dir=env_dir)
  File "/usr/lib/python3.10/venv/__init__.py", line 75, in create
    self._setup_pip(context)
  File "/usr/lib/python3.10/venv/__init__.py", line 336, in _setup_pip
    sys.exit(1)
SystemExit: 1

Desktop (please complete the following information):
Ubuntu 22.04 but running CALDERA as a container using its own Dockerfile, which uses ubuntu:latest

Additional context
Add any other context about the problem here.

Plugin won't load: error installing counterfit

Describe the bug
Arsenal plugin errors on installing counterfit

To Reproduce
Steps to reproduce the behavior:

Set up caldera-atlas:

  1. git clone --recursive https://github.com/mitre-atlas/caldera-atlas
    If one has SSH keys uploaded, this should check out all submodules.
    Otherwise, there may be failures because some submodules have SSH URLs. To resolve:
    a. git checkout https to get the branch with HTTPS URLs
    b. git submodule sync --recursive && git submodule update --init

  2. docker-compose build
    If one gets an error about python3-venv not being found, add python3-venv to the list of packages installed by CALDERA itself - edit the RUN apt-get ... step in the caldera/Dockerfile, then re-run docker-compose build

The services stay up, but upon visiting localhost:8888, Arsenal is not in the list of plugins and there is an error in the service start:

From the Docker container log of CALDERA (the caldera-atlas service):

ERROR (c_plugin.py:70 enable) Error enabling plugin=arsenal, Command '['/root/venv/cf_venv/bin/python3', '-m', 'pip', 'install', 'counterfit[dev] @ git+https://github.com/Azure/counterfit.git@main']' returned non-zero exit status 1.

Expected behavior
Arsenal plugin loads without error and appears in the CALDERA sidebar.

Desktop (please complete the following information):
Ubuntu 22.04 but running CALDERA as a container using its own Dockerfile, which uses ubuntu:latest

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.