ashirt-ops / ashirt Goto Github PK

OS version (uname -a if on a Unix-like system. For windows run winver and copy the text starting "Version "): win 11

QT version if dynamically linking (qtdiag |head -n 3): qt 6.3.0

Description of the problem including expected versus actual behavior: If you change the shortcut key, press the shortcut, then click delete ashirt will crash.

Steps to reproduce:

1. Change shortcut key for clipboard
2. Press new shortcut
3. Click delete button

Provide logs (if relevant):

 # Child-SP          RetAddr               Call Site
00 000000d2`d34fb348 00007ff6`4fbe0ebb     Qt6Core!QMimeType::staticMetaObject+0x1ffd8
01 000000d2`d34fb350 00007ff6`4fbda405     ashirt+0x50ebb
02 000000d2`d34fb380 00007fff`20bae340     ashirt+0x4a405
03 000000d2`d34fb3b0 00007fff`20bb05f4     Qt6Core!QObject::qt_static_metacall+0x17c0
04 000000d2`d34fb500 00007ff6`4fbddf70     Qt6Core!QMetaObject::activate+0x84
05 000000d2`d34fb530 00007ff6`4fbe13d9     ashirt+0x4df70
06 000000d2`d34fb570 00007ff6`4fbe0a0a     ashirt+0x513d9
07 000000d2`d34fb660 00007fff`20bae340     ashirt+0x50a0a
08 000000d2`d34fb6e0 00007fff`20bb05f4     Qt6Core!QObject::qt_static_metacall+0x17c0
09 000000d2`d34fb830 00007ff6`4fbdcaa0     Qt6Core!QMetaObject::activate+0x84
0a 000000d2`d34fb860 00007ff6`4fbe54c1     ashirt+0x4caa0
0b 000000d2`d34fb8c0 00007fff`20bae340     ashirt+0x554c1
0c 000000d2`d34fb960 00007fff`20bb05f4     Qt6Core!QObject::qt_static_metacall+0x17c0
0d 000000d2`d34fbab0 00007fff`621cba16     Qt6Core!QMetaObject::activate+0x84
0e 000000d2`d34fbae0 00007fff`621c9b06     Qt6Network!QNetworkConnectionMonitor::isMonitoring+0x8c86
0f 000000d2`d34fbb80 00007ff6`4fbac06a     Qt6Network!QNetworkConnectionMonitor::isMonitoring+0x6d76
10 000000d2`d34fbbe0 00007ff6`4fbe551e     ashirt+0x1c06a
11 000000d2`d34fbc10 00007ff6`4fbdf56d     ashirt+0x5551e
12 000000d2`d34fbc40 00007ff6`4fbd93af     ashirt+0x4f56d
13 000000d2`d34fbc70 00007ff6`4fbd95f4     ashirt+0x493af
14 000000d2`d34fbca0 00007ff6`4fbc11d8     ashirt+0x495f4
15 000000d2`d34fbcd0 00007fff`20bb6863     ashirt+0x311d8
16 000000d2`d34fbd00 00007fff`23affc36     Qt6Core!QObject::event+0xd3
17 000000d2`d34fbf20 00007fff`23ac2f6e     Qt6Widgets!QWidget::event+0xe76
18 000000d2`d34fc000 00007fff`23ac1f9f     Qt6Widgets!QApplicationPrivate::notify_helper+0x10e
19 000000d2`d34fc030 00007fff`20b734e5     Qt6Widgets!QApplication::notify+0x187f
1a 000000d2`d34fc500 00007fff`20b75d3f     Qt6Core!QCoreApplication::notifyInternal2+0xc5
1b 000000d2`d34fc570 00007fff`23690f6f     Qt6Core!QCoreApplicationPrivate::sendPostedEvents+0x21f
1c 000000d2`d34fc650 00007fff`20cceda0     Qt6Gui!QWindowsGuiEventDispatcher::sendPostedEvents+0xf
1d 000000d2`d34fc680 00007fff`23690f49     Qt6Core!QEventDispatcherWin32::processEvents+0x90
1e 000000d2`d34ff7e0 00007fff`20b78bef     Qt6Gui!QWindowsGuiEventDispatcher::processEvents+0x19
1f 000000d2`d34ff810 00007fff`20b712fd     Qt6Core!QEventLoop::exec+0x19f
20 000000d2`d34ff8b0 00007ff6`4fb9baa2     Qt6Core!QCoreApplication::exec+0x15d
21 000000d2`d34ff910 00007ff6`4fbee3a7     ashirt+0xbaa2
22 000000d2`d34ffb00 00007ff6`4fbd7bb2     ashirt+0x5e3a7
23 000000d2`d34ffb90 00007fff`9efd54e0     ashirt+0x47bb2
24 000000d2`d34ffbd0 00007fff`9fbc485b     KERNEL32!BaseThreadInitThunk+0x10
25 000000d2`d34ffc00 00000000`00000000     ntdll!RtlUserThreadStart+0x2b

For evidence that hasn't yet been submitted it should be possible to edit the description and tags.

Remove the close button from the bottom right of the evidence viewer. Normal hotkeys and windows buttons should be sufficient.

Due to potential complexity and differences between platforms automatic updates may not be trivial and may require extra work. This is especially true on macos where screen recording permissions need to be reset. A good first step is simply notifying a user that there is a newer version available and directing the user to the releases page on GitHub.

Automatically update the ashirt client as new versions are released. This should be implemented with cross platform support in mind and take care to leave the system in a state where any tooling will continue to work (eg. uninstallation, application permissions, etc.)

Currently ashirt and aterm each communicate directly with the ashirt-server application. This has worked well for now, and there is an advantage of allowing aterm to be installed on headless hosts that cannot run ashirt.

As we consider adding multi-server support, as well as new client applications, complexity around this approach has come to light. Aterm can read and import configuration from ashirt, however more issues arise with setting changes like deleted servers, the currently selected server, and the current operation.

One possible solution would be to have ashirt be responsible for launching aterm and storing output, similar to how it integrates with various screenshot clients. While this seems simple on the surface, it does not fully account for 3rd party integrations like burp to manage HAR evidence.

Another solution could be to always leverage ashirt as the source-of-truth for server, operation, and evidence management. Local interfaces would need to be created between ashirt and aterm. The ashirt evidence manager would also need to process all types of evidence that ashirt-server supports. This approach would be even more successful if ashirt could be installed and configured on headless hosts, where it mostly functions as an "evidence gateway" to ashirt-server.

This issue serves to collect feedback around the following:

• Do you need to use multiple ashirt servers?
• Do operators require aterm, or future cli clients, to function on hosts that do not support a gui?
  (I personally launch aterm from my workstation, and then ssh to something else after recording has started)
• Do operators require only aterm, and will not use ashirt for other reasons not considered?
• Are there other approaches that can be taken to simplify the interaction between ashirt and aterm?

We already have space for a version number and we should include the commit hash as well. It seems like qmake has support for specifying version information. We might be able to leverage this to pass at compile time and fill it in via the pre-processor. We should look into options here.

The current app bundle generation does not bundle Qt5 with the app bundle. This requires that the user have Qt5 installed on their system, such as through brew, to actually run it. We should have the packaging step provide any libraries necessary to run the bundle.

Note: need to verify that this doesn't effect the licensing in any negative ways. I believe that since we are using the GPLv3 license we are permitted but this needs verification.

Currently creating a new operation requires going into the ashirt web application. Provide the ability to create a new one directly from the ashirt application. This will depend on an endpoint being getting added into the api server to allow for this functionality. The design is currently such that the api server has limited read/edit access so that it can be permitted to live in a more permissive network space (eg. the internet) while the web interface can live somewhere more restrictive (corp). There isn't currently any plan to switch from this but we may need to evaluate that depending on if new management functionality is desired to be added to ashirt (eg. adding users, editing/delete tags, etc.)

Currently the delete option for specific evidence is performed by a button at the bottom right of the evidence viewer. Instead move this functionality into a "right-click" context-menu.

OS version (uname -a if on a Unix-like system. For windows run winver and copy the text starting "Version "): windows 10 and 11

QT version if dynamically linking (qtdiag |head -n 3): qt 6.3.0

Description of the problem including expected versus actual behavior: On key down two dialogs will pop up rather than one for evidence capture. This has only been tested with clipboard and not screenshot.

Steps to reproduce:

1. Assign shortcut for clipboard evidence capture
2. Press shortcut

Provide logs (if relevant):

Currently there is no easy way to switch between ashirt servers. It involves manually swapping your configuration files and restarting ashirt or swapping out your connection information in the settings each time you want to. This make testing or working in multiple environment somewhat difficult. It would be nice to have multiple profiles all with their own keys, hostname, root evidence directory, etc. available to let a user easily switch between servers.

Things to consider:

1. root evidence directories are likely needed to avoid name collisions between servers
2. are multiple databases needed? Aside from compartmentalization are there any collisions that can occur?
3. how are servers switched between? Add a menu item, like operation selection, but for servers? Is there a better way?
4. would you ever want the evidence manager to allow you to look at evidence across multiple profiles?
5. this would likely be a breaking change to the config and directory layout; what is the upgrade path?

Currently running ashirt for the first time on MacOS causes the warning dialog to pop up because it's not signed and requires the extra step to even get it to run. We should be signing and notarizing our official (and maybe dev) releases that are available on github. It probably makes sense to backport the implementation of this to any previous supported releases aside from just pushing into the main branch for future releases and tag point releases for all supported major.minor.

Note: This is blocked on getting an Apple developer account. Need to look into whether we're going to one of Verizon Media's existing accounts, create a new one for ashirt, or I'll just register one for the project.

The tool should theoretically work on Windows and has been built with cross platform support in mind. We should provide official builds and potentially some sort of installer, since that's how windows software is typically distributed. Likely also need to provide guidance on setting up the screenshot helper tools.

We need to decide if we want to build this statically or dynamically. We SHOULD be able to provide a static build with Qt, so long as it can be recompiled against another version, but unclear what makes the most sense. It's probably safe to build this on ubuntu but we should probably do some research into how people distribute official linux binaries. As always, users are free to compile from source.

Move to ubuntu 20.04

OS version (uname -a if on a Unix-like system. For windows run winver and copy the text starting "Version "):
Darwin 19.6.0 Darwin Kernel Version 19.6.0: Thu Jun 18 20:49:00 PDT 2020; root:xnu-6153.141.1~1/RELEASE_X86_64 x86_64

QT version if dynamically linking (qtdiag |head -n 3):
Qt 5.15.0 (x86_64-little_endian-lp64 shared (dynamic) release build; by Clang 11.0.3 (clang-1103.0.32.62) (Apple)) on "cocoa" OS: macOS 10.15 [darwin version 19.6.0]

Description of the problem including expected versus actual behavior:
When clicking on the system tray icon on the primary display (in a multidisplay environment) nothing happens. It works fine on the non-primary display. This might just be a bug in Qt on macos and not an application bug. It's probably just worth asking in #qt on freenode and seeing if this is a known issue and/or if there are workarounds/something we missed. I haven't see anything that directly references this issue but I haven't done a ton of googling or know exactly what we'd search for to find meaningful results.

Steps to reproduce:

1. Start application on a system with two displays
2. Click traybar icon on primary display

If there are no operations loaded (application is offline) and it doesn't know of any operations ashirt will fail to save any evidence. Code blocks will silently fail, never opening the getinfo dialog, and screenshots will launch the screenshot helper but then silently fail. The images and code blocks will be saved to the directory but there will be no acknowledgment of the issue or that it was saved.

could not write to the database: NOT NULL constraint failed: evidence.operation_slug Unable to fetch row

OS version (uname -a if on a Unix-like system. For windows run winver and copy the text starting "Version "): MacOS

QT version if dynamically linking (qtdiag |head -n 3): 5.15.1

Description of the problem including expected versus actual behavior:
In the evidence manager if you have a row selected then edit the filter such that the evidence is no longer listed the description and tags of the previously selected evidence will be cleared out but the preview will remain.

Steps to reproduce:

1. Open evidence manager
2. Select evidence
3. Change filter such that that evidence will no longer be visible

For some workflows with ashirt, and operator might need to capture a large area, and then annotate the image to draw attention to important parts.

Ashirt should perhaps have a separate hotkey combo for taking a screen, opening it in a system image editor for annotation, and then finally forward to ashirt-server.

Currently if you close the window via the close button (or other methods other than the delete button) it does not cause the evidence to be delete. This should not be the case. Any action, other than submit, should result in the evidence being deleted.

Qt has released the new major version 6. It sounds like there's still a handful of features that are not yet available in 6 yet but none of them seem relevant to us. We should evaluate what it will take to support Qt6 and get it building. Given that most Linux distros are unlikely to be shipping it yet, we don't want to move away from Qt5 yet but we do want to make sure that we're developing with the future in mind.

Some big changes include:

• shift in upstream build system from qmake to cmake (we probably want to move to cmake as well)
• upstream has moved to c++17 (we should as well)

Add an option to copy the file path of the evidence to the clipboard to the context menu. This should only be possible when selecting a single entry (if multiselect is added).

OS version (uname -a if on a Unix-like system. For windows run winver and copy the text starting "Version "):
darwin
QT version if dynamically linking (qtdiag |head -n 3):
5.15.2

Description of the problem including expected versus actual behavior:
If you attempt to enter a hotkey combination in the settings menu that is already being used by ashirt, it will not be entered and the application will trigger a screenshot.

Steps to reproduce:
set window hotkey: ctrl+meta+3
set area hotkey: ctrl+meta+4
save settings
attempt to set window hotkey again to ctrl+meta+3, or swap to ctrl+meta+4.

Debug builds or separate debug data would be nice for all platforms for tracking down crashes, especially when it's difficult to reproduce for others. What's the best way to achieve this? Should we make all builds that aren't releases debug builds? Produce separate dwarf/pdb/etc. files and provide them in the artifacts? Should we just not do this?

OS version (uname -a if on a Unix-like system. For windows run winver and copy the text starting "Version "): win 11 (though likely others)

QT version if dynamically linking (qtdiag |head -n 3): qt 6.3.0

Description of the problem including expected versus actual behavior: Reproducible crash on exit from ashirt. Not sure of the exact conditions for replication but I believe it has something to do with in flight http requests attempting to contact the server to retrieve the operations when opening the context menu from the tray icon and it not being complete.

Steps to reproduce:

1. Run ashirt with a server configured but not running
2. Right click on tray icon
3. Select quit

Provide logs (if relevant):

0:000> k
 # Child-SP          RetAddr               Call Site
00 000000ab`1d3df6c0 00007ff6`4fbf87c5     ashirt+0x1c064
01 000000ab`1d3df6f0 00007fff`9d95bea0     ashirt+0x687c5
02 000000ab`1d3df720 00007fff`9d95bdb7     ucrtbase!<lambda_f03950bc5685219e0bcd2087efbe011e>::operator()+0xb0
03 000000ab`1d3df770 00007fff`9d95bd6d     ucrtbase!__crt_seh_guarded_call<int>::operator()<<lambda_7777bce6b2f8c936911f934f8298dc43>,<lambda_f03950bc5685219e0bcd2087efbe011e> &,<lambda_3883c3dff614d5e0c5f61bb1ac94921c> >+0x3b
04 000000ab`1d3df7a0 00007fff`9d95a17a     ucrtbase!execute_onexit_table+0x3d
05 000000ab`1d3df7e0 00007fff`9d95a10f     ucrtbase!<lambda_ad52fe89635f51ec3b38e9c3ac6dac81>::operator()+0x42
06 000000ab`1d3df820 00007fff`9d95a0af     ucrtbase!__crt_seh_guarded_call<void>::operator()<<lambda_123965863b7b46a3332720573f9ce793>,<lambda_ad52fe89635f51ec3b38e9c3ac6dac81> &,<lambda_8d528b66de6ae1e796d7f5e3101fca72> >+0x3b
07 000000ab`1d3df850 00007ff6`4fbd7c14     ucrtbase!common_exit+0x67
08 000000ab`1d3df8b0 00007fff`9efd54e0     ashirt+0x47c14
09 000000ab`1d3df8f0 00007fff`9fbc485b     KERNEL32!BaseThreadInitThunk+0x10
0a 000000ab`1d3df920 00000000`00000000     ntdll!RtlUserThreadStart+0x2b

Currently the evidence manager has a submit button on the button the dialog. Move this functionality into a "right-click" context menu on the evidence line in the table above.

Currently we're using qmake for building but Qt has since moved to cmake and most of the c/c++ ecosystem has standardized around cmake. Let's make the transition. There is a bunch of stuff we get for free with qmake (eg. filling in all the stuff for us for plists, specific automatic qt configurations, etc.) Let's make sure we get all the right ones added.

This shouldn't require changes to the signing process of macos but we need to make sure nothing breaks.

C++ system() on Windows passes our screenshot string to cmd /c for execution. This pops a command window in the foreground and blocks the screenshot itself.

We should investigate moving to QProcess (which can automatically quote paths and provide slashes properly), or windows-specific APIs for launching the program only.

Currently to set global shortcuts requires typing the values (eg. Alt+4) into the text box in the settings. This is error prone and not always clear what the correct thing to type should be. Instead we should allow that text box to read keyboard input and automatically fill in the value based on the key codes read. This will also serve as canonicalization to ensure that the values are the same.

There was discussion a while back about redoing the dmz/zip/app bundle distribution for MacOS and some initial work for it. This was put on the back burner for a variety of reasons but we should re-visit it now that signing is done and we are switching to cmake.

Currently we rely on qmake to produce an app bundle, qtmacdeploy to make the app bundle distributable (providing Qt and updating paths), and do signing separately. The app bundle is stored in a dmg because GitHub actions can't upload a directory to store files between steps and we need to preserve permissions. This all then gets zipped with the README and licensing info because it's easier than adding to the dmg. This requires first unzipping then mounting the dmg to install which is less than ideal.

Moving forward we should create a single dmg, which is the distribution artifact, that includes the app bundle, README, license, and any other relevant distribution info and create a nice background image with links to the Applications directory for easy installation.

This should probably be done after #72 and #93 are completed to avoid having to go through the effort multiple times. This will likely require completely moving away from qtmacdeploy as it doesn't seem configurable enough and probably rewriting some of the signing steps for the new workflow.

I'm currently using 1.1.0 build from source on Linux.

The documentation in README.md documents how to supply commands for capturing screenshots, and that the expected filename will come from ashirt, but it does not mention that the expected screenshots should be saved as PNGs. Depending on the capture program being used, that might influence the command line arguments.

This isn't really needed anymore now that we're not relying on the the built in screenshot capabilities and hijacking the location of the output. It doesn't seem to be working anyway when tested during #11 as the global hotkeys still work when the operation is paused. Let's just rip the functionality out all together.

The esc key seems to work, as you would expect on (maybe) windows but on mac the standard cmd+w hotkey doesn't close the windows. This should work for the about, settings, and evidence manager dialogs. We should check to see if there is something that we missed/need to do before just setting up a shortcut key.

Description of the problem including expected versus actual behavior:
Changing the content that has been read from the clipboard in the evidence submission window does not work. Instead it still submits just what is on the clipboard.

Steps to reproduce:

1. Copy text onto clipboard
2. Press button to submit from clipboard
3. Edit text in evidence submission dialog
4. Submit evidence

Change the old filenames from screenshot.json and screenshot.sqlite to config.json and evidence.sqlite. Might as well do this ahead of tagging a 1.0.

The shirt icon is present on the dock when running on MacOS. We don't really need it there since there's nothing that it can do and it really is just a tray icon application. Our own known issues list this and have this link. keepassxreboot/keepassxc@45344bb#diff-a9e708931297992b08350ff7122fcb91R157. Unsure if this is the proper or best way to address this but maybe worth looking into. If this ends up not being trivial that's fine and we can ignore. This definitely isn't high priority, just a polish thing.

Currently when cleaning up an operation a user must go through and remove all of the evidence one by one and remove them (file and database entry). This is pretty tedious and leaves behind the directory in the evidence repo once everything is gone. #48 will help make this better but we should probably create some one click (plus a confirmation that you actually want to do it) way to delete all the evidence for an operation (files and rows) and the directory in the evidence repository.

Currently the layout causes images to not be displayed within the evidence viewer. This seems to be because the size of the window is just too small on 13" macs. This whole UI should change somewhat with the new tagging system (#9) and some layout changes for buttons and action handling (#6, #7, #8).

Windows builds from main and releases should be signed similar to macos

It would be nice to change the table in the evidence manager to a multi-select table to allow selecting multiple rows at the same time. The only option that really makes sense for a multi-select is probably going to be delete but going through and deleting a bunch of evidence, like everything from an operation, is currently pretty tedious.

This is going to be needed if/when we eventually support Windows due to the path separator being different. We might as well do this now and have it in place so it's there when we need it. Doing do requires that we bump the C++ standard up to C++17 but I don't anticipate that being an issue. We should be able to build the paths with std::filesystem::path and convert it into a String/c_str which can be fed into the constructors that currently use the path.

#142 Migrated to a common dialog class which handles adding the flags for specify visible chrome and behavior. About, Settings, evidence capture no longer have the minimize, maximize, and close icons. The hints likely just need to be added in for this dialogs.

The current tagging UI takes up a lot of space and isn't as simple as it is on the web interface. We want to build something that more closely resembles what we have in the web ui.

Requirements

• Colors of tags match the colors in the web UI
• Ability to type tags in a bar and have them autocomplete if exist
• Creation of new tags upon hitting return if the tag does not already exist for the operation
• Ability to pin/remember tags to have them automatically show up on future evidence until unpinned. This can either use explicit pinning or just remember the last tags used for an initial version, whichever is easier

Golang version (go version):
go version go1.14.1 darwin/amd64
OS version (uname -a if on a Unix-like system):
19.6.0 Darwin Kernel Version 19.6.0: Thu Jun 18 20:49:00 PDT 2020
Description of the problem including expected versus actual behavior:

Ashirt client gives "Connected" when accessing the URL for the UI, even if incorrect keys are submitted. Ashirt client should ensure it is connected to the API before returning "Connected"

Steps to reproduce:

Please include a minimal but complete recreation of the problem,
including (e.g.) index creation, mappings, settings, query etc. The easier
you make for us to reproduce it, the more likely that somebody will take the
time to look at it.

Provide logs (if relevant):

This is the last form that was built with the designer once some form of #29 goes in.

Currently the only way to capture a code block is to select the option within the traybar. We need to add support for global hotkeys like with screenshots. This should include the UI changes in the settings as well to support setting the hotkey.

OS version (uname -a if on a Unix-like system. For windows run winver and copy the text starting "Version "): Windows and Linux

QT version if dynamically linking (qtdiag |head -n 3): Qt 6.3.0

Description of the problem including expected versus actual behavior: When changing from light to dark mode or vice versa the icon in the tray bar will not update. The code to support this has been implemented in #138 but there is an upstream bug in Qt causing it not to work. The issues is tracked https://bugreports.qt.io/browse/QTBUG-103093. We need to wait for either Qt to fix the bug or provide a new solution to implement this.

Steps to reproduce:

1. Start ashirt
2. Switch from light to dark or dark to light

OS version: N/A (all oses)

QT version: N/A (>5.14)

Description of the problem including expected versus actual behavior:

If a user rapidly scrolls through the evidence menu (via selecting evidence), then multiple requests are sent on the same object, producing warnings to the console

Impact: Unknown. Possible system instability, but unclear. Possibly no impact.

Steps to reproduce:
(Should be done while viewing the console logs)

1. Open the application, have a reasonably substantial number of evidence (10-20+) -- type does not matter
2. Open the "View Accumulated Evidence" view
3. Select the first piece of evidence
4. Hold down the "down arrow" button to rapidly change the selection

Provide logs (if relevant):
error logs will look like the following:

QIODevice::read (QNetworkReplyHttpImpl): device not open
QCoreApplication::postEvent: Unexpected null receiver
QIODevice::read (QNetworkReplyHttpImpl): device not open
QCoreApplication::postEvent: Unexpected null receiver
QNetworkReplyImplPrivate::error: Internal problem, this method must only be called once.

Research:
Internally, we are only providing 1 request object for all requests. Each request made tries to re-use this object. Upon completion of the request, the object is deleted and can be re-used for a future request. This works fine in situations where requests can be gated. Unfortunately, in this situation, we cannot use this technique.

One possible solution is to cache requests for each operation for a period of time. As each request is just to gather the full set of tags, this data is unlikely to rapidly change with each request, and it should be reasonable to keep data around for seconds or minutes, or longer, as needed. Doing this, we can properly ignore repeated requests for the same operation, and allow multiple requests for multiple operations. Services requiring tag lookups can then wait for a signal from the caching monitor to properly fill in this data.