ghostmanager / ghostwriter Goto Github PK
View Code? Open in Web Editor NEWThe SpecterOps project management and reporting engine
Home Page: https://ghostwriter.wiki
License: BSD 3-Clause "New" or "Revised" License
The SpecterOps project management and reporting engine
Home Page: https://ghostwriter.wiki
License: BSD 3-Clause "New" or "Revised" License
ERROR 2020-03-09 18:01:52,444 log 13 139773665434952 Internal Server Error: /reporting/reports/6/generate_docx/
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/python3.7/contextlib.py", line 74, in inner
return func(*args, **kwds)
File "/usr/local/lib/python3.7/site-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view
return view_func(request, *args, **kwargs)
File "/app/ghostwriter/reporting/views.py", line 577, in generate_docx
docx = spenny.generate_word_docx()
File "/app/ghostwriter/modules/reportwriter.py", line 1312, in generate_word_docx
report_json)
File "/app/ghostwriter/modules/reportwriter.py", line 757, in process_text_xml
self.process_inline_text(line, p)
File "/app/ghostwriter/modules/reportwriter.py", line 489, in process_inline_text
run = p.add_run()
AttributeError: 'NoneType' object has no attribute 'add_run'
When there is an inline <code>
section in the findings
Fairly recently the bluecoat sitereview process was updated with additional checks to prevent automated use. I'm assuming this is why my bluecoat categorised domains return 'No Results' in the domain health section?
Getting the following error when running docker-compose -f local.yml up -d
Building wheel for lxml (setup.py): started
ERROR: Command errored out with exit status 1:
command: /usr/local/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-s37it2bz/lxml/setup.py'"'"'; file='"'"'/tmp/pip-install-s37it2bz/lxml/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-z2rnz7g3 --python-tag cp37
cwd: /tmp/pip-install-s37it2bz/lxml/
Complete output (88 lines):
Building lxml version 4.4.0.
Building without Cython.
Using build configuration of libxslt 1.1.33
Building against libxml2/libxslt in the following directory: /usr/lib
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/lxml
copying src/lxml/doctestcompare.py -> build/lib.linux-x86_64-3.7/lxml
copying src/lxml/ElementInclude.py -> build/lib.linux-x86_64-3.7/lxml
copying src/lxml/builder.py -> build/lib.linux-x86_64-3.7/lxml
copying src/lxml/usedoctest.py -> build/lib.linux-x86_64-3.7/lxml
copying src/lxml/pyclasslookup.py -> build/lib.linux-x86_64-3.7/lxml
copying src/lxml/sax.py -> build/lib.linux-x86_64-3.7/lxml
copying src/lxml/cssselect.py -> build/lib.linux-x86_64-3.7/lxml
copying src/lxml/init.py -> build/lib.linux-x86_64-3.7/lxml
copying src/lxml/_elementpath.py -> build/lib.linux-x86_64-3.7/lxml
creating build/lib.linux-x86_64-3.7/lxml/includes
copying src/lxml/includes/init.py -> build/lib.linux-x86_64-3.7/lxml/includes
creating build/lib.linux-x86_64-3.7/lxml/html
copying src/lxml/html/soupparser.py -> build/lib.linux-x86_64-3.7/lxml/html
copying src/lxml/html/formfill.py -> build/lib.linux-x86_64-3.7/lxml/html
copying src/lxml/html/builder.py -> build/lib.linux-x86_64-3.7/lxml/html
copying src/lxml/html/html5parser.py -> build/lib.linux-x86_64-3.7/lxml/html
copying src/lxml/html/_diffcommand.py -> build/lib.linux-x86_64-3.7/lxml/html
copying src/lxml/html/usedoctest.py -> build/lib.linux-x86_64-3.7/lxml/html
copying src/lxml/html/clean.py -> build/lib.linux-x86_64-3.7/lxml/html
copying src/lxml/html/ElementSoup.py -> build/lib.linux-x86_64-3.7/lxml/html
copying src/lxml/html/_setmixin.py -> build/lib.linux-x86_64-3.7/lxml/html
copying src/lxml/html/_html5builder.py -> build/lib.linux-x86_64-3.7/lxml/html
copying src/lxml/html/init.py -> build/lib.linux-x86_64-3.7/lxml/html
copying src/lxml/html/diff.py -> build/lib.linux-x86_64-3.7/lxml/html
copying src/lxml/html/defs.py -> build/lib.linux-x86_64-3.7/lxml/html
creating build/lib.linux-x86_64-3.7/lxml/isoschematron
copying src/lxml/isoschematron/init.py -> build/lib.linux-x86_64-3.7/lxml/isoschematron
copying src/lxml/etree.h -> build/lib.linux-x86_64-3.7/lxml
copying src/lxml/etree_api.h -> build/lib.linux-x86_64-3.7/lxml
copying src/lxml/lxml.etree.h -> build/lib.linux-x86_64-3.7/lxml
copying src/lxml/lxml.etree_api.h -> build/lib.linux-x86_64-3.7/lxml
copying src/lxml/includes/c14n.pxd -> build/lib.linux-x86_64-3.7/lxml/includes
copying src/lxml/includes/uri.pxd -> build/lib.linux-x86_64-3.7/lxml/includes
copying src/lxml/includes/xmlschema.pxd -> build/lib.linux-x86_64-3.7/lxml/includes
copying src/lxml/includes/xinclude.pxd -> build/lib.linux-x86_64-3.7/lxml/includes
copying src/lxml/includes/htmlparser.pxd -> build/lib.linux-x86_64-3.7/lxml/includes
copying src/lxml/includes/xslt.pxd -> build/lib.linux-x86_64-3.7/lxml/includes
copying src/lxml/includes/xmlerror.pxd -> build/lib.linux-x86_64-3.7/lxml/includes
copying src/lxml/includes/init.pxd -> build/lib.linux-x86_64-3.7/lxml/includes
copying src/lxml/includes/relaxng.pxd -> build/lib.linux-x86_64-3.7/lxml/includes
copying src/lxml/includes/dtdvalid.pxd -> build/lib.linux-x86_64-3.7/lxml/includes
copying src/lxml/includes/config.pxd -> build/lib.linux-x86_64-3.7/lxml/includes
copying src/lxml/includes/xpath.pxd -> build/lib.linux-x86_64-3.7/lxml/includes
copying src/lxml/includes/tree.pxd -> build/lib.linux-x86_64-3.7/lxml/includes
copying src/lxml/includes/xmlparser.pxd -> build/lib.linux-x86_64-3.7/lxml/includes
copying src/lxml/includes/etreepublic.pxd -> build/lib.linux-x86_64-3.7/lxml/includes
copying src/lxml/includes/schematron.pxd -> build/lib.linux-x86_64-3.7/lxml/includes
copying src/lxml/includes/etree_defs.h -> build/lib.linux-x86_64-3.7/lxml/includes
copying src/lxml/includes/lxml-version.h -> build/lib.linux-x86_64-3.7/lxml/includes
creating build/lib.linux-x86_64-3.7/lxml/isoschematron/resources
creating build/lib.linux-x86_64-3.7/lxml/isoschematron/resources/rng
copying src/lxml/isoschematron/resources/rng/iso-schematron.rng -> build/lib.linux-x86_64-3.7/lxml/isoschematron/resources/rng
creating build/lib.linux-x86_64-3.7/lxml/isoschematron/resources/xsl
copying src/lxml/isoschematron/resources/xsl/XSD2Schtrn.xsl -> build/lib.linux-x86_64-3.7/lxml/isoschematron/resources/xsl
copying src/lxml/isoschematron/resources/xsl/RNG2Schtrn.xsl -> build/lib.linux-x86_64-3.7/lxml/isoschematron/resources/xsl
creating build/lib.linux-x86_64-3.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_skeleton_for_xslt1.xsl -> build/lib.linux-x86_64-3.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_dsdl_include.xsl -> build/lib.linux-x86_64-3.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_message.xsl -> build/lib.linux-x86_64-3.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_svrl_for_xslt1.xsl -> build/lib.linux-x86_64-3.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_abstract_expand.xsl -> build/lib.linux-x86_64-3.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/readme.txt -> build/lib.linux-x86_64-3.7/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
running build_ext
building 'lxml.etree' extension
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/src
creating build/temp.linux-x86_64-3.7/src/lxml
gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -DTHREAD_STACK_SIZE=0x100000 -fPIC -DCYTHON_CLINE_IN_TRACEBACK=0 -I/usr/include/libxml2 -Isrc -Isrc/lxml/includes -I/usr/local/include/python3.7m -c src/lxml/etree.c -o build/temp.linux-x86_64-3.7/src/lxml/etree.o -w
ERROR: Failed building wheel for lxml
Building wheel for lxml (setup.py): finished with status 'error'
When performing an update for the domain health statuses, categories and DNS records for all domains within Ghostwriter slack provides repetitive updates. See below for example:
@ghostwriter domain.com has been flagged as burned because: Tied to a VirusTotal detected URL, Tied to a VirusTotal detected URL, Tied to a VirusTotal detected URL, Tied to a VirusTotal detected URL, Tied to a VirusTotal detected URL (Bad categories: uncategorized, No Tags, Phishing URLs)
Getting the following error after pulling #25
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
psycopg2.errors.UndefinedTable: relation "rolodex_objectivestatus" does not exist
LINE 1: UPDATE "rolodex_objectivestatus" SET "objective_status" = 'A...
In the instructions is written that the CSV file should have the header whois
and the template has a column with the same name.
I couldn't upload a file, so checking the code I figured it out that the actual name of the column is whois_status
Ghostwriter/ghostwriter/shepherd/views.py
Line 345 in 25aa057
Changing the name of the column header in my CSV file worked fine.
I have pull the merge request #7 into my STG system but I am now getting the following error when accessing any items that was created before that PR.
ERROR
django_1 | ERROR 2019-09-10 13:25:17,294 log 14 140591908080968 Internal Server Error: /rolodex/projects/5
django_1 | Traceback (most recent call last):
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in execute
django_1 | return self.cursor.execute(sql, params)
django_1 | psycopg2.errors.UndefinedColumn: column shepherd_transientserver.name does not exist
django_1 | LINE 1: ...r"."id", "shepherd_transientserver"."ip_address", "shepherd...
django_1 | ^
django_1 | HINT: Perhaps you meant to reference the column "shepherd_transientserver.note".
django_1 |
django_1 |
django_1 | The above exception was the direct cause of the following exception:
django_1 |
django_1 | Traceback (most recent call last):
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
django_1 | response = get_response(request)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", line 145, in _get_response
django_1 | response = self.process_exception_by_middleware(e, request)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", line 143, in _get_response
django_1 | response = response.render()
django_1 | File "/usr/local/lib/python3.7/site-packages/django/template/response.py", line 106, in render
django_1 | self.content = self.rendered_content
django_1 | File "/usr/local/lib/python3.7/site-packages/django/template/response.py", line 83, in rendered_content
django_1 | content = template.render(context, self._request)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/template/backends/django.py", line 61, in render
django_1 | return self.template.render(context)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 171, in render
django_1 | return self._render(context)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 163, in _render
django_1 | return self.nodelist.render(context)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 937, in render
django_1 | bit = node.render_annotated(context)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 904, in render_annotated
django_1 | return self.render(context)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/template/loader_tags.py", line 150, in render
django_1 | return compiled_parent._render(context)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 163, in _render
django_1 | return self.nodelist.render(context)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 937, in render
django_1 | bit = node.render_annotated(context)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 904, in render_annotated
django_1 | return self.render(context)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/template/loader_tags.py", line 62, in render
django_1 | result = block.nodelist.render(context)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 937, in render
django_1 | bit = node.render_annotated(context)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/template/base.py", line 904, in render_annotated
django_1 | return self.render(context)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/template/defaulttags.py", line 308, in render
django_1 | if match:
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 278, in bool
django_1 | self._fetch_all()
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 1242, in _fetch_all
django_1 | self._result_cache = list(self._iterable_class(self))
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 55, in iter
django_1 | results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1100, in execute_sql
django_1 | cursor.execute(sql, params)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 67, in execute
django_1 | return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
django_1 | return executor(sql, params, many, context)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
django_1 | return self.cursor.execute(sql, params)
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/utils.py", line 89, in exit
django_1 | raise dj_exc_value.with_traceback(traceback) from exc_value
django_1 | File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in execute
django_1 | return self.cursor.execute(sql, params)
django_1 | django.db.utils.ProgrammingError: column shepherd_transientserver.name does not exist
django_1 | LINE 1: ...r"."id", "shepherd_transientserver"."ip_address", "shepherd...
django_1 | ^
django_1 | HINT: Perhaps you meant to reference the column "shepherd_transientserver.note".
django_1 |
After adding a finding to a report, the finding Guidance is missing when editing the finding.
When trying to import the domains using the provided template, I receive the following error:
Unable to read rows: KeyError('ip_address')
I'll dig in the code and make a PR if I can fix it
I am having trouble finding how to create and pre-load users with passwords. Logging in with the administrator account, I can create users but there is no password field or options. I can set Full Name, User Name, Email Address and the two advanced options. Only way I have been able to find to create a user with specific password is to set DJANGO_ACCOUNT_ALLOW_REGISTRATION to true.
It would be nice to have support for domain fronting: keeping track of the different domains we use for that and which server they redirect to.
For example {{.client}} isn't filled in pptx, xlsx or json.
It would be interesting to allow using custom report templates. This could potentially be achieved by implementing something similar as done with Serpico
Just had an idea to improve the tool: it would be helpful to have a timeline functionality to track key events when performing a red team assessment.
Example of structure:
I'll see if I can make a PR with this functionality, but wanted to check with you first is this is something you would like to integrate / are already working on.
Currently, the project page doesn't allow for you to import or provide a list of IP Addresses, domain names, or other asset items as a section denoting the scope of the project. This could assist with the findings reporting by allowing for autocomplete or in-scope items / hosts.
This is an idea for further usability.
Give the user the ability to populate their personas used on engagements. For instance when simulating multiple attacks I create an individual character for each team member on the attacking team. It’s kinda like building a DnD character. How I respond to certain situations depends on which persona is on keyboard.
Some stats I track:
Best OS
nano vs vim vs emacs
Sysadmin skill
Developer skill
Operator skill
Self taught or military or college
Keep updated on TTP?
Go-to attack
Are they moonlighting?
Typing ticks.
The idea would be to track personas in use on each test so operators can check them in and out like a domain. If a persona gets caught to many times he gets blown like domain names do.
Generate a .docx file without any issue, either ignoring the <br>
tag or by adding a linebreak.
An error shows in the web interface stating:
Failed to generate the Word report for an unknown reason: list index out of range
The docker logs show the error
Unknown nested tag - br
<br>
tag to one of the fieldsI am using the latest code from Github and running everything in the production docker image.
The issue started to appear after copy and pasting some output from my terminal into one of the fields. The WYSIWYG text editor automatically added the <br />
tags and caused the issue to appear.
This idea is to add tasks which allow the automated import of servers from one or more providers, such as:
etc.
This would allow the server page to stay up to date without the need for manually importing.
👋
TLDR; Is there a space we can collaborate on Ghostwriter improvements and QA? For example, an IRC channel? I'd be happy to help coordinate or assist however I can.
Long Story
We've (https://dragos.com/) tested Ghostwriter on several engagements and the results were great. We learned a lot and have made some tweaks specific to us. For example, we made certain sections not generate in the docx if they are empty (dbuentello@43776ed). Maybe others could benefit from this and I should submit a PR or maybe I'm "Ghostwriter-ing" wrong. As we start to roll this out to more users (and developers) it would be nice to have another mechanism to communicate other than Github issues.
When running the task ghostwriter.shepherd.tasks.fetch_namecheap_domains to pull domains from namecheap, the following error appears in the logs. The task completes successfully without error in the interface, but no values populate and there are no exceptions thrown.
postgres_1 | 2019-10-15 20:54:43.859 UTC [385] DETAIL: Failing row contains (84, domain.net, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, 1, 1, null, 1).
postgres_1 | 2019-10-15 20:54:43.859 UTC [385] STATEMENT: INSERT INTO "shepherd_domain" ("name", "registrar", "dns_record", "health_dns", "creation", "expiration", "all_cat", "ibm_xforce_cat", "talos_cat", "bluecoat_cat", "fortiguard_cat", "opendns_cat", "trendmicro_cat", "mx_toolbox_status", "note", "burned_explanation", "whois_status_id", "health_status_id", "domain_status_id", "last_used_by_id") VALUES ('domain.net', NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, 1, 1, 1, NULL) RETURNING "shepherd_domain"."id"
postgres_1 | 2019-10-15 20:54:43.865 UTC [385] ERROR: null value in column "creation" violates not-null constraint
When using the DNS record update function in the Domain Update Controls, I get the following error:
Error: value too long for type character varying(500) : Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) psycopg2.errors.StringDataRightTruncation: value too long for type character varying(500) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/django_q/cluster.py", line 381, in worker res = f(*task['args'], **task['kwargs']) File "/app/ghostwriter/shepherd/tasks.py", line 373, in update_dns domain_instance.save() File "/usr/local/lib/python3.7/site-packages/django/db/models/base.py", line 741, in save force_update=force_update, update_fields=update_fields) File "/usr/local/lib/python3.7/site-packages/django/db/models/base.py", line 779, in save_base force_update, using, update_fields, File "/usr/local/lib/python3.7/site-packages/django/db/models/base.py", line 851, in _save_table forced_update) File "/usr/local/lib/python3.7/site-packages/django/db/models/base.py", line 900, in _do_update return filtered._update(values) > 0 File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 760, in _update return query.get_compiler(self.db).execute_sql(CURSOR) File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1429, in execute_sql cursor = super().execute_sql(result_type) File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1100, in execute_sql cursor.execute(sql, params) File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 99, in execute return super().execute(sql, params) File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 67, in execute return self._execute_with_wrappers(sql, params, many=False, executor=self._execute) File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers return executor(sql, params, many, context) File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/usr/local/lib/python3.7/site-packages/django/db/utils.py", line 89, in __exit__ raise dj_exc_value.with_traceback(traceback) from exc_value File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) django.db.utils.DataError: value too long for type character varying(500)
This is due to the DNS records (including the TXT records) being longer than 500 characters.
Please, let me know if you need any more info.
Error: 'NoneType' object has no attribute 'split' : Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/django_q/cluster.py", line 377, in worker res = f(*task['args'], **task['kwargs']) File "/app/ghostwriter/shepherd/tasks.py", line 220, in check_domains lab_results = domain_review.check_domain_status() File "/app/ghostwriter/modules/review.py", line 482, in check_domain_status 'timestamp': address['last_resolved']. AttributeError: 'NoneType' object has no attribute 'split'
Task logs:
Id:
aec500e4fe1c433e909495f9e04d31c3
Name:
mobile-november-item-hotel
Func:
ghostwriter.shepherd.tasks.check_domains
Hook:
ghostwriter.shepherd.tasks.send_slack_complete_msg
Args:
Kwargs:
{}
Result:
'NoneType' object has no attribute 'split' : Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/django_q/cluster.py", line 377, in worker
res = f(*task['args'], **task['kwargs'])
File "/app/ghostwriter/shepherd/tasks.py", line 220, in check_domains
lab_results = domain_review.check_domain_status()
File "/app/ghostwriter/modules/review.py", line 482, in check_domain_status
'timestamp': address['last_resolved'].
AttributeError: 'NoneType' object has no attribute 'split'
Group:
Domain Updates
Started:
Aug. 27, 2019, 6:11 p.m.
Stopped:
Aug. 27, 2019, 8:45 p.m.
Success:
False
When running the mkdir .envs && cp .envs_template/.* .envs
command as the first step to setting up the application, my system included ./
and ../
in .envs_template/.*
so it copied all of the application files. I didn't catch the mistake at first and ran into an infinite recursive build...
I would recommend changing the command in the wiki to mkdir .envs && cp -r .envs_template/.local .envs && cp -r .envs_template/.production .envs
.
The date handler on the projects does not seem to like those that natively use ISO8601 type dates (yyyy/mm/dd or dd/mm/yyyy).
So when I try to add dates as per the displayed (mm/dd/yyyy) mask the system rejects the input.
To correct this I add a EU type date (dd/mm/yyyy) and the system accepts it and shows it as 9 Feb 2019 in the date picker window and then as 2nd Sept on the Project summary.
The idea is to add the option to activate 2FA for users to login.
This could be implemented with either Duo or Google Authenticator.
I have domains that have expired but the status is not changing to expired. When I look at a single domain the expired icon appears. I have tried to mark the domain as expired but nothing happens to the status. The only way to change the status is through the admin page.
There is an error in the page to import the findings: the template doesn't exist.
Fixed in PR #7
The current CSV template doesn't include the auto_renew field, despite it being present in the code.
Also, if you include it on purpose, it is not parsed properly (it seems the logic is flawed)
Here's a diff that works for me if you want patch the code:
diff --git a/ghostwriter/shepherd/views.py b/ghostwriter/shepherd/views.py
index 16777ef..1926deb 100644
--- a/ghostwriter/shepherd/views.py
+++ b/ghostwriter/shepherd/views.py
@@ -489,9 +489,11 @@ def import_domains(request):
domain_status = DomainStatus.objects.get(
domain_status='Available')
entry['domain_status'] = domain_status
- # Accept any auto_renew value (True, X, Yes, ...) to mean True
- if 'auto_renew' in entry:
+ # Accept any auto_renew value (True, Enabled, Yes, ...) to mean True
+ if any(yes_option in entry['auto_renew'].lower() for yes_option in ["yes", "enabled", "true"]):
entry['auto_renew'] = True
+ else:
+ entry['auto_renew'] = False
# The last_used_by field will only be set by Shepherd at check-out
if 'last_used_by' in entry:
entry['last_used_by'] = None
When checking out a domain/server, it would be convenient to automatically have the start and end date populated with start and end date from the project.
Hey,
Currently having an issue when I git clone or download manually via .zip. Seems the .envs_templates are not saving to disk the folder nor any of the files inside .envs_templates, even if downloaded / extracting individually. Maybe I'm missing something? Tested on Kali, Ubuntu.
Thanks
We are finally running Ghostwriter in production mode after some changes we made.
We were trying to upload evidence true multiple options. First in the finding during the reporting stage and also in the django database section. But on both options we received an: Ooops something went wrong error.
After looking in the django logs I found the following:
django_1 | File "/usr/local/lib/python3.7/os.py", line 223, in makedirs
django_1 | mkdir(name, mode)
django_1 | PermissionError: [Errno 13] Permission denied: '/app/ghostwriter/media/evidence'
I logged into the docker it self and the media directory existed but the strange thing was that every folder in the ghostwriter directory had the owner "django" except for the media directory.
I fixed this issue by logging in to the docker with root:
docker exec -ti -u root ghostwriter_django_1 /bin/sh
moved into the directory ghostwriter and run the following command:
chown -R media/
Now we can upload evidence as needed. The only issue that remains is that when we remove the evidence from ghostwriter is seems to be deleted, but the uploaded file remains on disk in the docker. If you re-upload that file again then it uploads a second one under a different name.
I checked if the django user has the rights to delete the file and he does. when i remove them straight from the docker it works.
When running the initial database setup, model population fails with the following...
PostgreSQL is available
Loading /app/ghostwriter/reporting/fixtures/initial.json...
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
psycopg2.errors.UndefinedTable: relation "reporting_severity" does not exist
LINE 1: UPDATE "reporting_severity" SET "severity" = 'Informational'...
^
This error cascades into rolodex and shepherd.
Args:
Kwargs:
{}
Result:
No connection adapters were found for '"https://hooks.slack.com/services/{SNIP}"' : Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/django_q/cluster.py", line 377, in worker
res = f(*task['args'], **task['kwargs'])
File "/app/ghostwriter/shepherd/tasks.py", line 175, in release_servers
send_slack_msg(message, project.project.slack_channel)
File "/app/ghostwriter/shepherd/tasks.py", line 63, in send_slack_msg
'application/json'})
File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 116, in post
return request('post', url, data=data, json=json, **kwargs)
File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 640, in send
adapter = self.get_adapter(url=request.url)
File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 731, in get_adapter
raise InvalidSchema("No connection adapters were found for '%s'" % url)
requests.exceptions.InvalidSchema: No connection adapters were found for '"https://hooks.slack.com/services/{SNIP}"'
Group:
Server Release
Hello,
There is a typo in report writer.
File "/app/ghostwriter/reporting/views.py", line 665, in generate_docx
docx = spenny.generate_word_docx()
File "/app/ghostwriter/modules/reportwriter.py", line 874, in generate_word_docx
finding_color = self.critcal_color
AttributeError: 'Reportwriter' object has no attribute 'critcal_color'
You can fix it by replacing self.critcal_color
to self.critical_color
Hi,
Thank you for making this tool. I am impressed.
However, I have an issue. None of the reports that I generate seem to automatically add the picture evidence to the report. I don't doubt that this is a self-inflicted issue. I just don't fully understand how to resolve it. I read the wiki. But, I'm missing something somewhere. Are there any walkthroughs that aren't on the wiki that I can look at?
I'm currently trying to add a Project in Ghostwriter, and under "Project Type" There is nothing displayed there to enter just a check mark? Any ideas on why nothing is showing up? There is also nothing under "Finding type" neither. Am I missing something? Steps I've done so far:
But there is nothing under Project Type or can be selected.
Thanks!
Hi,
It seems utilizing inline_code keyword nested within another keyword (e.g. numbered_list or bulleted_list) doesn't render properly. Example here:
{{.numbered_list}}Browse target system for {{.inline_code}}.txt{{.end_inline_code}} files containing encrypted password Step 2, bla bla bla {{.end_numbered_list}}
Thanks.
On first installation, when running the seed data command, I get this error:
(Note: I made sure to run docker-compose -f local.yml stop; docker-compose -f local.yml rm -f; docker-compose -f local.yml build; docker-compose -f local.yml up -d
first)
~/ghostwriter/ master docker-compose -f local.yml run --rm django /seed_data
Starting ghostwriter_postgres_1 ... done
PostgreSQL is available
Loading /app/ghostwriter/reporting/fixtures/initial.json...
Installed 12 object(s) from 1 fixture(s)
Loading /app/ghostwriter/shepherd/fixtures/initial.json...
Installed 26 object(s) from 1 fixture(s)
Loading /app/ghostwriter/rolodex/fixtures/initial.json...
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
psycopg2.errors.UndefinedTable: relation "rolodex_objectivestatus" does not exist
LINE 1: UPDATE "rolodex_objectivestatus" SET "objective_status" = 'A...
^
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "manage.py", line 30, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 375, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 323, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 364, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/loaddata.py", line 72, in handle
self.loaddata(fixture_labels)
File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/loaddata.py", line 114, in loaddata
self.load_label(fixture_label)
File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/loaddata.py", line 181, in load_label
obj.save(using=self.using)
File "/usr/local/lib/python3.7/site-packages/django/core/serializers/base.py", line 223, in save
models.Model.save_base(self.object, using=using, raw=True, **kwargs)
File "/usr/local/lib/python3.7/site-packages/django/db/models/base.py", line 779, in save_base
force_update, using, update_fields,
File "/usr/local/lib/python3.7/site-packages/django/db/models/base.py", line 851, in _save_table
forced_update)
File "/usr/local/lib/python3.7/site-packages/django/db/models/base.py", line 900, in _do_update
return filtered._update(values) > 0
File "/usr/local/lib/python3.7/site-packages/django/db/models/query.py", line 760, in _update
return query.get_compiler(self.db).execute_sql(CURSOR)
File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1429, in execute_sql
cursor = super().execute_sql(result_type)
File "/usr/local/lib/python3.7/site-packages/django/db/models/sql/compiler.py", line 1100, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 99, in execute
return super().execute(sql, params)
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 67, in execute
return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python3.7/site-packages/django/db/utils.py", line 89, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/local/lib/python3.7/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: Problem installing fixture '/app/ghostwriter/rolodex/fixtures/initial.json': Could not load rolodex.ObjectiveStatus(pk=1): relation "rolodex_objectivestatus" does not exist
LINE 1: UPDATE "rolodex_objectivestatus" SET "objective_status" = 'A...
^
Any insight as to why this might be?
When using cloud providers to setup re-directors for red teaming, we usually take multiple IPs per server (to be more cost-efficient).
It might be interesting to have that view in the tool, I guess we are not the only ones doing that
Failed to generate the Word report: 'NoneType' object is not subscriptable
Caused by traceback:
ERROR 2020-02-26 17:55:08,714 log 16 140432287571272 Internal Server Error: /reporting/reports/5/generate_docx/
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/usr/local/lib/python3.7/site-packages/django/core/handlers/base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/python3.7/contextlib.py", line 74, in inner
return func(*args, **kwds)
File "/usr/local/lib/python3.7/site-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view
return view_func(request, *args, **kwargs)
File "/app/ghostwriter/reporting/views.py", line 577, in generate_docx
docx = spenny.generate_word_docx()
File "/app/ghostwriter/modules/reportwriter.py", line 1284, in generate_word_docx
self.process_text_xml(finding['description'], finding, report_json)
File "/app/ghostwriter/modules/reportwriter.py", line 917, in process_text_xml
self.process_inline_text(line, p, report_type)
File "/app/ghostwriter/modules/reportwriter.py", line 453, in process_inline_text
self.span_classes = match[0].split(' ')
TypeError: 'NoneType' object is not subscriptable
Hello
thx for Ghost
How can I upload a picture in the description?
It would be great to have support for LDAP/Active Directory integration for the authentication (and potentially authorization - cf. #67).
Ghostwriter has been inventoried on Rawsec's CyberSecurity Inventory.
https://inventory.rawsec.ml/tools.html#Ghostwriter
An inventory of tools and resources about CyberSecurity. This inventory aims to help people to find everything related to CyberSecurity.
More details about features here.
Note: the inventory is a FLOSS (Free, Libre and Open-Source Software) project.
Mainly because this is giving visibility to your tool and improve its referencing.
The badge shows to your community that your are inventoried. It looks good but also shows you care about your project, that your tool is referenced.
Feel free to claim your badge here: http://inventory.rawsec.ml/features.html#badges, it looks like that , but there are several styles available.
If you want to thank us, you can help make our open project better known by tweeting about it! For example:
That's all, this message is just to notify you if you care. Else you can close this issue.
I'm currently trying to move to the production side of Ghostwriter, and everytime I start the production mode the webserver 80, and 443 shut off right away. I check with "nmap localhost" and I will only see it open for about 5 seconds then stops right away. I've looked at my configs in .envs and etc but haven't come to any conclusion.
My production .django is set to allow ghostwriter.local, and the production.yml is set too 0.0.0.0:80:80 and 0.0.0.0:443:443. I've also changed this and still shuts off.
Commands used:
docker-compose -f production.yml stop; docker-compose -f production.yml rm -f; docker-compose -f production.yml build; docker-compose -f production.yml up -d
docker-compose -f local.yml up -d
Thanks
The idea would be to create access-controls per project and only give specific operators access to specific projects and its findings, reports, etc.
Not sure how easy this is to implement, but happy to help there where needed.
Currently, if you want to add or remove fields from a finding, you'll need to change both the model and the corresponding templates. It would be nice if there was a way to add your own fields, or at the very least, that the templates generate themselves from the model in a dynamic way.
Using the HTML markup of the WYSIWYG editor (e.g., links, br, etc), render properly in Ghostwriter however, when exporting a report HTML formatting is placed in the document literally (e.g., <a href="" etc) instead of creating a link.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.