mushorg / tanner Goto Github PK
View Code? Open in Web Editor NEWHe who flays the hide
License: GNU General Public License v3.0
He who flays the hide
License: GNU General Public License v3.0
Input should be a sensor UUID (part of every request to tanner). Something like /api/<uuid>/stats
Should show some basic stats regarding the activity from the sensor.
Need to write tests for config class to avoid problems like this #104 (Bad using of StringIO)
When TANNER determine the type of vulnerability, it incorrectly handles xss with links (it marks them as rfi).
For example, these links:
'/default.ph/?pageweb=<LINK+REL="stylesheet"+HREF="http://ha.ckers.org/xss.css">'
'/default.ph/?pageweb=<STYLE>@import\\'http://ha.ckers.org/xss.css\\';</STYLE>'
'/default.ph/?pageweb=Redirect+302+/a.jpg+http://victimsite.com/admin.asp&deleteuser'
'/default.ph/?pageweb=<IMG+SRC="http://www.thesiteyouareon.com/somecommand.php?somevariables=maliciouscode">'
'/default.ph/?pageweb=<A+HREF="javascript:document.location=\\'http://www.google.com/\\'">XSS</A>'
'/default.ph/?pageweb=<A+HREF="http://66.102.7.147/">XSS</A>'
'/default.ph/?pageweb=<IFRAME+SRC=http://ha.ckers.org/scriptlet.html+<'
'/default.ph/?pageweb=<META+HTTP-EQUIV="refresh"+CONTENT="0;+URL=http://;URL=javascript:alert(\\'XSS\\');">'
'/default.ph/?pageweb=<XML+SRC="http://ha.ckers.org/xsstest.xml"+ID=I></XML>+<SPAN+DATASRC=#I+DATAFLD=C+DATAFORMATAS=HTML></SPAN>'
We need to validate config before using its values
Currently, Tanner just responds to events by detecting some known patterns, and responding with payloads (if they exist.)
Do we also plan to store these events (in some database perhaps) for analysis later?
I'm also just guessing that this will be done on Tanner's side, rather than Snare's.
I'd be interested in working on this feature (if it is needed, of course,)
P.S: It'd be nice if you could maybe just add some issues about functionalities that you feel makes sense in Snare/Tanner.
It seems that new version of aiohttp fails when trying to connect over ftp.
tanner.rfi_emulator.RfiEmulator: ERROR: Error during downloading the rfi script [Errno 101] Cannot connect to host mirror.yandex.ru:None ssl:False [Can not connect to mirror.yandex.ru:None [Network is unreachable]]
We can use another library for ftp file downloading or try to solve the problem with aiohttp (if it's possible)
Add TANNER to Travis so we get continuous testing.
We can get differ types of messages from TANNER, so we should check the type and existence of some dict keys to process the page. https://github.com/mushorg/snare/blob/master/snare.py#L187
TANNER should return unified messages to make processing the result of the emulation easier. For example, now payload can be dict or str, so we should make a lot of checks to choose the action. If TANNER returns unified structured messages we will have the possibility to parse it properly. TANNER should have clear, proper api. Also we can return error messages if something goes wrong and make logs (as a option) of these errors on the SNARE side.
Maybe we can make the types of the messages, for example:
So, if we get the type 4, we definitely know that we need to get "error" key from the message.
It's a bit dirty description. Any advice and suggestions will be greatly appreciated
This lookup can cause a KeyError:
Line 12 in 110a1d2
Right now we have a fixed collection of dorks: https://github.com/mushorg/tanner/blob/master/dorks.pickle
We want to increase the collection of dorks to make the SNARE sensors more attractive.
How should we organize logging system? What info should we write to logs? Should we replace print messages with logs?
I don't like that all the files into the root of the project. Maybe we should make the project structure better: for example, move emulators into emulators/handlers
folder, change naming and so on
$ sudo tanner
Traceback (most recent call last):
File "/usr/local/bin/tanner", line 4, in
import('pkg_resources').run_script('Tanner==0.1.0', 'tanner')
File "/usr/local/lib/python3.5/dist-packages/pkg_resources/init.py", line 744, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/usr/local/lib/python3.5/dist-packages/pkg_resources/init.py", line 1499, in run_script
exec(code, namespace, namespace)
File "/usr/local/lib/python3.5/dist-packages/Tanner-0.1.0-py3.5.egg/EGG-INFO/scripts/tanner", line 21, in
main()
File "/usr/local/lib/python3.5/dist-packages/Tanner-0.1.0-py3.5.egg/EGG-INFO/scripts/tanner", line 16, in main
config.TannerConfig.set_config(args.config)
File "/usr/local/lib/python3.5/dist-packages/Tanner-0.1.0-py3.5.egg/tanner/config.py", line 12, in set_config
TannerConfig.create_default_config(cfg, config_path)
File "/usr/local/lib/python3.5/dist-packages/Tanner-0.1.0-py3.5.egg/tanner/config.py", line 23, in create_default_config
with open(config_path, 'w') as configfile:
FileNotFoundError: [Errno 2] No such file or directory: '/opt/tanner/config.cfg'
We should have two log files: one for the events and one for the errors
Related to #5 Create some rules to detect a remote file inclusion.
With the capability of tracking a session #7 can we return injected malicious code to the attacker?
POST /posts?comment=<script>alert('xss');</script>
GET /posts contains the JS
FAIL: test_ftp_download (tanner.tests.test_rfi_emulation.TestRfiEmulator)
Traceback (most recent call last):
File "/home/travis/build/mushorg/tanner/tanner/tests/test_rfi_emulation.py", line 26, in test_ftp_download
self.assertIsNotNone(data)
AssertionError: unexpectedly None
tanner.rfi_emulator.RfiEmulator: ERROR: Error during downloading the rfi script [Errno 101] Cannot connect to host mirror.yandex.ru:None ssl:False [Can not connect to mirror.yandex.ru:None [Network is unreachable]]
For now LFI adds files to virtualdocs folder only if the linux directory empty. I think it will be great if we will add new files when new object is put to vdocs.json
Hi, can you update tanner run instructions in the Readme file?. Thanks
We should add a SQLi section here: https://github.com/mushorg/tanner/blob/master/docs/source/emulators.rst
How many requests can we serve per second? Which APIs are slow? Which handlers are slow? Does https://github.com/MagicStack/uvloop speed up things?
How do I run tanner?
I need to see the .log file
Where I find it?
thanks
All operational data should be in the /opt/tanner directory.
I think this project has not been selected as a part of GSoC'16.
Will you be working on this in your personal time?
If not, then I'd like to ask you the same thing I've asked before, Can I port this (& snare) to flask? 🙏
I have no technical reason to do this. Just that I'm really well versed with flask (& things like how its projects should be structure etc.)
I actually want to work on this idea in any free time I can find, as I think there's a lot I could learn. But, the project involves a lot of things that are completely foreign to me, and I don't want the choice of a web framework etc. weighing me down (asyncio
just feels like one more thing I'll have to learn.) I am just trying to move to towards a comfortable territory so I can focus on stuff that actually matters - the honeynet, vulnerabilities etc.
Let me know what you feel.
Store the file with the file name being a md5 hash of the file.
Think about if we want to pass the path to the file or the file content to PHPOX.
This line failes if the IP in None:
Line 39 in 110a1d2
By using PHPox as a service we should be able to run injected PHP code and return the results to the adversary in a timely manner. We are primarily interested in executing those PHP scripts that try to evaluate the vulnerability of the web application. Bot's are interesting too but they usually require longer execution time (this could be done separately).
I'm testing SNARE/TANNER with grabber.
grabber --spider 1 --include --url <snare host url>
,
and OWASP Zap. LFI testing going well, but when starting testing RFI, it fails with this error:
Traceback (most recent call last):
File "server.py", line 128, in <module>
loop.run_forever()
File "/usr/lib/python3.5/asyncio/base_events.py", line 331, in run_forever
self._run_once()
File "/usr/lib/python3.5/asyncio/base_events.py", line 1262, in _run_once
event_list = self._selector.select(timeout)
File "/usr/lib/python3.5/selectors.py", line 441, in select
fd_event_list = self._epoll.poll(timeout, max_ev)
OverflowError: timeout is too large
Task was destroyed but it is pending!
task: <Task pending coro=<ServerHttpProtocol.start() running at /usr/local/lib/python3.5/dist-packages/aiohttp-0.22.0a0-py3.5.egg/aiohttp/server.py:266> wait_for=<Future pending cb=[BaseSelectorEventLoop._sock_connect_done(10)(), Task._wakeup()]>>
Exception ignored in: <generator object ServerHttpProtocol.start at 0x7ff2725cbbf8>
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/aiohttp-0.22.0a0-py3.5.egg/aiohttp/server.py", line 312, in start
File "/usr/lib/python3.5/asyncio/selector_events.py", line 568, in close
File "/usr/lib/python3.5/asyncio/base_events.py", line 483, in call_soon
File "/usr/lib/python3.5/asyncio/base_events.py", line 492, in _call_soon
File "/usr/lib/python3.5/asyncio/base_events.py", line 320, in _check_closed
RuntimeError: Event loop is closed
With some basic test to ensure functionality we are able to see quickly if a pull request breaks the tanner. Probably best to use nosetests and unittest.
Add the capability to track client sessions. Be it through the IP address, user agent, cookies... We had something along those lines in Glastopf.
This is a bit of a tricky one and we made various attempts in Glastopf.
First attempt was based on tokenizing know SQL injection queries and using them to match queries against Glastopf. This had limited success as we had a small database. Response was generally just a default MySQL error message.
Second attempt can be found here and here (IIRC libinjection has now it's own Python bindings).
Finally we have this fork of Glastopf using a sandboxed database: https://github.com/rebeccan/glastopf
I think an initial implementation should be based on libinjection.
With session tracking #7 we can:
Have a look at http://user-mode-linux.sourceforge.net/old/hppfs.html
I've cloned wordpress site which contains some .css files in the /wp-content structure.
When hosting the clone with SNARE a request to such a css leads to a 'FileNotFound' Error in Tanner and a response 500 Internal Server error on SNARE.
The tanner log contains following error log:
2017-02-08 15:20 INFO:tanner.server.HttpRequestHandler:handle_event: Requested path /wp-content/themes/quark/css/normalize.css?ver=3.0.2
2017-02-08 15:20 ERROR:tanner.server.HttpRequestHandler:log_exception: Error handling request
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/aiohttp/server.py", line 261, in start
yield from self.handle_request(message, payload)
File "/usr/lib/python3.6/site-packages/Tanner-0.1.0-py3.6.egg/tanner/server.py", line 92, in handle_request
response_msg = yield from self.handle_event(data, self.redis_client)
File "/usr/lib/python3.6/site-packages/Tanner-0.1.0-py3.6.egg/tanner/server.py", line 56, in handle_event
detection = yield from self.base_handler.handle(data, session, path)
File "/usr/lib/python3.6/site-packages/Tanner-0.1.0-py3.6.egg/tanner/emulators/base.py", line 74, in handle
detection = yield from self.emulate(data, session, path)
File "/usr/lib/python3.6/site-packages/Tanner-0.1.0-py3.6.egg/tanner/emulators/base.py", line 64, in emulate
detection = yield from self.handle_get(path)
File "/usr/lib/python3.6/site-packages/Tanner-0.1.0-py3.6.egg/tanner/emulators/base.py", line 53, in handle_get
sqli = yield from self.emulators['sqli'].check_get_data(path)
File "/usr/lib/python3.6/site-packages/Tanner-0.1.0-py3.6.egg/tanner/emulators/sqli.py", line 61, in check_get_data
sqli = yield from self.check_sqli(query[1])
File "/usr/lib/python3.6/site-packages/Tanner-0.1.0-py3.6.egg/tanner/emulators/sqli.py", line 39, in check_sqli
res = yield from _run_cmd(command)
File "/usr/lib/python3.6/site-packages/Tanner-0.1.0-py3.6.egg/tanner/emulators/sqli.py", line 33, in _run_cmd
proc = yield from asyncio.wait_for(asyncio.create_subprocess_exec(*cmd, stdout=PIPE), 5)
File "/usr/lib/python3.6/asyncio/tasks.py", line 352, in wait_for
return fut.result()
File "/usr/lib/python3.6/asyncio/subprocess.py", line 212, in create_subprocess_exec
stderr=stderr, **kwds)
File "uvloop/loop.pyx", line 2164, in __subprocess_run (uvloop/loop.c:39415)
File "uvloop/handles/process.pyx", line 549, in uvloop.loop.UVProcessTransport.new (uvloop/loop.c:93764)
File "uvloop/handles/process.pyx", line 87, in uvloop.loop.UVProcess._init (uvloop/loop.c:85151)
FileNotFoundError: [Errno 2] No such file or directory
2017-02-08 15:20 INFO:aiohttp.access:log: 127.0.0.1 - - [08/Feb/2017:15:20:49 +0000] "POST /event HTTP/1.1" 500 170 "-" "Python/3.6 aiohttp/1.2.0"
I'm not sure if it's an TANNER issue because of the ?ver=3.0.2 or an aiohttp/server.py issue.
I would fix it and propose a Pull Request, but I'm not sure what exactly causes the issue.
I have cloned site, which has Cyrillic symbols in paths. And I got this error:
--- Logging error ---
Traceback (most recent call last):
File "/usr/lib/python3.6/logging/__init__.py", line 989, in emit
stream.write(msg)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 112-115: ordinal not in range(128)
Call stack:
File "/opt/tanner/bin/tanner", line 33, in <module>
main()
File "/opt/tanner/bin/tanner", line 29, in main
tanner.server.run_server()
File "/opt/tanner/tanner/server.py", line 132, in run_server
loop.run_forever()
File "/usr/lib/python3.6/site-packages/aiohttp/server.py", line 261, in start
yield from self.handle_request(message, payload)
File "/opt/tanner/tanner/server.py", line 89, in handle_request
response_msg = yield from self.handle_event(data, self.redis_client)
File "/opt/tanner/tanner/server.py", line 54, in handle_event
self.logger.info('Requested path %s', path)
Message: 'Requested path %s'
Arguments: ('/sites/default/files/images/фото яковец.jpg',)
Instead of using a hash of user agent and IP (
Line 40 in 110a1d2
self.sensor = data['uuid']
and self.uuid = uuid()
Now we have some hard coded values: handler's directories, db name, redis host/port. I think it will be great, if we create config file for these purposes.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.