13o-bbr-bbq / machine_learning_security Goto Github PK
View Code? Open in Web Editor NEWSource code about machine learning and security.
Source code about machine learning and security.
problems with tensorflow
Traceback (most recent call last):
File "DeepExploit.py", line 2263, in <module>
SESS = tf.Session() # Start TensorFlow session.
AttributeError: module 'tensorflow' has no attribute 'Session'
This is on another VM (VMWorkstation) Kali linux, I get this error
Traceback (most recent call last):
File "/home/drwho/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1322, in _do_call
return fn(*args)
File "/home/drwho/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1307, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "/home/drwho/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1409, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [200,539] rhs shape= [200,504]
[[Node: save/Assign_9 = Assign[T=DT_FLOAT, _class=["loc:@local_thread1/dense_9/kernel"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](local_thread1/dense_9/kernel, save/RestoreV2:9)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "DeepExploit.py", line 1994, in
saver.restore(SESS, env.save_file)
File "/home/drwho/.local/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1802, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/home/drwho/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 900, in run
run_metadata_ptr)
File "/home/drwho/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1135, in _run
feed_dict_tensor, options, run_metadata)
File "/home/drwho/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1316, in _do_run
run_metadata)
File "/home/drwho/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1335, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [200,539] rhs shape= [200,504]
[[Node: save/Assign_9 = Assign[T=DT_FLOAT, _class=["loc:@local_thread1/dense_9/kernel"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](local_thread1/dense_9/kernel, save/RestoreV2:9)]]
Caused by op 'save/Assign_9', defined at:
File "DeepExploit.py", line 1971, in
saver = tf.train.Saver()
File "/home/drwho/.local/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1338, in init
self.build()
File "/home/drwho/.local/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1347, in build
self._build(self._filename, build_save=True, build_restore=True)
File "/home/drwho/.local/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1384, in _build
build_save=build_save, build_restore=build_restore)
File "/home/drwho/.local/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 835, in _build_internal
restore_sequentially, reshape)
File "/home/drwho/.local/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 494, in _AddRestoreOps
assign_ops.append(saveable.restore(saveable_tensors, shapes))
File "/home/drwho/.local/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 185, in restore
self.op.get_shape().is_fully_defined())
File "/home/drwho/.local/lib/python3.6/site-packages/tensorflow/python/ops/state_ops.py", line 283, in assign
validate_shape=validate_shape)
File "/home/drwho/.local/lib/python3.6/site-packages/tensorflow/python/ops/gen_state_ops.py", line 60, in assign
use_locking=use_locking, name=name)
File "/home/drwho/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/home/drwho/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3392, in create_op
op_def=op_def)
File "/home/drwho/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1718, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access
InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [200,539] rhs shape= [200,504]
[[Node: save/Assign_9 = Assign[T=DT_FLOAT, _class=["loc:@local_thread1/dense_9/kernel"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](local_thread1/dense_9/kernel, save/RestoreV2:9)]]
oops! I spent 2 hours but it still errors. qwq!!!!!
python3 DeepExploit.py -h
Traceback (most recent call last):
File "DeepExploit.py", line 19, in
import tensorflow as tf
File "/usr/local/lib/python3.7/dist-packages/tensorflow/init.py", line 24, in
from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/init.py", line 49, in
from tensorflow.python import pywrap_tensorflow
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in
from tensorflow.python.pywrap_tensorflow_internal import *
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 114
def TFE_ContextOptionsSetAsync(arg1, async):
^
SyntaxError: invalid syntax
The link directs to a 404.
Hello, i'm trying to run the DeepExploit but it always throw an error, just like:
[*] Port scanning: 192.168.37.120 [Elapsed time: 5 s]
[*] Executing keep_alive..
[!] type:<class 'AttributeError'>
[!] args:("'NoneType' object has no attribute 'decode'",)
[!] 'NoneType' object has no attribute 'decode'
[!] Failed: version
I debugged the code and found the 'console_id' in class Msgrpc is None, help please, thanks.
Could not find a version that satisfies the requirement tensorflow>=1.8.0 (from -r requirements.txt (line 10)) (from versions: )
No matching distribution found for tensorflow>=1.8.0 (from -r requirements.txt (line 10))
uname -a
5.2.0-kali2-686-pae #1 SMP Debian 5.2.9-2kali1 (2019-08-22) i686 GNU/Linux
python3 -V
Python 3.7.5
When using Tidy for HTML matching check, the regular expression to get the count of warnings and errors might not work with all Tidy versions. Therefore I had to switch from:
str_pattern = r'.*Tidy found ([0-9]+) warnings and ([0-9]+) errors.*$'
to:
str_pattern = r'.*([0-9]+) warnings, ([0-9]+) error were found!.*$'
ModuleNotFoundError: No module named 'tensorflow'
root@vultr:/opt/machine_learning_security/DeepExploit# pip3 install tensorflow
Collecting tensorflow
Downloading https://files.pythonhosted.org/packages/f4/28/96efba1a516cdacc2e2d6d081f699c001d414cc8ca3250e6d59ae657eb2b/tensorflow-1.14.0-cp37-cp37m-manylinux1_x86_64.whl (109.3MB)
99% |████████████████████████████████| 109.3MB 42.2MB/s eta 0:00:01Killed
python3 -V
Python 3.7.5
uname -a
5.3.0-kali2-amd64 #1 SMP Debian 5.3.9-3kali1 (2019-11-20) x86_64 GNU/Linux
pip3 -V
pip 18.1 from /usr/lib/python3/dist-packages/pip (python 3.7)
root@soup-natzi:~/Desktop/newtools/machine_learning_security/DeepExploit# python3 DeepExploit.py -t 192.168.1.102 -m train
Illegal instruction
i do have installed every requirements and do have python3 installed.
[root:...arning_security/DeepExploit]# python DeepExploit.py -t 192.168.32.144 -m train (master✱)
Using TensorFlow backend.
[*] Invalid IP address: 192.168.32.144
[root:...arning_security/DeepExploit]#
Any idea? This is running on Kali 2018.1
Why use 'x = (x-mean)/mean' standardization In line 15 of file analyze_kmeans.py?
Why not use max-min standardization or z-score standardization ?
Hi, After sucessful installation following are the steps i did
[] Find product=joomla/ from http://5.107.217.144:80/templates/joomla/
[!] type:<class 'urllib3.exceptions.MaxRetryError'>
[!] args:('HTTPConnectionPool(host='5.107.217.144', port=80): Max retries exceeded with url: /mt.cgi (Caused by ReadTimeoutError("HTTPConnectionPool(host='5.107.217.144', port=80): Read timed out. (read timeout=3.0)"))',)
[!] HTTPConnectionPool(host='5.107.217.144', port=80): Max retries exceeded with url: /mt.cgi (Caused by ReadTimeoutError("HTTPConnectionPool(host='5.107.217.144', port=80): Read timed out. (read timeout=3.0)"))
[!] Access is failure : http://5.107.217.144:80/mt.cgi
Thanks in advance for your help.
When I run python3 main.py TRAIN I get the following error:
/$PATH/lib/python3.6/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
Using TensorFlow backend.
Illegal instruction
This error does not halt the activity of the program. However, during every session I receive this message.
#<Thread:0x00005563295fc8a0@/usr/share/metasploit-framework/lib/msf/core/thread_manager.rb:93 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
6: from /usr/share/metasploit-framework/lib/msf/core/thread_manager.rb:100:in block in spawn' 5: from /usr/share/metasploit-framework/lib/msf/ui/web/console.rb:72:in
block in initialize'
4: from /usr/share/metasploit-framework/lib/rex/ui/text/shell.rb:191:in run' 3: from /usr/share/metasploit-framework/lib/rex/ui/text/shell.rb:375:in
get_input_line'
2: from /usr/share/metasploit-framework/lib/rex/ui/text/bidirectional_pipe.rb:151:in pgets' 1: from /usr/share/metasploit-framework/lib/rex/ui/text/bidirectional_pipe.rb:121:in
gets'
/usr/share/metasploit-framework/lib/rex/ui/text/input/buffer.rb:56:in `gets': no implicit conversion of nil into String (TypeError)
When running DeepExploit, I'm getting the following output..
Using TensorFlow backend.
RuntimeError: module compiled against API version 0xc but this version of numpy is 0xb
RuntimeError: module compiled against API version 0xc but this version of numpy is 0xb
[*] Invalid IP address: 192.168.2.7
Am I doing something wrong?
xml will display on msf rpc server's stdout
client of msf rpc will get
"{"data"=>"[*] exec: more nmap_result_192.168.7.167.xml\n\n", "prompt"=>"\x01\x02msf5\x01\x02 \x01\x02> ", "busy"=>false}"
Hello,
When I launch the deep exploit on metasploitable 3 win2k8 everything goes well up to:
[+] Analyzing gathered HTTP response using ML.
[*] Executing keep_alive...
[!] Product Not Found.
[+] Explore unnecessary content.
then it gets stuck on the:
[+] Confirm string matching.
Is it a PC resource issue? I have 16giga of ram with an i5.
Thanks!
Hey,
When I want to launch it on debian 9 with metasploit framework installed on I get this error:
Traceback (most recent call last):
File "DeepExploit.py", line 2051, in
env.execute_nmap(env.rhost, nmap_command, env.nmap_timeout)
File "DeepExploit.py", line 726, in execute_nmap
self.util.print_message(OK, 'Start time: {}'.format(self.get_current_time()))
File "DeepExploit.py", line 716, in get_current_time
now = datetime.datetime.now()
AttributeError: type object 'datetime.datetime' has no attribute 'datetime'
In line 322 of file
machine_learning_security/Generator/gan_main.py /
I had to use append
instead of extend
to work
lst_scripts.extend(self.train(target_sig_list))
hello im coming across DeepExploit and i need some help if its still here the admin, after ruining the Exploit i have issue like
[] Start time: 2019/10/31 16:52:58
[] Port scanning: 192.168.1.247 [Elapsed time: 0 s]
[] Executing keep_alive..
[] End time : 2019/10/31 16:53:03
[+] Get port list from /root/nmap_result_192.168.1.247.xml.
[!] No open port.
[!] Shutdown Deep Exploit..
AFTER changing some files
# status = ret.get(b'busy')
# if status is False:
# break
with open(nmap_result_file) as fi:
nmap_result = fi.read()
i WAS able to READ THE NMAP FILES , BUT I HAVE A ISSUE AGAIN LIKE
[] Getting 21/tcp info: unknown
[] Getting 22/tcp info: unknown
[] Getting 23/tcp info: unknown
[] Getting 25/tcp info: unknown
[] Getting 53/tcp info: unknown
[] Getting 80/tcp info: unknown
[] Getting 111/tcp info: unknown
[] Getting 139/tcp info: unknown
[] Getting 445/tcp info: unknown
[] Getting 512/tcp info: unknown
[] Getting 513/tcp info: unknown
[] Getting 514/tcp info: unknown
[] Getting 1099/tcp info: unknown
[] Getting 1524/tcp info: unknown
[] Getting 2049/tcp info: unknown
[] Getting 2121/tcp info: unknown
[] Getting 3306/tcp info: unknown
[] Getting 5432/tcp info: unknown
[] Getting 5900/tcp info: unknown
[] Getting 6000/tcp info: unknown
[] Getting 6667/tcp info: unknown
[] Getting 8180/tcp info: unknown
[+] Get exploit list.
[*] Loading exploit list from Metasploit.
[-] MsfRPC: Not Authenticated.
HOW CAN I DO THIS TO WORKS , IM CONNECTED TO THE MSF CONSOLE AND IM RUINING THE RPC SERVER!!! SOME HELP SHALL BE GREAT , THANK YOU~!
While training DeepExploit, the XML file of NMAP is not found. At metasploit end I see following error:
cat: nmap_result_172.28.128.3.xml: No such file or directory
Hi,
Any help on this one?
[root:...arning_security/DeepExploit]# python DeepExploit.py -t 192.168.1.254 -m train (master)
[1] 4567 illegal hardware instruction python DeepExploit.py -t 192.168.1.254 -m train
It is a Kali-rolling machine running on VMWare ESXI 6.5
Dear developer!
When Deep Exploit finishes work, do not create a report in the format of HTML, several CSV files are created with descriptions of the vulnerabilities found.
The system is Kali Linux 2018.2.
The Way to Deep Exploit - /root/Pentest/machine_learning_security/DeepExploit
python DeepExploit.py -t 192.168.147.135 -m train
/usr/local/lib/python2.7/dist-packages/h5py/init.py:36: FutureWarning: Conversion of the second argument of issubdtype from float
to np.floating
is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type
.
from ._conv import register_converters as _register_converters
Using TensorFlow backend.
[*] Invalid IP address: 192.168.147.135
Hi,
I'm getting the following errors when running the test mode as below:
root@kali:~/machine_learning_security/DeepExploit# python3 DeepExploit.py -t 192.168.88.133 -m test
...
[+] Execute Nmap against 192.168.88.133
[] Nmap already scanned.
[+] Get port list from nmap_result_192.168.88.133.xml.
[] Loaded target tree from : /root/machine_learning_security/DeepExploit/data/target_info_192.168.88.133.json
[+] Get exploit list.
[] Loaded exploit list from : /root/machine_learning_security/DeepExploit/data/exploit_list.csv
[+] Get payload list.
[] Loaded payload list from : /root/machine_learning_security/DeepExploit/data/payload_list.csv
[+] Get exploit tree.
[] Loaded exploit tree from : /root/machine_learning_security/DeepExploit/data/exploit_tree.json
[+] Get target info.
[] Loaded target tree from : /root/machine_learning_security/DeepExploit/data/target_info_192.168.88.133.json
[*] Restore learned data.
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1322, in _do_call
return fn(*args)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1307, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1409, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.NotFoundError: Key local_thread1/dense_11/bias not found in checkpoint
[[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "DeepExploit.py", line 2237, in
saver.restore(SESS, env.save_file)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py", line 1802, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 900, in run
run_metadata_ptr)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1135, in _run
feed_dict_tensor, options, run_metadata)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1316, in _do_run
run_metadata)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/client/session.py", line 1335, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: Key local_thread1/dense_11/bias not found in checkpoint
[[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
Caused by op 'save/RestoreV2', defined at:
File "DeepExploit.py", line 2214, in
saver = tf.train.Saver()
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py", line 1338, in init
self.build()
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py", line 1347, in build
self._build(self._filename, build_save=True, build_restore=True)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py", line 1384, in _build
build_save=build_save, build_restore=build_restore)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py", line 835, in _build_internal
restore_sequentially, reshape)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py", line 472, in _AddRestoreOps
restore_sequentially)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py", line 886, in bulk_restore
return io_ops.restore_v2(filename_tensor, names, slices, dtypes)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/gen_io_ops.py", line 1463, in restore_v2
shape_and_slices=shape_and_slices, dtypes=dtypes, name=name)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py", line 3392, in create_op
op_def=op_def)
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py", line 1718, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access
NotFoundError (see above for traceback): Key local_thread1/dense_11/bias not found in checkpoint
[[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
root@kali:~/machine_learning_security/DeepExploit#
Any ideas where the issue is? I'm using kali-linux-2018.3-vm-amd64.
Thank you.
Hi
Sorry to disturb you. I would love to test this app and installed it on server. When it try to launch in test mode or train, i have this error :
Traceback (most recent call last):
File "DeepExploit.py", line 2229, in
com_port_list, proto_list, info_list = env.get_port_list(nmap_result, env.rh ost)
File "DeepExploit.py", line 937, in get_port_list
info_list[idx]))
IndexError: list index out of range
I suppose, i can modify the DeepExploit.py to correct it but how?
Thank you
I have tried your DeepExploit system and finished the configuration.
After training, I used the system to exploit one machine, it show some failed message below.
[!] type:<class 'AttributeError'>
[!] args:("'int' object has no attribute 'decode'",)
[!] 'int' object has no attribute 'decode'
[!] Failed: module.info
Is there something i missed or ?
$ python3 DeepExploit.py -t 192.168.86.22 -m train
///
[!] 1/3 Retry "auth.login" call. reason: [Errno 60] Operation timed out
[!] 2/3 Retry "auth.login" call. reason: [Errno 60] Operation timed out
[!] type:<class 'TimeoutError'>
[!] args:(60, 'Operation timed out')
[!] [Errno 60] Operation timed out
[!] Retry count is over.
Thank you for publishing your code.
I have some questions to DeepExploit.py.
I think that it is necessary to handle the state function (port, protocol, etc.) as one hot vector, is it correct? (I think that distance in feature space can not be handled correctly)
Although it seems that reinforcement learning is applied while state does not change after performing action, is this good? (In this problem, state transition by Markov decision process is not done)
What are the system minimum requirements?
Hi,
I was trying to train the model targeting Metasploitable 3 VM. However, after complete scan and training, not a single Bingo (award > 0) was found. I tried it on both versions of metasploitable 3 (ubuntu and win) with no success.
I also tried it on another linux VM and still could not find any bingo.
I was wondering if that's a known issue or there might be a solution for it.
For reference, I have attached nmap output as well as the log file from DeepExploit.py.
train_met3_win.log
everything was wonderful these days training with metasploitable, since before yesterday I scan but can not find open ports despite creating the report as an example: nmap_result_192.168.56.101
Starting Nmap 7.70 ( https://nmap.org ) at 2019-07-19 20:40 -04
Nmap scan report for 192.168.56.101
Host is up (0.0015s latency).
Not shown: 65506 closed ports
PORT STATE SERVICE VERSION
21/tcp open ftp vsftpd 2.3.4
22/tcp open ssh OpenSSH 4.7p1 Debian 8ubuntu1 (protocol 2.0)
23/tcp open telnet Linux telnetd
25/tcp open smtp Postfix smtpd
53/tcp open domain ISC BIND 9.4.2
80/tcp open http Apache httpd 2.2.8 ((Ubuntu) DAV/2)
111/tcp open rpcbind 2 (RPC #100000)
139/tcp open netbios-ssn Samba smbd 3.X - 4.X (workgroup: WORKGROUP)
445/tcp open netbios-ssn Samba smbd 3.X - 4.X (workgroup: WORKGROUP)
512/tcp open exec netkit-rsh rexecd
513/tcp open login
514/tcp open shell Netkit rshd
1099/tcp open rmiregistry GNU Classpath grmiregistry
1524/tcp open bindshell Metasploitable root shell
2049/tcp open nfs 2-4 (RPC #100003)
2121/tcp open ftp ProFTPD 1.3.1
3306/tcp open mysql MySQL 5.0.51a-3ubuntu5
3632/tcp open distccd distccd v1 ((GNU) 4.2.4 (Ubuntu 4.2.4-1ubuntu4))
5432/tcp open postgresql PostgreSQL DB 8.3.0 - 8.3.7
5900/tcp open vnc VNC (protocol 3.3)
6000/tcp open X11 (access denied)
6667/tcp open irc UnrealIRCd
6697/tcp open irc UnrealIRCd
8009/tcp open ajp13 Apache Jserv (Protocol v1.3)
8180/tcp open http Apache Tomcat/Coyote JSP engine 1.1
8787/tcp open drb Ruby DRb RMI (Ruby 1.8; path /usr/lib/ruby/1.8/drb)
35544/tcp open mountd 1-3 (RPC #100005)
41383/tcp open nlockmgr 1-4 (RPC #100021)
46105/tcp open status 1 (RPC #100024)
57343/tcp open rmiregistry GNU Classpath grmiregistry
Service Info: Hosts: metasploitable.localdomain, localhost, irc.Metasploitable.LAN; OSs: Unix, Linux; CPE: cpe:/o:linux:linux_kernel
Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 125.48 seconds
I will copy this report from the msfconsole, after executing the script in training mode, and the script returns me:
[+] Execute Nmap against 192.168.56.101
[*] nmap -p0-65535 -T4 -Pn -sV -sT --min-rate 1000 -oX nmap_result_192.168.56.101.xml 192.168.56.101
[] Start time: 2019/07/19 20:40:24
[] Port scanning: 192.168.56.101 [Elapsed time: 0 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 5 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 10 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 15 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 20 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 25 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 30 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 35 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 40 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 45 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 50 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 55 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 60 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 65 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 70 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 75 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 80 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 85 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 90 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 95 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 100 s]
[] Executing keep_alive..
[] Port scanning: 192.168.56.101 [Elapsed time: 105 s]
[] Executing keep_alive..
[] End time : 2019/07/19 20:42:30
[+] Get port list from nmap_result_192.168.56.101.xml.
[!] No open port.
[!] Shutdown Deep Exploit...
reinstall the requirements of pip and remains the same, i used metasploit from another server using msgrpc and get the same result, some idea of how to fix it, I do not believe the json, there must be some dependency that will have been updated in my system that is not working correctly.
When using Tidy for HTML matching check, the regular expression to get the count of warnings and errors might not work with all Tidy versions. Therefore I had to switch from:
str_pattern = r'.*Tidy found ([0-9]+) warnings and ([0-9]+) errors.*$'
to:
str_pattern = r'.*([0-9]+) warnings, ([0-9]+) error were found!.*$'
Hi
Im trying to install the requirements with pip3 install -r requirements.txt and I manage to install some of them but others don't
I get
Complete output from command python setup.py egg_info:
IMPORTANT WARNING:
pkg-config is not installed.
matplotlib may not be able to find some of its dependencies
============================================================================
Edit setup.cfg to change the build options
BUILDING MATPLOTLIB
matplotlib: yes [2.0.2]
python: yes [3.7.3rc1 (default, Mar 13 2019, 11:01:15) [GCC
8.3.0]]
platform: yes [linux]
REQUIRED DEPENDENCIES AND EXTENSIONS
numpy: yes [version 1.16.2]
six: yes [using six version 1.12.0]
dateutil: yes [using dateutil version 2.7.3]
functools32: yes [Not required]
subprocess32: yes [Not required]
pytz: yes [using pytz version 2019.1]
cycler: yes [using cycler version 0.10.0]
tornado: yes [using tornado version 5.1.1]
pyparsing: yes [using pyparsing version 2.2.0]
libagg: yes [pkg-config information for 'libagg' could not
be found. Using local copy.]
freetype: no [The C/C++ header for freetype2 (ft2build.h)
could not be found. You may need to install the
development package.]
png: no [pkg-config information for 'libpng' could not
be found.]
qhull: yes [pkg-config information for 'qhull' could not be
found. Using local copy.]
OPTIONAL SUBPACKAGES
sample_data: yes [installing]
toolkits: yes [installing]
tests: no [skipping due to configuration]
toolkits_tests: no [skipping due to configuration]
OPTIONAL BACKEND EXTENSIONS
macosx: no [Mac OS-X only]
qt5agg: no [PyQt5 not found]
qt4agg: yes [installing, Qt: 4.8.7, PyQt: 4.8.7; PySide not
found]
gtk3agg: yes [installing, version 3.5.24]
gtk3cairo: yes [installing, version 3.5.24]
gtkagg: no [Requires pygtk]
tkagg: yes [installing; run-time loading from Python Tcl /
Tk]
wxagg: no [requires wxPython]
gtk: no [Requires pygtk]
agg: yes [installing]
cairo: yes [installing, pycairo version 1.16.2]
windowing: no [Microsoft Windows only]
OPTIONAL LATEX DEPENDENCIES
dvipng: no
ghostscript: yes [version 9.27]
latex: yes [version 3.14159265]
pdftops: yes [version 0.71.0]
OPTIONAL PACKAGE DATA
dlls: no [skipping due to configuration]
============================================================================
* The following required packages can not be built:
* freetype, png
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-znp_ozre/matplotlib/
I have installed pkgconfig 1.5.1 (pip install pkgconfig), matplotlib 3.1.0 (pip install matplotlib) and panda (pip install pandas) manually
when I tried to run deepexploit it returned me
Traceback (most recent call last):
File "DeepExploit.py", line 18, in
import pandas as pd
ModuleNotFoundError: No module named 'pandas'
How can I bypass this and proceed with the installation? How freetype and png installed and all the other requirements?
Thanks
2018-07-17 22:05:12 [scrapy.utils.log] INFO: Scrapy 1.5.0 started (bot: scrapybot)
2018-07-17 22:05:12 [scrapy.utils.log] INFO: Versions: lxml 4.2.3.0, libxml2 2.9.8, cssselect 1.0.3, parsel 1.5.0, w3lib 1.19.0, Twisted 18.7.0, Python 3.5.3 (default, Jan 19 2017, 14:11:04) - [GCC 6.3.0 20170118], pyOpenSSL 18.0.0 (OpenSSL 1.1.0h 27 Mar 2018), cryptographyform Linux-4.9.0-3-amd64-x86_64-with-debian-9.5
2018-07-17 22:05:12 [scrapy.crawler] INFO: Overridden settings: {'FEED_FORMAT': 'json', 'SPIDER_LOADER_WARN_ONLY': True, 'FEED_URI': 'crawl_result/20180717220511_crawl_result.json'}
2018-07-17 22:05:12 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
'scrapy.extensions.feedexport.FeedExporter',
'scrapy.extensions.logstats.LogStats',
'scrapy.extensions.memusage.MemoryUsage',
'scrapy.extensions.telnet.TelnetConsole']
[*] Save log to /opt/machine_learning_security/DeepExploit/crawl_result/some-ip1_80.log
2018-07-17 22:05:13 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
'scrapy.downloadermiddlewares.retry.RetryMiddleware',
'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
'scrapy.downloadermiddlewares.stats.DownloaderStats']
2018-07-17 22:05:13 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
'scrapy.spidermiddlewares.referer.RefererMiddleware',
'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
'scrapy.spidermiddlewares.depth.DepthMiddleware']
2018-07-17 22:05:13 [scrapy.middleware] INFO: Enabled item pipelines:
[]
2018-07-17 22:05:13 [scrapy.core.engine] INFO: Spider opened
2018-07-17 22:05:13 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2018-07-17 22:05:13 [scrapy.extensions.telnet] DEBUG: Telnet console listening on 127.0.0.1:6023
2018-07-17 22:05:13 [scrapy.core.engine] DEBUG: Crawled (200) <GET http://some-ip:80/> (referer: None)
2018-07-17 22:05:13 [scrapy.core.engine] INFO: Closing spider (finished)
2018-07-17 22:05:13 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 212,
'downloader/request_count': 1,
'downloader/request_method_count/GET': 1,
'downloader/response_bytes': 564,
'downloader/response_count': 1,
'downloader/response_status_count/200': 1,
'finish_reason': 'finished',
'finish_time': datetime.datetime(2018, 7, 17, 20, 5, 13, 150929),
'log_count/DEBUG': 2,
'log_count/INFO': 7,
'memusage/max': 55209984,
'memusage/startup': 55209984,
'response_received_count': 1,
'scheduler/dequeued': 1,
'scheduler/dequeued/memory': 1,
'scheduler/enqueued': 1,
'scheduler/enqueued/memory': 1,
'start_time': datetime.datetime(2018, 7, 17, 20, 5, 13, 10283)}
2018-07-17 22:05:13 [scrapy.core.engine] INFO: Spider closed (finished)
Traceback (most recent call last):
File "DeepExploit.py", line 2064, in
target_tree = env.get_target_info(rhost, proto_list, info_list)
File "DeepExploit.py", line 525, in get_target_info
web_target_info = self.util.run_spider(rhost, web_port_list)
File "/opt/machine_learning_security/DeepExploit/util.py", line 159, in run_spider
dict_json = json.load(fin)
File "/usr/lib/python3.5/json/init.py", line 268, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/usr/lib/python3.5/json/init.py", line 319, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.5/json/decoder.py", line 339, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.5/json/decoder.py", line 357, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
when i run deepexploit,i find kali system is stop. i can't operate it.
i want to know deepexploit will use how many memory and cpu?
I remember seeing something about this from DEFCON 26 this year and finally got around to checking out this tool. took me forever to get it installed properly, had some issues getting it to actually run in training or test mode. had some Errors once I got it running. but once I did everything looked pretty smooth.
ran a few training sessions against a patched windows 7 machine ( figured there wouldn't be too much to discover there) but the machine did have a bunch of extraneous services and applications running.
ran a test against it and it found nothing. So I decided to train/test against an old windows 2000 advanced server that I had hanging around, once again everything looked fine, in training mode, when that completed I ran it in Test mode and I expected it to be able to exploit a few things on that old 2000 server if for no other reason than it was full of holes when it was retired.
so my question is I am not sure how I can tell if this tool is even functioning properly, or if it just didn't find anything or if maybe eevn though I thought it was setup correctly it wasn't anyway great idea in thought but so far I am not able to verify the practice.
thanks!
Hi I was trying out the DeepExploit.py and I ran into some errors that I was able to solve and some errors I couldn't.
Regarding the issue before this, I found out that you should use 'python3-pip' and use 'pip3 install libraryname' in order to run DeepExploit.py properly. And you need to type python3 DeepExploit.py in order to run the python file with python3(Kali most up to date version).
Also I found a typo in DeepExploit.py which causes a value error.
In line 1282 or 1288, there is a period(.) after exploit_tree.json (so it would be 'exploit_tree.json.' instead of 'exploit_tree.json' which causes a value error when running in the training mode.
Now the errors that I couldn't fix:
Everything runs smoothly but I hit this error at almost the end. Any idea if this is a bug?
Traceback (most recent call last):
File "DeepExploit.py", line 2309, in <module>
saver.restore(SESS, env.save_file)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/training/saver.py", line 1268, in restore
+ compat.as_text(save_path))
ValueError: The passed save_path is not a valid checkpoint: /root/machine_learning_security/DeepExploit/trained_data/DeepExploit.ckpt
To start off I'm writing this using a fresh install of Kali 2018 4.15.0-kali3-amd64.
After running your install script there is a few workarounds ( I cant remember specifically the issues I had installing to start but I had to do some removal dist-packages that came with Kali using rm -rf and re running pip install -r requirements.txt -I).
Past that the first thing I get when running python DeepExploit.py -t xxx.xxx.xxx.xxx -m train I get "Invalid IP address" which is caused by ipaddress.ip_address(arg) -> ipaddress.ip_address expects a unicode IP see below
ipaddress.ip_address('0.0.0.0')
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python2.7/dist-packages/ipaddress.py", line 163, in ip_address
' a unicode object?' % address)
ipaddress.AddressValueError: '0.0.0.0' does not appear to be an IPv4 or IPv6 address. Did you pass in a bytes (str in Python 2) instead of a unicode object?
ipaddress.ip_address(u'0.0.0.0')
IPv4Address(u'0.0.0.0')
Which can be fixed by converting the rhost to a unicode variable. I changed
rhost, mode, port, service = command_parse()
to
rhost, mode, port, service = command_parse()
rhost = unicode(rhost)
When finding a successful exploit a UnicodeDecodeError is thrown from show_banner_bingo which can be resolved by changing
banner = """
to
banner = u"""
When using test mode ( python Deepexploit.py -t xx.xx.xx.xx -m test) this error is thrown. I don't have a fix on this one as I do not understand completely the code that is being used for the tensorflow learning algorithms, however it appears that the apply_gradients call is throwing it.
WARNING:tensorflow:From DeepExploit.py:1321: calling reduce_sum (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
Traceback (most recent call last):
File "DeepExploit.py", line 1819, in
rhost=rhost))
File "DeepExploit.py", line 1626, in init
self.environment = Environment(thread_name, thread_type, parameter_server, rhost)
File "DeepExploit.py", line 1470, in init
self.agent = Agent(name, parameter_server)
File "DeepExploit.py", line 1396, in init
self.brain = LocalBrain(name, parameter_server)
File "DeepExploit.py", line 1296, in init
self._build_graph(name, parameter_server)
File "DeepExploit.py", line 1337, in _build_graph
parameter_server.optimizer.apply_gradients(zip(self.grads, parameter_server.weights_params))
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/optimizer.py", line 598, in apply_gradients
([str(v) for _, _, v in converted_grads_and_vars],))
ValueError: No gradients provided for any variable: ["<_RefVariableProcessor(<tf.Variable 'parameter_server/dense_1/kernel:0' shape=(7, 50) dtype=float32_ref>)>", "<_RefVariableProcessor(<tf.Variable 'parameter_server/dense_1/bias:0' shape=(50,) dtype=float32_ref>)>", "<_RefVariableProcessor(<tf.Variable 'parameter_server/dense_2/kernel:0' shape=(50, 100) dtype=float32_ref>)>", "<_RefVariableProcessor(<tf.Variable 'parameter_server/dense_2/bias:0' shape=(100,) dtype=float32_ref>)>", "<_RefVariableProcessor(<tf.Variable 'parameter_server/dense_3/kernel:0' shape=(100, 200) dtype=float32_ref>)>", "<_RefVariableProcessor(<tf.Variable 'parameter_server/dense_3/bias:0' shape=(200,) dtype=float32_ref>)>", "<_RefVariableProcessor(<tf.Variable 'parameter_server/dense_4/kernel:0' shape=(200, 538) dtype=float32_ref>)>", "<_RefVariableProcessor(<tf.Variable 'parameter_server/dense_4/bias:0' shape=(538,) dtype=float32_ref>)>", "<_RefVariableProcessor(<tf.Variable 'parameter_server/dense_5/kernel:0' shape=(200, 1) dtype=float32_ref>)>", "<_RefVariableProcessor(<tf.Variable 'parameter_server/dense_5/bias:0' shape=(1,) dtype=float32_ref>)>"].
The last thing is that I've always had errors when using the data folder that it came with. You should probably just ship the data folder empty by default and let it enumerate the exploits through Metasploit since your code already does that.
After solving the problems that I can solve, I have encountered some other problems.
That's the way he is.
[] 95/1148 exploit:linux/http/pandora_fms_exec, targets:1
[] 96/1148 exploit:linux/http/pandora_fms_sqli, targets:1
[] 97/1148 exploit:linux/http/pineapp_ldapsyncnow_exec, targets:1
[] 98/1148 exploit:linux/http/pineapp_livelog_exec, targets:1
Traceback (most recent call last):
File "DeepExploit.py", line 1599, in
exploit_tree = get_exploit_tree(env)
File "DeepExploit.py", line 1293, in get_exploit_tree
print('[*} Timeout: {0}'.format(show_cmd))
ValueError: Single '}' encountered in format string
The error is random and does not repeat at one point.
In addition, for some beginners, or for the first installation, I strongly recommend that the author's script download the environment dependency package required for the tool automatically when it first runs in order to ensure that the error is reduced.
For example, the following:
pip install --upgrade pip
python3 -m pip install docopt
python3 -m pip install theano;
python3 -m pip install tensorflow;
python3 -m pip install keras;
python3 -m pip install h5py==2.8.0rc1
python3 -m pip install keepdims
In addition, there is an abandoned parameter in the 928934 row of deep-exploit.
I replaced it with this keepdims
I don't know what the consequences will be, but I have been trying to solve these problems. If you come across this problem, I hope you can communicate with you more. Please forgive me, I'm not good at English.
When I enter the IP address for the target... it gives the error "Invalid IP address"
Hi, I have tried to add an exploit that is not inside the current pool of exploit list based of your rank filter. So I made the edit to this line of code:
for module in module_list:
if module[1] in {'excellent', 'great', 'good'}:
for module in module_list:
if module[1] in {'excellent', 'great', 'good', 'normal', 'average', 'low'}:
Doing so still does not add the exploit with rank normal/average into the exploit tree.
Could you let me how I can manually add the exploit into the tree.
Thanks.
Regards,
Cheng Lim
It seems that using GAN will be fine to achieve this, or maybe why you chose RL. Or if we can use the tradition ML to extract the features from gathering stage...
Hi
I would like to know how can we define a range of targets
can we select a sub-net?
can we import from a list?
or is a single target?
If not possible to use multiple targets, a leave here my suggestion
About the modules/exploits, just to confirm, are those that included in our metaploit, right?
Thanks
Hi,
On a debian 9 system as well as on a kali system I keep getting
[root:...arning_security/DeepExploit]# python3 ./DeepExploit.py (master✱)
[1] 1728 illegal hardware instruction python3 ./DeepExploit.py
I did install -r requirements, it is using tenserflow 1.8
I see error on line 139 in file
machine_learning_security/Generator/util.py
when trying to obj_browser.switch_to.alert.accept()
The error message is:
selenium.common.exceptions.NoAlertPresentException: Message: no such alert
Can you help me with that?
Hello,
I am interested in auto-generated code.
when I run generator.py, there is an error:
File "Generator\gan_main.py", line 324, in main
lst_scripts.extend(self.train(target_sig_list))
File "\Generator\gan_main.py", line 141, in train
d_loss = discriminator.train_on_batch(X, y)
ValueError: Error when checking input: expected dense_8_input to have shape (10,) but got array with shape (5,)
What should I do?
Thank you.
Best wishes,
~Tristan
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.