unbaiat / apache-scalp Goto Github PK

Just would like to see a verbosity level on output if possible.  I don't know 
if anything's happening right now while it's scanning.  

The default filter file is no longer available at the URL in the error message.

Old URL: https://svn.php-ids.org/svn/trunk/lib/IDS/default_filter.xml
New URL: https://dev.itratos.de/svn/php-ids/trunk/lib/IDS/default_filter.xml

The PHP IDS guys seem to have lost their old domain but are back up and running 
on https://phpids.org/

https://phpids.org/2011/03/30/we-are-back/

What steps will reproduce the problem?
1.run bellow
[xxxx@Keroro /]$ ./scalp-0.4.py --help
  File "./scalp-0.4.py", line 318
    total_nb_lines = sum(1 for line in open(access))
                             ^
SyntaxError: invalid syntax

What steps will reproduce the problem?
1. Python 2.7.3
2. scalp-0.4
3. RHEL4

What is the expected output? This is my first time using this tools.  What do 
you see instead?  Loading XML file './default_filter.xml'...
The rule 
'(?:union\s*(?:all|distinct|[(!@]*)?\s*[([]*\s*select)|(?:\w\s+like\s+\")|(?:lik
e\s*"\%)|(?:"\s*like\W*["\d])|(?:"\s*(?:n?and|x?or|not 
|\|\||\&\&)\s+[\s\w]+=\s*\w+\s*having)|(?:"\s*\*\s*\w+\W+")|(?:"\s*[^?\w\s=.,;)(
]+\s*[(@"]*\s*\w+\W+\w)|(?:select\s*[\[\]()\s\w\.,"-]+from)|(?:find_in_set\s*\()
' cannot be compiled properly

Hi,

With the latest PHPIDS rules, I get the following error with Scalp (Python
version):

The rule
(?:union\s*(?:all|distinct|[(!@]*)?\s*[([]\s*select)|(?:\w+\s+like\s+\")|(?:like
\s*"\%)|(?:"\s*like\W*["\d])|(?:"\s*(?:n?and|x?or|not
|\|\||\&\&)\s+[\s\w]+=\s*\w+\s*having)|(?:"\s*\*\s*\w+\W+")|(?:"\s*[^?\w\s=.,;)(
]+\s*[(@]*\s*\w+\W+\w)|(?:select\s*[\[\]()\s\w\.,-]+from)]]
cannot be compiled properly

The rules are a bit too complex for me to try debugging :) For now, I've
just removed this rule from the filter file. 

Is there an easy way to make it compile with Scalp?

Thanks

What steps will reproduce the problem?
1. /usr/bin/python2.5/bin/python scalp-0.4.py --log 
/home/webserver/httpd/error.log -o output --html --period 
14/Nov/2011:06*;*/Nov/2011
2.
3.

What is the expected output? What do you see instead?


What version of the product are you using? On what operating system?
scalp-0.4.py
python 2.5

Please provide any additional information below.

What steps will reproduce the problem?
1. Compile
2. Run: scalp -l ./tmp/$filename -f ./default_filter_mod.xml -o ./scalp-output 
--html


What is the expected output? What do you see instead?

- Expected an html report (as the python script does)
- Got only a log file



What version of the product are you using? On what operating system?

SVN version (latest)


Please provide any additional information below.

Hi,

Nice work!

I have tried out your software today and I found 1-2 interesting things I 
thought you might want to know. I had to modify a bit the C/C++ version in 
order to compile:

A. added some missing headers
B. Changed the Makefile (all libs ($OFLAGS) at the end of the line, remove 
architecture)

Attached is the diff file (System info ad the end)... 

Running scalp as mentioned above created a log file in the same directory but 
no html output so the C version does not work for me. (I don't know if it is in 
early dev stage or so...)


Something that may also be interesting is the exec. times. I may have messed up 
by changing the make file but it seems that python runs faster! 

 - C output:
507975 lines analyzed in 329.02 seconds
4328 possible warnings found

 - python output
Loading XML file './default_filter_mod.xml'...
Processing the file './tmp/access.log'...
Scalp results:
        Processed 507460 lines over 507975
        Found 5049 attack patterns in 277.271566 s
Generating output in ./scalp-output/access.log_scalp_*

real    4m38.187s
user    4m37.505s
sys     0m0.088s


(The errors/warnings above are all for xss)


My System Info:

* uname -a
Linux urban-uni 3.2.0-30-generic #48-Ubuntu SMP Fri Aug 24 16:52:48 UTC 2012 
x86_64 x86_64 x86_64 GNU/Linux

* cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=12.04
DISTRIB_CODENAME=precise
DISTRIB_DESCRIPTION="Ubuntu 12.04.1 LTS"

* g++ -v
Using built-in specs.
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.6/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 
4.6.3-1ubuntu5' --with-bugurl=file:///usr/share/doc/gcc-4.6/README.Bugs 
--enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr 
--program-suffix=-4.6 --enable-shared --enable-linker-build-id 
--with-system-zlib --libexecdir=/usr/lib --without-included-gettext 
--enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.6 
--libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu 
--enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object 
--enable-plugin --enable-objc-gc --disable-werror --with-arch-32=i686 
--with-tune=generic --enable-checking=release --build=x86_64-linux-gnu 
--host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5)

Hope it helps a bit. Let me know if you need any more info...

Regards,

Andreas

What steps will reproduce the problem?
1.
2.
3.

What is the expected output? What do you see instead?


What version of the product are you using? On what operating system?


Please provide any additional information below.

What steps will reproduce the problem?
1.Run script as
2. ./scalp-0.4.py  -l /var/log/apache2/access.log -f./default_filter.xml -o 
./scalp-output --html
3.

What is the expected output? What do you see instead?
The rule 
'(?:union\s*(?:all|distinct|[(!@]*)?\s*[([]*\s*select)|(?:\w+\s+like\s+\")|(?:li
ke\s*"\%)|(?:"\s*like\W*["\d])|(?:"\s*(?:n?and|x?or|not 
|\|\||\&\&)\s+[\s\w]+=\s*\w+\s*having)|(?:"\s*\*\s*\w+\W+")|(?:"\s*[^?\w\s=.,;)(
]+\s*[(@"]*\s*\w+\W+\w)|(?:select\s*[\[\]()\s\w\.,"-]+from)|(?:find_in_set\s*\()
' cannot be compiled properly


What version of the product are you using? On what operating system?

0.4


Please provide any additional information below.

What steps will reproduce the problem?
1../scalp.py -e -l ./access_log -f ./default_filter.xml -o ./scalp-output --html

What is the expected output? What do you see instead?
AFAIK, expected output would be that the script processed n number of lines

What is see instead is
Processing the file 'access_log'...
Scalp results:
    Processed 0 lines over 0
    Found 0 attack patterns in 0.524253 s

What version of the product are you using? On what operating system?
Version Used: scalp-0.4
OS: RHEL 5.7

Please provide any additional information below.
If I grep for directory traversing, the log file shows the grep parameters. But 
the same is not reflected in scalp through the directory traversing patterns 
are listed in the default_filter.xml file

--Syd

It would be helpfull to show the ip of each match or to be able to export ips 
and import to /etc/hosts.deny

What steps will reproduce the problem?
1.
/usr/bin/python2.5/bin/python scalp-0.4.py --log 
/home/webserver/httpd/access_log -o output --html --period -p 
14/Nov/2011:06*;*/Nov/2011
2.
3.

What is the expected output? What do you see instead?
I get the below error:
Traceback (most recent call last):
  File "scalp-0.4.py", line 633, in <module>
    main(len(sys.argv), sys.argv)
  File "scalp-0.4.py", line 601, in main
    preferences['period'] = analyze_date(argv[i+1])
  File "scalp-0.4.py", line 508, in analyze_date
    l_end  = l_date[1].split('/')
IndexError: list index out of range


What version of the product are you using? On what operating system?
OS:GNU/Linux
scalp-0.4.py
python 2.5


Please provide any additional information below.
 All other options work, accept --period, am i doing something wrong here? or is there any changes required?

What steps will reproduce the problem?
1. ./scalp-0.4.py -l /var/log/apache2/access.log -f ./default_filter.xml -o
./scalp-output --html

What is the expected output? What do you see instead?
Expected : Unsure, never ran.
Actual : 

  File "./scalp-0.4.py", line 328
    with open(access) as log_file:
            ^
SyntaxError: invalid syntax


What version of the product are you using? On what operating system?
Scalp : 0.4
OS : Debian 4 with kernel 2.6.18-6-686
Python : 2.4.4

Please provide any additional information below.
md5sum of scalp : 90f87b11fccb21028c60634cc1c5f305

Hi Romain!,

    I've been testing scalp with a log file that I got from a friend and
it's sending me lots of false positives, I'm reporting them, hoping that
you fix them in the 0.5 version =)

    ### Impact 5
    67.195.37.122 - - [04/Dec/2008:02:36:04 -0200] "GET
/QP/index.php?view=article&id=1:principal&tmpl=component&print=1&page=
HTTP/1.0" 200 4053 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp/3.0;
http://help.yahoo.com/help/us/ysearch/slurp)"
    Reason: "Detects JavaScript with(), ternary operators and XML predicate
attacks"


    ### Impact 4
    190.27.11.202 - - [01/Dec/2008:15:21:58 -0200] "GET
/QP/index.php?view=article&id=3%3Aiso-9000&tmpl=component&print=1&page=&option=c
om_content&Itemid=3
HTTP/1.1" 200 16143
"http://www.google.com.co/search?hl=es&q=motivacion+implementacion+iso+9000&star
t=30&sa=N"
"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)"
    Reason: "Detects JavaScript object properties and methods"

    ### Impact 3
    201.252.60.230 - - [01/Dec/2008:00:04:18 -0200] "GET
/QP/index.php?option=com_content&view=article&id=6&Itemid=6 HTTP/1.1" 200
9062 "http://qperformance.com.ar/QP/" "Mozilla/4.0 (compatible; MSIE 7.0;
Windows NT 5.1; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ;
.NET CLR 2.0.50727; .NET CLR 1.1.4322)"
    Reason: "Detects very basic XSS probings"

    201.252.60.230 - - [01/Dec/2008:00:02:45 -0200] "GET
/QP/templates/system/css/error.css HTTP/1.1" 200 1672
"http://qperformance.com.ar/QP/index.php?option=com_content&view=article&id=4#co
ntent"
"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Mozilla/4.0
(compatible; MSIE 6.0; Windows NT 5.1; SV1) ; .NET CLR 2.0.50727; .NET CLR
1.1.4322)"
    Reason: "Detects specific directory and path traversal"

What steps will reproduce the problem?
1. Go to http://code.google.com/p/apache-scalp/
2. Click on "You will then need this file 
https://svn.php-ids.org/svn/trunk/lib/IDS/default_filter.xml "
3. The link is broken

What is the expected output? 

I'm unsure, but probably:

http://dev.itratos.de/projects/php-ids/repository/raw/trunk/lib/IDS/default_filt
er.xml

What steps will reproduce the problem?
1. scalp-0.4.py -l access.log
2. wget https://svn.php-ids.org/svn/trunk/lib/IDS/default_filter.xml
3.

What is the expected output? What do you see instead?
some attack info
error: the filters file (XML) doesn't exist
please download it at 
https://svn.php-ids.org/svn/trunk/lib/IDS/default_filter.xml
Resolving svn.php-ids.org... failed: Name or service not known.
wget: unable to resolve host address “svn.php-ids.org”


What version of the product are you using? On what operating system?
0.4
linux


Please provide any additional information below.

What version of the product are you using? On what operating system?
Scalp-0.4 on Microsoft Windows

Please provide any additional information below.
Newbie question.
Is it possible to run Apache-scalp on Windows operating system with Python 
installed ?

What steps will reproduce the problem?
1. Downloaded default xml from 
https://dev.itratos.de/projects/php-ids/repository/raw/trunk/lib/IDS/default_fil
ter.xml
2.
./scalp.py -l /var/log/apache2/access.log -f ./default_filter.xml -o file --html
3.

What is the expected output? What do you see instead?
something...ERROR: "(XML)...cannot be compiled properly"

What version of the product are you using? On what operating system?
0.4

Please provide any additional information below.

As root, I have done :
./scalp-0.4.py /var/log/apache2/access.log -f ./default_filter.xml -o
./public_html/ --html
error: the log file doesn't exist

But access.log exists and it s readable by root. I am not Python fluent,
using under ubuntu gutsy :
# python --version
Python 2.5.2

Ty!