Comments (6)
hey man,
thanks for feedback - if u can attach example text file with this line i'll
try to debug through sometime in the next few days
Adam
On Mon, Apr 9, 2012 at 12:05 PM, bkc <
[email protected]
wrote:
Hi,
Great package. I have hit one small problem:
using logtools==0.8
got this exception:
File
"/home/bkc/Python_Environments/weblog/lib/python2.7/site-packages/logtools/_parse.py",
line 100, in logparse
yield key_func(line)
File
"/home/bkc/Python_Environments/weblog/lib/python2.7/site-packages/logtools/parsers.py",
line 47, in multiindex_getter
data = parser(line.strip())
File
"/home/bkc/Python_Environments/weblog/lib/python2.7/site-packages/logtools/parsers.py",
line 69, in call
return self.parse(line)
File
"/home/bkc/Python_Environments/weblog/lib/python2.7/site-packages/logtools/parsers.py",
line 183, in parse
raise ValueError("Could not parse log line: '%s'" % logline)
ValueError: Could not parse log line: '122.118.199.146 -
"car5941 " [09/Apr/2012:14:35:48 -0400] "GET /favicon.ico HTTP/1.1" 200
125 "-" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/535.19 (KHTML, like
Gecko) Chrome/18.0.1025.151 Safari/535.19"'Editing the input file and removing the space character, e.g. "car5941
" to "car5941" fixes the problem.
Reply to this email directly or view it on GitHub:
#4
from logtools.
I can't find a way to attach a file to the git issue.
I have attached a file to this note.
Brad Clements, [email protected] (315)268-1000
Jabber/XMPP: [email protected]
from logtools.
Brad,
I take it you tried using the logparser utility / api with this line. Can you supply the command-line you tried, or otherwise the parser / format string you're using for this?
from logtools.
Hi,
Sorry I did not include this information before:
the command line is:
(weblog)[bkc@server5 scripts]$ cat /tmp/test.log | logparse --parser
AccessLog --format '%h %l %u %t "%r" %>s %b "%{Referer}i"
"%{User-agent}i"' -f1,2
2012-04-09 19:01:16,903 - root - ERROR - Could not match fields for
parsed line: 69.18.99.146 - "car5941 " [09/Apr/2012:14:35:48 -0400]
"GET /favicon.ico HTTP/1.1" 200 125 "-" "Mozilla/5.0 (Windows NT 5.1)
AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.151 Safari/535.19"
Traceback (most recent call last):
File "/home/bkc/Python_Environments/weblog/bin/logparse", line 9, in
load_entry_point('logtools==0.8', 'console_scripts', 'logparse')()
File
"/home/bkc/Python_Environments/weblog/lib/python2.7/site-packages/logtools/_parse.py",
line 112, in logparse_main
for row in logparse(options, args, fh=sys.stdin):
File
"/home/bkc/Python_Environments/weblog/lib/python2.7/site-packages/logtools/_parse.py",
line 100, in logparse
yield key_func(line)
File
"/home/bkc/Python_Environments/weblog/lib/python2.7/site-packages/logtools/parsers.py",
line 47, in multiindex_getter
data = parser(line.strip())
File
"/home/bkc/Python_Environments/weblog/lib/python2.7/site-packages/logtools/parsers.py",
line 69, in call
return self.parse(line)
File
"/home/bkc/Python_Environments/weblog/lib/python2.7/site-packages/logtools/parsers.py",
line 183, in parse
raise ValueError("Could not parse log line: '%s'" % logline)
ValueError: Could not parse log line: '69.18.99.146 - "car5941 "
[09/Apr/2012:14:35:48 -0400] "GET /favicon.ico HTTP/1.1" 200 125 "-"
"Mozilla/5.0 (Windows NT 5.1) AppleWebKit/535.19 (KHTML, like Gecko)
Chrome/18.0.1025.151 Safari/535.19"'
On 04/09/2012 06:13 PM, Adam Ever-Hadani wrote:
Brad,
I take it you tried using the logparser utility / api with this line. Can you supply the command-line you tried, or otherwise the parser / format string you're using for this?
Reply to this email directly or view it on GitHub:
#4 (comment)
Brad Clements, [email protected] (315)268-1000
Jabber/XMPP: [email protected]
from logtools.
Hey Brad,
sorry for taking so long to respond. I played around with the script and it looks like if removing the backslashes on quotes and specifying them in the format (e.g "%u" instead of %u) works. Question is - are you specifically trying to use quoted usernames that might have spaces in them that u'd like to differentiate?
here is what worked for me (after as said, changed original logs from " to " where appropriate ):
cat data/buglog.txt | logparse --parser AccessLog --format '%h %l "%u" %t "%r" %>s %b "%{Referer}i" "%{User-agent}i"' -f1,2,3
122.118.199.146 - car5941
122.118.199.146 - car5941
from logtools.
Hi,
Thanks for the response.
The field we are logging doesn't contain quote marks in the values to be
logged, so I have no idea why apache is adding quotes around that field
when recording them in the log. It's also perplexing that it then adds
backslashes to the quotes it's added to the field.
We do not expect the %u column to contain spaces. I suspect that one
user accidentally typed in a trailing space when they logged in, the
authentication routine must have stripped that space when checking the
password, but the cookie was set to the value with the trailing space.
does that help?
On 05/01/2012 04:46 PM, Adam Ever-Hadani wrote:
Hey Brad,
sorry for taking so long to respond. I played around with the script and it looks like if removing the backslashes on quotes and specifying them in the format (e.g "%u" instead of %u) works. Question is - are you specifically trying to use quoted usernames that might have spaces in them that u'd like to differentiate?here is what worked for me (after as said, changed original logs from " to " where appropriate ):
cat data/buglog.txt | logparse --parser AccessLog --format '%h %l "%u" %t "%r" %>s %b "%{Referer}i" "%{User-agent}i"' -f1,2,3
122.118.199.146 - car5941
122.118.199.146 - car5941
Reply to this email directly or view it on GitHub:
#4 (comment)
Brad Clements, [email protected] (315)268-1000
Jabber/XMPP: [email protected]
from logtools.
Related Issues (7)
- Where can I download "bots_hosts.txt" and "bots_useragents.txt" file ? HOT 1
- Unrecognized qps --format argument in README HOT 1
- Better formatting for the qps value HOT 1
- SQLSoup has been refactored out of latest SQLAlchemy as a separate module HOT 3
- documentation on percentiles script
- Moved to Python3, latest SQLAlchemy (supports Mysql8.0), plus some extensions HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from logtools.