daveqqq / web-log-parser

An open source analysis web log tool

web-log-parser

web-log-parser is an open source analysis web log tool, developed in python language, with flexible log format configuration

中文说明文档

---

Example

---

Instructions

1. store the logs files in the data directory ./data
2. modify config.ini
3. install requirements pip install -r requirements.txt
4. start the analysis in bin dircd ./bin && python start.py
5. check html result in result dir result/index.html

---

Configuration

Format

• log-pattern: regular expression for parsing logs
• log-format: log content format, corresponding log-pattern match. The following configuration items are supported by default.：
  • ip： request IP
  • real_ip： user real IP. If this option is configured, ip configuration will be ignored, and real_ip will be used for statistics.
  • datetime： request access time
  • url： request url
  • method： request method
  • protocol： request protocol
  • cost: request cost

Filter

• support_method: supported request method, otherwise no statistics
• is_with_parameters: enable flag for statistics url parameter，By default, the parameters in the URL will be converted, such as? Key = 123 when converting to key = {key}
• urls_most_number: the maximum number of URLs separately counted
• custom_parameters: configure special parameter key-value pairs, separated by commas, when configured t = {timeStamp}, the value of t key will be replaced with a fixed {timeStamp}
• fixed_parameter_keys: configure special parameter key values, separated by commas, when configured key = 123, will not be replaced with key = {key}
• ignore_urls: urls will be ignore
• static-file: file suffixes will be ignore

Report

• language: the language use in the report
  • chinese
  • english
• second_line_flag: enable flags for PV curve graph per second
• cost_time_flag: enable flags for response time interval proportion map and response time distribution map

GoAccess

• goaccess_flag: whether to get GoAccess analysis results, set to 1: Get, 0: Do not get (requires GoAccess installed)
• time-format: time format, for GoAccess
• date-format: date format, for GoAccess
• goaccess-log-format: log format，for GoAccess