ezPAARSE is an open-source software that can ingest your (proxy) log files and show how users access subscribed electronic resources. It filters, extracts and enriches the consultation events that were spotted and produces a CSV file following COUNTER codes of practice. This document describes how to install and run ezPAARSE on your computer.
Moreover, have a look to the ezpaarse demo, it will show you a nice user interface where you can register and test to process your own proxy logs.
Built-in proxies supported log formats are: ezproxy, bibliopam, and squid
Table of content
- Recommended system requirements
- Prerequisites
- Installation quickstart
- Test the installation
- Usage
- Go further
- Advanced parameters
- Use with docker
- a linux box or VM (eg: Ubuntu)
- 50Gb disk space (to be adjusted, depending on the quantity and size of logfiles to be simultaneously processed)
- 2 cores of CPU
- 2 to 4 Gb of RAM space
The tools you need to let ezPAARSE run are:
- Linux OS: See the prerequisites for those OSes
- Standard Linux tools: bash, make, grep, sed ...
- python
- gcc and g++
- curl (used by nvm)
- git >= 1.7.10 (required to clone ezpaarse from github and to keep your ezpaarse copy up to date)
- MongoDB >= 3.2
ezPAARSE then comes with all the elements it needs to run. When the prerequesites are met, you can run the make command (see below) that will run all installation steps.
If you are a Windows user, you can install ezPAARSE on your computer as a docker image. Please refer to the docker section below.
To install the latest stable version of ezPAARSE on a Unix-type system, open a terminal and type:
git clone https://github.com/ezpaarse-project/ezpaarse.git
cd ezpaarse
git checkout `git describe --tags --abbrev=0`
makeIf you want to install the version in development (unstable), open a terminal and type:
git clone https://github.com/ezpaarse-project/ezpaarse.git
cd ezpaarse
makeThis step allows you to validate that your install is working.
make start
make testAnonymised example logfiles are made available in the repositories of ezPAARSE.
You need to make sure that ezPAARSE is started. To do so, type the following command:
make startIf you are not computer-savvy, the easiest way to work with ezPAARSE is to use its HTML form, accessible from your favorite webbrowser and open the following URL: http://localhost:59599/
If you are computer-savvy, you can use an HTTP client (like curl) to send a logfile (for this example, we will use ./test/dataset/sd.2012-11-30.300.log) to ezPAARSE's Web service and get a CSV stream of consultation events as a response.
curl -X POST http://127.0.0.1:59599 \
-v --proxy "" --no-buffer \
--data-binary @./test/dataset/sd.2012-11-30.300.logOr you can use the command ./bin/loginjector ezPAARSE provides you with
to send the logfile to the web service in a simpler way:
. ./bin/env
cat ./test/dataset/sd.2012-11-30.300.log | ./bin/loginjectorYou can also see quick countings on your data if you add the command
./bin/csvtotalizer at the end of the command line.
Doing so, you will get an overview of the consultation events extracted
from your logs by ezPAARSE:
. ./bin/env
cat ./test/dataset/sd.2012-11-30.300.log | ./bin/loginjector | ./bin/csvtotalizerTo stop ezPAARSE, you have to type the following command:
make stopTo go further, you can consult the full documentation
The default ezPAARSE parameters can be found in the config.json file. All these parameters can be changed. A good practice is to define a new file called config.local.json containing just the parameters you need to override.
For example, to change the ezPAARSE listening port (59599 by default), you can override the EZPAARSE_NODEJS_PORT by defining a new config.local.json file this way:
{
"EZPAARSE_NODEJS_PORT": 45000
}ezPAARSE is available as a docker image.
You need:
- Docker (Version >= 1.12)
- Docker Compose (Version >= 1.7)
Then, you can run the dockerized ezpaarse this way:
mkdir ezpaarse/
wget --no-check-certificate https://raw.githubusercontent.com/ezpaarse-project/ezpaarse/master/docker-compose.yml
test -f config.local.json || echo '{}' > config.local.json
# compose v1
docker-compose pull
docker-compose up -d
# compose v2
docker compose pull
docker compose up -d
Then ezpaarse is available at this URL: http://127.0.0.1:59599
To have a look to the ezpaarse system logs, you can run: docker logs -f ezpaarse
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent