abdulmajeedalomari / log-analysis-project

Building an informative summary from logs is a real task that comes up very often in software engineering. For instance, at Udacity we collect logs to help us measure student progress and the success of our courses. The reporting tools we use to analyze those logs involve hundreds of lines of SQL.

Log Analysis Project

You've been hired onto a team working on a newspaper site. The user-facing newspaper site frontend itself, and the database behind it, are already built and running. You've been asked to build an internal reporting tool that will use information from the database to discover what kind of articles the site's readers like.

The database contains newspaper articles, as well as the web server log for the site. The log has a database row for each time a reader loaded a web page. Using that information, your code will answer questions about the site's user activity.

The program you write in this project will run from the command line. It won't take any input from the user. Instead, it will connect to that database, use SQL queries to analyze the log data, and print out the answers to some questions.

This project is required for Full-Stack Nanodegree program.

It was developed by:

Abdulmajeed Alomari

Installation

• Download and install Vagrant.
• Download and install VirtualBox.
• Download FSND Virtual Machine.

Prerequisites

• Having python3 installed.
• You need to use Terminal on Mac or Linux systems. On windows, you I recommend using Git Bash.
• You need to download the project's data "newsdata.sql" from here.

Usage

Once you get the above software installed and the needed data downloaded:

1. Navigate to your FSND Virtual Machine folder and run the following commands:

cd vagrant
vagrant up
vagrant ssh
cd /vagrant
git clone https://github.com/AbdulmajeedAlOmari/log-analysis-project.git log-analysis-project
cd log-analysis-project

2. You need to install "psycopg2" module by running the following command: pip3 install psycopg2
3. Move the "newsdata.sql" to your project folder "log-analysis-project".
4. Load the data from the “newsdata.sql” by using the following command: psql -d news -f newsdata.sql
5. Connect to the database using the command: psql -d news
6. Now, you can run the python script using the command: python log-analysis.py

Sample Output

You can visit "sample_output" to see the output of the python script "log-analysis.py"

References

I used the following references to help me finish this project:

• Join using Like operator
• HTTP status codes
• Calculate percentage in an SQL query
• Creating and using subqueries
• Format dates
• Round a number
• README.md syntax