andeevosters / movies_etl

Movies-ETL

Scope

Amazing Prime wants to use and update a dataset we created on a daily basis. We created an automated pipeline that takes in new data, performs the appropriate transformations, and loads the data into existing PostgreSQL tables.

We use data from three external places: Wikipedia, Kaggle, and MovieLens.

What We Did

1: We created an ETL Function to Read Three Data Files. 2: We Extracted and Transformed the Wikipedia Data. 3: We Extracted and Transformed the Kaggle data. 4: We created the Movie Database to hold the cleaned movie and ratings data.

The cleaned movie data contains 6,052 rows, and the ratings data contains 26,024,289 rows of data. ![Movies Query]( ![Ratings Query](# Movies_ETL