Coder Social home page Coder Social logo

big_data_integration_project's Introduction

Big_Data_Integration_Project

A project to find the seller engagement for a fictional e-commerce company called Dualcore

Given a challenging messy dataset for a fictional e-commerce company called Dualcore, we were asked to integrate it with any external dataset, identify a business opportunity and come up with a unique data-driven solution for it.

We used an Amazon dataset consisting of ~3 million rows. This dataset consisted of data extracted from the Q&A section of electronic products listed on Amazon. The business problem we chose to identify the level of seller engagement in the Q&A section so that the company can know which sellers they need to push to be more proactive in answering product questions from potential customers to increase the level of confidence in a product and quicken the decision making process for the customer before they can change their mind.

The real challenge was to integrate the external dataset with the Dualcore database as the granularity was different. We used Hue to ingest the data. We also used a Spark enabled virtual machine cluster to run our queries and perform all the text processing required to perform the kind of analysis we wanted. We used the Hive command line interface, HiveQL functions as well as the Hue editor to write and run our queries.

big_data_integration_project's People

Contributors

tanvikini avatar

Watchers

James Cloos avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.