ashok-amsamani Goto Github PK

credit-card-defaulters

dataengineering

This repo contains commands that data engineers use in day to day work.

gps-usecase

hackerrank

My solutions to HackerRank problems

hive-dml

Documented my learnings - how to perform DML operations in HIVE.

hive-etl-olap

Documented my learnings about ETL operations on various table, flattening data from different table and aggregating on top of it.

hive-orc-parquet-benchmark

Finding storage space requirement and data retrieval time for ORC and Parquet.

hive-schema-evolution

Detailed about how we can dynamically load columns in HIVE using AVRO.

hive-schema-migration-xml-to-json

hive-sqoop-integration

Listed steps about how to move data from Mysql to HIVE using Sqoop and Hive to Mysql using sqoop.

hive-sqoop-integration-2

Listed steps about how to move data from Mysql to HIVE using Sqoop and Hive to Mysql using sqoop. Follow steps given in script.txt. It has HIVE/SQOOP/Linux and Hadoop scripts. You should have a knowledge on where to run what to try this excercise. It covers:

hive-xml-serde

XML Serializer/Deserializer for Apache Hive

pyspark-cheatsheet

PySpark Cheat Sheet - learn PySpark and develop apps faster

pyspark-examples

Pyspark RDD, DataFrame and Dataset Examples in Python language

pysparkcheatsheet

PySpark Cheatsheet

retail-challenge

scala-beginners

The official repository for the Rock the JVM Scala beginners course

spark-core-usecases

spark-sqoopcdc

Change data capture realization using Spark and Sqoop

spark-the-definitive-guide

Spark: The Definitive Guide's Code Repository

sqoop-create-hive-table-

Creating hive table directly from sqoop.