Coder Social home page Coder Social logo

data-engineering-with-aws---udacity's Introduction

Data-Engineering-with-AWS---Udacity

Quick start

This course has been taken in the platform Udacity, and takes over 160 hours to complete de degree and get the certificate. The issues related to the database structures, have fulfilled a lack of my knowledge and have taught me the bases of relational and no relational databases, data warehouses, data lakes and data pipelines specifically in the Amazon Web Services environment. I did not like too much the AWS drag&drop interface, in my opinion, the best way to acquire this knowledge comes from programming in console (CLI), and not from selecting buttons and building graphs. However, this is possible from the AWS Cloud Shell and the python scripts automatically generated. Another issue to improve is the support help from Udacity In particular, Apache Airflow raised exceptions that were not contemplated in the Udacity Frequent Answer and Questions, and the human help is too slow and not focused on the problem. Although Udacity-GPT is a great way to solve problems, it could be better to optimize the contact between tutor and student. However, I absolutely recommend this online course, the teachers are great and can solve any problem related to the content.

1. Data Modeling

Learn to create relational and NoSQL data models to fit the diverse needs of data costumers. Use ETL to build databases in PostgreSQL and Apache Cassandra:

  • Introduction to Data Modeling.
  • Relational Data Models with SQL and PostgreSQL.
  • NoSQL Data Models with Apache Cassandra.
  • Final Project: Data Modeling with Apache Cassandra.

2. Cloud Data Warehouses

In this course, we will learne to create cloud-based data warehouses. We will sharpen our data warehouses skills, deepen our understanding of data infrastructure, and be introduced to data engineering on the cloud using Amazon Web Services (AWS):

  • Introduction to Cloud Data Warehouses.
  • Introduction to Data Warehouses.
  • ETL and Data Warehouse Technology in the Cloud.
  • AWS Data Warehouse Technologies.
  • Implementing a Data Warehouse on AWS.
  • Final Project: Data Warehouse.

3. Spark and Data Lakes

In this course, we will learn about the big data ecosystem and how to use Spark to work with massive datasets. We will also learn about how to store big data in a data lake and query it with Spark:

  • Introduction to Spark and Data Lakes.
  • Big Data Ecosystem, Data Lakes and Spark.
  • Spark Essentials.
  • Using Spark in AWS.
  • Ingesting and Organizing Data in a Lakehouse.
  • Final Project: STEDI Human Balance Analytics.

4. Automate Data Pipelines

In this course, we will build pipelines leveraging Airflow DAGs to organize our tasks along with AWS resources such as S3 and Redshift:

  • Introduction to Automating Data Pipelines.
  • Data Pipelines.
  • Airflow and AWS.
  • Data Quality.
  • Production Data Pipelines.
  • Final Project: Data Pipelines.

data-engineering-with-aws---udacity's People

Contributors

huunhat1703tkbn avatar

Stargazers

 avatar

Watchers

 avatar

Forkers

huunhat1703

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.