Coder Social home page Coder Social logo

ovokpus / aws-etl-pipeline Goto Github PK

View Code? Open in Web Editor NEW
1.0 2.0 2.0 503 KB

Data Engineering Batch Pipeline with scheduled API calls as Ingestion, transformation with Glue Workflows, querying with Athena and consumption set up for Quicksight

Python 100.00%
api-rest aws aws-athena aws-eventbridge aws-glue aws-lambda aws-s3 aws-sqs data-engineering data-pipeline

aws-etl-pipeline's Introduction

Hi there, Welcome to my Profile Page! I am glad you made it this far... ๐Ÿ‘‹

ovokpus

I enjoy working as a Data Engineering Consultant in the cloud, building Analytics workflows and discovering valuable insights that help solve problems for client businesses and other types of organizations.

I have a keen interest in ETL and ELT data Pipelines, Machine Learning Systems, Analytics Engineering and Data Warehousing, as well as Cloud Development Operations. I am on a career path that leads to becoming a seasoned Data and Analytics Engineer with useful Machine Learning Operations(MLOps) Engineering, and Cloud computing skills.

With an Educational Background in Engineering Technology and Applied Sciences, I have acquired a broad and rich skillset that overlaps the fields of Data and Machine Learning Engineering, Software Development, and Cloud Operations. I have worked on more than a few Engineering and Cloud projects, both individually and as part of Agile Development teams. My experience covers building data products in Retail, Energy, Telco, Banking and Financial services, and also HR Analytics.

I enjoy working with data, discovering valuable insights that help solve problems for businesses and other types of organizations.

I also love programming and am enhancing my skills in Python and and SQL, database design, data warehouse modelling, as well as Machine Learning model development, experimentation, packaging and deployment. I also have marginal exposure to JavaScript, Microsoft C#(.NET Core), and a tiny bit of Java.

I am also gaining real world experience with Big Data and Cloud computing platforms that are utilized in Machine Learning and Business Intelligence Analytics use cases. These use cases are especially present in various sectors of Industry where digital transformation is playing a huge role in determining business outcomes.



This is a sampling of the work I have been doing for the past couple of years, since I made a major career pivot into Data Science. Programming and developing solutions within the data space has become my passion and pursuit. I place a high value on personal growth and making positive contributions in a friendly team environment, and I am looking to do just that to help organizations build and develop their data strategy.


Some things you should know about me ๐Ÿ‘‡

  • ๐Ÿ‘จโ€๐Ÿ’ป I'm currently a Senior Data Engineer at Badal.io, the foremost Canadian GCP consulting company.
  • ๐Ÿ‘จโ€๐Ÿ’ป I used to be a Data Scientist and eventually, a Data Engineer at Totogi (A TelcoDR company).
  • ๐Ÿ‘จโ€๐Ÿ”ฌ On the side (after hours, casually) I help out as a Data Science Mentor with The Lighthouse Labs Data Bootcamp.
  • ๐Ÿ‘จโ€๐Ÿ”ฌ Before that, I was an Applied Machine Learning Specialist with ReVisionz Inc.
  • ๐Ÿ‘จโ€๐Ÿ”ฌ And Before that, I was a 2021 Data Science Fellow , and helped develop a Recommender System PoC model with Cybera Inc and Hockey AI(Actionable Insights).
  • โ˜ I have been studying and working on various Data Science and Machine Learning Learning programs, individual and team projects, internships and fellowships since late 2019.
  • ๐Ÿ‘จโ€๐ŸŽ“ Making this switch into Data Science has become one of the best career decisions I have made.

My Technical Knowdledge Areas and Skillsets include ๐Ÿ‘จโ€๐Ÿ’ป



  • ๐Ÿ”ญ I am now working on a very complex Data Migration Project on Google Cloud Platform, implementing data models and Data Warehousing designs using dbt and airflow with Google Cloud BigQuery, for a major Enterprise Banking Client in Canada. I am also building pipelines for Apache Hive lift-and-shift workloads with Python and HiveQL and shell scripting. This is high-end GCP consulting at its best!

  • ๐ŸŒฑ I was working on Platform Configuration, Backend Development (Flask) and Telco Data Migration projects, implementing Telecom Charging Software Systems hosted on the Public Cloud (AWS)

  • ๐ŸŒฑ Iโ€™m currently learning Cloud Computing and Data Migration on GCP, Productionizing Machine Learning models, building data pipelines, DevOps and infrastructure Engineering best practices, as it relates to Data and Machine Learning Engineering.

  • ๐ŸŒฑ Previously, I was working on applying Computer Vision (Object Detection and Optical Character Recognition) models using the YOLO Object Detector and Microsoft Azure Cognitive Services. Models were used to extract technical information from industrial design documents and blueprints.

  • ๐Ÿ’ฌ Ask me about how to pivot into a tech career

  • ๐Ÿ“ซ How to reach me: linkedin.com/in/ovokpus

  • ๐Ÿ˜„ Pronouns: He/Him


  • โšก Fun fact: I still have not yet seen "Star Wars"! Maybe someday, don't hold your breath! -

Certifications and Credentials

You can find my professional certifications in Credly and also in Accredible


Find below links to some of my projects and repositories ๐Ÿ‘‡.

My all time favorites are linked below in the Pinned Repositories. But here are others as well:

Data Engineering Projects

  1. AWS ETL Pipeline
  2. Azure Streaming Pipeline
  3. Airflow Learning Project - Astronomer
  4. Document Streaming App with fastAPI, Kafka, Spark & MongoDB
  5. Analytics Engineering Prototype with dbt and BigQuery
  6. Contact Tracing using Elasticsearch and Streamlit Frontend
  7. Time Series Analytics Pipeline with Python, InfluxDB and Grafana
  8. Data Engineering with Hadoop - A Learning Project

Machine Learning Engineering Projects

  1. Income Prediction Pipeline - MLOps
  2. Python-Azure-AI-REST-APIs
  3. Azure Machine Learning Project
  4. Azure AI Engineering Code Library
  5. My MLOps Learning Repository

Data Science & Analytics Projects

  1. Salary Prediction Prototype
  2. Car Manufacturing Test
  3. Customer Segmentation using RFM modelling and K-Means Clustering
  4. And here is my Business Intelligence Gallery

Software Projects (Frontend, Backend, FullStack)

  1. US Cities API Backend

I Hope you have a great time going through them. Feedback is highly appreciated. -

aws-etl-pipeline's People

Contributors

ovokpus avatar

Stargazers

 avatar

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.