nellyexey / python_eda

Exploratory data analysis

Don't forget to hit the ⭐ if you like this repo.

About Us

The information on this Github is part of the materials for the subject High Performance Data Processing (SECP3133). This folder contains general Exploratory Data Analysis (EDA) information as well as EDA case studies using Malaysian datasets. This case study was created by a Bachelor of Computer Science (Data Engineering), Universiti Teknologi Malaysia student. In addition, my research group also contributed materials and case studies. Thank you to the collaborators who shared their knowledge in this github.

• Li Jing
• Eman Al Jabarti

Exploratory Data Analysis

✅️ EDA involves using graphics and visualizations to explore and analyze a data set. The goal is to explore, investigate and learn, as opposed to confirming statistical hypotheses.

✅️ EDA is used by data scientists to analyze and explore datasets and summarize the main characteristics of them.

✅️ EDA makes it easier for data scientists to discover patterns, spot anomalies, test a hypothesis, or check assumptions.

✅️ EDA is primarily used to provide a better understanding of dataset's variables and their relationships.

✅️ EDA can also help determine whether the statistical techniques you are considering are appropriate for data analysis.

✅️ Developed by the American mathematician John Tukey in the 1970s, EDA techniques continue to be a widely used method in the data exploration process today.

Why is EDA so important in data science?

✅️ The main purpose of EDA is to help you look at the data before making any assumptions. In addition to better understanding the patterns in the data or detecting unusual events, it also helps you find interesting relationships between variables.

✅️ Data scientists can use exploratory analysis to ensure that the results they produce are valid and relevant to desired business outcomes and goals.

✅️ EDA also helps stakeholders by verifying that they are asking the right questions.

✅️ EDA can help to answer questions about standard deviations, categorical variables, and confidence intervals.

✅️ After the exploratory analysis is completed and the predictions are determined, its features can be used for more complex data analysis or modeling, including machine learning.

📖 Notes

Basic Concept

• developers.google: Good Data Analysis
• Towardsdatascience: What is Exploratory Data Analysis?
• Wikipedia: Exploratory data analysis
• r4ds: Exploratory Data Analysis
• careerfoundry:What Is Exploratory Data Analysis?
• How To Conduct Exploratory Data Analysis in 6 Steps
• A Five-Step Guide for Conducting Exploratory Data Analysis

Code & Practice

• simplilearn: What is Exploratory Data Analysis? Steps and Market Analysis
• Exploratory Data Analysis (EDA): Types, Tools, Process
• projectpro: Exploratory Data Analysis in Python-Stop, Drop and Explore
• medium.com: 10 Things to do when conducting your Exploratory Data Analysis (EDA)
• towardsdatascience.com: An Extensive Step by Step Guide to Exploratory Data Analysis
• EDA - Exploratory Data Analysis: Using Python Functions
• Step-by-Step Exploratory Data Analysis (EDA) using Python

Videos

• Exploratory Data Analysis Tutorial | What Is EDA | How EDA Works | EDA In Python | Intellipaat
• Live Day 1-Live Session On EDA And Feature Engineering- Zomato Dataset
• Live Day 2-Live Session On EDA And Feature Engineering- Black Friday Dataset
• Live Day 3-Live Session On EDA And Feature Engineering- Flight Price Prediction Dataset
• Step By Step Process In EDA And Feature Engineering In Data Science Projects
• Exploratory Data Analysis(EDA) of Titanic dataset
• Exploratory Data Analysis (EDA) Using Python | Python Data Analysis | Python Training | Edureka
• Exploratory Data Analysis with Pandas Python
• How to Do Data Exploration (step-by-step tutorial on real-life dataset)

Kaggle: Notebook

• Exploratory Data Analysis (Step by Step)
• A Simple Tutorial on Exploratory Data Analysis
• Intro to Exploratory data analysis (EDA) in Python
• Topic 1. Exploratory Data Analysis with Pandas
• Detailed exploratory data analysis with python
• EDA using Python Pandas
• Pandas: EDA of Cars Dataset
• Step-by-step Data Preprocessing & EDA

Github

• PacktPublishing/Hands on Exploratory Data analysis with Python
• code4kunal/eda-python-examples
• SouRitra01/Exploratory-Data-Analysis-EDA-in-Banking-Using-Python
• sandipanpaul21/EDA-in-Python
• vharivinay/python-eda-viz
• demonpratapdemon/Exploratory-Data-Analysis-EDA-and-PreProcessing
• PacktPublishing/Python-for-Data-Analysis-step-by-step-with-projects-
• sandyy2505/Cardio Good Fitness Project
• ajaymache/Data analysis of used car database

📖 Lab

No	Dataset	Colab	GitHub
1	Boston
2	Car Features and MSRP
3	Housing Dataset
4	United Nations Development Corporation

🌟 Case Study: Exploratory Data Analysis

Team	Title	Colab	GitHub
404 Error	Property in Kuala Lumpur
Alrite	The Exportation of Plantation in Sarawak
BEFE	Covid-19 Clusters in Malaysia
Boboiboy	Property Listings in Kuala Lumpur
COLBY	Malaysia GE-14 Result
FANTOM	Daily recorded COVID-19 cases at state level In Malaysia
HAHA	Foreign Direct Investment In Malaysia
HD	Guna Tanah Tampin 2021
KIA	Malaysia State Election 2018
LAB	Malaysia Air Pollution Analysis
MAAM	Malaysia Hospital Patient Movement Analysis
MEOW	Capacity and utilisation of Intensive Care Unit (ICU) beds during COVID-19
MM	Malaysia's 14th State Election Result
PIXALATED	Number of deaths in Malaysia from 2001 to 2018
POTATO	Death by state, sex and age group Malaysia 2001-2018
QnX	Real Estate Kuala Lumpur Malaysia
SAMVERSE	Restaurant Rating in Malaysia
SMOL	Population in Malaysia from 2010-2019
SQ	Number of Cases and Incidents Rate of Communicable Disease by State
TUK	Number of Government School Pupils by District Education Office and State 2017-2018
UWU	Property Listings in Kuala Lumpur

🌟 Case Study

Name	Title	Colab	GitHub
Li Jing	ABC
Saleh Dhekre Saber Saleh	ABC
Eman Al Jabarti	ABC
Anwar Said Salim Al Talaii	ABC
Zhu Caihua	ABC
Shiekhah AL Binali	ABC
Li Haopeng	ABC

Contribution 🛠️

Please create an Issue for any improvements, suggestions or errors in the content.

You can also contact me using Linkedin for any other queries or feedback.