tanlull / datamining

data mining project for everyone

DataMining

data mining project for everyone

Index : Data Preparation using Python

1. Import libraries
2. Import data

• Read from csv
• Connect to a database

3. Metadata and data types

• Metadata
• Modify data types

4. Duplicate detection

• Find duplicates
• Remove duplicates

5. Dataframe manipulation

• Columns and rows
• Aggregation, filtering and sorting
• Combine dataframes

6. Outlier detection

• Tukey’s test for extreme values
• Kernel density estimation

7. Variable generation and manipulation

• Generate new variables
• Bucket variables
• Encode categorical variables
• Generate dummy variables

8. Prepararation of data for modeling

• Draw samples and split dataset
• Reshape data for modeling