mikeferko Goto Github PK

1. Generate a data set with the simple Saddle Point or the Ackley Function
• Saddle Point:

• Ackley:

2. Add uniform random noise and visualize the 3D meshgrid
3. Reshape the generated data to be a tensor input vector (shape will be: sample rows by feature columns)
4. Regression MLP Model Parameters:
• Stochastic Gradient Descent optimizer
• Neural Network Architecture:
• Input Layer = 10 neurons with a sigmoid activation function
• Output Layer = 1 neuron
• Learing Rate = 0.1
• Exponential Decay Factor = 0
• Momentum = 0.1
• Train Duration: 50 Epochs
• Batch Size = 10
• Mean Square Error Loss Function
5. Create a predicted Saddle point and Ackley Function from the Regression MLP trained Neural Network
6. Plot the Results

Multiple Linear Perceptron (MLP) Model

Results of Saddle Function Predictions

Results are shown in the above image Left-to-Right, Top-to-Bottom

1. Real vs. Predicted Saddle
2. z-x cross section @ y = 2
3. z-x cross section @ y = 0
4. Model Loss Vs. Epochs
5. Topological Heat Map

Results of Ackley Function Predictions

Results are shown in the above image Left-to-Right, Top-to-Bottom

1. Real vs. Predicted Ackley
2. z-x cross section @ y = 2
3. z-x cross section @ y = 0
4. Model Loss Vs. Epochs
5. Topological Heat Map

• Google Coolab Notebook: Jupyter Notebook
• Github Repository: Respository
• Paper: Real Estate Evaluation of housing prices in Taipei Taiwan
• We are using the same sequential MLP model used for the Saddle Point and Ackley Function preditctions.

Multiple Linear Perceptron (MLP) Model

We will be examining real estate valuation which will help us understand where people tend to live in a city. The higher the price, the greater the demand to live in the property. Predicting real estate valuation can help urban design and urban policies, as it could help identify what factors have the most impact on property prices. Our aim is to predict real estate value, based on several features.

• Regression MLP Machine Learning on Taipei Taiwan Algorithm:
1. Load the Real estate valuation data set
2. Independent feature vector containing:
2. X2 house age
3. X3 distance to the nearest MRT station
4. X4 number of convenience stores
5. X5 latitude
6. X6 longitude
3. Train/Test split the data at a ratio of 80:20, respectively
4. Min/Max Scale the dataset with a range of 0 to 1
5. Normalise the scaled features
6. Regression MLP Model Parameters:
• Neural Network Architecture:
• Input Layer = 10 neurons with a sigmoid activation function
• Output Layer = 1 neuron
• Stochastic Gradient Descent optimizer
• Learing Rate = 0.1
• Exponential Decay Factor = 0
• Momentum = 0.1
• Mean Square Error Loss Function
• Train Duration: 50 Epochs
• Batch Size = 10
7. Create a predicted House Price Prediction of the unit area from the Regression MLP trained Neural Network
8. Plot the Results

Regression MLP Model Loss

Regression MLP Predictions in New Taiwan Dollars (NT$)

• Google Coolab Notebook: Jupyter Notebook
• Github Repository: Respository
• Paper: Classification of MNIST 70,000 Handwritten Digits 0-9 Image Data Set
• Categorical Cross Entropy Algorithm:
1. Load the Modified National Institute of Standards and Technology (MNIST) Handwritten digits 0-9 data set
2. Train/Test split the data at a ratio of 6:1, respectively
3. Reshape the images from 28x28 pixels to 784x1 pixels
4. Normalise the image pixels by dividing by the gray scale image intensity level set L:



5. Create 10 Categories for the 10 digits 0-9 to be classified





Creating classes 10 classes for the 10 digits 0-9 of handwritten digits





6. Categorical Cross Entropy (CE) Model Parameters:
• categorical cross entropy (CE) Loss Function:


Where: The formula can be seen as above, where ti refers to the i -th element of the target vector and si refers to the i -th element of the models output vector, and C the number of classes.



Visualization of Log Loss (Cross Entropy)



Cross Entropy between probability distributions for each Class


• Model Accuracy:


Where: M is the number of samples in the dataset, tk is the target vector for the k-th sample, and sk is the models output vector for the k-th sample.


• Neural Network Architecture:
• Input Layer = 16 hyperbolic tangent activation (tanh) neurons with an input shape of 784x1
• Hidden Layer = 16 hyperbolic tangent activation (tanh) neurons with an input shape of 16x1
• Output Layer = 10 softmax neurons


Classification Neural Network Architecture


• Stochastic Gradient Descent optimizer
• Learing Rate = 0.4
• Exponential Decay Factor = 0
• Momentum = 0.5
• Train Duration: 10 Epochs
• Batch Size = 128
• training samples = 60,000
• testing samples = 10,000

Visualization of Model Loss and Accuracy (0.1532 and 95.49% Respectively)

Visualization of First Layer Weights W1 from Neural Network Architecture

Visualization of Second Layer Weights W2 from Neural Network Architecture

Visualization of Third Layer Weights W3 from Neural Network Architecture

• Google Coolab Notebook: Jupyter Notebook
• Github Repository: Respository
• Paper: Classification of Fashion MNIST Image Data Set
• Categorical Cross Entropy Algorithm:
1. Load the Modified National Institute of Standards and Technology (MNIST) Fashion data set
2. Train/Test split the data at a ratio of 6:1, respectively
3. Reshape the images from 28x28 pixels to 784x1 pixels
4. Normalise the image pixels by dividing by the gray scale image intensity level set L:



5. Create 10 Categories for class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']





Creating 10 classes for the 10 types of clothing in the Image Data Set





6. Categorical Cross Entropy (CE) Model Parameters:
• categorical cross entropy (CE) Loss Function:


Where: The formula can be seen as above, where ti refers to the i -th element of the target vector and si refers to the i -th element of the models output vector, and C the number of classes.



Visualization of Log Loss (Cross Entropy)



Cross Entropy between probability distributions for each Class


• Model Accuracy:


Where: M is the number of samples in the dataset, tk is the target vector for the k-th sample, and sk is the models output vector for the k-th sample.


• Neural Network Architecture:
• Input Layer = 64 ReLu activation neurons with an input shape of 784x1
• Hidden Layer = 64 ReLu activation neurons with an input shape of 64x1
• Output Layer = 10 softmax neurons


Classification Neural Network Architecture


• Stochastic Gradient Descent optimizer
• Learing Rate = 0.1
• Exponential Decay Factor = 0
• Momentum = 0
• Train Duration: 10 Epochs
• Batch Size = 128
• training samples = 60,000
• testing samples = 10,000

Visualization of Model Loss and Accuracy (0.3090 and 88.66% Respectively)

Visualization of First Layer Weights W1 from Neural Network Architecture

Visualization of Second Layer Weights W2 from Neural Network Architecture

Visualization of Third Layer Weights W3 from Neural Network Architecture