moinul7002 / facial-expression-analysis

Facial-Expression-Analysis

Task 1. Face preprocessing

There are three subtasks you need to do:

Task 1.1. Extract facial landmarks

1. Load the example image example_img.jpg, using the plt.imread() function for example.
2. As a sanity check visualize the image to see what you are working with.
3. Initialize a face detector and a face landmarks detector. We have provide the code of this part, please learn to use them.
4. Detect the face
5. Detect the face landmarks
6. Transfrom the detected result to a 2-D numpy array by using the provided shape2points function from the face_lib.py.
7. Visualize the landmarks with the image

Task 1.2. Perform a face registration task using a set of fixed landmarks from a standard model and extract face from the registered image.

1. Load the landmark position of a standard face model. We provide these positions in a csv file, and also the code block to read these positions.
2. Calculate the transformation between your detected landmarks position and the standard face model landmark positions using the skimage.transform.PolynomialTransform() class and its estimate() methods.
   1. Instantiate a PolynomialTransform object by calling transform.PolynomialTransform()
   2. Call its estimate() method to calculate the transformation between the two sets of points. The manual of this method can be found in the same page which introduced of this class.
3. Transform the example image using the calculated transformation to register (map) the example image into a space of the standard face model. You can use the skimage.transform.warp() function to perform this.
4. Crop the face from the registered face using the standard face model landmarks. The cropping function is provided in the face_lib.py, you can directly use it after importing.
5. Also extract the face from the example image using your detected landmarks.

Task 1.3. Visualize the result using subplots.

Draw a figure with 3 x 2 subplots using matplotlib.pyplot.subplots(). Read the manual of it and also the matplotlib.pyplot. Each of the subplots should have the following images:

• subplot [0, 0]: the original example image and detected landmarks.
• subplot [1, 0]: the face cropped from the example image.
• subplot [2, 0]: the histogram of the face cropped from the example.

As an example, the three subplots are given above. Then you need to implement:

• subplot [0, 1]: the registered face image.
• subplot [1, 1]: the face cropped from the registered face image.
• subplot [2, 1]: the histogram of the face cropped from the registered face image.

Task 2. Feature Classification

For feature classification the SVM (Support Vector Machine) will be utilized. See the documentation for sklearn.svm.SVC(). Mainly you will use its two methods: fit() to training the classifier and predict() to use the classifer for classification. There are following three subtasks you need to complete:

Task 2.1. Load data

Firstly, you need to read .mat files using python. You can use the scipy.io.loadmat() function to read .mat file. In the provided Task3_data.mat file, different data are packed by different dictionaries which are list below:

• training_data
• testing_data
• training_class
• testing_class

Task 2.2. Train Classifiers

Use the sklearn.svm library to train Support Vector Machine (SVM) classifiers. The sample_train and sample_test matrices contain the calculated LBP-TOP features for the training and testing sets, respectively. The block size for LBP-TOP used for training and testing data are 2x2x1. The label_test group vector contains the class of samples: 1 = happy, 2 = sadness, corresponding to the rows of the training data matrices.

1. Construct an SVM classifier object using a linear kernel. See sklearn.svm.SVC().
2. Use the fit() method and the sample_train and label_train to train your classifier.

Task 2.3. Evaluate Classifiers

1. Use your trained classifer to classify the sample_train and sample_test, using the predict() method.
2. Calculate the classification accuracies when classifying the sample_train and sample_test, respectively. The correct class labels corresponding with the rows of the training and testing data matrices are in the variables label_train and label_test, respectively.
3. Calculate the confusion matrices when evaluating both the dataset by using sklearn.metrics.confusion_matrix().