Facial Expression Recognition using CNN

Overview

This program aims to develop a Convolutional Neural Network (CNN) model for face expression recognition in grayscale photos which are 48x48 using the Keras library. The grayscale photos are imported from the training and validation folders locally present on the computer, the data are normalized and reshaped, and then the data are encoded.

The Keras ImageDataGenerator class uses the optimized data. There are three convolutional layers,also there are three max-pooling layers, and three dropout layers which basically builds the whole CNN model here. The network is then provided with 2 dense layers, the second one among those of which contains seven output units that responds to the seven facial expression categories.

I have used the Adam method, which is used to train the model at a learning rate of 0.001.

Models implemented

The model used was convolutional brain organizations, a type of brain organization (CNN).

The epoch size of this model is different from the original reference code model, and the kfold method is not used. The batch size is chosen, and each input image’s goal size is set to (48, 48).

Among its numerous layers are convolutional, pooling, and fully linked layers. Following a sequence of Conv2D layers with varied filter sizes and activation functions, the MaxPooling2D levels with a predetermined pool size and a dropout layer are added.

The output of the layer above is transformed into a 1-dimensional vector using a flatten layer, that is subsequently processed by a dense layer with a specific number of units and an activation function prior to actually passing through one subsequent dropout layer.

Data loading and putting labels

The first 10 layers of this design were made untrainable for transfer learning purposes using the Keras Sequential API. Using the ImageDataGenerator class from the Keras API, data augmentation was also provided to the training data to improve the model’s capacity to generalize to new data. The model was trained using the categorical cross-entropy loss function and 0.001 learning rate of the Adam method. A 64-person batch size resulted in 25 epochs of instruction being finished. In order to classify images into one of seven categories, this CNN model uses three Conv2D layers, three MaxPooling 2D layers, and two Dense layers.

To prepare the images, we divided the data into training and validation groups, normalized the pixel values, and reshaped the arrays to have a fourth dimension. (for the number of channels).

We standardized the pixel values, split the data into training and validation groups, then completely remodeled the arrays to add a fourth dimension in order to produce the images. (with respect to how many channels there are).

Then, data improvement techniques were used using the Keras ImageDataGenerator. We used 20 degrees of rotation, 0.1 degrees of height and width shifts, 0.1 degrees of shear, 0.1 degrees of zoom, and 0.1 degrees of horizontal flip.

Image Augumentation

We then created a K-fold cross-validation function to evaluate our model’s performance. We used a K value of 5 and trained the model for 25 epochs with a batch size of 64.

My Contribution

I developed the load data function, standardized and rearranged the data, and encoded the label array before importing and preparing the image data for training and validation. To offer additional training data and avoid overfitting, I also used ImageDataGenerator to augment the data.

Additionally, I built a Convolutional Neural Network (CNN) model with LeakyReLU activation, dropout regularization, and the Adam algorithm using Sequential from Keras. I added additional density and convolutional layers to the model to increase its complexity and accuracy, and I froze the first few levels to prevent overfitting.

I also added different learning rates for optimization, increased the batch size for faster training, and increased the number of epochs for additional training to the original code. Both model accuracy and training efficiency are enhanced by these adjustments. In the original code, I also included different learning rates for optimization, a larger batch size for faster training, and a larger number of epochs for further training. These modifications improve model accuracy and training effectiveness. Overall, my efforts have resulted in increased performance and effectiveness of the model.

Testing

I have trained a model and validated it with an improved training set. In 25 training epochs, the accuracy of our validation set was around 64%. A practice model has been kept to be used in the future.

Final results

Training Accuracy Vs Validation Accuracy

Training Loss Vs Validation Loss

In conclusion, validation accuracy gained was 62.55% , with training accuracy at 56.39%. Training loss at 1.088 and Validation loss at 0.952.

Technical Challenges

The management of image data is difficult, as it requires a lot of memory and processing power. The code imports the photos using OpenCV, a powerful tool but one that can be slow when handling with large datasets.

Also, a lot of RAM is consumed when NumPy is used to create image data. Another obstacle is the implementation of coherent neural networks. (CNN). The computational cost of running CNNs can be high, particularly when training with huge datasets.

To address this issue, the code employs K-fold cross-validation and image data augmentation, which improves model accuracy and minimizes overfitting.

K-fold cross-validation and picture data augmentation are utilized in the code to solve this issue, improving model accuracy and lower overfitting.

Submission Details

GitHub link: https://github.com/Abhismoothie/Data-Mining-Asgnmnt-1/blob/main/nsjbc%20%20(2).ipynb

References

1. Facial Emotion Expressions by Samaneh Eslamifar https://www.kaggle.com/datasets/samaneheslamifar/facial-emotion-expressions
2. OpenCV Library (2021). OpenCV-Python Tutorials. Available at: https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_tutorials.html [Accessed 26 March 2023].
3. TensorFlow (2021). TensorFlow API Documentation. Available at: https://www.tensorflow.org/api_docs [Accessed 26 March 2023].
4. Keras (2021). Keras API Documentation. Available at: https://keras.io/api/ [Accessed 26 March 2023].
5. Brownlee, J. (2019). How to Develop a Convolutional Neural Network to Classify Photos of Dogs and Cats. Machine Learning Mastery. Available at: https://machinelearningmastery.com/how-to-develop-a-convolutional-neural-network-to-classify-photos-of-dogs-and-cats/ [Accessed 26 March 2023].
6. Chollet, F. (2018). Deep Learning with Python. Manning Publications.
7. Flower Image Classifier, Aunsh Arekar
   https://aunsharekar.wixsite.com/my-site/post/flower-image-classifier
8. https://matplotlib.org/stable/contents.html