Hyperparameter Optimization in Deep Learning

Abstract

Deep learning is an artificial neural network-based machine learning technique that has been successfully employed in many applications, including image analysis, speech recognition, and text comprehension. It consists of multi-level layers, many capable of concurrently running processes with high-level features defined and automatically extracted from low-level ones. A major challenge in building deep learning models is determining the best setting for their hyperparameters, i.e., finding the combination of adjustment values that maximizes the model's performance during the training phase. In this project, we aim to investigate hyperparameter optimization methods in deep learning models, especially in fine-tuning pre-trained convolutional neural networks, which will be used to classify small and large microorganism image sets. The efficiency of the researched hyperparameter optimization methods will be evaluated in terms of accuracy, running time, and number of iterations considering custom hardware.