5 Tips Before Starting Your First Deep Learning Image Classification Project with Keras
Deep learning applications in image classification have shown very impressive results in recent years, with advances rapidly happening all the time. They are being applied in many fields, such as: security and facial recognition, diagnostic support in medical imaging, and the development of driverless car technologies. This makes deep learning an increasingly valuable skill set for data scientists to be familiar with. If you have worked extensively with data, and especially if you have built neural networks before and are familiar with keras, but have not attempted a thorough image classification project before, this article is for you!
Much of how an image classification project is conducted is similar to how we might model any set of features to classify a target. However, if you have not worked extensively with imaging data, here are a few tips to keep in mind before actually modeling on your data that can help your project move along more smoothly.
Manage your expectations on the data
Fine tuning the perfect combination of layers and hyperparameters in a convolutional neural network can be a challenge. It is helpful to keep in mind what kind of challenges you will expect to see in your data before getting started. For example, what are the classes you are trying to identify? Are you easily able to identify the classes on your own? Building a model that identifies the difference between classes you can easily differentiate, such as dogs and cats, is likely to be less challenging than a model that has to identify very subtle changes, such as a positive finding in an x ray image, which the untrained eye may not be qualified to identify. This little bit of intuition can help set you up with an expectation of how deep your model may have to be, how long it may take to train, and what sort of benchmarks to expect in early model attempts.
Collect a large amount of data
If your goal is to train a model from scratch that can be put into production to classify similar images in the future, be prepared to collect a large library of images. With image classification, 5,000 images — or even 50,000 images — is often not an overly large dataset to train a model on, particularly in more difficult to identify classes. Truly comprehensive trained models are often trained on hundreds of thousands, or even over a million, images, in order to find weights that lead to highly accurate and generalizable results. This can be daunting on many levels. Image acquisition will obviously be a difficult task, but such a large number of files will also lead to challenges in finding sufficient storage, processing power, and training times. It is not unheard of for such deep learning models to take days or even weeks to properly train.
Having said all of this, if you find that this is too challenging to be realistically accomplished with the resources and time you have alloted for a project, don’t stop reading and give up yet! There are a few more tips that can make a good project possible, even with a relatively small amount of data.
Consider applying Transfer Learning in your model
If collecting massive numbers of images if beyond the scope of your realistic expectations, transfer learning can still yield good results. Keras applications are a series of pre trained models that can be imported directly into your workspace to implement a base model. The best part is that you can import model weights that were trained on ImageNet, which has pre trained these models on millions of random images. You can include the dense layers, or only the convolutional layers, adding your own dense layers at the end for yourself (for example, if you are trying to predict a different number of classes). By freezing the weights of the imported model, you can ensure that your project will use strong feature extraction, even if you only have access to a small amount of data. This is important in transfer learning, as you do not actually want to update the weights that ImageNet has already created for you, but use them for the purpose of feature extraction to benefit your current model.
Utilize Keras’ Image Data Preprocessing
Keras’ Image Data Preprocessing is a very helpful tool that any data scientist planning to use Keras for image classification should be familiar with. Creating a master array of all images in your project script will likely be impossible to create, given the size and memory constraints of such a task (or at least be prohibitively long in runtime to create), and this tool helps to get around this issue.
By using the Imagedatagenerator from this module, you can create a generator object that will let you read in image objects batch by batch, resize them (if needed) appropriately for your neural network inputs, and separate them by classes (or apply classes from a dataframe column). These generators can then pass your data into your convolutional neural network batch by batch, allowing you to run models on massive amounts of data even if you do not have a large RAM allowance to hold the entirety of the data at once.
More than this, this keras generator can do data augmentation on the fly. One of the best methods of regularization (the prevention of overfitting) with image classification models involves altering your images before passing them into the model — adjusting their brightness, flipping them horizontally or vertically, rotating them, zooming in and out, etc. By doing this, your model is less likely to overfit on the same images as they pass through the model in subsequent epochs. Normally, you would have to create new files that alter your original image files in order to accomplish this, but your keras generator will randomly do this for you without the need to create new files.
Utilize model checkpoints and saved model format
Even if you decide to employ transfer learning in your project, the reality is that achieving optimal accuracy could still require many hours, or beyond, of training time. If your computer crashes, or your remote kernel is disconnected, at any point during this process, your entire model could be lost, requiring you to start over.
Keras’ callback module (the same module that allows you to implement early stopping) has a modelcheckpoint import that allows you to save the weights of your model after every epoch, or, if preferred, after a set number of batches have passed per epoch. It also contains an argument that allows you to only save the weights if there is an improvement in accuracy, or a decrease in the loss metric. Should the worst happen and your computer crash 9/10 of the way into training, all is not lost!
Most models, such as those built in sklearn, can be “pickled,” saving the model as a file that can be imported into other scripts, or used in flask, streamline, or other sources for web deployment. This can technically also be used in keras, but best practice for keras models is using keras’ saved model format. This allows you to save your trained model as a file once you have it optimized to your liking and are ready to deploy.
Understanding these tools before starting a project can help save a lot of hardships, headaches, and struggles as you get started with your first image classification project, and help you get past error messages and into modeling sooner, and with hopefully greater success. Happy fitting!
