TensorFlow Tutorial Using Fashion MNIST (A Beginners Guide)¶
Figure 1 Fashion-MNIST samples (by Zalando, MIT License).
Introduction¶
Machine learning (ML)/Deep Learning(DL)/Neural Network (NN) tools have recently made a huge splash with applications in data analysis, image classification, and data generation. Although ML methods have existed for decades, recent advancements in hardware have generated systems powerful enough to run these algorithms.
The typical "hello world" example for ML is a classifier trained over the MNIST(Modified National Institute of Standards and Technology database) dataset; a dataset of the handwritten digits 0-9. This dataset is getting a little stale and is no longer impressive due to both its seeming simplicity and to the plethora of existing tutorials on the topic. Here we will use a newer dataset to perform our ML "hello world", the Fashion MNIST dataset!
The Zeroth Step of ML (that should be completed before ever putting a hand to mouse, or finger to key) is understanding the format and sizes of your data. This step is often referred to as feature engineering. Feature engineering, typically, includes selecting and preprocessing the particular aspects of training data to give to your algorithm. You and I will start a good habit of examining the data and its format to make decisions concerning the appropriate size and format for our NN.
The Fashion MNIST dataset is comprised of 70,000 grayscale images of articles of clothing. The greyscale values for a pixel range from 0-255 (black to white). Each low-resolution image is 28x28 pixels and is of exactly one clothing item. Alongside each image is a label that places the article within a category; these categories are shown in Figure 2 with an example image belonging to the class.
Figure 2 class numbers are shown next to image labels
TL;DR¶
Open this tutorial on Google colab:
And, walk through the tutorial yourself.
Also check out DataCamp if you want to learn Data Science and Machine Learning Online for FREE like I did.
Code walk-through¶
To complete this tutorial we will to install a few packages; pip is a fast and easy way to install packages and their dependencies. For Machine Learning projects we use a library called TensorFlow, it handle all the complicated maths for you so you do not need to worry much on the maths side of things. For visualization we using matplotlib & nnv and for numerical-computations we using numpy.
!pip install tensorflow
!pip install matplotlib
!pip install numpy
!pip install nnv
import tensorflow as tf
from tensorflow import keras
import matplotlib.pyplot as plt
import numpy as np
Import MNIST¶
Now we are ready to roll! First, we must admit that it takes a lot of data to train a NN, and 70,000 examples is an anemic dataset. So instead of doing a more traditional 70/20/10 or 80/10/10 percent split between training/validating/testing, we will do a simple 6:1 ratio of training:testing (note that this is not best practices, but when there is limited data it may be your only recourse).
We first load in the dataset from the Keras package:
fashion_mnist = keras.datasets.fashion_mnist
(train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()
The first line merely assigns the name fashion_mnist to a particular dataset located within Keras' dataset library. The second line defines four arrays containing the training and testing data, cleaved again into separate structures for images and labels, and then loads all of that data into our standup of Python. The training data arrays will be used to --you guessed it-- train the model, and the testing arrays will allow us to evaluate the performance of our model.
Look at data¶
It's always nice to be able to show that we've actually done something; ever since kindergarten there has been no better way than with a picture! You'll note that we pip installed and imported MatPlotLib, a library for plots and graphs. Here we'll use it to visualize an example of the Fashion MNIST dataset.
print(f"Train images dimensions: {train_images.shape}")
print(f"Test images dimensions: {test_images.shape}")
The first command basically generates a figure object that will be manipulated by commands 2 through 4. Command 2 specifies what it is that we shall be plotting: the first element from the train_images array. NOTE: Recall that python is an inclusive counting language, meaning that it numbers/indexes things starting from zero, not one! And the final command, "show()", tells Python to generate this figure in an external (from CMD) window.
Your window should contain a plot that looks similar to Figure 3. Also, be aware that after plt.show(), Python will not return you to a command line until the newly generated window (containing our super nice picture) is closed. Upon closing the window, you will be able to continue entering Python commands.
plt.figure(figsize=(10,5))
plt.imshow(train_images[0])
plt.colormaps()
plt.show()
Preprocessing the dataset¶
The greyscale assigned to each pixel within an image has a value range of 0-255. We will want to flatten (smoosh… scale…) this range to 0-1. To achieve this flattening, we will exploit the data structure that our images are stored in, arrays. You see, each image is stored as a 2-dimensional array where each numerical value in the array is the greyscale code of particular pixel. Conveniently, if we divide an entire array by a scalar we generate a new array whose elements are the original elements divided by the scalar.
train_images = train_images /255.
test_images = test_images / 255
Model Generation¶
Every Neural Network is constructed from a series of connected layers that are full of connection nodes. Simple mathematical operations are undertaken at each node in each layer, yet through the volume of connections and operations, these ML models can perform impressive and complex tasks.
To read and understand more about the underlying maths, checkout some of the notes I made when I took the deeplearning.ai course here.
Our model will be constructed from 3 layers. The first layer – often referred to as the Input Layer – will intake an image and format the data structure in a method acceptable for the subsequent layers. In our case, this first layer will be a Flatten layer that intakes a multi-dimensional array and produces an array of a single dimension, this places all the pixel data on an equal depth during input. Both of the next layers will be simple fully connected layers, referred to as Dense layers, with 128 and 10 nodes respectively. These fully connected layers are the simplest layer in the sense of understanding, yet allow for the greatest number of layer-to-layer connections and relationships.
The final bit of hyper-technical knowledge you'll need to learn is that each layer can have its own particular mathematical operation. These activation functions determine the form and relationship between the information provided by the layer. The first dense layer will feature a Rectified Linear Unit (ReLU) Activation Function that outputs values between zero and 1; mathematically, the activation function behaves like f(x)=max(0,x). The final layer uses the softmax activation function. This function also produces values in the 0-1 range, BUT generates these values such that the sum of the outputs will be 1!
This makes the softmax a layer that is excellent at outputting probabilities.
Read more about hyperparameters optimisation (especially about Softmax Regression) on some of my notes.
I found this lecture video very insightful into why we need to flatten the array.
model = keras.Sequential([
keras.layers.Flatten(input_shape=(28, 28)),
keras.layers.Dense(128, activation=tf.nn.relu),
keras.layers.Dense(10, activation=tf.nn.softmax)
])
Running the code below creates a file model_plot.png with a plot of the created model.
keras.utils.plot_model(model, to_file='model.png', show_shapes=True)
Let's visualise how the neural network feed fordward would look:
from nnv import NNV
plt.rcParams["figure.figsize"] = 200,50
layersList = [
{"title":"Input\n(784 flatten)", "units": 784, "color": "Blue"},
{"title":"Hidden 1\n(relu: 128)", "units": 128},
{"title":"Output\n(softmax: 10)", "units": 10,"color": "Green"},
]
NNV(layersList, spacing_layer=10, max_num_nodes_visible=20, node_radius=1, font_size=24).render()
Training the Model¶
Models must be both compiled and trained prior to use. When compiling we must define a few more parameters that control how models are updated during training (optimizer), how the model's accuracy is measured during training (loss function), and what is to be measured to determine the model's accuracy (metrics). These values were selected for this project, yet are generally dependent on the model's intent and expected input and output.
model.compile(
optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy']
)
Now we can begin training our model! Now, with already having generated and compiled the model, the code required to train the model is a single line.
model.fit(train_images, train_labels, epochs=5)
This single line completes the entire job of training our model, but let's take a brief look at the arguments provided to the model.fit command.
- The first argument is input data, and recall that our input Flatten layer takes a (28,28) array, conforming to the dimensionality of our images.
- Next we train the system by providing the correct classification for all the training examples.
- The final argument is the number of epochs undertaken during training; each epoch is a training cycle over all the training data. Our setting the epoch value to 5 means that the model will be trained overall 60,000 training examples 5 times. After each epoch, we get both the value of the loss function and the model's accuracy (88.97% after epoch 5) at this epoch.
Evaluating Our Model¶
Now we are working with a functional and trained Neural Network model. Following our logic from the top, we have built a Neural Network that intakes a (28,28) array, flattens the data into a (784) array, compiled and trained 2 dense layers, and the softmax activation function of the final output layer will provide a probability that the image belongs to each of the 10 label categories.
Our model can be evaluated by using the model.evaluate command, that takes in the images and labels so that it can compare its predictions to the ground truth provided by the labels. model.evaluate provides two outputs, the value of the loss function over the testing examples, and the accuracy of the model over this testing population. The important output for us is the model's accuracy.
test_loss, test_acc = model.evaluate(test_images, test_labels)
print(f"Model Accuracy: {test_acc * 100}%")
This is great! Our model performs at an accuracy of 87.80%. As good as that is, it is lower than the model accuracy promised above (89.01%). This lower performance is due to the model overfitting on the training data. Overfitting occurs when there are too many parameters within the model when compared to the number of training instances; this allows the model to over learn on those limited examples. Overfitting leads to better model performance over non-training data.
That said, 87.80% is a decent number! Let's finally learn how you can feed our model the series of test examples from the test_images array, and have it provide its predictions.
predictions = model.predict(test_images)
predictions[1]
As we can see, most of the entries in our prediction array are very close to 0. They are written in scientific notation--the value after the e being the number decimal places to adjust the value (for example 5.1 e-04 is actually 0.00051). The entry that stands out is predictions[1][9] at .9951, or 99.51%, certainty that this image should be classified as a shirt!
If you prefer to not look through a list to determine the class label, we can simplify the output by:
np.argmax(predictions[1])
Finally, we can verify this prediction by looking at the label ourselves:
test_labels[1]
plt.figure(figsize=(10,5))
plt.imshow(test_images[1])
plt.colormaps()
plt.show()
There you have it! You have built and trained your first neural network from scratch, and properly classified a shirt as a shirt!
Learn more about Machine Learning/Deep Learning and Data Science at DataCamp for FREE.