Image recognition with Tensorflow: Model components¶

Adapted from Tensorflow Image Classification Walkthrough.

This is the first part of the Image Recognition workshop series, where we will be making a first pass at a model to differentiate flower types by images.

Machine learning and deep learning¶

Machine learning (ML) and deep learning (DL) are tools to made predictions. We train models up on data; they become good at making predictions on those data, and then we feed them new information and see how they do on the new information.

../../_images/machine-vs-deep-learning.jpg

We use ML and DL to model complex systems with complicated relationships. If your data is relatively simple, machine learning and deep learning are likely not the best choice; will take longer and may do worse than more simple models.

Building a model¶

Now that we gone through our preprocessing workflow, we are going to construct a basic Keras model, which contains several layers. Layers take information, process them in some way, and then pass the output on to different layers.

We are going to build a sequential model, which puts layers in a defined order, and feeds data through the layers in that order. Each layer will have a single tensor input and a single tensor output.

We are starting our basic model with the following layers:

Rescaling layer: works like above example. Our data contains 3 dimensions: x position, y position, and RGB channel
Flatten layer: removes dimensional component into a single dimension; only reformats our data
Dense layer: a layer that is fully connected to the previous layer.
- Our Dense layer has 32 neurons or nodes. Every node is receives information about all pixels
- Use relu activation. Activation functions determines how strongly each neuron “fires” -> to what degree each node gets used to make predictions
ends with another Dense layer with 5 nodes, one for each class. It will contain the odds of the images being each flower type.

Python

num_classes = len(class_names)

model = Sequential([
layers.Rescaling(1./255, input_shape=(img_height, img_width, 3)),
layers.Flatten(),
layers.Dense(32, activation='relu'),
layers.Dense(num_classes)
])

We then compile our model with model.compile(), adding in a few more important options.

Loss is how the training process determines how well it is doing. We want loss to be as close to zero as possible. There are many possible loss functions; here we use one called sparse categorical cross entropy.

Our optimizer tries to decide how to make changes to our model to decrease loss.

In the example below, the optimizer is trying to find the lowest point on the parabola. It tries to take larger steps when it’s far away from the minimum, and smaller steps when it’s near.

If it takes steps that are too large, however, the model may have a hard time finding the minimum loss due to overshooting.

../../_images/gradient-descent-learning-rate.png

Reality is more complicated than this simple case. Here we show a more complicated gradient. It contains many places for the minimization process to get stuck (local minima). Therefore, making sure our step size is large enough to get out of local minima is also important.

We also will keep track of the accuracy of our model. This is the proportion of images that the model correctly classifies. The model does not use this information; it is purely for us, the users.

Loss and accuracy metrics

We use a type of loss called sparse categorical cross entropy. However, there are many different kinds of loss we can use.

There are also different metrics we can track besides accuracy, as well. If we add them to the list, we can track multiple metrics at the same time.

We use an optimizer called “Adam” commonly used in neural networks. Other optimers can be usedm as well.

Python

model.compile(optimizer='adam',
            loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
            metrics=['accuracy'])

We can print model summary, which shows our layers and how many parameters we have for each layer.

Python

model.summary()

Output

Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #
=================================================================
 rescaling_1 (Rescaling)     (None, 180, 180, 3)       0

 flatten (Flatten)           (None, 97200)             0

 dense (Dense)               (None, 32)                3110432

 dense_1 (Dense)             (None, 5)                 165

=================================================================
Total params: 3,110,597
Trainable params: 3,110,597
Non-trainable params: 0
_________________________________________________________________

We are going to run the model for 10 epochs. An epoch is one iteration through the model pipeline where the model can adjust itself throughout. This means that we will pass our entire data set through our model 10 times. After the first epoch, future epochs will build upon the model created in prior epochs and refine it to minimize the loss.

Here, we use model.fit() to actually fit the model that we have defined. We will call the output of the model fitting history, as it will store a record of the fitting process over time.

Python

epochs=10
history = model.fit(
  train_ds,
  validation_data=val_ds,
  epochs=epochs
)

Output

Epoch 1/10
92/92 [==============================] - 1s 7ms/step - loss: 3.8806 - accuracy: 0.2016 - val_loss: 1.6087 - val_accuracy: 0.2398
Epoch 2/10
92/92 [==============================] - 0s 4ms/step - loss: 1.6073 - accuracy: 0.2459 - val_loss: 1.6065 - val_accuracy: 0.2398
Epoch 3/10
92/92 [==============================] - 0s 4ms/step - loss: 1.6051 - accuracy: 0.2459 - val_loss: 1.6048 - val_accuracy: 0.2398
Epoch 4/10
92/92 [==============================] - 0s 4ms/step - loss: 1.6034 - accuracy: 0.2459 - val_loss: 1.6036 - val_accuracy: 0.2398
Epoch 5/10
92/92 [==============================] - 0s 4ms/step - loss: 1.6022 - accuracy: 0.2459 - val_loss: 1.6028 - val_accuracy: 0.2398
Epoch 6/10
92/92 [==============================] - 0s 4ms/step - loss: 1.6014 - accuracy: 0.2459 - val_loss: 1.6023 - val_accuracy: 0.2398
Epoch 7/10
92/92 [==============================] - 0s 4ms/step - loss: 1.6009 - accuracy: 0.2459 - val_loss: 1.6021 - val_accuracy: 0.2398
Epoch 8/10
92/92 [==============================] - 0s 4ms/step - loss: 1.6005 - accuracy: 0.2459 - val_loss: 1.6019 - val_accuracy: 0.2398
Epoch 9/10
92/92 [==============================] - 0s 4ms/step - loss: 1.6003 - accuracy: 0.2459 - val_loss: 1.6019 - val_accuracy: 0.2398
Epoch 10/10
92/92 [==============================] - 0s 4ms/step - loss: 1.6002 - accuracy: 0.2459 - val_loss: 1.6017 - val_accuracy: 0.2398

We can visualize the results of our model in matplotlib, looking both at the training and validation sets. For each we look at accuracy, as well as loss.

Here is an example of how we want our plot to look:

Here, the validation accuracy slowly increases to be around 75%. It is a little smaller than the training accuracy, because we are always more accurate on the data that the model has already seen than on new data.

When we look at training and validation loss, the absolute values are less important. However, we want to see loss decrease as we train the model. Smaller loss is better. We will see validation loss be larger than training loss, similar to how validation accuracy is always smaller than training accuracy.

Model history

We saved the record of the fitting process and the resulting model in a variable called history. This variable has an attribute .history, which is a dictionary containing information about our fitting.

For instance, history.history['accuracy'] contains the training accuracy across our epochs, while history.history['val_accuracy'] contains the validation accuracy. Likewise, history.history['loss'] is the training loss, and history.history['val_loss'] is the validation loss.

We then plot each one in their own subplots.

Python

acc = history.history['accuracy']
val_acc = history.history['val_accuracy']

loss = history.history['loss']
val_loss = history.history['val_loss']

epochs_range = range(epochs)

plt.figure(figsize=(8, 8))
plt.subplot(1, 2, 1)
plt.plot(epochs_range, acc, label='Training Accuracy')
plt.plot(epochs_range, val_acc, label='Validation Accuracy')
plt.ylim(0, 1)
plt.legend(loc='lower right')
plt.title('Training and Validation Accuracy')

plt.subplot(1, 2, 2)
plt.plot(epochs_range, loss, label='Training Loss')
plt.plot(epochs_range, val_loss, label='Validation Loss')
plt.legend(loc='upper right')
plt.title('Training and Validation Loss')
plt.show()

Output

Our model isn’t performing particularly well. Next time, we will go over ways to fix it to be more accurate.

Homework: TensorFlow Playground¶

Using TensorFlow Playground, create the best model possible for the spiral data set.

We will be judging models based on their test loss and the number of epochs it takes to get that loss.

You should experiment with using different features, different numbers of nodes and layers, and other settings to create the best model.

Image recognition with Tensorflow: Model components¶

Machine learning and deep learning¶

Setup¶

Load data as a keras dataset¶

Image batching¶

Image data and normalization¶

Building a model¶

Homework: TensorFlow Playground¶