How to Deploy Machine Learning models? End-to-End Dog Breed Identification Project! | by Gurami Keretchashvili | Apr, 2023

By Jessie Hobb On Apr 3, 2023

Simplest way to deploy your ML model on the web.

In this article I will discuss step-by-step tutorial about the easiest and fastest way to deploy your ML project on the web using Streamlit. The project is about dog breed identification, which classifies a dog out of 120 types of breeds. I will focus more on the deployment part of the project, rather than building a complex machine learning model.

Before discussing any further, let’s see the demo of the project below:

deploy machine learning model demo — Deployment Demo (gif by the author)

You can play with the demo — HERE,

The GitHub link of the project is available-HERE

Outline of the article:

Background
Project tutorial
A. Build and Train model (Build_AND_Save_DL_model.ipynb)
B. Streamlit application (streamlit.py)
C. Deployment
Common errors and troubleshooting
Conclusion and future work

Building machine learning model in the Jupyter notebook is one thing and deploying the model is — another, that requires creating a service that another user can access over the internet. There are plenty of methods and tools to deploy an ML model, such as building REST API with lightweight web framework like Flask or Django. The API can be called from a web or mobile application to get prediction from the model. In addition, you can deploy ML models as serverless functions using platforms such as Google Cloud Functions or AWS Lambda. Also, you can package machine learning model and dependencies into a docker container and deploy it to a certain management platform such as Docker Swarm or Kubernetes. Deployment options are depend on project needs, such as budget, scalability and performance. Based on these needs, the developer should choose the methods. In our case, we build a machine learning model using Streamlit, which is a python framework for building data science web applications. We will also use the same free Streamlit platform to host it.

The project repository contains following files/folders.

Two important files:

A. Build_AND_Save_DL_model.ipynb -> train and save ML model.

B. Streamlit.py -> Build interactive web app using Streamlit and make custom prediction.

Other files/folders:

data -> The folder stores custom datasets and dog breed names (Training dataset is large you can download it from Kaggle directly)
pretrained_models -> Stores pretrained fine tuned efficientnet3 model.
README.md -> Text about project information
custom_prediction.ipynb -> (Optional) Experimental Jupiter notebook which does custom inference on the pretrained model.
requirements.txt -> Package requirements to make inference on the Streamlit website.

Import libraries and helper functions.

I used google Colab which has python libraries already installed.

# I use tensorflow 2.9.1 version to train model
!pip install tensorflow==2.9.1import pandas as pd
import pickle
from PIL import Image
from google.colab import drive
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
import tensorflow as tf
import numpy as np
import zipfile
import os
import warnings
import shutil
import seaborn as sns
import random
from tensorflow.keras import layers
from tensorflow.keras.layers.experimental import preprocessing
warnings.filterwarnings("ignore")
drive.mount('/content/gdrive')
print(tf.__version__)
def unzip_data(filename):
"""
Unzips filename into the current working directory.
Args:
filename (str): a filepath to a target zip folder to be unzipped.
"""
zip_ref = zipfile.ZipFile(filename, "r")
zip_ref.extractall()
zip_ref.close()
def plot_value_count(df, column):
"""
plots value count of dataframe column
"""
sns.set(style='darkgrid')
plt.figure(figsize= (20,10))
sns.countplot(x=column, data=df, order = df
.value_counts().index)
plt.xticks(rotation=90)
plt.show()
unzip_data("/content/gdrive/MyDrive/dog-breed-identification/dog-breed-identification.zip")

If you try to train the model locally, make sure you create a new python virtual environment using batch script:

python -m venv /path/to/new/virtual/environment

and then install libraries in order not to happen package dependencies issues.

2. Understand the data

Building ML models is important, but beforehand it is crucial to visualize the data and its statistics. Let’s see what is the total number of training and test samples and total number of dog breed. Also, we visualize each dog breed frequency.

labels_df=pd.read_csv('labels.csv')train_images = os.listdir("/content/train/")
test_image = os.listdir("/content/test/")
# how many dog breed? 
target_labels = sorted(labels_df['breed'].unique().tolist())
print('train label shape:', labels_df.shape)
print('train dataset and test dataset sizes:',len(train_images), len(test_image))
print('total number of dog breeds: ',len(target_labels))
# lets plot distribution of dog breeds in training data
plot_value_count(labels_df, 'breed')

Each dog breed count (Image by the author)

3. Create Structured of the data

The raw data only contained images of all the breeds in one folder, therefore, I created each folder for each breed to create a specific format to store the data and load it later.

# 1.create class folders
parent_folder = "data"
datasets = ["all_data"]
for i in datasets:
for j in target_labels:
os.makedirs(parent_folder + "/" + i + "/" + j,  exist_ok=True)# all data
for _, name, label in labels_df.itertuples():
original_path = "/content/train/" + name +".jpg"
dest_path = "/content/data/all_data/" + label
shutil.copy(original_path, dest_path)

4. Visualize Random Images

Here I just visualize the training random training images.

all_data_dir = "/content/data/all_data/"def view_25_random_image(target_dir, target_classes):
plt.figure(figsize=(18, 12))
random_images = []
for i, target_class in enumerate(target_classes):
# setup the target directory
target_folder = target_dir + target_class
# get the random image path
random_image = random.sample(os.listdir(target_folder),1)
img = mpimg.imread(target_folder + "/" + random_image[0])
#print(target_folder + "/" + random_image[0])
#print(random_image)
plt.subplot(5,5,i+1)
plt.imshow(img)
plt.title(f"{target_class}")
plt.axis("off");
random_images.append(img)
#print(f"image shape: {img.shape}")
return random_images
random_images = view_25_random_image(target_dir=all_data_dir,
target_classes = random.sample(target_labels, 25))

5. Get The Data

As we created specific data storing format, we can easily generate tensorflow dataset from the image folder.

tf.random.set_seed(42)
IMG_SIZE  = (256,256)
IMG_SHAPE = IMG_SIZE +(3,)
BATCH_SIZE = 32
data_all = tf.keras.preprocessing.image_dataset_from_directory(all_data_dir,
label_mode = "categorical",
image_size=IMG_SIZE,
batch_size=BATCH_SIZE,
)
print(f"all_data batches: {len(data_all)}")

6. Train the model

I use several data augmentation techniques, such as RandomFlip, RandomCrop, Randomrotation, etc. Also, I use two callbacks, such as reduce the learning rate callback, that reduces learning rate when validation accuracy stops improving and early stopping, which stops training after validation accuracy does not improve anymore. Reduce learning rate helps the model to converge more slowly and therefore potentially find a better and more generalizable solution. Whereas, early stopping helps the model to avoid overfitting and stop training earlier. As for the ML model architecture, I used pretrained EfficientNetB3 model and then I fine tuned with our new data. EfficientNetB3 is a convolutional neural network architecture that was proposed in 2019 by Tan et al. that achieved state-of-the-art accuracy while using fewer parameters and less computational resources than other ML models.

#data augmentation
data_augmentation_layer = tf.keras.models.Sequential([
preprocessing.RandomFlip("horizontal"),
preprocessing.RandomCrop(height=224, width=224),
preprocessing.RandomRotation(0.2),
preprocessing.RandomZoom(0.1),
preprocessing.RandomContrast(factor=0.1),
])# EarlyStopping Callback (stop training in val accuracy not improves)
early_stopping = tf.keras.callbacks.EarlyStopping(monitor="val_accuracy", 
patience=5,
restore_best_weights=True)
#ReduceLROnPlateau Callback (Creating learning rate)
reduce_lr = tf.keras.callbacks.ReduceLROnPlateau(monitor="val_accuracy",  
factor=0.2, # multiply the learning rate by 0.2 (reduce by 5x)
patience=2,
verbose=1, # print out when learning rate goes down 
min_lr=1e-5)
baseline_model = tf.keras.applications.EfficientNetB3(include_top=False)
# trainable freeze
baseline_model.trainable = False
inputs = tf.keras.Input(shape = IMG_SIZE+(3,), name = "input_layer")
#x = data_augmentation_layer(inputs)
x = baseline_model(inputs, training=False) # weights whhich need to stay frozen, stay frozen
x = layers.GlobalAveragePooling2D(name="global_average_pool_layer")(x)
x = layers.Dense(len(data_all.class_names))(x)
outputs = layers.Activation("softmax", dtype=tf.float32)(x)
model_0 = tf.keras.Model(inputs, outputs)
train_size = int(0.9 * len(data_all))
print(train_size)
model_0.compile(optimizer=tf.keras.optimizers.Adam(),
loss="categorical_crossentropy",
metrics = ["accuracy"])
history_0 = model_0.fit(data_all.take(train_size),
epochs=5,
validation_data = data_all.skip(train_size),
callbacks = [
early_stopping,
reduce_lr
])

7. Save the model and class names

This is the last step. After fine tuning the model, I saved the trained model and breed names to use them later.


#save model
model_0.save('/content/gdrive/MyDrive/dog-breed-identification/EfficientNetB3_Model.h5')breed_names = data_all.class_names
breed_names = [n.replace('_',' ').capitalize() for n in breed_names]
print(breed_names[0:3])
#save dog breed names
with open('/content/gdrive/MyDrive/dog-breed-identification/class_names', 'wb') as fp:
pickle.dump(breed_names, fp)

Import libraries, pretrained model and breed class names

import streamlit as st
import tensorflow as tf
from tensorflow.keras.preprocessing.image import load_img
from tensorflow.keras.preprocessing.image import img_to_array
import pickle
import numpy as np
import pathlib
import matplotlib.pyplot as plt
from PIL import ImageIMG_SIZE  = (256, 256)
IMG_SHAPE = IMG_SIZE +(3,)
#1. load model
@st.cache_resource
def load_model():
return tf.keras.models.load_model('pretrained_models/EfficientNetB3_Model.h5')
loaded_model = load_model()
#2. load class names
with open ('data/class_names', 'rb') as fp:
class_names = pickle.load(fp)

2. Create web app

I create an interactive web interface with the Streamlit scripts. I create the title of the web and file uploader object.

st.header("Dog Identification Application")uploaded_file = st.file_uploader("Upload an image of your dog and identify its breed...", type=['jpg', 'jpeg', 'png'])

3. Make prediction

When the file is uploaded, the image is first resized and then fed into the loaded pretrained model. If the prediction confidence score is more than 50%, one prediction result will be shown at the top of the image, but if the maximum confidence is less than 50%, two results based on the top two highest confidence scores. At the end of the website, there is a list of all 120 available dog breeds that our model knows.


# If an image is uploaded, display it and make a prediction
if uploaded_file is not None:
img = Image.open(uploaded_file).resize(IMG_SIZE)
img = np.array(img, dtype=np.float32)
img_g = np.expand_dims(img, axis=0)
custom_predict = loaded_model.predict(img_g)# Display the uploaded image and the predictions
# st.write(f"{class_names[np.argmax(custom_predict[0])]}  ({round(np.max(custom_predict[0]*100))}% confidence)")
if round(np.max(custom_predict[0]*100))>50:
st.write(f'<p style="font-size:26px; color:green;"> {class_names[np.argmax(custom_predict[0])]}  ({round(np.max(custom_predict[0]*100))}% confidence)</p> ', unsafe_allow_html=True)
else:
argsorts = np.argsort(custom_predict[0])
sorts = np.sort(custom_predict[0])
st.write(f'<p style="font-size:26px; color:red;"> {class_names[argsorts[-1]]}  ({round(sorts[-1]*100)}% confidence) OR {class_names[argsorts[-2]]}  ({round(sorts[-2]*100)}% confidence)</p> ', unsafe_allow_html=True)
st.image(img/255,  use_column_width=True)
st.write('Notes:')
st.write('1. The model is a first simplest baseline version, the prediction result will be improved in later versions')
st.write('2. The model can identify up to 120 types of dog breeds')
option = st.selectbox(
'All Breeds',
class_names)

You can visualize result in a local host by running the bash command

streamlit run Streamlit.py

C. Deployment

As all the project is created locally, now it is time to deploy it on the web. This step is so easy.

Upload to GitHub

First, we upload the following files/folders:

Streamlit.py,
pretrained_models/EfficientNetB3_Model.h5
data/class_names
requirements.txt

in a GitHub repository.

2. Connect GitHub repository to Streamlit.

After that, log in Streamlit website, create account and press ‘New app’

Streamlit platform (image by the author)

and we just follow instructions

New app instructions on Streamlit (image by the author)

Here are some most probable error and proposed solution during building a project.

Packaging dependency issue: It might happen if you training the model locally, make sure to create python virtual environment and then install the libraries.
Packaging bugs: When using Google Colab, some Tensorflow versions had issue during saving a model, therefore use version 2.9.1 of Tesnorflow in Google Colab.
Streamlit application does not work after connecting the GitHub account: It is mainly because of lack of packages. Make sure you have requirements.txt file uploaded in the GitHub with specific packages. Streamlit will automatically look for the file and install all the required packages to run the application.

In summary, I have developed a simple deep learning model that identifies dog out of 120 breeds. Also, I hosted the model on the web using Streamlit. The model achieves about 94% accuracy on validation data, which is a pretty decent result. Even though the model works well, it has some drawbacks. For example, the model confidence score is not calibrated, that means that the confidence score does not represent real prediction probability. Also, the model only knows 120 dog breeds, so it can not identify all the dog breeds. In addition, the model does not know when it does not know. For example, if you upload the image of an airplane, the model can not tell you that it is not a dog. I will try to overcome these issues in a later version of the application.