Deploy Your Own MLflow Workspace On-Premise with Docker | by Patrick Tinz | Apr, 2023
Manage the lifecycle of your ML models like a Pro
MLflow is an open source platform to manage the lifecycle of ML models end to end. It tracks the code, data and results for each ML experiment, which means you have a history of all experiments at any time. A dream for every Data Scientist. Moreover, MLflow is library-agnostic, which means you can use all ML libraries like TensorFlow, PyTorch or scikit-learn. All MLflow functions are available via a REST API, CLI, Python API, R API and Java API.
As a Data Scientist, you spend a lot of time optimizing ML models. The best models often depend on an optimal hyperparameter or feature selection, and it is challenging to find an optimal combination. Also, you have to remember all experiments, which is very time-consuming. MLflow is an efficient platform to address these challenges.
In this post, we briefly introduce the basics of MLflow and show how to set up an MLflow workspace on-premise. We set up the MLflow environment in a Docker stack so that we can run it on all systems. In this context, we have the services Postgres database, SFTP server, JupyterLab and MLflow tracking server UI. Let’s start.
Currently, MLflow offers four components for managing the ML lifecycle. The following figure shows an overview.
MLflow Tracking is used to track and query experiments. It tracks the model parameters, code, data and model artifacts. In addition, MLFlow’s tracking server provides a web UI, which shows the history of all experiments. The MLflow library already provides the web UI. The tracking server distinguishes between different experiments. You can compare your ML models in the experiments by visualising the results.
MLflow Project is a component used for packaging data science code in a reusable and reproducible way.
The MLflow Models format provides a uniform storage format for ML models created with different libraries (e.g. TensorFlow, PyTorch or scikit-learn). The uniform format enables deployment in diverse environments.
The Model Registry component allows providing of produced models from staging to production. It enables the management of ML models in a central model repository.
You can learn more about the components in the official documentation or the MLflow GitHub repo.
You will need the following prerequisites:
- The latest version of Docker must be installed on your machine. If you do not have it installed yet, please follow the instructions.
- The latest version of Docker Compose must be installed on your machine. Please follow the instructions.
- Access to a terminal (macOS, Linux or Windows).
First, you should check that you have Docker and Docker Compose installed correctly. Open the terminal of your choice and enter the following command:
$ docker --version
# Output: $ Docker version 20.10.23
If the installation is correct, you can see the Docker version (Maybe you have a different version.). Next, you can check the same for your Docker Compose installation.
$ docker-compose --version
# Output: Docker Compose version v2.15.1
Yeah. Everything is ok. Now we can start with the MLflow Workspace.
There are several ways to use MLflow. You can use MLflow with localhost or you deploy a complete stack for a production environment. We will focus on the second option in this article. We have set up a production-ready MLflow workspace, which we explain in this article.
Please download or clone the MLflow Workspace from our GitHub repo. The project contains four services, which you can see in the following figure.
You can start all services by executing the following command in the terminal.
$ sh start_docker_stack.sh
The first time you start it, it takes a while until all the Docker images are pulled. It’s the right time to get a coffee. ☕️
After everything has started, please open a web browser of your choice. Next, you can access the Jupyter server by entering the following URL http://127.0.0.1:8888 into your web browser. You can log in with the password mlflow. Next, visit the MLflow UI website at http://127.0.0.1:5001. Note that we are on localhost. If you run the MLflow Workspace on a remote server, please specify the IP address of your server.
You can test the MLflow Workspace by running the notebook /notebooks/mlflow_example.ipynb. If no error appears, the setup was successful. Congratulations!
When you have finished working on a notebook, you can shut down the Workspace. The Workspace saves your work persistently so that you can continue working the next time. You can shut down the Workspace with the following command:
$ sh stop_docker_stack.sh
After we have completed the setup, we can now take a closer look at the individual services.
JupyterLab
This service provides a JupyterLab environment. In the MLflow Workspace, we use the jupyter/scipy-notebook image from DockerHub. You can also use other Jupyter images if you want. In the future, we plan to replace the image with a JuypterHub image so that each Data Scientist has their own account. An advantage of this approach is that the accounts also communicate with the MLflow tracking server. It allows us a better user management. Look forward to more updates on the Workspace.
SFTP Server
This service provides remote data storage. SFTP (Secure File Transfer Protocol) is a file transfer protocol that provides secure access to a remote computer. For more information about SFTP, please read the following article.
In this project, we use the atmoz/sftp image from DockerHub. You can also use other storage technologies, such as AWS S3 or HDFS. In the MLflow Workspace, the SFTP server provides the artifact store. In this store, we save the ML models and other artifacts such as Jupyter notebooks or Python scripts.
MLflow Tracking Server
This service provides the MLflow UI. This web UI enables the management of your ML experiments. You can also compare different ML models via the web UI. To build this service we used the Docker image python. The web UI is accessible via http://127.0.0.1:5001.
Postgres database
This service provides a Postgres database for the backend store. PostgreSQL is a free and open-source relational database management system. We use it to store parameters and evaluation metrics. The MLflow Workspace uses the official postgres Docker image from DockerHub.
The MLflow Workspace is running, and we have understood the basic functionality of the services. It’s time to implement a practical example with the Workspace.
We explain the functionality of the workspace with a small ML example. In this example, we will try to separate the two classes in the sklearn moon dataset. We use the Random Forest as ML model. First, we load the data and visualise it.
n = 1000
test_size = 0.25
data_seed = 73 X, y = datasets.make_moons(
n_samples = n,
noise = 0.25,
random_state = data_seed)
X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size = test_size,
random_state = 42)
plt.scatter(
x = X[:,0],
y = X[:,1],
s = 40,
c = y,
cmap = plt.cm.Accent);
You can see that we have two classes. We want to separate these two classes with a Random Forest. In addition, we also want to track the metrics and parameters of the individual experiments. We achieve this by including MLflow commands in our code. You can check out the full code in our GitHub repo.
with mlflow.start_run(run_name='random_forest_model') as run:
model_rf = RandomForestClassifier(random_state = 42)
model_rf.fit(X_train, y_train)
y_pred = model_rf.predict(X_test)# metrics
precision_0 = classification_report(
y_true=y_test,
y_pred=y_pred,
target_names=target_names,
output_dict=True)["0"]["precision"]
...
f1_score_1 = classification_report(
y_true=y_test,
y_pred=y_pred,
target_names=target_names,
output_dict=True)["1"]["f1-score"]
err, acc, tpr, tnr, fnr, fpr = get_confusion_matrix_metrics(y_test=y_test, y_pred=y_pred)
# log metrics
mlflow.log_metric("precision_0", precision_0)
...
mlflow.log_metric("f1_score_1", f1_score_1)
mlflow.log_metric("err", err)
mlflow.log_metric("acc", acc)
mlflow.log_metric("tpr", tpr)
mlflow.log_metric("tnr", tnr)
mlflow.log_metric("fnr", fnr)
mlflow.log_metric("fpr", fpr)
...
mlflow.log_artifact("logging_example.ipynb")
After running the code with different parameters for the Random forest model, our experiments appear in the web UI.
You can see two runs. Next, we can compare these two runs. To do this, click on the checkboxes of the two runs and click on Compare. A new web page opens where you can carry out extensive comparisons. The following figure shows an example.
We see the details for the two runs. MLflow distinguishes the runs via run IDs. It also saves the start time, the end time and the duration of the run. The second section lists the model parameters. In the first run we used 100 trees, and in the second run only two trees. It is practical that you can filter by differences. The third section lists all the metrics we tracked. The metrics show that both models work similarly well, with the first model performing slightly better. MLflow offers many more comparison functions, e.g. you can compare different metrics or parameters in graphs (Parallel Coordinates Plot, Scatter Plot, Box Plot and Contour Plot). The MLflow Workspace is extremely helpful for Data Scientists because it tracks all model combinations with metrics and code. It avoids chaos with many experiments, and you always keep an overview.
In this article we presented a production-ready MLflow Workspace. We briefly described the setup with Docker and run an example in the Workspace. In addition, we invite all Data Scientists to test the MLflow Workspace so that we can improve it continuously. Feedback is always welcome. Write your thoughts in the comments. Thanks.
👉🏽 Use the MLflow Workspace for your next Data Science project. It’s free!
Manage the lifecycle of your ML models like a Pro
MLflow is an open source platform to manage the lifecycle of ML models end to end. It tracks the code, data and results for each ML experiment, which means you have a history of all experiments at any time. A dream for every Data Scientist. Moreover, MLflow is library-agnostic, which means you can use all ML libraries like TensorFlow, PyTorch or scikit-learn. All MLflow functions are available via a REST API, CLI, Python API, R API and Java API.
As a Data Scientist, you spend a lot of time optimizing ML models. The best models often depend on an optimal hyperparameter or feature selection, and it is challenging to find an optimal combination. Also, you have to remember all experiments, which is very time-consuming. MLflow is an efficient platform to address these challenges.
In this post, we briefly introduce the basics of MLflow and show how to set up an MLflow workspace on-premise. We set up the MLflow environment in a Docker stack so that we can run it on all systems. In this context, we have the services Postgres database, SFTP server, JupyterLab and MLflow tracking server UI. Let’s start.
Currently, MLflow offers four components for managing the ML lifecycle. The following figure shows an overview.
MLflow Tracking is used to track and query experiments. It tracks the model parameters, code, data and model artifacts. In addition, MLFlow’s tracking server provides a web UI, which shows the history of all experiments. The MLflow library already provides the web UI. The tracking server distinguishes between different experiments. You can compare your ML models in the experiments by visualising the results.
MLflow Project is a component used for packaging data science code in a reusable and reproducible way.
The MLflow Models format provides a uniform storage format for ML models created with different libraries (e.g. TensorFlow, PyTorch or scikit-learn). The uniform format enables deployment in diverse environments.
The Model Registry component allows providing of produced models from staging to production. It enables the management of ML models in a central model repository.
You can learn more about the components in the official documentation or the MLflow GitHub repo.
You will need the following prerequisites:
- The latest version of Docker must be installed on your machine. If you do not have it installed yet, please follow the instructions.
- The latest version of Docker Compose must be installed on your machine. Please follow the instructions.
- Access to a terminal (macOS, Linux or Windows).
First, you should check that you have Docker and Docker Compose installed correctly. Open the terminal of your choice and enter the following command:
$ docker --version
# Output: $ Docker version 20.10.23
If the installation is correct, you can see the Docker version (Maybe you have a different version.). Next, you can check the same for your Docker Compose installation.
$ docker-compose --version
# Output: Docker Compose version v2.15.1
Yeah. Everything is ok. Now we can start with the MLflow Workspace.
There are several ways to use MLflow. You can use MLflow with localhost or you deploy a complete stack for a production environment. We will focus on the second option in this article. We have set up a production-ready MLflow workspace, which we explain in this article.
Please download or clone the MLflow Workspace from our GitHub repo. The project contains four services, which you can see in the following figure.
You can start all services by executing the following command in the terminal.
$ sh start_docker_stack.sh
The first time you start it, it takes a while until all the Docker images are pulled. It’s the right time to get a coffee. ☕️
After everything has started, please open a web browser of your choice. Next, you can access the Jupyter server by entering the following URL http://127.0.0.1:8888 into your web browser. You can log in with the password mlflow. Next, visit the MLflow UI website at http://127.0.0.1:5001. Note that we are on localhost. If you run the MLflow Workspace on a remote server, please specify the IP address of your server.
You can test the MLflow Workspace by running the notebook /notebooks/mlflow_example.ipynb. If no error appears, the setup was successful. Congratulations!
When you have finished working on a notebook, you can shut down the Workspace. The Workspace saves your work persistently so that you can continue working the next time. You can shut down the Workspace with the following command:
$ sh stop_docker_stack.sh
After we have completed the setup, we can now take a closer look at the individual services.
JupyterLab
This service provides a JupyterLab environment. In the MLflow Workspace, we use the jupyter/scipy-notebook image from DockerHub. You can also use other Jupyter images if you want. In the future, we plan to replace the image with a JuypterHub image so that each Data Scientist has their own account. An advantage of this approach is that the accounts also communicate with the MLflow tracking server. It allows us a better user management. Look forward to more updates on the Workspace.
SFTP Server
This service provides remote data storage. SFTP (Secure File Transfer Protocol) is a file transfer protocol that provides secure access to a remote computer. For more information about SFTP, please read the following article.
In this project, we use the atmoz/sftp image from DockerHub. You can also use other storage technologies, such as AWS S3 or HDFS. In the MLflow Workspace, the SFTP server provides the artifact store. In this store, we save the ML models and other artifacts such as Jupyter notebooks or Python scripts.
MLflow Tracking Server
This service provides the MLflow UI. This web UI enables the management of your ML experiments. You can also compare different ML models via the web UI. To build this service we used the Docker image python. The web UI is accessible via http://127.0.0.1:5001.
Postgres database
This service provides a Postgres database for the backend store. PostgreSQL is a free and open-source relational database management system. We use it to store parameters and evaluation metrics. The MLflow Workspace uses the official postgres Docker image from DockerHub.
The MLflow Workspace is running, and we have understood the basic functionality of the services. It’s time to implement a practical example with the Workspace.
We explain the functionality of the workspace with a small ML example. In this example, we will try to separate the two classes in the sklearn moon dataset. We use the Random Forest as ML model. First, we load the data and visualise it.
n = 1000
test_size = 0.25
data_seed = 73 X, y = datasets.make_moons(
n_samples = n,
noise = 0.25,
random_state = data_seed)
X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size = test_size,
random_state = 42)
plt.scatter(
x = X[:,0],
y = X[:,1],
s = 40,
c = y,
cmap = plt.cm.Accent);
You can see that we have two classes. We want to separate these two classes with a Random Forest. In addition, we also want to track the metrics and parameters of the individual experiments. We achieve this by including MLflow commands in our code. You can check out the full code in our GitHub repo.
with mlflow.start_run(run_name='random_forest_model') as run:
model_rf = RandomForestClassifier(random_state = 42)
model_rf.fit(X_train, y_train)
y_pred = model_rf.predict(X_test)# metrics
precision_0 = classification_report(
y_true=y_test,
y_pred=y_pred,
target_names=target_names,
output_dict=True)["0"]["precision"]
...
f1_score_1 = classification_report(
y_true=y_test,
y_pred=y_pred,
target_names=target_names,
output_dict=True)["1"]["f1-score"]
err, acc, tpr, tnr, fnr, fpr = get_confusion_matrix_metrics(y_test=y_test, y_pred=y_pred)
# log metrics
mlflow.log_metric("precision_0", precision_0)
...
mlflow.log_metric("f1_score_1", f1_score_1)
mlflow.log_metric("err", err)
mlflow.log_metric("acc", acc)
mlflow.log_metric("tpr", tpr)
mlflow.log_metric("tnr", tnr)
mlflow.log_metric("fnr", fnr)
mlflow.log_metric("fpr", fpr)
...
mlflow.log_artifact("logging_example.ipynb")
After running the code with different parameters for the Random forest model, our experiments appear in the web UI.
You can see two runs. Next, we can compare these two runs. To do this, click on the checkboxes of the two runs and click on Compare. A new web page opens where you can carry out extensive comparisons. The following figure shows an example.
We see the details for the two runs. MLflow distinguishes the runs via run IDs. It also saves the start time, the end time and the duration of the run. The second section lists the model parameters. In the first run we used 100 trees, and in the second run only two trees. It is practical that you can filter by differences. The third section lists all the metrics we tracked. The metrics show that both models work similarly well, with the first model performing slightly better. MLflow offers many more comparison functions, e.g. you can compare different metrics or parameters in graphs (Parallel Coordinates Plot, Scatter Plot, Box Plot and Contour Plot). The MLflow Workspace is extremely helpful for Data Scientists because it tracks all model combinations with metrics and code. It avoids chaos with many experiments, and you always keep an overview.
In this article we presented a production-ready MLflow Workspace. We briefly described the setup with Docker and run an example in the Workspace. In addition, we invite all Data Scientists to test the MLflow Workspace so that we can improve it continuously. Feedback is always welcome. Write your thoughts in the comments. Thanks.
👉🏽 Use the MLflow Workspace for your next Data Science project. It’s free!
Denial of responsibility! Techno Blender is an automatic aggregator of the all world’s media. In each content, the hyperlink to the primary source is specified. All trademarks belong to their rightful owners, all materials to their authors. If you are the owner of the content and do not want us to publish your materials, please contact us by email – [email protected]. The content will be deleted within 24 hours.