Vegiraju

Applying MLOps best practices to advanced serving optionsImage from Unsplash by GrowtikaMLOps is an essential practice to productionizing your Machine Learning workflows. With MLOps you can establish workflows that are catered for the ML lifecycle. These make it easier to centrally maintain resources, update/track models, and in general simplify the process as your ML experimentation scales up.A key MLOps tool within the Amazon SageMaker ecosystem is SageMaker Pipelines. With SageMaker Pipelines you can define workflows…

Infrastructure as Code With TerraformImage from Unsplash by Krishna PandeyInfrastructure as Code (IaC) is an essential concept to optimize and take your resources and infrastructure to production. IaC is an age old DevOps/Software practice and has a few key benefits: Resources are maintained centrally via code, which in turn optimizes the speed and collaboration required to take your architecture to production.This software best practice like many other also applies to your Machine Learning tooling and infrastructure. For…

Test TensorFlow ResNet50 on SageMaker Real-Time EndpointsImage from Unsplash by Amokrane Ait-KaciIn the past I’ve written extensively about the importance of load testing your Machine Learning models before deploying them into production. When it comes to real-time inference use-cases in specific it’s essential to ensure your solution meets your target latency and throughput. We’ve also explored how we can use the Python library, Locust to define scripts that can simulate our expected traffic patterns.While Locust is an…

Utilize Locust to Distribute Traffic Weight Across ModelsImage from Unsplash by Luis ReyesProductionizing Machine Learning models is a complicated practice. There’s a lot of iteration around different model parameters, hardware configurations, traffic patterns that you will have to test to try to finalize a production grade deployment. Load testing is an essential software engineering practice, but also crucial to apply in the MLOps space to see how performant your model is in a real-world setting.How can we load test? A…

Rapid production-grade deployment of Transformers modelsImage from Unsplash by Towfiqu barbhuiyaA constant theme in my articles has been the deployment of your Machine Learning models. As Machine Learning grows in popularity so has the range of model deployment options for users. HuggingFace in particular has become a leader in the Machine Learning space and for Data Science Practitioners it’s incredibly likely you’ve used a Transformers model in the past.HuggingFace has partnerships with both AWS and Azure and has…

Deploy TensorFlow ResNet50 on SageMaker InferenceImage from Unsplash by Possessed PhotographyIn the past I’ve written about deploying pre-trained models on Amazon SageMaker. While the artifact I published is replicable it requires understanding of AWS SDKs and the appropriate API calls, along with an understanding of the higher level SageMaker Python SDK.This can be a lot to digest for new users and to make the experience simpler, I’ve helped build an API that automates a lot of the lower level work a user had to…

Infrastructure As Code With SageMakerImage from Unsplash by Abraham BarreraIn the past I’ve worked with SageMaker Deployment through Jupyter Notebooks and Python scripts. This is completely fine, but often times in the scope of a larger applications, you need to be able to define your SageMaker resources with the rest of your infrastructure in a central template. This brings in the idea of Infrastructure as Code, which then brings in AWS CloudFormation. When it comes to productionizing applications it’s essential to be…

Stop Waiting For Your Endpoints To CreateImage from Unsplash by Ben WhiteFor frequent users of SageMaker Inference a common frustration is being able to debug endpoints quickly. Often times with SageMaker Endpoints you end up with a custom inference script that helps you control the pre and post processing of your model.Initially when I first started with SageMaker I would re-deploy the endpoint every time I had to make a change to the inference script. This would be time consuming as I’d wait 2–3 minutes without…

Guide On Deploying ML Models With Flask and Containerizing Your WorkImage from Unsplash by Jonas SmithDeploying ML models is an essential step in the ML Lifecycle that’s often overlooked by Data Scientists. Without model deployment/hosting, there is no usage of Machine Learning models in real-world applications. There’s a variety of ways to host your ML models and one of the simplest, yet most effective is Flask.Flask is a micro-web framework that’s implemented in Python. Using Flask, we’ll walk through an example of how…

Step by Step GuideImage from Unsplash by Dominik LückmannAmazon Elastic Container Registry (ECR) is a container image registry that we can use push Docker images to on AWS. Why use a Container Registry? It helps make it easy to manage your various images and separate projects. For example, when I first started working with Docker locally I didn’t realize how many images I had built. It was hard for me to track my images for different projects, I needed a central resource and ECR was my registry of choice as I worked with…