serverless

Building a lean and robust model serving layer for vector embedding and search with Hugging Face’s new Candle FrameworkPhoto by Clay Banks on Unsplash1. IntroThe incredible progress in AI research and tooling over the past decade has given rise to more accurate and more reliable machine learning models, as well as to libraries and frameworks that make it increasingly simpler to integrate AI capabilities into existing applications.However, one area that still remains a considerable challenge in demanding production…

A simple routine that can save you loads of moneyPhoto by Ziyou Zhang on UnsplashThis blog post was co-authored with my colleague Shay Margalit. It summarizes his research into how AWS Lambda functions can be used to increase the control over the usage and costs of the Amazon SageMaker training service. Interested? Please read on :).We are fortunate (or very unfortunate — depending on who you ask) to be sharing a front row seat to an AI revolution that is expected by many to change the world as we know it. Powered by…

Amazon kicked off AWS re:Invent, its annual customer conference, in Las Vegas tonight with a few new serverless offerings designed to make it easier to manage Aurora, Elastic Cache and Redshift serverless services. Matt Wood, AWS VP, says that Aurora Serverless is great for getting up and running very quickly with a cloud database, but over time, once you get to very, very high scale, and you’re  dealing with tens of millions of customers, or millions of different records, it becomes challenging…

Large language models (LLMs) like chatGPT has been a hot topic for business world since last year. Thus, the number of these models have drastically increased. Yet, one major LLM challenge prevents more enterprises adopting it, system costs for developing these models. For instance, Megatron-Turing from NVIDIA and Microsoft is estimated to hold a cost of approximately $100 million for the entire project. Serverless GPU space can reduce this cost by helping with inference phase of large language models (LLMs). Serverless…

Is Serverless Hard to Adopt?Understand the simple measures to make your serverless adoption a successPhoto by Mikhail Nilov on PexelsMore than a year ago, hiking at some altitude in the September setting of Semmering, Austria, we were lost at a junction on our trail. As the signposts were inconclusive, I reached out to my mobile for clues. Amongst several Twitter (X) notifications, a reference to Paul Johnston’s article Learning Serverless (and why it is hard) just flashed past.On a normal day, I would’ve read it…

In this blog post, you will learn how to build a serverless solution for invoice processing using Amazon Textract, AWS Lambda, and the Go programming language. Invoices and expense receipt images uploaded to Amazon S3 will trigger a Lambda function which will extract invoice metadata (ID, date, amount, etc.) using the AWS Go SDK and persist it to an Amazon DynamoDB table. You will also use Go bindings for AWS CDK to implement "Infrastructure-as-code" for the entire solution and deploy it with the AWS Cloud Development Kit…

An open-source implementation of a Data Lake with DuckDB and AWS LambdasA duck in the cloud. Photo by László Glatz on UnsplashIn this post we will show how to build a simple end-to-end application in the cloud on a serverless infrastructure. The purpose is simple: we want to show that we can develop directly against the cloud while minimizing the cognitive overhead of designing and building infrastructure. Plus, we will put together a design that minimizes costs compared to modern data warehouses, such as Big Query or…

How to create serverless containerized inference endpoint for your machine learning models with Azure Container AppPhoto by Jan Canty on UnsplashServerless container architecture is an approach to building and running containerized applications and services without having to manage the underlying infrastructure. In this architecture, containers are used to package and deploy applications, and these containers are run in a fully managed environment provided by a cloud provider.The cloud provider is responsible for the…

Prefect is a global coordination plane for any data stack. It allows you to design, build, schedule, and monitor your dataflows, covering the full spectrum from observability to orchestration. Here is an example of a simple Prefect flow:With a single CLI command, you can build a scheduled deployment that can run on any infrastructure — a local process, a Docker container, a Kubernetes job, or a serverless task on ECS Fargate. In this post, we’ll focus on the latter. But before we can deploy flows, we need an agent that…

Why serverless? And how? A practical dive into AWS ECSPhoto by authorIn this post we will design and deploy a FastAPI application on AWS.We will architect the application using a serverless compute service coupled with a managed database service. This microservice architecture helps make a decoupled system which increases robustness. Next, we use the AWS CDK to have our entire infrastructure bundled as code. The benefits of this are numerous. In a nutshell, it makes deployments and infrastructure updates easier. It allows…