PyTorch

Deep Dive into Automatic Speech Recognition: Benchmarking Whisper JAX and PyTorch Implementations Across PlatformsIn the world of Automatic Speech Recognition (ASR), speed and accuracy are of great importance. The size of the data and models has been growing substantially recently, making it hard to be efficient. Nonetheless, the race is just starting, and we see new developments every week. In this article, we focus on Whisper JAX, a recent implementation of Whisper using a different backend framework that seems to run…

How to use differential equations layers in pytorchDifferential equations are the mathematical foundation for most of modern science. They describe the state of a system using an equation for the rate of change (differential). It is remarkable how many systems can be well described by equations of this form. For example, the physical laws describing motion, electromagnetism and quantum mechanics all take this form. More broadly, differential equations describe chemical reaction rates through the law of mass action,…

Building a Transformer model step by step in PytorchFigure 1. Photo by Kevin Ku on UnsplashIn this tutorial, we will build a basic Transformer model from scratch using PyTorch. The Transformer model, introduced by Vaswani et al. in the paper “Attention is All You Need,” is a deep learning architecture designed for sequence-to-sequence tasks, such as machine translation and text summarization. It is based on self-attention mechanisms and has become the foundation for many state-of-the-art natural language processing…

A primer on deep learning compiler technologies in PyTorch for graph capture, intermediate representations, operator fusion, and optimized C++ and GPU code generationillustration by authorComputer programming is magical. We write code in human readable languages, and as though by magic, it gets translated into electric currents through silicon transistors making them behave like switches and allowing them to implement complex logic — just so we can enjoy cat videos on the internet. Between the programming language and…

Data augmentation techniques for Computer Vision implemented in PythonCutmix image augmentation (Background image drawn by the author, artificial photograph of statue generated with DALLE)It’s almost guaranteed that applying data augmentations will improve the performance of your neural network. Augmentations are a regularization technique that artificially expands your training data and helps your Deep Learning model generalize better. Thus, image augmentations can improve the model performance.Image augmentations can…

An end-to-end use-case combining PyTorch Lightning and BentoMLPhoto by SpaceX on UnsplashAs an ML engineer, I often encounter two main challenges when working with deep learning models in production.The first one is the need to rewrite boilerplate code for each project to handle tasks like training loops, data loading, or metric computation. As this work often makes the codebase more complex with unnecessary abstractions, it slows down the iterative process.The second challenge involves the wide range of skills or tools…

An introduction to PyTorch’s training loop and general approach to tackle the library’s steeper initial learning curveImage by author.In this post we will cover how to implement a logistic regression model using PyTorch in Python.PyTorch is one of the most famous and used deep learning frameworks by the community of data scientists and machine learning engineers in the world, and thus learning this tool becomes an essential step in your learning path if you want to build a career in the field of applied AI.It joins…

Build an object detection model, compare it to intensity thresholds, evaluate it and explain it using DeepSHAP(source: author)If the world was less chaotic self-driving cars would be simple. But it’s not. To avoid serious harm, AI has to consider many variables — speed limits, traffic and obstacles in the road (such as a distracted human). AI needs to be able to detect these obstacles and take appropriate actions when encountered.Thankfully, our application is not as complicated. Even more, thankfully, we will be using…

Photo by Greyson Joralemon on UnsplashFive Tips for Writing Clean, Efficient and readable Code in PyTorchIntroductionIn the field of data science and programming in general, it is very important to be able to write code that is easy to read and maintain. Surely you too have had the experience of writing code that seemed to work fine and that was pretty clear, but then you reread it a day or a week later and it looked incomprehensible. Even more obvious is this difficulty when you have to review code written by other…

PyTorch is not only used for research but also for production purposes, with billions of requests being served and trained daily. The PyTorch community has made remarkable strides in recent times. Last year, contributors of PyTorch also introduced BetterTransformer inference optimizations for transformer models such as GPT, which have significantly improved the performance of these models. This collection of highly optimized code is designed specifically to accelerate transformer models in production workloads, allowing…