Jessie Hobb 7582 posts 0 comments

Two nifty ways to estimate unobserved variablesPhoto by Yannis Papanastasopoulos on UnsplashI recently started working my way through “Probabilistic Programming and Bayesian Methods for Hackers”, which has been on my to-do list for a long time. Speaking as someone who’s taken my fair share of statistics and machine learning classes (including Bayesian stats), I find that I’m understanding things through this coding-first approach that were never clear before. I highly recommend this read!Some generous souls updated the…

Asking ChatGPT about PetrophysicsImage generated by DALL-E 2 by the author.OpenAI has recently released their latest Artificial Intelligence (AI) chatbot prototype powered by a model from the GPT-3.5 series. It provides a service where you can ask questions and it comes back with a detailed answer in a conversational way. Almost as if you were talking to a human!ChatGPT is based on a trained model using Reinforcement Learning from Human Feedback which allows it to simulate conversation, answer follow-up questions and even…

When creating a classification model, many algorithms offer the function predict_proba() to give us the probability of that observation being classified under each class. Thus, it is common to see an output like this:In the previous case, the model is 92.5% sure that the observation pertains to class 0, and only 7.5% of chance to be from class 1.If we, therefore, request this same model to give us a binary prediction using the predict() function, we’ll just get a as the result, correct?In this example, it is most likely…

Written with Co-Author: Daniel Shin, MSCS at StanfordPhoto by Firmbee.com on UnsplashSometimes as engineers and scientists, we think of data only as bytes on RAM, matrices in GPUs, and numeric features that go into our predictive black-box. We forget they represent changes in some real-world patterns.For example, when real world events and trends arise, we tend to defer to Google first to acquire related information (i.e where to go for a hike, what does term X mean) — which makes Google Search Trends a very good source…

Why and how to attempt certificationConsidering taking the AWS Machine Learning Specialty certification? I took the test in October 2022 and wasn’t able to find much updated information about it. This post is designed to help you decide whether to take the exam and how to invest your time if you decide to go for it.Source: pixabay.comLet’s get to it! 🚀To see if Machine Learning specialty exam is a good fit for you, let’s look at reasons to take it and my suggested prerequisites.Why take this test?Sign up for this test if…

Photo by Scott Graham on UnsplashUsing Python and the mindee API to automate expense reportsIt used to be that employees had to file expense reports by hand, or at least draft them on a spreadsheet. Then computers came along, with built-in spreadsheets that made it easier to generate and track reports. But what if you could automate the entire data entry process itself to facilitate generating expense reports from images of receipts?In this article, I will show you the steps to automate expense reports using Python.The…

OpinionThe Importance of Customization and Flexibility in Machine Learning InfrastructureDALL·E - A illustration of a vendor lock in costing a company too much moneyIn the recent year, my inbox was flooded by MLOps companies trying to sell their one-size-fits-all infrastructure. But not only there, in comparison to pre-covid years, exhibitors at conferences were also dominated by ML(Ops) platforms. You can also find a lot of spam under every related article here on medium, and even a paid workshop was simply a sales demo…

Spark Structured Streaming supports a Kafka source and a file source, meaning it can treat a folder as a source of streaming messages. Can a solution, entirely based on files, really compare to a streaming platform such as Kafka?Photo by Edward Koorey on UnsplashIn this article, we explore using an Iceberg table as a source of streaming messages. To do this, we create a Java program that writes messages to an Iceberg table and a pySpark Structured Streaming job that reads these messages.Azure Event Hubs is a big data…

Probabilistic deep learningThis article belongs to the series “Probabilistic Deep Learning”. This weekly series covers probabilistic approaches to deep learning. The main goal is to extend deep learning models to quantify uncertainty, i.e. know what they do not know.We develop our models using TensorFlow and TensorFlow Probability. TensorFlow Probability is a Python library built on top of TensorFlow. We are going to start with the basic objects that we can find in TensorFlow Probability and understand how can we…

Conformal Prediction in Julia — Part 2Conformal Predictions sets with varying degrees of uncertainty. Image by author.Deep Learning is popular and — for some tasks like image classification — remarkably powerful. But it is also well-known that Deep Neural Networks (DNN) can be unstable (Goodfellow, Shlens, and Szegedy 2014) and poorly calibrated. Conformal Prediction can be used to mitigate these pitfalls.In the first part of this series of posts on Conformal Prediction, we looked at the basic underlying methodology and…