Jessie Hobb 7582 posts 0 comments

Point forecasts are good for making decisions. With probabilistic forecasts, you can also make the right ones.Photo by Aron Visuals on UnsplashIn Data Science, forecasting often involves creating the best possible model for predicting future events. Usually, the “best” model is one that minimizes a given error metric such as the Mean-Squared Error (MSE). The end result is then a list of values that depicts the predicted trajectory of the time-series. A statistician or econometrician would call this a point forecast.More…

A single reference for EDA, feature engineering, model fitting, and morePhoto by James Harrison on UnsplashI recently completed an internal transfer process to join Meta’s Core Data Science team. While studying for these interviews with my colleagues, Barry Zhang and Nicolas Lepore, we found that there was no singular resource for typical data science coding functions like EDA plotting, data cleaning, data transformation, hyperparemeter optimization, model fitting, unsupervised learning etc. so we decided to create one.…

TutorialCalculating The 6 Month Moving AverageTima Miroshnichenko, PexelsWhen I look at the work I do, I consider myself to be a builder. Like most builders, I have tools I use when working. The more I understand those tools, the easier it is to do the work. One of those tools is the Window function. I like its versatility; its ability to slice, dice, and control data.In this tutorial, we’ll dig into the frame clause in a window function. We use the frame clause to control the number of rows included in the calculation.…

Deep Learning in NILMA CNN for Sequence modelling? Let’s see how it faresPhoto by Riccardo Annandale on UnsplashIntroductionNILM (non-intrusive load monitoring) has been one of the important processes in understanding customer behavior and building scalable infrastructure solutions. The most basic use case of NILM is to provide the end-user a better understanding of their own household’s energy consumption. This in turn helps users develop better consumption habits, which results in a 5–10% reduction in consumption. So…

Understand the Importance of Graphics Card for the Advance of Machine LearningPhoto by Christian Wiediger on UnsplashThe graphics card, also known as Graphics Processing Unit (GPU), is responsible for calculating images in a computer, which can then be displayed on a monitor. It represents the interface between the processor’s calculations and the monitor. However, the development of graphics cards is now so far advanced that, in addition to this function, they can also support and relieve the CPU during calculations.The…

The era of large language models is here nowPhoto by Anil Sharma on PexelsSince the groundbreaking release of BERT in October 2018, machine learning has achieved ever greater heights through clever optimization and augmented compute. BERT, which stands for Bidirectional Encoder Representations from Transformers, introduced a new paradigm in neural network architecture. The transformer has served as a significant unlock in machine learning capabilities.Further advancements in the field of Natural Language Processing (NLP)…

Adapting federated learning to your own datasetsPhoto by DeepMind on UnsplashIn my earlier post, I described the importance of federal learning from a data scientist’s view. I will now get you started on FL using your own datasets. There are several FL frameworks available, along with tutorials and user guides. However, adapting these frameworks on your own datasets is not a simple task. In this article, I will provide you the concise solution to start your FL journey with one of the popular frameworks and that is…

Searching for the best solution (on the highest top). Image by Dall-E 2.Local search, genetic algorithms, and more.There are many different ways to solve mathematical optimization problems. You can use greedy algorithms, constraint programming, mixed integer programming, genetic algorithms, local search, and others. Depending on the size and type of the problem, and the solution quality desired, one technique may work better than the other.This post gives an overview of different heuristics for solving discrete…

In-depth explanation with Python examples of unsupervised learning evaluation metricsPhoto by Markus Spiske on UnsplashIn Supervised Learning, the labels are known and evaluation can be done by calculating the degree of correctness by comparing the predicted values against the labels. However, in Unsupervised Learning, the labels are not known, which makes it hard to evaluate the degree of correctness as there is no ground truth.That being said, it is still consistent that a good clustering algorithm has clusters that…

Ensure high-quality machine learning across the ML lifecycleQuality management istock image. Credit NicoElNino, Stock photo ID:1357020474Machine learning has been booming in recent years. It is becoming more and more integrated into our everyday lives, and is providing a huge amount of value to businesses across industries. PWC predicts AI will contribute $15.7 trillion to the global economy by 2030. It sounds too good to be true…However, with such a large potential value-add to the global economy and society, why are we…