Linear

In machine learning, prediction is a critical component. It is the process of using a trained model to make predictions on new data. PyTorch is an open-source machine learning library that allows developers to build and train neural networks. One common use case in PyTorch is using linear classifiers for prediction tasks. In this article, we will go through the steps to build a linear classifier in PyTorch and use it to make predictions on new data.Linear Classifier:A linear classifier is a type of machine learning model…

Regression models are out there for a long time now, much longer before Machine Learning was a thing. Statisticians have been using these models to understand the relationship between variables even before the 1900s, when Sir Francis Galton (1885) introduced the idea.Fortunately, the theory has developed so much since then, and so have the computers and technology, up to the point that we can say that it is an easy (if not the easiest) model to create these days.However, do not be fooled by its easiness to implement. It…

Now, things are becoming more interesting. While raw i.i.d. noise cannot account for auto-correlation between observations, integrated noise can. Before we do a demonstration, let us introduce the differencing operator:(image by author)If you haven’t heard about differencing for time-series problems yet — great! If you have, then you can hopefully still learn something new.Definition of an integrated time-seriesWith the difference operator in our toolbox, we can now define an integrated time-series:(image by author)There…

Explore how to estimate regression parameter using R’s matrix operatorsPhoto by Breno Machado on UnsplashIntroductionI recently wrote an article that explored the matrix algebra and mathematical operations that sit behind linear regression. Now, while it’s certainly important to have a firm grasp of theoretical principles, nothing actually beats doing those calculations. So, in this follow up article, we’re going to look at how to implement those matrix operations using R.This article should be treated as a companion to…

Looking under the hood at the matrix operations behind linear regressionPhoto by Mingwei Lim on UnsplashIntroductionFor most, simple linear regression is a common starting point for understanding model-based estimation and model comparison. Regardless of whether you’re taking an introductory statistics or data science course, you can bet your bottom dollar that linear regression will crop up at some point. And there’s a good reason for it.Simple linear regression provides a natural extension of simple descriptive…

Not very much, but we can improve it.Photo by Greg Rakozy on UnsplashThe F-test statistic for joint significance of the slope coefficients of a regression is routinely reported in regression outputs, along with other key statistics such as R² and t-ratio values.The question is whether it is useful or informative as a key statistic. Does it add any value to your regression results? While it is routinely reported, one may observe that the F-statistic almost always rejects H0 in practical applications. What does it tell us…

A gentle refresher on essential concepts and operations for data sciencePhoto by Viktor Forgacs on UnsplashIntroductionIn my previous post, I introduced some of the operations and concepts that are fundamental to linear algebra. This included vectors and matrices, as well as the transpose, dot product, and matrix multiplication operators. In this post, I’ll introduce some additional concepts that complement those discussed previously. If you haven’t already seen my primer on linear algebra you can check it out here.Linear…

How the humble prediction method shows us the way to Generative AIThere are numerous writings about Generative AI. There are essays dedicated to its applications, ethical and moral issues, and its risk to human society. If you want to understand the technology itself, there is a range of available material from the original research papers to introductory articles and videos. Depending on your current level and interest, you can find the right resources for study.This article is written for a specific class of readers.…

Deep Dive into Multiple Linear Regression with Examples in PythonPhoto by ThisisEngineering RAEng on UnsplashIn the first part of this article we formally defined the linear regression problem and showed how to solve simple linear regression problems, where the data set contains only one feature. In the second part of the article, we will discuss multiple linear regression problems, where the data set may contain any number of features.We will first generalize the closed-form solution we have found for simple linear…

Deep Dive into the Theory and Implementation of Linear Regression ModelsPhoto by Enayet Raheem on UnsplashLinear regression is one of the most basic and commonly used type of predictive models. It dates back to 1805, when Legendre and Gauss used linear regression to predict the movement of the planets.The goal in regression problems is to predict the value of one variable based on the values of other variables. For example, we can use regression to predict the price of a stock based on various economic indicators or the…