Linear

The most important, overlooked, and predictable regression model.Linear Regression Image — By AuthorLinear regression is a simple, yet powerful tool that should be in every data scientist’s back pocket. In this article, I will use a real-world example to showcase the power of linear regression on real-world data.CalCOFI DatasetWe are going to be working with the CalCOFI Dataset. This dataset contains comprehensive oceanographic measurement data from California from 1949 up to the present day. The dataset includes features…

A mathematical technique to Machine LearningPhoto by Isaac Smith on UnsplashWith graduation season just around the corner, I’ve had a few family members ask how often they would use the information they learned over the years. One cousin, in particular, was not a huge fan of math. However, he built his gaming PC and otherwise enjoys learning about computer hardware. At one point it was mentioned that computers will do all the necessary math, so why memorize formulas? While I saw his perspective to a degree, math is a…

Understand OLS Linear Regression with a bit of mathImage by AuthorHeteroskedasticity and Autocorrelation are unavoidable issues we need to address when setting up a linear regression. In this article, let’s dive deeper into what are Heteroskedasticity and Autocorrelation, what are the Consequences, and remedies to handle issues.A typical linear regression takes the form as follows. The response variable (i.e., Y) is explained as a linear combination of explanatory variables (e.g., the intercept, X1, X2, X3, …) and ε is…

The connection between probability definition and machine learning explanationAre variance and bias of an estimator mathematical definitions and machine learning explanations reconcilable? Source: Robot hand photo created by rawpixel.com — www.freepik.comI had trouble to clearly understand the connection between the definitions of the variance and the bias of an estimator in my probability courses and the explanation from the machine learning courses I then followed. In the first case, the mathematical definitions are…

Understanding linear regression mechanics via the Frisch-Waugh-Lovell TheoremPhoto by AuthorApplied econometrics is generally interested in establishing causality. That is, what is the “treatment” effect of T on some outcome y. In a simple bivariate case, we can imagine randomly assigning treatment T=1 to some individuals and T=0 to others. This can be represented by the following linear regression model:(1)If we assume the treatment is truly randomly assigned, then T is orthogonal to the error term or, in the economists…

Understand OLS Linear Regression with a bit of mathImage by AuthorThe OLS estimator is known to be unbiased, consistent and BLUE (Best Linear Unbiased Estimator). But what do these properties mean? Why are they important for a linear regression model? In this article, we will discuss these properties.A typical linear regression looks like something as follows. The response variable (i.e., Y) is explained as a linear combination of explanatory variables (e.g., the intercept, X1, X2, X3, …) and ε is the error term (i.e., a…