Bex

Let’s catch those high-dimensional outliersPhoto by João JesusWe’ve all used those simple techniques — plot a scatterplot or a KDE, and the data points farthest from the group are outliers. Now, tell me — how would you use these methods if you were to find outliers in, say, 100-dimensional datasets? Right off the bat, visual outlier detection methods are out of the question.So, fancy machine learning algorithms like Local Outlier Factor or Isolation Forest come to mind, which are effective against outliers in…

Get that model onlinePhoto by Suzy HazelwoodIntroductionPeople say that the best thing about Python is the abundance of its open-source libraries. I beg to differ. Having many alternatives for doing a single task might give you a lot of flexibility, but they might just end up creating a giant mess that leaves programmers confused and angry.Right now, it is the case regarding Machine Learning Operations (MLOps). It has become such an essential field in recent years that the Python community has risen to the occasion…

ExcelWriter, factorize, explode, squeeze, T, mask, idxmax, clip, …Photo by Caleb Oquendo from Pexels“I wish I could do this operation in Pandas….”Well, chances are, you can!Pandas is so vast and deep that it enables you to execute virtually any tabular manipulation you can think of. However, this vastness sometimes comes at a disadvantage.Many elegant, advanced features that solve rare edge cases, and unique scenarios are lost in the documentation, shadowed by the more frequently used ones.This article aims to rediscover…

Learn new packages like never beforeAmong productivity gurus, there is a term called “flow state.” It happens when you concentrate on what you are doing so hard that you go into some type of a trance. You can’t keep your fingers away from the keyboard. They start working on their accord, producing your best quality code without thinking for hours at a time.You are doing what’s called “deep work” — a period of time where you are most productive and creative and in the “flow.” The more times you go into a flow during a…

Get rich in NumPyPhoto by Ashin K Suresh on UnsplashEvery data scientist admires someone. For some, it might be people who create killer data visualizations; for others, it is simply anyone who answers their StackOverflow questions. For me, it was people who used NumPy like a ninja.I don't know. I have always thought that the ability to use a long forsaken function buried deep inside the documentation on rare edge cases spoke a lot about a programmer's skill. Reinventing the wheel for a particular task is challenging, but…

Get to know Python's standard libraries like never beforePhoto by DidsIntroductionMost people think Python's mass dominance is due to its powerful packages like NumPy, Pandas, Sklearn, XGBoost, etc. These are third-party packages written by professional developers, often with the help of other faster programming languages like C, Java, or C++.So, one of the feeble arguments haters might throw against Python is that it won't be as popular once you strip away all the glory these third-party packages bring. I am here to say…

Learn to control model output based on what's important to the problem using a confusion matrixMaster the fundamentals of the confusion matrix using Sklearn and build a practical intuition for three of the most common metrics used in binary classification: precision, recall, and F1 score.Photo by Thomas Skirde on PixabayIntroductionClassification is a massive part of machine learning. Its benefits and applications are endless — ranging from detecting new asteroids and planets to identifying cancerous cells, all are done…

All CV procedures you need to know as a data scientist, explainedPhoto by Anni RoenkaeUntil I start selling the related merchandise, I gotta advertise the main idea. Here it goes.Let’s imagine a world where you don’t know what a cross-valıdatıon procedure is. In that crazy world, you obviously split your data into a single train and test sets. The model learns from the training data, and you test its performance by predicting on the so-called unseen data that is your test set. If you are not satisfied with the score, you…

Use Matplotlib like never beforePhoto by Ruthson Zimmerman on UnsplashWhen I was a beginner learning data visualization, I used to have a rule — never, ever visit the Matplotlib documentation.Nothing good ever came out of there. When I was on its pages, my eyes would process English words but, somehow, they were interpreted into my brain like I was deciphering a foreign language. Then I would stumble upon some masterpieces like these, and I would think, do I even use the same library?Even if you are a seasoned programmer,…

Hidden treasuresPhoto by Jonny LewI hated Sklearn. The “first introduction to ML” course I took explained SVMs so badly that I judged the entire library based on that. I abandoned the course and ML for good for a few months after that.Little did I know that Sklearn would turn out to be my favorite library in the vast data science stack. I also didn’t know that I would be writing 22 articles about it. This is my 23rd. The last one and the one before that were flying successes. Thousands have read them.I am writing this one…