Conor

How to effectively run and debug code in a resource-constrained environmentPhoto by Martijn Baudoin on UnsplashQueue job, wait 24 hours, cuda runtime error: out of memoryQueue job, wait 24 hours, FileNotFoundError: No such file or directoryQueue job, wait 24 hours, RuntimeError: stack expects each tensor…AHHHGH!!!Debugging code on a high-performance computing (HPC) cluster can be incredibly frustrating. To make matters worse, at university you will be sharing resources with other students. Jobs will be added to a queue.…

Build an object detection model, compare it to intensity thresholds, evaluate it and explain it using DeepSHAP(source: author)If the world was less chaotic self-driving cars would be simple. But it’s not. To avoid serious harm, AI has to consider many variables — speed limits, traffic and obstacles in the road (such as a distracted human). AI needs to be able to detect these obstacles and take appropriate actions when encountered.Thankfully, our application is not as complicated. Even more, thankfully, we will be using…

How skip connections allow CNNs to perform accurate semantic segmentation with less data(source: author)U-Net is a popular deep-learning architecture for semantic segmentation. Originally developed for medical images, it had great success in this field. But, that was only the beginning! From satellite images to handwritten characters, the architecture has improved performance on a range of data types. Yet, other CNN architectures can also do segmentation, so what makes U-Net so special?To answer this, we will explore the…

Liam Neeson opened up about how he really feels about UFC fighting — and Conor McGregor. The actor recently sat down with Men’s Health to answer some of the internet’s most burning questions for him, starting with his thoughts on the mixed martial arts fighting. “UFC, I can’t stand,” Neeson said on an episode of “Don’t Read the Comments.” “That, to me, is like a bar fight. I know the practitioners are like, ‘No, you’re wrong — the months of training we do …’ Why don’t you just grab a beer…

How understanding your model can lead to trust, knowledge and better performance in production(source: DALL.E 2)We seem to be in the golden era of AI. Every week there is a new service that can do anything from creating short stories to original images. These innovations are powered by machine learning. We use powerful computers and vast amounts of data to train these models. The problem is, this process leaves us with a poor understanding of how they actually work.Ever increasing abilities? No idea how they work? Sounds…

How to incorporate the scientific process in your data science analysisPhoto by Greg Rakozy on UnsplashBusiness science, sports science, food science… we love throwing the word science behind other words. It makes those fields sound, well, more scientific. Yet, if we are not careful they will have more in common with pseudoscience.Data science is no different.On the surface, it may seem close to science. Both are about seeking knowledge. By nature, machine learning is also a repetitive process. We constantly collect new…

Using DeepShap to understand and improve the model powering an autonomous car(source: author)Autonomous cars terrify me. Big hunks of metal flying around with no humans to stop them if something goes wrong. To reduce this risk it is not enough to evaluate the models powering these beasts. We also need to understand how they are making predictions. This is to avoid any edge cases that would cause unforeseen accidents.Okay, so our application is not so consequential. We will be debugging the model used to power a…

Cropping, grayscale, RGB channels, intensity thresholds, edge detection and colour filters(source: flaticon)With feature engineering, we immediately think about tabular data. Yet, we can also get features for image data. The goal is to extract the most important aspects of the image. Doing so will make it easier to find a mapping between our data and the target variable.This means you can use less data and train smaller models. A smaller model reduces the time needed to make predictions. This is particularly useful when…

Creating a tool that selects the RGB channels from image pixels(source: kaggle)I know, I know… there are loads of colour-picking tools out there. Yet, I think you will find advantages to using one that works directly in a notebook. This tool is also fully customisable.We will build two colour pickers:Simple picker — selects a colour from a single imageComplex picker—selects a list of colours from multiple images & displays the coloursTo end, we discuss some applications in data science. These are ways that I’ve used…

Using Python to augment data by flipping, adjusting brightness, color jitter and random noise(source: author)Data collection can be time-consuming, expensive and, honestly, boring. When our ability to collect data is limited, data augmentation can play an important role. It can help us to build a more robust dataset, reduce overfitting and increase the amount of training data.We will discuss data augmentation and its benefits. We will also keep things practical. That is by walking through Python code used to augment…