SelfSupervised

How to train models with only a few labeled examplesA large portion of the value delivered by AI so far comes from supervised models trained on increasingly large datasets. Many of these datasets have been labeled by humans, a job that is mundane, time-consuming, error-prone, and sometimes expensive. Self-supervised learning (SSL) is a different learning paradigm allowing machines to learn from unlabeled data. In this article, we will discuss how SSL works and how to apply it to computer vision. We will compare simple…

Distill Fine-Grained Features Without SupervisionFigure 1. Self-supervised learning is an important step to true artifical intelligence. Image retrieved from Unsplash.Previously, I have written several articles briefly discussing self-supervised learning and, in particular, contrastive learning. What was not yet covered, however, was a concurrent branch of self-supervised approach using interactions of multiple networks that seems to emerge and excel recently. As of today, one of the state-of-the-art training methods is a…

Anil Ananthaswamy / Quanta Magazine: Some computational neuroscientists say self-supervised AI learning models have shown a closer correspondence to brain function than supervised-learning models — Self-supervised learning allows a neural network to figure out for itself what matters. The process might be what makes our own brains so successful. Anil Ananthaswamy / Quanta Magazine: Some computational neuroscientists say self-supervised AI learning models have shown a closer correspondence to…

Why It Matters, What It Is & Different Types of Self-Supervised LearningA robot is “self-supervising”. Photo by Brett Jordan on Unsplash.The term “self-supervised learning” came from the quote of Yann Lecun on April 30, 2019 (tweet and post):I now call it “self-supervised learning”, because “unsupervised” is both a loaded and confusing term.In this post, I will explain what it is, why it is important, how it can be used and different categories of self-supervised learning in multiple domains including text, image,…

Exploiting longitudinal measurements to learn better representationsLongitudinal Self-Supervised Learning is an exciting new avenue for learning on large datasets where repeated measures are taken. This is especially fitting for the medical domain (imaging, tabular health record data, etc.), where patients may have several data points over months or years but not all have exact labels (e.g. presence of a specific disease).Photo by Aron Visuals on UnsplashSelf-supervised learning (SSL) has been one of the hottest areas of…