TabTransformer

Deep learning for tabular data with FT-TransformerPhoto by Nick Hillier on UnsplashIn the previous post about TabTransformer I’ve described how the model works and how it can be applied to your data. This post will build on it, so if you haven’t read it yet, I highly recommend starting there and returning to this post afterwards.TabTransformer was shown to outperform traditional multi-layer perceptrons (MLPs) and came close to the performance of Gradient Boosted Trees (GBTs) on some datasets. However, there is one…

Making sense of out TabTransformer and learning to apply itPhoto by Samule Sun on UnsplashToday, Transformers are the key blocks in most of the state-of-the-art Natural Language Processing (NLP) and Computer Vision (CV) architectures. Nevertheless, tabular domain is still mainly dominated by gradient boosted decision trees (GBDT), so it was only logical that someone will attempt to bridge this gap. The first transformer-based models was introduced by Huang et al. (2020) in their TabTransformer: Tabular Data Modeling Using…