Transformers for Tabular Data (Part 3): Piecewise Linear & Periodic Encodings | by Anton Rubert | Nov, 2022
Advanced numerical embeddings for better performancePhoto by Pawel Czerwinski on UnsplashThis is the third part in my exploration of Transformers for Tabular Data.In the Part 2 I’ve described linear numerical embeddings and how they are used in the FT-Transformer model. This post is going to explore more complex versions of the numerical embeddings, so if you haven’t read the previous part, I highly recommend starting there and returning to this post afterwards.FT-Transformer. Image by author.As a reminder, above you can…