Furnieles

Understanding Transformer architecture and its key insights in 5 minutesThis is the first part of the article’s extended version, soon you will find its continuation here.Author’s note. For this very first part, I’ve decided to introduce the notions and concepts necessary to get a better understanding of Transformer models and to make it easier to follow the next chapter. If you are already familiar with Transformers, you can check the last section to get a summary of this article and feel free to jump to the second part,…

The math behind two of the most used activation functions in Machine LearningPhoto by Tomáš Malík on Unsplash. Edited by authorThe Sigmoid and SoftMax functions define activation functions used in Machine Learning, and more specifically in the field of Deep Learning for classification methods.Activation function: Function that transforms the weighted sum of a neuron so that the output is non-linearNote. The sigmoid function is also called The Logistic Function since it was first introduced with the algorithm of Logistic…