Training CausalLM Models Part 1: What Actually Is CausalLM?
The first part of a practical guide to using HuggingFace’s CausalLM classCausal langauge models model each new word as a function of all previous words. Source: PexelsIf you’ve played around with recent models on HuggingFace, chances are you encountered a causal language model. When you pull up the documentation for a model family, you’ll get a page with “tasks” like LlamaForCausalLM or LlamaForSequenceClassification.If you’re like me, going from that documentation to actually finetuning a model can be a bit confusing.…