Tuning-Free Longer Context Lengths For LLMs — A Review of Self-Extend (LLM Maybe LongLM)
Tuning-Free Longer Context Lengths For LLMs — A Review of Self-Extend (LLM Maybe LongLM)A simple strategy to enable LLMs to consume longer context length inputs during inference without the need for finetuning.Source: Image created by the author using DALL-E 3In this article we will look at the paper “LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning” by Hongye Jin et. al. that was just released on arxiv 2 days ago (2nd Jan 2024).Github: https://github.com/datamllab/LongLMLLMs like GPT-3 or BERT are…