While ChatGPT and similar tools have monopolized much of our collective attention in recent months, large language models (LLMs)—the infrastructure behind all those chat outputs we endlessly share, screenshot, and occasionally shake our heads at—have remained in (relative) obscurity.
On a certain level, it makes sense: when you watch a show on your laptop, you focus on the plot and the characters, not on the electric grid that makes those moving pixels possible. If you’re eating a cookie right now (like… some TDS editors, most likely), you’re probably thinking about the flavors and textures you’re enjoying rather than, say, about wheat farmers or the history of cocoa beans.
As data professionals, however, we can only benefit by developing a deeper and more nuanced understanding of LLMs. There’s a professional angle to this, of course, but also a more amorphous pleasure of knowing how something complex and magic-like actually functions behind the curtain. To give you a leg up, we’ve assembled a stellar lineup of recent articles that look into the past, present, and future of LLMs—and tackle both their incredible abilities and nontrivial limitations. Enjoy!
- Machine learning research in a post-ChatGPT world. The inner workings of some of the most ubiquitous LLMs (like OpenAI’s GPT models) remain tightly guarded by corporate entities. What does that mean for NLP researchers whose future projects are likely to hit a proprietary wall sooner or later? Anna Rogers proposes a powerful idea: “That which is not open and reasonably reproducible cannot be considered a requisite baseline.”
- Assessing the impact of recent advances in conversational AI. The size and power of LLMs have made it possible for human-machine interaction to take a giant leap in mere months. Gadi Singer reflects on how we got here—and on what is still missing before the machines start communicating in a human-like manner.
- Welcome to a new ethical (and practical) quagmire. LLMs require massive amounts of training data, which has turned out to be a major vector of risk. These models are currently facing intense scrutiny for the potential presence of biased, private, or harmful text and the unauthorized inclusion of copyrighted material. What are users to do in the meantime? Lingjuan Lyu’s debut TDS article presents a thoughtful overview of the stakes and dangers we should all be aware of if we’re to produce AI-generated content responsibly.
- The wide horizon of LLM-powered tools. Just a few months ago, the LangChain library was flying under most practitioners’ radars. Well, not anymore: it’s become a go-to resource for many tinkerers who want to leverage LLMs to build new apps. Dr. Varshita Sher’s latest deep dive is a helpful, hands-on introduction to the library’s core building blocks.
- Identifying drift and detecting anomalies with LLMs. As the novelty of generating clunky poems with ChatGPT fades, fresh use cases continue to emerge — and many of them might end up streamlining data science workflows. Case in point: Aparna Dhinakaran, Jason Lopatecki, and Christopher Broan’s latest post, which outlines a promising approach for using using LLM embeddings for anomaly and drift detection.
- Bonus read: go one level deeper. If LLMs make user-facing applications like ChatGPT possible, transformer neural networks are the architecture that made LLMs possible in the first place. To get your bearings around this crucial (and often complex) topic, explore Soran Ghaderi’s detailed “map” of past and current transformers research.
While ChatGPT and similar tools have monopolized much of our collective attention in recent months, large language models (LLMs)—the infrastructure behind all those chat outputs we endlessly share, screenshot, and occasionally shake our heads at—have remained in (relative) obscurity.
On a certain level, it makes sense: when you watch a show on your laptop, you focus on the plot and the characters, not on the electric grid that makes those moving pixels possible. If you’re eating a cookie right now (like… some TDS editors, most likely), you’re probably thinking about the flavors and textures you’re enjoying rather than, say, about wheat farmers or the history of cocoa beans.
As data professionals, however, we can only benefit by developing a deeper and more nuanced understanding of LLMs. There’s a professional angle to this, of course, but also a more amorphous pleasure of knowing how something complex and magic-like actually functions behind the curtain. To give you a leg up, we’ve assembled a stellar lineup of recent articles that look into the past, present, and future of LLMs—and tackle both their incredible abilities and nontrivial limitations. Enjoy!
- Machine learning research in a post-ChatGPT world. The inner workings of some of the most ubiquitous LLMs (like OpenAI’s GPT models) remain tightly guarded by corporate entities. What does that mean for NLP researchers whose future projects are likely to hit a proprietary wall sooner or later? Anna Rogers proposes a powerful idea: “That which is not open and reasonably reproducible cannot be considered a requisite baseline.”
- Assessing the impact of recent advances in conversational AI. The size and power of LLMs have made it possible for human-machine interaction to take a giant leap in mere months. Gadi Singer reflects on how we got here—and on what is still missing before the machines start communicating in a human-like manner.
- Welcome to a new ethical (and practical) quagmire. LLMs require massive amounts of training data, which has turned out to be a major vector of risk. These models are currently facing intense scrutiny for the potential presence of biased, private, or harmful text and the unauthorized inclusion of copyrighted material. What are users to do in the meantime? Lingjuan Lyu’s debut TDS article presents a thoughtful overview of the stakes and dangers we should all be aware of if we’re to produce AI-generated content responsibly.
- The wide horizon of LLM-powered tools. Just a few months ago, the LangChain library was flying under most practitioners’ radars. Well, not anymore: it’s become a go-to resource for many tinkerers who want to leverage LLMs to build new apps. Dr. Varshita Sher’s latest deep dive is a helpful, hands-on introduction to the library’s core building blocks.
- Identifying drift and detecting anomalies with LLMs. As the novelty of generating clunky poems with ChatGPT fades, fresh use cases continue to emerge — and many of them might end up streamlining data science workflows. Case in point: Aparna Dhinakaran, Jason Lopatecki, and Christopher Broan’s latest post, which outlines a promising approach for using using LLM embeddings for anomaly and drift detection.
- Bonus read: go one level deeper. If LLMs make user-facing applications like ChatGPT possible, transformer neural networks are the architecture that made LLMs possible in the first place. To get your bearings around this crucial (and often complex) topic, explore Soran Ghaderi’s detailed “map” of past and current transformers research.