What can and can't language models do? Lessons learned from BIGBench
Por um escritor misterioso
Descrição
So what exactly can and can’t language models do? What's the least impressive thing GPT-4 won't be able to do? What will GPT-4 be incapable of?
BIGBench is kind of a way to figure this out. BigBench, aka “The Beyond the Imitation Game” Benchmark, is an attempt to explore the capabilities of large language models over a wide variety of tasks. All the tasks are enumerated here.
I looked through every BIGBench task and took the ones that compared both GPT3 and PaLM against humans.
* Spreadsheet
Key Takeaways from NeurIPS 2022 Top Papers
Xinyun Chen (@xinyun_chen_) / X
Generative AI and large language models: background and contexts
New Benchmarks Test the Limits of Large Language Models
Emergent Abilities in AI: Are We Chasing a Myth?
Frontiers Language models and psychological sciences
Language Models Perform Reasoning via Chain of Thought – Google
PDF) Language Models Don't Always Say What They Think: Unfaithful
R] 85% of the variance in language model performance is explained
The Flan Collection: Advancing open source methods for instruction
444 Authors From 132 Institutions Release BIG-bench: A 204-Task