Sign up for NVIDIA GTC2025 here!
Join The RTX4080 SUPER Giveaway (enter between March 17-21st)
In this video, you will learn about the fascinating ways of how AI companies can rig LLM benchmarks, for educational purposes of course… Plus the reason why people don’t trust “chatbot arena” as much as more.
My Newletter
My Patreon
I was really vague about the sources in this video to fit it in the narrative, but most facts presented are backed by the following papers
Changing Answer Order Can Decrease MMLU Accuracy
[Paper] arxiv.org/abs/2406.19470
Catch me if you can! How to beat GPT-4 with a 13B model
[Blog]
Idiosyncrasies in Large Language Models
[Paper]
Improving Your Model Ranking on Chatbot Arena by Vote Rigging
[Paper]
This video is supported by the kind Patrons & YouTube Members:
🙏Andrew Lescelius, Ben Shaener, Chris LeDoux, Miguilim, Deagan, FiFaŁ, Robert Zawiasa, Marcelo Ferreira, Owen Ingraham, Daddy Wen, Tony Jimenez, Panther Modern, Jake Disco, Demilson Quintao, Penumbraa, Shuhong Chen, Hongbo Men, happi nyuu nyaa, Carol Lo, Mose Sakashita, Miguel, Bandera, Gennaro Schiano, gunwoo, Ravid Freedman, Mert Seftali, Mrityunjay, Richárd Nagyfi, Timo Steiner, Henrik G Sundt, projectAnthony, Brigham Hall, Kyle Hudson, Kalila, Jef Come, Jvari Williams, Tien Tien, BIll Mangrum, owned, Janne Kytölä, SO, Richárd Nagyfi, Hector, Drexon, Claxvii 177th, Inferencer, Michael Brenner, Akkusativ, Oleg Wock, FantomBloth, Thipok Tham, Clayton Ford, Theo, Handenon, Diego Silva, mayssam, Kadhai Pesalam, Tim Schulz, jiye, Anushka, Henrik Sundt, Julian Aßmann, Thomas Lin, Sid_Cypher, Mark Buckler, Kevin Tai, NO U, Gonzalo Fidalgo, Igor Alvarez, Alon Pluda, Clément Veyssière, Sander Zwaenepoel, etrotta, Binnie Yiu, Matej Macak, c zhou, Berhane-Meskel, sai sandeep mandava, Leo, Asad Dhamani, Charlie C, tantan assawade, Ângelo Fonseca, Stefan Lorenz, Paperboy, mika, Leo, Utsav Soi
[Discord]
[Twitter]
[Patreon]
[Business Inquiries] bycloudai@gmail.com
[Music] Massobeats – Glimmer
[Music] Massobeats – Honey jam
[Music] Massobeats – Lush
[Profile & Banner Art]
[Video Editor] @Booga04
[Bitcoin (BTC)] 3JFMJQVGXNA2HJE5V9qCwLiqy6wHY9Vhdx
[Ethereum (ETH)] 0x3d784F55E0bE5f35c1566B2E014598C0f354f190
[Litecoin (LTC)] MGHnqALjyU2W6NuJSSW9fTWV4dcHfwHZd7
[Bitcoin Cash (BCH)] 1LkyGfzHxnSfqMF8tN7ZGDwUTyBB6vcii9
[Solana (SOL)] 6XyMCEdVhtxJQRjMKgUJaySL8cGoBPzzA2NPDMPfVkKN
[Ko-fi]
