Download Tanka today and enjoy 3 months of free Premium!
You can also get $20 / team for each referrals
I’ve been planning for a bitnet video for the longest time, and with the release of bitnet b1.58 2B4T gave me the perfect chance to brief you on the history of 1-bit LLM! Fun fact, the major bitnet research is mostly done by the same researchers.
My Newsletter
my project: find, discover & explain AI research semantically
My Patreon
Quantifying the Capabilities of LLMs across Scale and Precision
[Paper]
BitNet: Scaling 1-bit Transformers for Large Language Models
[Paper]
The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
[Paper]
BitNet a4.8: 4-bit Activations for 1-bit LLMs
[Paper]
Efficient Construction of Model Family through Progressive Training Using Model Expansion
[Paper]
BitNet b1.58 2B4T Technical Report
[Paper]
[Web Demo]
[HuggingFace]
[Code]
[Additional Recs]
T-MAC: CPU Renaissance via Table Lookup for Low-Bit LLM Deployment on Edge
FBI-LLM: Scaling Up Fully Binarized LLMs from Scratch via Autoregressive Distillation
Matmul or No Matmul in the Era of 1-bit LLMs
1-bit AI Infra: Part 1.1, Fast and Lossless BitNet b1.58 Inference on CPUs
Bitnet.cpp: Efficient Edge Inference for Ternary LLMs
Continual Quantization-Aware Pre-Training: When to transition from 16-bit to 1.58-bit pre-training for BitNet language models?
(NEW!) BitNet v2: Native 4-bit Activations with Hadamard Transformation for 1-bit LLMs
(NEW!) BitVLA: 1-bit Vision-Language-Action Models for Robotics Manipulation
Try out my new fav place to learn how to code
This video is supported by the kind Patrons & YouTube Members:
🙏Nous Research, Chris LeDoux, Ben Shaener, DX Research Group, Poof N’ Inu, Andrew Lescelius, Deagan, Robert Zawiasa, Ryszard Warzocha, Tobe2d, Louis Muk, Akkusativ, Kevin Tai, Mark Buckler, NO U, Tony Jimenez, Ângelo Fonseca, jiye, Anushka, Asad Dhamani, Binnie Yiu, Calvin Yan, Clayton Ford, Diego Silva, Etrotta, Gonzalo Fidalgo, Handenon, Hector, Jake Disco very, Michael Brenner, Nilly K, OlegWock, Daddy Wen, Shuhong Chen, Sid_Cipher, Stefan Lorenz, Sup, tantan assawade, Thipok Tham, Thomas Di Martino, Thomas Lin, Richárd Nagyfi, Paperboy, mika, Leo, Berhane-Meskel, Kadhai Pesalam, mayssam, Bill Mangrum, nyaa
[Discord]
[Twitter]
[Patreon]
[Business Inquiries] bycloud@smoothmedia.co
[Profile & Banner Art]
[Video Editor] Abhay
[Ko-fi]
