annual letter 2024: the coral reef of instructing llamas

25 Dec, 2024

Huge congrats to @AIatMeta on the Llama 3.1 release!
Few notes:

Today, with the 405B model release, is the first time that a frontier-capability LLM is available to everyone to work with and build on. The model appears to be GPT-4 / Claude 3.5 Sonnet grade and the weights are…

— Andrej Karpathy (@karpathy) July 23, 2024

Contents

• llama, the application
• tinygrad, the language
• vllm, the runtime

llama, the application

The second year after ChatGPT's initial release has passed, and it was a big one for open source.

llama2 (mid 2023): https://arxiv.org/pdf/2307.09288 llama3 (2024): https://arxiv.org/pdf/2407.21783 zuck: https://www.youtube.com/watch?v=bc6uFV9CJGg

tinygrad, the language

vllm, the runtime

big picture scaling is done (ilya neurips): https://www.youtube.com/watch?v=1yvBqasHLZs

next; qstar