The Pre-Training Wall and the Treadmill After It
Failed to add items
Sorry, we are unable to add the item because your shopping cart is already at capacity.
Add to basket failed.
Please try again later
Add to wishlist failed.
Please try again later
Remove from wishlist failed.
Please try again later
Adding to library failed
Please try again
Follow podcast failed
Unfollow podcast failed
The Pre-Training Wall and the Treadmill After It
-
Narrated by:
-
By:
Summary
I've been confusing Don with frontier-lab links late at night for a bit.
Ilya Sutskever told a NeurIPS audience that pre-training as we know it would unquestionably end. There's only one internet, and the data isn't growing. The frontier labs call this the pre-training wall.
A leaked Google memo from 2023 argued they had no moat. R1 is on GitHub. Llama is on Hugging Face. OpenAI's secondary-market valuation has climbed past $850 billion.
Don was confused. So he came over and we made an episode about it.
- Episode Page
- Support The Show
- Subscribe To The Podcast
- Join The Newsletter
adbl_web_anon_alc_button_suppression_c
No reviews yet