Richard Sutton, the author of The Bitter Lesson, explains why large language models may not truly embody his idea of scalable AI — and why methods that learn directly from experience will eventually win.
“People get locked into the human knowledge approach… and their lunch gets eaten by the methods that are truly scalable. The scalable method is you learn from experience. You try things, you see what works — no one has to tell you.”
Richard Sutton argues that next-token prediction isn’t the same kind of prediction that truly intelligent beings make about the world.
"Something is intelligent if it can achieve goals. If you’re just sitting there predicting and being happy with yourself that you’re predicting accurately—you’re not intelligent."
“Gradient descent will not make you generalize well. It will make you solve the problem… but there’s nothing in it that will cause you to generalize well.” — Richard Sutton
Jacob Kimmel describes how Shinya Yamanaka’s Nobel Prize winning experiment was shockingly straightforward — you could literally see the results with your naked eye.
Honestly the thing that motivated me to do this episode was learning that there's less than $200M/year of smart philanthropy on factory farming - GLOBALLY.
Just to explain how crazy that is:
1. It's insane how cheap the interventions that will spare BILLIONS of animals from gruesome, painful fates have been.
For example, less than $200M has been spent getting corporate commitments that have already spared more than 400M hens from battery cages, and securing pledges that will spare billions more over the years to come. That’s < $1 per 10 years of animal well-being improved!
Another example: In-ovo sexing (which determines the sex of eggs pre-birth) has already saved 200M male chicks from maceration at birth (with the potential to spare 7 billion every year). And it only cost ~$10M to get off the ground.
2. 80 billion land animals are factory farmed every year. That means the ratio is $1 donated : 40,000 animals.
3. Compared to the amount of private philanthropy alone on global health ($50b+/year) or climate change ($15b+/year), the <$200M/year of smart money spent on factory farming is nothing.
The way we treat factory farmed animals is one of the worst atrocities in history. And unfortunately, the problem is on track to get worse every year.
There is a case for optimism though: Given how neglected this issue is, the scope of impact even one individual can have is absolutely massive.
I decided to give $250,000 as a donation match to FarmKind after learning about the outsized opportunities to help. It’s truly insane that just $1 can help avert 10 years of farmed animal suffering.
If you have the means, please consider contributing, even if it’s just a small amount. We can double each other’s impact and give a total of $500,000 together.
“The Russians had this fancy fleet that their aristocrats wanted to buy—they were good at cutting checks. But they weren’t doing any training on it. If you look at days at sea for the Russians, it’s laughable.
Meanwhile, the Japanese are constantly deploying their forces. They’re learning, doing all kinds of practice. So you can imagine—if you don’t do any practice, it’s not going to go very well for you.”
Dwarkesh Patel
Dwarkesh Patel
Tomorrow
2 weeks ago | [YT] | 1,097
View 63 replies
Dwarkesh Patel
Richard Sutton, the author of The Bitter Lesson, explains why large language models may not truly embody his idea of scalable AI — and why methods that learn directly from experience will eventually win.
“People get locked into the human knowledge approach… and their lunch gets eaten by the methods that are truly scalable. The scalable method is you learn from experience. You try things, you see what works — no one has to tell you.”
1 month ago | [YT] | 90
View 0 replies
Dwarkesh Patel
“Succession to digital intelligence or augmented humans is inevitable.”
1 month ago | [YT] | 23
View 2 replies
Dwarkesh Patel
Richard Sutton argues that next-token prediction isn’t the same kind of prediction that truly intelligent beings make about the world.
"Something is intelligent if it can achieve goals. If you’re just sitting there predicting and being happy with yourself that you’re predicting accurately—you’re not intelligent."
1 month ago | [YT] | 48
View 9 replies
Dwarkesh Patel
“Gradient descent will not make you generalize well. It will make you solve the problem… but there’s nothing in it that will cause you to generalize well.” — Richard Sutton
1 month ago | [YT] | 95
View 10 replies
Dwarkesh Patel
Jacob Kimmel describes how Shinya Yamanaka’s Nobel Prize winning experiment was shockingly straightforward — you could literally see the results with your naked eye.
2 months ago | [YT] | 64
View 2 replies
Dwarkesh Patel
Pretended to know things about AI with @ChrisWillx. Go watch!
https://youtu.be/RXcYIae6TH8?si=db43x...
2 months ago | [YT] | 70
View 3 replies
Dwarkesh Patel
Honestly the thing that motivated me to do this episode was learning that there's less than $200M/year of smart philanthropy on factory farming - GLOBALLY.
Just to explain how crazy that is:
1. It's insane how cheap the interventions that will spare BILLIONS of animals from gruesome, painful fates have been.
For example, less than $200M has been spent getting corporate commitments that have already spared more than 400M hens from battery cages, and securing pledges that will spare billions more over the years to come. That’s < $1 per 10 years of animal well-being improved!
Another example: In-ovo sexing (which determines the sex of eggs pre-birth) has already saved 200M male chicks from maceration at birth (with the potential to spare 7 billion every year). And it only cost ~$10M to get off the ground.
2. 80 billion land animals are factory farmed every year. That means the ratio is $1 donated : 40,000 animals.
3. Compared to the amount of private philanthropy alone on global health ($50b+/year) or climate change ($15b+/year), the <$200M/year of smart money spent on factory farming is nothing.
The way we treat factory farmed animals is one of the worst atrocities in history. And unfortunately, the problem is on track to get worse every year.
There is a case for optimism though: Given how neglected this issue is, the scope of impact even one individual can have is absolutely massive.
So, if you have the means, please go to farmkind.giving/dwarkesh to donate.
2 months ago | [YT] | 194
View 21 replies
Dwarkesh Patel
I decided to give $250,000 as a donation match to FarmKind after learning about the outsized opportunities to help. It’s truly insane that just $1 can help avert 10 years of farmed animal suffering.
If you have the means, please consider contributing, even if it’s just a small amount. We can double each other’s impact and give a total of $500,000 together.
Go to farmkind.giving/dwarkesh to donate with my match.
2 months ago | [YT] | 240
View 24 replies
Dwarkesh Patel
“The Russians had this fancy fleet that their aristocrats wanted to buy—they were good at cutting checks. But they weren’t doing any training on it. If you look at days at sea for the Russians, it’s laughable.
Meanwhile, the Japanese are constantly deploying their forces. They’re learning, doing all kinds of practice. So you can imagine—if you don’t do any practice, it’s not going to go very well for you.”
3 months ago | [YT] | 340
View 34 replies
Load more