Really enjoyed this post. I was wondering, as I read it, "Why not both?" AWS's origins were serving some of Amazon's consumer-facing business lines. Amazon was (is?) AWS's first and best customer, which ensured immediate demand for its most advanced features.
What would prevent a similar framing for OpenAI? Ie consumer-facing offerings in the short term push the backend forward as fast as possible, then rolling those features out at the platform level makes them widely available for recombination by developers. (And in the short term, keeping things more closed allows OpenAI to get a better handle on safety.)
This is a great read, Benn. But the logic breaks down when training and running an LLM stops being as expensive as building data centers :).
I am curious to hear what you think about OSS LLMs like Databrick’s Dolly that can do instruction following, brainstorming, and summarization just like ChatGPT.
BUT….
can be created for $30 using one server for 3 hours on a small dataset using a 2 year old open source base LLM as opposed to training for 100,000 GPU hours at ~$10 million price point.
Hi Benn, I enjoyed reading about your thoughts in this post, and have two questions I hope you can address.
1. Along the lines of what "smakubi" is saying, what about the argument that LLMs will become commodities? The Dolly example is a good one, but even before that, James Currier of NfX wrote about how LLMs are likely to become commoditized and (at least some) open sourced (link here: https://www.nfx.com/post/generative-ai-tech-5-layers). Do you tend to agree with this LLMs as commodities future?
2. On the application layer, our collective imagination seems to be constrained so far, and while the apps are pretty nifty, we haven't seen the truly breakthrough ones. Some have compared our current apps to calculator apps in the early days of iPhone, and the Ubers and DoorDashes are no where to be seen. My view on this is that we will see a big leap in creativity AFTER advancements in hardware like AR/VR/XR and 5G. If we use the GPT as iPhone analogy, then we can also say that few people if any thought they'd disrupt the taxi industry when the first iPhone came out, and it was only after 4G's wider adoption did that become a killer app idea. What are your thoughts on when the truly killer apps will emerge in this new technological shift?
Really enjoyed this post. I was wondering, as I read it, "Why not both?" AWS's origins were serving some of Amazon's consumer-facing business lines. Amazon was (is?) AWS's first and best customer, which ensured immediate demand for its most advanced features.
What would prevent a similar framing for OpenAI? Ie consumer-facing offerings in the short term push the backend forward as fast as possible, then rolling those features out at the platform level makes them widely available for recombination by developers. (And in the short term, keeping things more closed allows OpenAI to get a better handle on safety.)
This is a great read, Benn. But the logic breaks down when training and running an LLM stops being as expensive as building data centers :).
I am curious to hear what you think about OSS LLMs like Databrick’s Dolly that can do instruction following, brainstorming, and summarization just like ChatGPT.
BUT….
can be created for $30 using one server for 3 hours on a small dataset using a 2 year old open source base LLM as opposed to training for 100,000 GPU hours at ~$10 million price point.
Hi Benn, I enjoyed reading about your thoughts in this post, and have two questions I hope you can address.
1. Along the lines of what "smakubi" is saying, what about the argument that LLMs will become commodities? The Dolly example is a good one, but even before that, James Currier of NfX wrote about how LLMs are likely to become commoditized and (at least some) open sourced (link here: https://www.nfx.com/post/generative-ai-tech-5-layers). Do you tend to agree with this LLMs as commodities future?
2. On the application layer, our collective imagination seems to be constrained so far, and while the apps are pretty nifty, we haven't seen the truly breakthrough ones. Some have compared our current apps to calculator apps in the early days of iPhone, and the Ubers and DoorDashes are no where to be seen. My view on this is that we will see a big leap in creativity AFTER advancements in hardware like AR/VR/XR and 5G. If we use the GPT as iPhone analogy, then we can also say that few people if any thought they'd disrupt the taxi industry when the first iPhone came out, and it was only after 4G's wider adoption did that become a killer app idea. What are your thoughts on when the truly killer apps will emerge in this new technological shift?
Thank you
I learned so much in this one today - thanks Benn! Also, the "agents of chaos" being 3 different YouTube video links really got me.
I wonder if OpenAI will switch from non-profit to for-profit