Scaling ChatGPT: Inside OpenAI's Rapid Growth and Technical Challenges
Plus, setting your goals for 2024,the role of clarity for high-performance teams, and how Section 174 will impact US startups
What can you learn from the scaling issues OpenAI experienced when Chat-GPT went viral?
On this week’s episode, guest host
is joined by Evan Morikawa, Engineering Manager at OpenAI. Join us for a first-hand look at the engineering challenges that came with Chat-GPT’s viral success, and the difficulties associated with scaling in response to the sudden platform popularity.They also discuss misconceptions around generative AI, OpenAI’s reliance on GPUs to carry out their complex computations, the key role of APIs in their success, and some fascinating use cases they’ve seen implementing GPT-4.
“It took a lot of tweaking to kind of find what the right utilization metrics were and then how to optimize those.
But for us, everything was framed in terms of every improvement that we have represents more users we could let onto the platform. As opposed to saying, ‘Oh it's driving down our margins or making it faster,’ we kept cost and latency relatively fixed and we got to get more users onto the system.”
Episode Highlights:
2:57 When ChatGPT Was A Single Developer Facing API
4:50 ChatGPT’s Viral Launch and the Scaling Issues That Followed
9:37 Misconceptions About AI
12:36 Ideal Use Cases for Generative AI
17:17 GPUs: What are They and Why Does OpenAI Need So Many
25:58 Scaling ChatGpt
34:04 APIs Role in the Success of OpenAI
38:40 Innovative Applications Using OpenAI’s Products
The Download
The Download is engineering leadership content we’re reading, watching, and attending that we think you might find valuable.
1. Need help setting your goals for 2024?
Amidst the stress of people finally “circling back,” it’s easy to delay the goal-setting session you need to have to set yourself up for success this year.
2. Don’t let a lack of clarity get in the way of your team’s performance.
If your team seems stuck, morale feels low, and people are unsure what to work on, you’re likely experiencing an absence of clarity.
’s article in points out the three areas to drive clarity for sustained high-performance teams. This framework and guided questions will help you define clarity for yourself, and your team going into the new year.“When a team can’t understand why they’re doing what they’re doing, it becomes a losing battle.”
3. How Section 174 will impact US startups
Until the last week, there’s been little discussion of a major pain point for R&D teams at US companies - the impact of an IRS tax code change in Section 174. This change gets rid of the ability for businesses to deduct R&D as an expense, massively impacting the financial viability of software startups and R&D in the United States.
Measure and Understand the Impact of GenAI on Software Development (Sponsor)
Uncover the transformative impact of GenAI on software delivery processes and the metrics used to measure that impact in this must-attend workshop for engineering leaders.
Workshop Topics Include:
In-depth analysis from LinearB’s GenAI Impact Report.
Success stories: How leaders are leveraging GenAI in their SDLC.
Key metrics: Adoption rates, benefits, and risk assessment.
Live Demo: Practical steps to measure your GenAI project’s impact.
Understand and quantify the real impact of Generative AI in your projects and elevate your strategy for the new year - join us January 25th or 30th.
4. How are you enabling your team?
Have you been following our short videos sharing engineering and leadership insights? Check out this video with
and Gene Kim on how to build high-performing teams, and head to our YouTube to watch more of our shortsUpcoming Events
January 25, 30 | Online
Highlights: GenAI is rapidly changing the software development landscape, but how do you measure its impact on your engineering org and the potential ROI? LinearB's upcoming digital workshop will help shine a light on the metrics you can use to measure the impact of your GenAI initiative.