Nvidia: Subject to Disruption
Q3 2024 ER Update
This newsletter is sponsored by me. This week I’m officially launching my 2 Hour Deep Diver course, following a successful pre-launch with tens of satisfied students at present.
By completing the course, you will walk away with the exact framework that I use to analyze companies and write deep dives, for just $150 and in under two hours. You will learn mental models that will help you spot multi-baggers early and know when to buy/sell a stock, among other valuable skills.
Edited by Brian Birnbaum.
1.0 An Innovator’s Dilemma
Although Nvidia’s Q3 results are astounding, the company faces the risk of being unsettled by a tectonic shift that is currently underway in the semiconductor industry.
Nvidia is currently riding an exponential growth curve. Reading through Microsoft’s Q3 results, I see that AI copilots are quickly becoming a fundamental utility for millions of people across the globe.
This is translating into an exponential increase in the demand for AI training and inference. Nvidia is a privileged benefactor of this, as the until recently sole provider of top performing GPUs.
For example, Microsoft’s Github Copilot now has more than one million paid users. When they code, the copilot makes suggestions which developers can then integrate into their code by hitting tab.
That’s all driven by GPUs.
Consumer Internet companies and enterprises drove exceptional sequential growth in Q3, comprising approximately half of our Data Center revenue and outpacing total growth.
-Nvidia CFO Colette Kress, during the Q3 FY2024 conference call.
Nvidia Compute and Networking revenue is up to $14.64B in Q3 FY2024 from $3.82B in Q3 FY2023. That is a whopping 284% increase, primarily driven by growth in Nvidia’s Data Center business, with revenue quadrupling YoY.
Copilots make the underlying tools far more effective for users. As a result, those that do not engage with the copilots stand at at tremendous disadvantage in terms of their productivity.
The AI revolution has started with copilots, but it is unlikely to end there.
As I have previously outlined, as we add more parameters to the LLMs (large language models) that power these models, new features emerge that were not explicitly coded for.
Inference is contributing significantly to our data center demand, as AI is now in full production for deep learning, recommenders, chatbots, copilots and text to image generation and this is just the beginning.
-Nvidia CEO Jensen Huang, during the Q3 FY2024 conference call.
This property makes it very likely that soon new AI applications will emerge, driving further demand for compute. But, there’s a caveat for Nvidia.
The industry is running up against the litographic reticle limit, whereby producing cutting edge chips is getting exponentially more expensive and even physically impossible.
The way around said limit is stitching together many smaller chips (chiplets), to create one big chip. This drastically decreases the complexity of making cutting edge chips, by enabling far higher yields.
This dynamic has been silently re-shaping the industry for almost a decade now and the broader market is only starting to realize it. Chiplets ultimately enable compute power on par with that of monolithic chips, but at a lower cost.
As Nvidia remains focused on monolithic chips (chips that are made from one big piece of silicon), it stands the risk of getting disrupted by players with far more experience in chiplets.
2.0 A Probabilistic Angle
Although Nvidia is displaying no signs of defense, I refuse to believe that a company of this caliber will go down easily.
From a fundamental point of view Nvidia is one of the world’s top organizations. Nvidia has just 28,000 employees and it serves pretty much everyone.
The odds that they allow themselves to be disrupted are low, but every quarter that goes by we see the following:
Nvidia’s competitor AMD making strides in the GPU space, with their chiplet-based GPUs.
Nvidia making no explicit mention of this dynamic and instead, carrying on down the monolithic route.
If my understanding of the litographic reticle limit is correct, which I believe it is, Nvidia will eventually be forced to pivot to chiplets and this will not be easy for them.
I've often heard people saying that pivoting to chiplets can be done quickly. But if you begin to unpack the production process, it's apparent that complexity is high.
For instance, testing individual chiplets (bare die) before packaging is crucial for chiplet-based systems, but it presents new challenges compared to testing packaged parts:
Probing: Fine pads on bare die require specialized probing techniques.
Self-sufficiency: Each chiplet needs built-in testing capabilities (clock, etc.) and cannot rely on other chiplets during testing.
Value at risk: Faulty chiplets in a packaged system can render the entire package unusable, wasting valuable resources. If you get testing slightly wrong, your yield quickly declines, which makes using chiplets in the first place pointless.
With chiplets, a small error at the testing level thus scales terribly.
Getting all the parts of the chiplet value chain is in fact a process moat: you can only get it right by going through many production cycles, seeing what goes wrong in each one of them, fixing the issues and learning from them.
No matter how smart you are, this takes time.
Therefore, going forward there is a very finite set of possibilities:
My understanding of Moore’s Law is totally wrong.
If it’s not, Nvidia is working on chiplets in secret.
If neither of the above are true, Nvidia is set to lose its dominance in the GPU space.
3.0 Nvidia’s Moat is Strong
Nvidia’s moat is very strong and even with a superior price/performance ratio, it may take a long time for AMD to erode it.
In Q3, the US government placed a set of export restrictions to China which made a severe dent in Nvidia’s Data Center business. The products subject to said regulations have ‘consistently contributed approximately 20% to 25% of Data Center revenue over the past few quarters’.
Unless one digs into the quarterly conference transcript, this hindrance is barely perceptible in the top line numbers.
The Data Center business has evolved spectacularly YoY, because ultimately the world’s thirst for computation can only be satisfied by a small list of companies.
If indeed AMD does take some share away from Nvidia in the GPU space, it is likely for it not to matter to Nvidia shareholders over the long term.
The increase in compute demand with the advent of LLMs has been marvelous, but with the universe being computable, we are only just scratching the surface. In other words, a rising tide lifts all boats.
Further, Nvidia has been leading the GPU space for 20 years, while AMD has only just gotten started. This has yielded an enormous installed base, which includes many millions of GPUs in the cloud and in people’s PCs.
Throughout this time, Nvidia has been doing the same with its GPUs as Tesla has been doing with its cars: fostering a retro-compatible architecture, enabled by software.
Nvidia’s CUDA (a software which enables developers to seamlessly interact with Nvidia GPUs) is effectively a network which ties together all of these seemingly disparate hardware units.
Every iteration of the software accrues to an ever larger installed base, which creates an ecosystem that draws talent towards it. The more people that use Nvidia GPUs, the more valuable each GPU becomes.
Additionally, Nvidia continues to improve its software at breakneck speed. This quarter it released TensorRT-LLM, which allegedly ‘without anybody touching anything, improves the performance [of a GPU] by a factor of two.’
During the quarter, Nvidia also announced the launch of the latest member of the Hooper family, the H200. It increases inference speed by a factor of two, with respect to the H100.
Thus, the combination of hardware and software has enabled Nvidia to increase the performance of its GPUs by a factor of 4 in a year. This would’t be possible without the software.
Additionally, Pandas (the world’s most popular data science framework) is now accelerated by Nvidia CUDA without a single line of code, thanks to the recently launched cuDF Pandas.
For AMD to truly disrupt Nvidia, therefore, it has to create a similar ecosystem and not just bring to the market products with superior price/performance specs.
Meanwhile, AMD is progressing fast on its CUDA equivalent (ROCm) and its GPUs are definitely coming in hot.
4.0 The Pursuit of Higher Levels of Software Abstraction
Nvidia is dabbling in a space which makes total strategic sense, but that may lead the company to lose (further?) sight of its hardware business.
In my original Nvidia deep dive I explain how CUDA radically increased Nvidia’s operating leverage when it was launched and how, a new software ‘feature’ of this sort could again revamp Nvidia’s ability to make money.
In Q3 I see Nvidia making considerable progress in that direction, with Nvidia on track to exit FY2024 with its Software and Services business at an annualized run rate of $1B.
Bu, Nvidia is operating in this domain in a way that resembles manual consulting.
And so our business model is help you create your custom models, you run those custom models on NVIDIA AI Enterprise. And it's off to a great start. NVIDIA AI Enterprise is going to be a very large business for us.
-Nvidia CEO Jensen Huang, during the Q3 FY2024 conference call.
And indeed, said consulting activity sets the company on a collision course with Palantir, Microsoft, UiPath and others:
[…] there is a lot of different ways that they could take a -- create their own business model, but ours is basically like a software license, like an operating system.
-Nvidia CEO Jensen Huang, during the Q3 FY2024 conference call.
Thus, while the competitive environment is looking somewhat convoluted at the hardware level, Nvidia is making strides into a highly competitive space.
The company makes explicit mention of its desire to help customers with the workloads prior to actually loading data and AI models on a GPU, but it makes no mention of the industry getting near the limit of Moore’s Law.
I am genuinely puzzled.
5.0 The Networking Business is Progressing Excellently
Nvidia’s networking business now exceeds a $10B annualized run rate, which is giving the company an edge in the data center.
In my AMD deep dive, I explain the concept of Gen 4 datacenters and how they are a requirement to bring AI to the world at scale. By becoming an indispensable part of these datacenters at the networking level, semiconductor companies can gain an additional moat.
Gen 4 are essentially stateful, meaning that they hold data about their state at all times and can use it to train AI models, so the datacenter gains autonomy.
For a datacenter to be stateful, it has to move data around the place very efficiently.
Nvidia’s acquisition of Mellanox in 2020 enabled it to onboard two key technologies:
The BlueField DPU: a data processing unit (DPU) is a specialized processor designed to offload networking, storage, and security tasks from general-purpose CPUs, that enables datacenters to hold information about themselves.
Infiniband: a high-performance networking technology that provides ultra-low latency, high bandwidth, and scalable connectivity for data centers. It is a key enabler for high-performance computing (HPC), artificial intelligence (AI), and other demanding workloads that require fast and efficient data transfer.
While AMD is pursuing a similar roadmap with the acquisition of Pensando, I see no particular progress made on this front.
On the other hand, Nvidia’s networking business now exceeds a $10 billion annualized revenue run rate, ‘driven by exceptional demand for InfiniBand, which grew fivefold year-on-year.’
AMD is nowhere near Nvidia on the networking side.
6.0 Financials
Income Statement
Nvidia reports three business segments:
Graphics: This segment is Nvidia's core business and includes the development and sale of graphics processing units (GPUs) for gaming, professional visualization, and data centers.
Compute & Networking: This segment includes the development and sale of high-performance computing (HPC) and networking solutions for data centers and other enterprise environments.
All Other: This segment includes revenue from a variety of products and services that don't fit neatly into the other two segments. These products and services include semiconductor foundry services, automotive solutions, professional visualization, and other miscellaneous products and services.
As mentioned, datacenter revenue (GPUs) came in at a record $14.5B, up 41% sequentially and up 279% YoY.
Gaming revenue came in at $2.86B, up 15% sequentially and up more than 80% YoY. Revenue has doubled relative to pre-Covid levels.
Networking now exceeds a $10B annualized run rate and software is on track to exit the year with a $1B annualized run rate. Automotive revenue came in at $261M, up 33% sequentially and up 4% YoY.
Nvidia’s margins have been expanding considerably with the onset of LLMs, as there is a scarcity of GPU engines worldwide.
Margins have also been aided by the company getting leaner, especially over the last two quarters:
As a result of the above, net income has evolved exponentially since October 2022.
Cash Flow Statement
Cash from operations is trailing Nvidia’s growing operating leverage.
And so is free cash flow per share:
Stock based compensation has been trending up, but at a far lower pace than revenue.
Balance Sheet
At the end of Q3 FY2024, cash and equivalents came in at $18.2B, with long term debt coming at $8.4B and capital leases at $1.09B.
Nvidia’s strong balance sheet, coupled with its healthy cash flows, give the company plenty of financial muscle to carry on operating.
7.0 Conclusion
The B100 rumours suggest that Nvidia is working on chiplets already. If it isn’t, its networking and software advantages afford it plenty of time to iterate its way towards chiplets over time.
The question is then, how quickly can AMD iterate its ROCm software and how likely is the industry to embrace it? If AMD can improve the hardware fast enough, the industry will likely want to have a second option.
In such a scenario, Nvidia is likely to lose a fair bit of market share to AMD over time.
From a capital allocation standpoint, as previously mentioned, the demand for compute is going to keep on increasing exponentially.
That is why another player coming into the GPU space may not, in any of its potential manifestations, deal a considerable blow to Nvidia shareholders, but it may yield meaningful upside for AMD shareholders.
Until next time!
⚡ If you enjoyed the post, please feel free to share with friends, drop a like and leave me a comment.
You can also reach me at:
Twitter: @alc2022
LinkedIn: antoniolinaresc