The most comfortable place in a gold rush is supposed to be the shop selling shovels. Silicon Valley has improved this arrangement. We now have companies selling shovels to companies selling access to shovels to startups building a conversational interface for the shovel. The person looking for gold has been invited to join the waitlist.
At GTC in March, NVIDIA presented Vera Rubin as a coordinated system spanning seven chips and five rack-scale systems. The announcement was about the machinery around computation as well as the processors themselves: a full infrastructure proposition, not merely a faster component. These are NVIDIA’s announcements and performance claims, not results from our own testing.1
A rack still has to be built, powered, connected, and kept at a temperature that permits it to remain a rack. The launch presentation can compress that work into a diagram. The people installing it cannot.
A beautifully integrated invoice
The founder needs to know whether a customer will pay enough, repeatedly, for a particular outcome. The infrastructure supplier needs to know whether customers will build and operate more infrastructure. Those questions can reinforce each other. They can also drift apart for longer than a pitch deck likes to acknowledge.
Imagine an entirely hypothetical business that uses an elaborate agent to prepare sales proposals. Better models make the proposals faster and more polished. Hardware improvements reduce the cost of each attempt. Excellent. But the customer may still need a salesperson to correct the terms, a manager to approve the discount, and someone with authority to promise delivery.
The remaining cost is not an insult to the model. It is part of the service. Removing ninety percent of the typing does not necessarily remove ninety percent of the responsibility.
The customer’s unfortunate habit
Customers have a habit of comparing products with their actual alternatives rather than with the most exciting slide at a developer conference. A customer might choose a less ambitious tool, a spreadsheet, an existing employee’s workflow, or simply not doing the proposed task.
This is extremely poor ecosystem behavior.
For an application company, lower input costs create an opportunity, not a margin guarantee. Competitors can receive the same improvement. The customer might demand a lower price. Usage might expand into work that was previously too expensive to attempt. The company might discover that quality assurance, distribution, and integration absorb more of the budget than the model call ever did.
A buyer can discover any of these problems after the supplier has booked a perfectly legitimate sale. The hardware can perform exactly as advertised while the business built on top of it disappoints.
The seductive version of the story collapses the chain. More capable chips mean more capable models; more capable models mean better products; better products mean adoption; adoption means attractive returns for everyone involved. Each arrow is doing work. Some arrows are engineering problems. Others require a person with a budget to change their mind.
Capacity is not a use case
So we borrow the certainty of the visible object and carry it downstream. If that much intelligence is arriving, surely the application will find its market. If that much money is being spent, surely someone has established the demand. Sometimes someone has. Sometimes several institutions are each assuming that a different institution has.
Building ahead of demand is how some useful infrastructure gets built. It is also how a company acquires an expensive facility whose most reliable customer is the electricity provider.
But building ahead of demand should be described as a bet on future demand, not as proof that demand already exists. The distinction is especially valuable when every layer of the stack uses another layer’s growth as its supporting evidence.
Show the rejected proposal
Alongside cost per token, show cost per accepted outcome. Alongside usage, show repeat use after the trial. Alongside the beautiful architecture, show the customer who stopped buying and explain why.
Huang’s industry is building remarkable machinery. The founders using it do not need to diminish that achievement to ask harder questions of their own companies. Quite the opposite: cheaper, more capable infrastructure makes the absence of a useful product harder to blame on infrastructure.
The shovel looks excellent. Before ordering another million of them, somebody might inspect the hole.