Most investors still treat AI chip spending as a training story, with giant clusters teaching ever-larger models. Gartner sees it differently. Its estimate puts 2026 spending on AI-optimised cloud infrastructure at US$23.3 billion for inference, the work of running a trained model to answer requests, against US$19 billion for training.
Training and inference workloads place opposite demands on a GPU cluster: training needs tight synchronisation across every chip, while inference routes requests independently, which is why identical hardware can show very different utilisation depending on the job it is given.
That gap is the backdrop to a deal announced two days ago. On 3 November 2026, SambaNova and Supermicro (Nasdaq: SMCI) said they would build and sell disaggregated AI inference systems. These systems split one AI workload across two different kinds of chip.
The design pairs NVIDIA GPUs with SambaNova’s SN50 chips in a single system. Read it as more than a product launch. It is a signal about where demand for AI infrastructure is heading.
Here is how the two-phase split works, why the pairing matters to the AI hardware market, and which parts of the announcement are confirmed versus still open.
What did SambaNova and Supermicro actually announce?
The headline is simple. The two companies will jointly develop, build and sell systems in which NVIDIA GPUs handle prefill, the stage that processes your prompt. SambaNova’s SN50 Reconfigurable Dataflow Units (RDUs) handle decode, the stage that writes the response.
An RDU is SambaNova’s own type of AI processor, designed around how data flows through a model rather than around general-purpose graphics work. The SN50, introduced on 24 February 2026, is its fifth generation. The systems will sit inside Supermicro’s Data Center Building Block Solutions (DCBBS) portfolio.
The fine print is where you should slow down. Supermicro is initially planning for 10,000 SN50 chips, with substantial additional capacity expected in 2027.
Planning figure, not a commitment StorageReview reported the 10,000-chip number as a planning quantity for a phased rollout. Neither company disclosed a binding purchase commitment.
Here is what is confirmed and what is not:
- Confirmed: the prefill/decode division, the DCBBS product line, and the first target buyers (NeoClouds and sovereign-cloud providers, then enterprises running their own data centres)
- Undisclosed: pricing, shipment dates, binding volumes, and any timeline for the enterprise expansion
The roles are clearer than the numbers.
| Supermicro | SambaNova |
|---|---|
| Integrates and tests the systems | Engineers get customer models running on SN50 |
| Delivers and supports, including on-site service | Optimises models and assists with deployments |
The performance claims need the same care. According to SambaNova, as reported by Datacenter Dynamics, the SN50 supports models of up to 10 trillion parameters, links up to 256 accelerators, and offers four times the network bandwidth of the prior generation. SambaNova also claims up to five times the speed and more than three times the throughput of NVIDIA’s B200 on selected agentic comparisons. No independent benchmark has confirmed this.
For you, the read is straightforward: this is a go-to-market alignment with a large server maker, not yet proof of revenue. Treat the chip count as a plan.
When big ASX news breaks, our subscribers know first
How does disaggregated AI inference work, and why split prefill from decode?
When you type a question into an AI chatbot, the model does two very different jobs. The split in this deal follows that difference.
First comes prefill. The model reads your whole prompt at once and builds the KV cache, which is a stored record of the prompt’s processed information that the model can reuse. It then produces the first word, or token, of the answer. This stage is generally compute-bound, meaning raw processing power limits it, and it sets your time to first token.
Decode comes next. The model generates the reply one token at a time, rereading its weights and the growing KV cache for every step. That makes it generally memory-bandwidth-bound: the limit is how fast data moves from memory, not how fast the chip calculates.
| Prefill | Decode |
|---|---|
| Processes the full prompt and builds the KV cache | Generates output one token at a time |
| Bottleneck: compute | Bottleneck: memory bandwidth |
| Drives time to first token | Drives inter-token latency |
| Suits compute-dense GPUs | Suits high-bandwidth, large-capacity memory |
This is not SambaNova’s theory alone. NVIDIA’s own documentation describes the same split, and its Dynamo software supports disaggregated serving. Placing each phase on its own pool of chips offers several benefits:
- Prefill and decode capacity can scale independently
- Prompt processing and token generation interfere with each other less
- Each pool can use hardware specialised for its job
- Utilisation improves, potentially lowering cost
- Latency becomes easier to control
The SN50’s memory design shows why SambaNova wants the decode role. It combines 432 MB of on-chip SRAM, 64 GB of HBM2E and a DDR5 tier. Sources conflict on that last tier: product materials cite up to 512 GB, while datasheet references cite up to 2 TB.
The catch: moving the KV cache between chips
Splitting the work creates a new job. The KV cache must travel from the prefill chip to the decode chip, and NVIDIA’s Dynamo uses RDMA, a method of moving data directly between machines’ memory, to do this.
If the interconnect, routing, cache placement or scheduling falls short, that transfer overhead can erase the gains. Mixed-chip racks also add work in partitioning models and handling failures across two hardware types.
So the NVIDIA-plus-challenger pairing is a design choice grounded in how workloads behave. Judge the deal on whether the interconnect and software hold up, not on chip specifications alone.
Why is inference spending rising, and where do agentic workloads fit?
Gartner’s figures, reported in September and October 2026, frame the demand.
Inference versus training, AI-optimised cloud infrastructure Gartner estimates US$23.3 billion for inference against US$19 billion for training in 2026. Inference accounts for 55% of the category this year, rising to 59% in 2027.
That covers one category, not all AI spending. Still, it suggests serving models in production is becoming the larger new infrastructure need.
Why agents make decode speed matter
AI agents are systems that complete tasks by planning, calling tools, checking results and replanning. Each of those steps is a separate model call, and the next step usually waits for the previous answer.
That produces more output tokens and more sequential dependencies. A small delay per token compounds across every turn of a task. Decode speed stops being a technical detail and becomes a cost and productivity variable, which is exactly the stage SambaNova is targeting.
Forecasters are already building this into capacity models: Citi estimates agentic workloads consume 20-30 times more compute per active user than traditional generative AI, which is lifting its global data centre projections.
SambaNova is one of several non-NVIDIA options competing for this kind of work:
- SambaNova SN50, with tiered memory and a dedicated decode role
- Groq, focused on predictable low-latency token speed
- Cerebras, using wafer-scale processors
- Google TPU Ironwood, aimed substantially at large-scale inference
- AMD Instinct, a potentially lower-cost route where software support suffices
SambaNova has capital behind its push. It closed the first US$1 billion of a Series F round at an US$11 billion valuation on 8 July 2026, led by General Atlantic. It also counts a JPMorganChase selection and SoftBank as its first SN50 deployment partner in Japan.
How NeoClouds and sovereign clouds buy
NeoClouds, specialist providers renting AI compute to developers, buy in bulk. They want delivery certainty and strong performance per dollar and per watt. Sovereign clouds, built for governments and domestic partners that must keep data within a jurisdiction, prioritise local control, compliance and long-term serviceability. Both value a vendor that integrates, delivers and supports on site, which is Supermicro’s role.
Supermicro’s latest quarter shows the volume it already handles.
| Measure | Figure | Comparison |
|---|---|---|
| Q1 FY2027 net sales | ~US$19.9B | ~US$5.0B a year earlier |
| AI share of revenue | ~71% | 59% prior quarter |
| Non-GAAP gross margin | 12.8% | Q1 FY2027 |
| Q2 FY2027 sales guidance | US$19.5-21.5B | Company guidance |
| FY2027 sales guidance | US$72-82B | AI ~70-80% of revenue |
Taken together, inference is becoming the bigger incremental need. That widens the room for specialised chips alongside NVIDIA, and Supermicro has the delivery scale to carry them.
What could limit the deal, and what should investors watch next?
The strongest objection is that NVIDIA can do this itself. Dynamo is open-source, supports TensorRT-LLM, vLLM and SGLang, and offers KV-aware routing, so a customer can run both phases on NVIDIA GPUs alone.
Other frictions are less obvious but real:
- Software porting: customers must validate models and production tools on a non-NVIDIA accelerator, which is why SambaNova’s tuning help is central to the pitch
- Integration complexity: mixed racks need cross-chip routing, synchronisation and failure handling
- Commercial uncertainty: planning-only volumes, undisclosed pricing and dates, and reliance on a small set of strategic customers
- Supermicro margins: management said large AI deployments carry lower margins than enterprise and channel business
- Governance history: Supermicro has faced prior scrutiny over accounting and internal controls, and no newly published resolution was found
The latest customer disclosures remain the JPMorganChase selection and the SoftBank relationship, with no new binding commitments or shipment schedules identified. SambaNova also has a separate route through its February 2026 Intel collaboration.
Signposts to watch
- Named customers: NeoCloud or sovereign buyers confirmed by name
- Shipment timing: dates for the first SN50-based systems
- Pricing and margin commentary: what Supermicro says about profitability on these systems
- Independent benchmarks: third-party tests of the B200 comparison
- 2027 capacity: whether the expected expansion becomes firm orders
Treat the announcement as an early positioning move. Its value depends on conversion into named orders and independent performance evidence.
This article is for informational purposes only and should not be considered financial advice. Investors should conduct their own research and consult with financial professionals before making investment decisions. Forward-looking statements are speculative and subject to change based on market developments and company performance.
What the pairing signals about the next phase of AI hardware
The SambaNova and Supermicro deal reflects a market where inference is being carved into stages and matched to specialised hardware. NVIDIA stays central to prefill, while challengers compete for decode, the stage agents lean on hardest.
What remains open is just as important: volumes, pricing, independent performance evidence and the software friction of running two chip types side by side.
For your own decision-making, look past the 10,000-chip headline. Named orders, shipment disclosures and independent benchmarks will tell you far more about whether this split architecture is turning into revenue than any planning figure can.
For readers weighing how much of this demand is priced in, our deep-dive into semiconductor cycle timing examines Truist’s finding that chip stocks have typically peaked about a quarter after revenue growth does.
