AI chip · AMD
AMD Instinct MI300X
CDNA 3 data-center accelerator with 192 GB HBM3, positioned as the primary alternative to Nvidia H100/H200 for LLM inference workloads. Fabricated by TSMC.
On the DEPLOY graph: 1 reference-design cluster include it.
Market position
MI300X is the chip that gave AMD real inference share against H100/H200: 192 GB HBM3 (2.4x H100 80GB) at $10-15K street. Meta, Microsoft (Copilot inference), Oracle and Vultr became public buyers. First real dual-source cloud GPU since Volta.
DEPLOY editorial. What the vendor PDF cannot tell you.
Physical-AI cross-link
Derived from claims already on record (vendor first-party TDP, throughput, price, and the reference-design cluster registry). The arithmetic is shown per row so it can be audited. No estimated inputs.
- Perf per watt
- 3.49 FP8 TFLOPS/W2,614 TFLOPS ÷ 750 W = 3.49
- List $ per FP8 TFLOP
- $5.74$15,000 ÷ 2,614 TFLOPS = $5.74
- Rack power (in this design)
- 6 kW per AMD Instinct MI300X Platform (8-GPU OAM baseboard) (~750 W per chip incl. overhead)6 kW ÷ 8 chips = 750 W each (system-level, includes CPU/mem/NIC/cooling)
- Cooling class (across reference designs)
- airObserved across 1 reference-design cluster containing this chip
What fits in 192 GB
Weights-only footprint for public open-weight LLMs. Pure arithmetic: params × bytes/param. Excludes KV cache and activation memory; add ~10-30% headroom for real serving. A model that does not fit at FP16 may still fit at INT8 or INT4 with quality trade-offs. Not a benchmark.
| Model | Params | FP16 | INT8 | INT4 |
|---|---|---|---|---|
| Llama 3.1 8B dense | 8 B | 16 GB ✓ | 8 GB ✓ | 4 GB ✓ |
| Llama 3.1 70B dense | 70 B | 140 GB ✓ | 70 GB ✓ | 35 GB ✓ |
| Llama 3.1 405B dense | 405 B | 810 GB ✗ | 405 GB ✗ | 203 GB ✗ |
| Llama 3.3 70B dense | 70 B | 140 GB ✓ | 70 GB ✓ | 35 GB ✓ |
| DeepSeek V3 MoE (671B total, 37B active per token) | 671 B | 1342 GB ✗ | 671 GB ✗ | 336 GB ✗ |
| DeepSeek R1 MoE (671B total, 37B active per token) | 671 B | 1342 GB ✗ | 671 GB ✗ | 336 GB ✗ |
| Qwen 2.5 7B dense | 7 B | 14 GB ✓ | 7 GB ✓ | 4 GB ✓ |
| Qwen 2.5 72B dense | 72 B | 144 GB ✓ | 72 GB ✓ | 36 GB ✓ |
| Mixtral 8x7B MoE (46.7B total, 12.9B active per token) | 46.7 B | 93 GB ✓ | 47 GB ✓ | 23 GB ✓ |
| Mixtral 8x22B MoE (141B total, 39B active per token) | 141 B | 282 GB ✗ | 141 GB ✓ | 71 GB ✓ |
| Gemma 2 27B dense | 27 B | 54 GB ✓ | 27 GB ✓ | 14 GB ✓ |
| Command R+ dense | 104 B | 208 GB ✗ | 104 GB ✓ | 52 GB ✓ |
| Kimi K2 MoE (1T total, 32B active per token) | 1000 B | 2000 GB ✗ | 1000 GB ✗ | 500 GB ✗ |
Math: FP16 = params × 2 bytes; INT8 = params × 1 byte; INT4 = params × 0.5 bytes. MoE models sum every expert (full weights on disk), not the per-token active subset.
Common questions
Answer-first, sourced. Every claim below traces to a specific field on this page or a cited datasheet. FAQPage schema is emitted so LLM crawlers can lift these verbatim.
How much does AMD Instinct MI300X cost?
List price at launch was $15,000 (2024-01-01). Street prices vary with generation age and supply. DEPLOY does not currently hold cloud rental $/hr as a first-party field.
How much memory does AMD Instinct MI300X have?
192 GB of HBM3 at 5,300 GB/s. Vendor datasheet, first-party.
How much power does one AMD Instinct MI300X draw?
750 W TDP (thermal design power) per chip. System-level draw is higher: see the reference-design section for rack-level kW.
Which open-weight LLMs fit on one AMD Instinct MI300X?
Of the 13 public open-weight LLMs DEPLOY tracks: 7 fit at FP16, 9 at INT8, 9 at INT4 (weights only, excludes KV cache). The full table is above with per-model math. Sparsity, offloading and multi-GPU serving change the picture; this row is single-chip weights-only.
Key facts
- Class
- Compute SoC (AI accelerator)
- Designer
- AMD
- Safety-critical?
- No (data-center inference / training)
- Record as of
- 2026-08-25
- Latest cited claim
- 2023-12-01 (across 3 inventory + benchmark rows)
Specifications (14 fields, click to expand)
Vendor datasheet figures (first-party). Dense throughput first; sparse (2:4) in parentheses where NVIDIA quotes it. The exhaustive spec sheet lives on the datasheet URL below the table: this row set covers what buyers actually compare on.
- Process node
- TSMC N5 + N6 (chiplet)
- Transistors
- 153 B
- CUDA cores
- 19,456
- TDP
- 750 W
- Memory
- 192 GB HBM3
- Memory bandwidth
- 5,300 GB/s
- PCIe
- Gen 5 x16 (896 GB/s)
- FP16 (dense)
- 1,307 TFLOPS
- FP8 (dense)
- 2,614 TFLOPS
- INT8 (dense)
- 2,614 TOPS
- Form factor
- OAM
- Announced
- 2023-12-06
- Released
- 2024-01-01
- Launch price
- $15,000 (list)
Source: vendor datasheet
Generation
Used in reference designs
The named rack-scale and pod-scale designs buyers actually order that contain this chip.
Compare with
Benchmarks
Published measurements per workload. Vendor datasheet numbers are first-party (verified posture); MLPerf / InferenceX / press results come in when their license terms + workload naming permit.
Sources
Adoption rows appear as we verify chip-integration claims per model. Every row is a public claim the maker or a tier-1 source has stated; verification tier (verified / reported / inferred) is shown per row. See every chip on record for the full catalog or the AMD record.