AI chip · NVIDIA
NVIDIA H200 Tensor Core GPU
Hopper refresh with 141 GB HBM3e (from 80 GB); the mid-generation memory-bandwidth bump between H100 and Blackwell.
On the DEPLOY graph: 4 campuses carry a cited inventory line for it · 1 reference-design cluster include it.
Market position
H200 kept H100's compute silicon and swapped HBM3 for 141 GB HBM3e at 4.8 TB/s: 1.4x memory bandwidth in the same socket. Inference shops (Together AI, Perplexity, hyperscaler tenants) upgraded here first; training shops mostly held for Blackwell.
DEPLOY editorial. What the vendor PDF cannot tell you.
Physical-AI cross-link
Derived from claims already on record (vendor first-party TDP, throughput, price, and the reference-design cluster registry). The arithmetic is shown per row so it can be audited. No estimated inputs.
- Perf per watt
- 2.83 FP8 TFLOPS/W1,979 TFLOPS ÷ 700 W = 2.83
- Rack power (in this design)
- 10.2 kW per NVIDIA HGX H200 (8-GPU baseboard) (~1275 W per chip incl. overhead)10.2 kW ÷ 8 chips = 1275 W each (system-level, includes CPU/mem/NIC/cooling)
- Cooling class (across reference designs)
- airObserved across 1 reference-design cluster containing this chip
What fits in 141 GB
Weights-only footprint for public open-weight LLMs. Pure arithmetic: params × bytes/param. Excludes KV cache and activation memory; add ~10-30% headroom for real serving. A model that does not fit at FP16 may still fit at INT8 or INT4 with quality trade-offs. Not a benchmark.
| Model | Params | FP16 | INT8 | INT4 |
|---|---|---|---|---|
| Llama 3.1 8B dense | 8 B | 16 GB ✓ | 8 GB ✓ | 4 GB ✓ |
| Llama 3.1 70B dense | 70 B | 140 GB ✓ | 70 GB ✓ | 35 GB ✓ |
| Llama 3.1 405B dense | 405 B | 810 GB ✗ | 405 GB ✗ | 203 GB ✗ |
| Llama 3.3 70B dense | 70 B | 140 GB ✓ | 70 GB ✓ | 35 GB ✓ |
| DeepSeek V3 MoE (671B total, 37B active per token) | 671 B | 1342 GB ✗ | 671 GB ✗ | 336 GB ✗ |
| DeepSeek R1 MoE (671B total, 37B active per token) | 671 B | 1342 GB ✗ | 671 GB ✗ | 336 GB ✗ |
| Qwen 2.5 7B dense | 7 B | 14 GB ✓ | 7 GB ✓ | 4 GB ✓ |
| Qwen 2.5 72B dense | 72 B | 144 GB ✗ | 72 GB ✓ | 36 GB ✓ |
| Mixtral 8x7B MoE (46.7B total, 12.9B active per token) | 46.7 B | 93 GB ✓ | 47 GB ✓ | 23 GB ✓ |
| Mixtral 8x22B MoE (141B total, 39B active per token) | 141 B | 282 GB ✗ | 141 GB ✓ | 71 GB ✓ |
| Gemma 2 27B dense | 27 B | 54 GB ✓ | 27 GB ✓ | 14 GB ✓ |
| Command R+ dense | 104 B | 208 GB ✗ | 104 GB ✓ | 52 GB ✓ |
| Kimi K2 MoE (1T total, 32B active per token) | 1000 B | 2000 GB ✗ | 1000 GB ✗ | 500 GB ✗ |
Math: FP16 = params × 2 bytes; INT8 = params × 1 byte; INT4 = params × 0.5 bytes. MoE models sum every expert (full weights on disk), not the per-token active subset.
Common questions
Answer-first, sourced. Every claim below traces to a specific field on this page or a cited datasheet. FAQPage schema is emitted so LLM crawlers can lift these verbatim.
How much does NVIDIA H200 Tensor Core GPU cost?
NVIDIA H200 Tensor Core GPU launch pricing is not on record in the DEPLOY registry. Vendor list prices are typically published in the vendor's own briefing rather than the datasheet.
How much memory does NVIDIA H200 Tensor Core GPU have?
141 GB of HBM3e at 4,800 GB/s. Vendor datasheet, first-party.
How much power does one NVIDIA H200 Tensor Core GPU draw?
700 W TDP (thermal design power) per chip. System-level draw is higher: see the reference-design section for rack-level kW.
Which data centers have NVIDIA H200 Tensor Core GPU?
4 campuses on the DEPLOY registry carry a cited inventory line for this chip. Full list below with per-campus quantity hints where published.
Which open-weight LLMs fit on one NVIDIA H200 Tensor Core GPU?
Of the 13 public open-weight LLMs DEPLOY tracks: 6 fit at FP16, 9 at INT8, 9 at INT4 (weights only, excludes KV cache). The full table is above with per-model math. Sparsity, offloading and multi-GPU serving change the picture; this row is single-chip weights-only.
Key facts
- Class
- Compute SoC (AI accelerator)
- Designer
- NVIDIA
- Safety-critical?
- No (data-center inference / training)
- Record as of
- 2026-08-25
- Latest cited claim
- 2026-01-15 (across 6 inventory + benchmark rows)
Specifications (17 fields, click to expand)
Vendor datasheet figures (first-party). Dense throughput first; sparse (2:4) in parentheses where NVIDIA quotes it. The exhaustive spec sheet lives on the datasheet URL below the table: this row set covers what buyers actually compare on.
- Process node
- TSMC 4N
- Transistors
- 80 B
- Die size
- 814 mm²
- CUDA cores
- 16,896
- Tensor cores
- 528 (4th gen (Hopper))
- TDP
- 700 W
- Memory
- 141 GB HBM3e
- Memory bandwidth
- 4,800 GB/s
- PCIe
- Gen 5 x16 (900 GB/s)
- FP32
- 67 TFLOPS
- TF32 (dense)
- 495 TFLOPS
- FP16 (dense)
- 989 TFLOPS (1,979 sparse)
- FP8 (dense)
- 1,979 TFLOPS (3,958 sparse)
- INT8 (dense)
- 1,979 TOPS (3,958 sparse)
- Form factor
- SXM5
- Announced
- 2023-11-13
- Released
- 2024-03-01
Source: vendor datasheet
Generation
Used in reference designs
The named rack-scale and pod-scale designs buyers actually order that contain this chip.
- ×8NVIDIA HGX H200 (8-GPU baseboard)platform
Compare with
Benchmarks
Published measurements per workload. Vendor datasheet numbers are first-party (verified posture); MLPerf / InferenceX / press results come in when their license terms + workload naming permit.
Data centers stocking NVIDIA H200 Tensor Core GPU
Who has the most? →Every campus on the DEPLOY registry with a public claim of NVIDIA H200 Tensor Core GPU on site. Quantity hints are unstructured because underlying disclosures vary by order of magnitude.
- xAI Colossus (Memphis)as of 2026-01-15partially energizedreported
Second-wave upgrade GPUs added late 2024
- Khazna QAJ1 Ajman AI-Optimized Data Centeras of 2025-03-01under constructioninferred
H200/GB200-generation capacity for G42 / UAE AI stack
G42 has committed multi-billion NVIDIA compute; per-campus attribution not published.
- Lake Mariner Data Campusas of 2025-01-01partially energizedinferred
H200-generation HPC capacity for AI-training tenants
AI-training-oriented HPC campus; specific chip mix not published, H200 inferred from current-gen positioning.
- Crusoe Childress Campusas of 2024-12-01announcedreported
Recent Crusoe expansions include H200 capacity
Sources
- NVIDIA H200 product pageNvidia
Adoption rows appear as we verify chip-integration claims per model. Every row is a public claim the maker or a tier-1 source has stated; verification tier (verified / reported / inferred) is shown per row. See every chip on record for the full catalog or the NVIDIA record.