Capability
Historical Token Throughput per GPU
Historical Token Throughput per GPU. Demo series compiled by The Latent research desk.
Historical Token Throughput per GPU
- H100 FP8
- H100 FP16
- MI300X FP8
View data
| Date | H100 FP8 | H100 FP16 | MI300X FP8 |
|---|---|---|---|
| 2025-08-01 | 20069.38 | 11352.63 | 16341.1 |
| 2025-09-01 | 20118.78 | 11370.57 | 16352.71 |
| 2025-10-01 | 22483.2 | 12706.02 | 18271.7 |
| 2025-11-01 | 22688.28 | 12836.42 | 18480.2 |
| 2025-12-01 | 22293.24 | 12607.14 | 18141.94 |
| 2026-01-01 | 22815.47 | 12908.67 | 18584.56 |
| 2026-02-01 | 21223.92 | 12010.56 | 17295.3 |
| 2026-03-01 | 21860.14 | 12353.24 | 17763.69 |
| 2026-04-01 | 23606.2 | 13347.17 | 19203.15 |
| 2026-05-01 | 23700.55 | 13399.24 | 19276.44 |
| 2026-06-01 | 25600.73 | 14471.17 | 20814.87 |
| 2026-07-01 | 25466.71 | 14413.65 | 20758.43 |
API access · Pro coming soon
Demo preview through July 2026 (tokens per second).
Methodology
Synthetic demo series generated to preview the data product layout.