Capability
Historical Token Throughput per GPU
Historical Token Throughput per GPU. Demo series compiled by The Latent research desk.
Historical Token Throughput per GPU
- H100 FP8
- H100 FP16
- MI300X FP8
View data
| Date | H100 FP8 | H100 FP16 | MI300X FP8 |
|---|---|---|---|
| 2023-08-01 | 14698.01 | 8317.65 | 11977.48 |
| 2023-09-01 | 14642.91 | 8280.72 | 11916.02 |
| 2023-10-01 | 14646.13 | 8287.77 | 11933.66 |
| 2023-11-01 | 13992.81 | 7912.37 | 11385.09 |
| 2023-12-01 | 15395.95 | 8698.93 | 12506.79 |
| 2024-01-01 | 16140.2 | 9128.18 | 13136.5 |
| 2024-02-01 | 16224.94 | 9172.79 | 13196.04 |
| 2024-03-01 | 17116.13 | 9681.07 | 13933.46 |
| 2024-04-01 | 16055.37 | 9088.56 | 13091.61 |
| 2024-05-01 | 15656.41 | 8852.19 | 12736.05 |
| 2024-06-01 | 16521.81 | 9343.06 | 13444.39 |
| 2024-07-01 | 16374.27 | 9258.05 | 13319.96 |
| 2024-08-01 | 17890.8 | 10108.48 | 14533.24 |
| 2024-09-01 | 18803.93 | 10637.36 | 15312.18 |
| 2024-10-01 | 17945.66 | 10151.99 | 14613.93 |
| 2024-11-01 | 18391.08 | 10401.81 | 14970.29 |
| 2024-12-01 | 17645.19 | 9986.64 | 14382.57 |
| 2025-01-01 | 17362.37 | 9813.58 | 14114.77 |
| 2025-02-01 | 19227.7 | 10867.5 | 15629.72 |
| 2025-03-01 | 19472.65 | 11012.33 | 15847.34 |
| 2025-04-01 | 20336.6 | 11494.84 | 16532.88 |
| 2025-05-01 | 21036.35 | 11902.89 | 17137.69 |
| 2025-06-01 | 19414.51 | 10987.17 | 15822.37 |
| 2025-07-01 | 19734.49 | 11157.51 | 16052.09 |
| 2025-08-01 | 20069.38 | 11352.63 | 16341.1 |
| 2025-09-01 | 20118.78 | 11370.57 | 16352.71 |
| 2025-10-01 | 22483.2 | 12706.02 | 18271.7 |
| 2025-11-01 | 22688.28 | 12836.42 | 18480.2 |
| 2025-12-01 | 22293.24 | 12607.14 | 18141.94 |
| 2026-01-01 | 22815.47 | 12908.67 | 18584.56 |
| 2026-02-01 | 21223.92 | 12010.56 | 17295.3 |
| 2026-03-01 | 21860.14 | 12353.24 | 17763.69 |
| 2026-04-01 | 23606.2 | 13347.17 | 19203.15 |
| 2026-05-01 | 23700.55 | 13399.24 | 19276.44 |
| 2026-06-01 | 25600.73 | 14471.17 | 20814.87 |
| 2026-07-01 | 25466.71 | 14413.65 | 20758.43 |
API access · Pro coming soon
Demo preview through July 2026 (tokens per second).
Methodology
Synthetic demo series generated to preview the data product layout.