nanoTitan benchmark
Parallelism comparison
Metric
Tokens per second
Step time
Forward time
Backward time
First-stage peak memory
Middle-stage peak memory
Last-stage peak memory
Number of GPUs
2 GPUs
4 GPUs
8 GPUs
Parallelism strategy comparison
A horizontal dot plot comparing experiment configurations for a selected metric and GPU count.
Metric
Tokens per second
Step time
Forward time
Backward time
First-stage peak memory
Middle-stage peak memory
Last-stage peak memory
Experiment configuration comparison
A horizontal dot plot comparing experiment configurations for the selected metric.