Bucket size sweep

AllReduce launches and average backward-pass time

Bucket size sweep One dual-axis plot compares bucket size with the number of AllReduce calls on the left axis and average backward-pass time in seconds on the right axis.
Bucket size sweep data
Bucket size (MB) Number of AllReduce calls Average backward time (seconds)