How fast you can train gigantic new AI models boils down to two words: up and out.
In data-center terms, scaling out means increasing how many AI computers you can link together to tackle a big problem in chunks. Scaling up, on the other hand, means jamming as many GPUs as possible into each of those computers, linking them so that they act like a single gigantic GPU, and allowing them to do bigger pieces of a problem faster.
The two domains rely on two different physical connections. Scaling out mostly…
