NVIDIA announced it’s making wholesale changes to the way CUDA operates. It will also transform the way programmers conceive and write AI programs for the company’s red-hot GPUs.
The past was about creating runtimes for standalone GPUs, but now NVIDIA wants to create a single runtime for hundreds of thousands of GPUs.
“We’ve been talking about programming a GPU — how do I get my program on one GPU. And the reality is no one runs on one GPU anymore,” said Stephen Jones, a CUDA architect at NVIDIA, during a technical session at the…