AI inference is poised to become the single largest compute workload. For some time now, after all, AI service providers like OpenAI have claimed that they are mostly constrained by compute availability. Even as these companies build massive, power-hungry data centers to run both their training and inference workloads, the need for optimized hardware and software solutions remains.
Meanwhile, it’s not just the frontier AI labs that are concerned with the cost of running these models, but increasingly, enterprises want full control over…
