The major driver behind Machine Learning going mainstream is the exponential growth of chip performance through Moore’s law over the last four decades. However, when it comes to ML for the future, Moore’s law is not enough. Mere scaling through going to a lower node is not sufficient to keep pace with the ever-increasing complexity of ML workloads for both training and inference. Add energy efficiency to the mix and we need a dramatically new class of hardware – silicon optimized for specific domains.

Many companies today are focused on trying to deliver peak efficiency in machine learning (ML) inference by encouraging customers to move from less efficient traditional processors, to purpose-built accelerators for ML inference. 

While this is directionally correct, oftentimes hardware specific solutions are unable to match customers’ performance and efficiency goals. The issue, solving for ‘peak efficiency’ cannot be accomplished by simply throwing a combination of silicon and power at the problem; this is especially true at the edge.