Former Intel CEO Pat Gelsinger has entered the AI hardware arena with a strategic investment in Fractile, a UK-based startup. The company’s focus is on AI hardware capable of running large language model (LLM) inference in memory rather than using traditional processing methods. This innovative approach aims to lessen compute overhead and enhance scalability.
Gelsinger’s transition follows a notable move in December, when he stepped down from Intel due to a lack of investor confidence in the company’s strategic direction. Now, his commitment to advancing AI continues through Fractile. In a recent LinkedIn post, he commented on the challenges faced in deploying large AI models. He noted that “inference of frontier AI models is bottlenecked by hardware,” addressing existing limitations and the need for more efficient inference solutions.
Scaling AI requires concurrent growth in model size, dataset size, and compute power. However, test-time scaling is emerging as a key factor driving efficiency during the inference phase without retraining models. Techniques like dynamic model adjustment and efficient batch processing are vital for optimizing AI performance.
The concept of edge AI, which involves deploying AI inference locally on devices or decentralized infrastructure, is gaining traction. This approach improves latency, reduces computing costs, and enhances data privacy, potentially aligning better with data sovereignty regulations. Gelsinger emphasized edge AI’s benefits during a keynote at CES 2024.
Under his leadership, this strategic shift was evident, embracing edge computing’s economic, physical, and regulatory advantages. “It’s cheaper to do it on your device,” Gelsinger pointed out, highlighting the cost-saving potential of localized computation, which bypasses the need for cloud-based processing.
Fractile’s innovative in-memory compute method offers a promising solution to current hardware bottlenecks. By overcoming memory constraints and minimizing power consumption, an ever-critical concern for data center expansion, Fractile aims to redefine inference performance. Gelsinger noted, “Being able to run any given model orders of magnitude faster, at a fraction of the cost and maybe most importantly at dramatically lower power envelope provides a performance leap.”
