In today’s fast-evolving AI landscape, performance and power efficiency are at the heart of the AI inference challenge. The second-generation speedAI®240 Slim accelerator card, verified by the industry-standard MLPerf® Inference 4.1 benchmarks, offers the industry’s best-in-class solution to these challenges by pushing the limits of performance and energy efficiency, setting new benchmarks for AI inference. As AI models grow in size and complexity, traditional architectures often force designers to choose between peak performance and energy consumption—a critical trade-off in both datacenters and edge computing. Untether AI, with its At-Memory compute architecture, solves this issue by eliminating the bottlenecks of data movement in AI processing.
What is MLPerf and Why Does It Matter?
MLPerf, developed by the MLCommons® consortium, is the gold standard for evaluating AI hardware, offering an objective measure of performance and power efficiency. Supported by leading AI chip developers like Nvidia, Google, and Intel, this peer-reviewed benchmark assesses AI systems across several categories, including Datacenter and Edge.
MLPerf’s rigorous benchmarks require detailed submissions that encompass metrics like latency, throughput, accuracy, and power consumption. Additionally, all submissions must declare the type and number of CPUs and AI accelerators used – and these results can be audited by the submitters themselves – ensuring a fair and accurate representation. Untether AI submitted results in these categories with speedAI240 Slim and speedAI240 Preview accelerator cards in real-world AI workloads like ResNet-50, an image classification model widely used for AI inference.
Unprecedented Datacenter Performance
The speedAI240 Preview card delivered a standout performance in the Datacenter Closed category. With a throughput of 70,348 samples/second, it established itself as the highest-performing single PCIe card in this category. This result underscores the card’s exceptional processing power for datacenter applications, where maximizing throughput is critical.
Moreover, in the Datacenter Closed Power category, Untether AI demonstrated 3X greater energy efficiency than its nearest competitor. With 309,752 Server Queries/Second at 986 Watts, it set a new standard for efficient AI computing, reducing both energy consumption and operating costs for datacenter operators—a key factor as the AI industry becomes increasingly mindful of sustainability.
Energy-Efficient Edge AI
Untether AI’s results in the Edge Closed category were equally impressive, further solidifying its leadership in AI acceleration at the edge. The speedAI240 Slim card achieved a latency of 0.12ms in single-stream and 0.17ms in multi-stream configurations, breaking records for the lowest-ever latencies in MLPerf submissions.
In terms of energy efficiency, the speedAI240 Slim card delivered 6X the efficiency of other accelerators in this category, a testament to its potential in edge computing environments that prioritize low power consumption and high performance. Whether it’s autonomous vehicles, robotics, or real-time video surveillance, the speedAI240 Slim card is designed to handle compute-intensive tasks with minimal energy overhead, making it an ideal choice for edge AI applications.
Take the Leap with speedAI240 Slim
Untether AI’s success in the MLPerf Inference v4.1 benchmarks demonstrates a significant leap forward for AI hardware, proving that performance and energy efficiency can coexist at the highest levels of AI inference. Whether you’re operating in the cloud or on the edge, Untether AI’s accelerator cards are designed to meet your most demanding AI workloads efficiently and effectively.
Ready to experience the next level of AI inference acceleration? Order your speedAI240 Slim card today and harness the full potential of AI, powered by Untether AI’s world-leading performance and energy efficiency. Visit Untether AI’s website to learn more and explore the benchmark results in detail.