TORONTO, Canada – October 29, 2020 – Today at the fall Linley Processor Conference, Untether AI unveiled its tsunAImi accelerator cards powered by the runAI devices. Using at-memory computation, Untether AI breaks through the barriers of traditional von Neumann architectures, offering industry-leading compute density with power and price efficiency.
The Need for Speed
Artificial Intelligence (AI) workloads for inference require increasing amounts of compute resources, far outstripping the gains available to traditional CPU and GPU architectures. The slowing of Moore’s Law and the end of Dennard scaling further limits future gains in performance from traditional computing approaches. Solving this dilemma is important, as inference acceleration in the datacenter, using AI accelerators, is estimated to be a $10 billion market by 2025, according to McKinsey & Company. Untether AI was founded to radically rethink how computation for machine learning is accomplished. In current architectures, 90 percent of the energy for AI workloads is consumed by data movement, transferring the weights and activations between external memory, on-chip caches, and finally to the computing element itself. By focusing on the needs for inference acceleration and maximizing power efficiency, Untether AI is able to deliver two PetaOperations per second (POPs) in a standard PCI-Express card form factor.
“For AI inference in cloud and datacenters, compute density is king. Untether AI is ushering in the PetaOps era to accelerate AI inference workloads at scale with unprecedented efficiency,” said Arun Iyengar, CEO of Untether AI.
The Most Efficient AI Compute Engine Available – runAI200 Devices
Tailored for inference acceleration, runAI200 devices operate using integer data types and a batch mode of 1. At the heart of the unique at-memory compute architecture is a memory bank: 385KBs of SRAM with a 2D array of 512 processing elements. With 511 banks per chip, each device offers 200MB of memory and operates up to 502 TeraOperations per second in its “sport” mode. It may also be configured for maximum efficiency, offering 8 TOPs per watt in “eco” mode. runAI200 devices are manufactured using a cost-effective, mainstream 16nm process.
“As AI compute requirements continue to explode, new architectures are needed to meet these demands,” said Linley Gwennap, principal analyst, The Linley Group. “Untether AI’s runAI200 devices, with their innovative at-memory compute architecture, break through traditional von Neumann architecture bottlenecks and represent a new breed of AI accelerators.”
2 PetaOps at the Lowest Price per TOP- tsunAImi Accelerator Cards
tsunAImi accelerator cards are powered by four runAI200 devices, providing 2 POPs of compute, more than two times any currently announced PCIe cards. This compute power translates into over 80,000 frames per second of ResNet-50 v 1.5 throughput at batch=1, three times the throughput of its nearest competitor. For natural language processing, tsunAImi accelerator cards can process more than 12,000 queries per second (qps) of BERT-base, four times faster than any announced product.
“When we founded Untether AI, our laser focus was unlocking the potential of scalable AI, by delivering more efficient neural network compute,” said Martin Snelgrove, co-founder and CTO of Untether AI. “We are gratified to see our technology come to fruition.”
Simple, Automatic Tool Flow – the imAIgine Software Development Kit
Until now, making neural networks perform optimally has been a manual process. The Untether AI imAIgine Software Development Kit (SDK) provides an automated path to running networks at high performance, with push-button quantization, optimization, physical allocation, and multi-chip partitioning. The imAIgine SDK frees data scientists from having to perform low-level optimization tasks and instead spend time where it matters to them – crafting their models. The imAIgine SDK also provides an extensive visualization toolkit, cycle-accurate simulator, and an easily integrated runtime API.
Availability
The imAIgine SDK is currently in Early Access (EA) with select customers and partners. The tsunAImi accelerator card is sampling now and will be commercially available in 1Q2021.
About Untether AI
Untether AI® provides energy-centric AI inference acceleration from the edge to the cloud, supporting any type of neural network model. With its at-memory compute architecture, Untether AI has solved the data movement bottleneck that costs energy and performance in traditional CPUs and GPUs, resulting in high-performance, low-latency neural network inference acceleration without sacrificing accuracy. Untether AI embodies its technology in runAI® and speedAI™ devices, tsunAImi® acceleration cards, and its imAIgine® Software Development Kit. Founded in Toronto in 2018, Untether AI is funded by CPPIB, GM Ventures, Intel Capital, Radical Ventures, and Tracker Capital. More information can be found at www.untether.ai.
All references to Untether AI trademarks are the property of Untether AI. All other trademarks mentioned herein are the property of their respective owners.