Supercomputing researchers document evolution of AI hardware

As artificial intelligence transforms industries and national security, understanding the latest hardware capabilities is important for maintainind technological advantage.

AI accelerators — specialized systems designed to speed up capabilities such as neural networks, deep learning, and machine learning — have been a major area of development for nearly a decade. Since 2018, team from the Lincoln Laboratory Supercomputing Center (LLSC) has been conducting the Lincoln AI Computing Survey (LAICS, pronounced "lace"). Six papers later, LAICS continues to summarize current commercial AI accelerators and compare their peak performance and peak power.

"About eight years ago, we saw a sharp rise in the number of research AI accelerators described in research papers and commercial accelerators being announced, and we started to get questions about them from government sponsors of the laboratory's work. That was motivation enough to start the survey," says Albert Reuther, a staff member at the LLSC, which operates and optimizes the high-performance computing systems used by thousands of laboratory research staff.

Although AI accelerators are frequently used for processes such as machine learning, they also can enable other parallel applications, such as modeling the functions of molecules and speeding up simulations of fluid dynamics — processes that are very computationally expensive.

AI accelerator technology can come in a number of forms: central processing units (CPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and dataflow accelerators. Each type of accelerator has slightly different capabilities. CPUs can be used for general-purpose computing, while ASICs can perform only very specific tasks. Dataflow accelerators, FPGAs, and GPUs are more flexible and can be configured for a variety of workloads. Efficiency and performance vary across the different types of accelerators depending on how they are designed. The goal of LAICS is to survey the technologies currently on the market and compare them to find the best accelerators for certain needs.

Led by Reuther, the LAICS team includes LLSC members Michael Jones, Peter Michaleas, Jeremy Kepner, and Vijay Gadepally. The team also collaborates with researchers across Lincoln Laboratory, including in the Advanced Technology Division and Intelligence, Surveillance, and Reconnaissance and Tactical Systems Division, to learn how accelerators support research and development for their missions.

The first paper in their series studied 57 accelerators, while the latest one looked at more than 120 accelerators. The main metrics the team uses to compare accelerators are the peak performance and power; then they sort accelerators by whether they're on a chip, card, or system. All the data in the papers are drawn from public sources, which can be challenging because some companies prefer to keep their performance and power data private. To keep up to date on the latest in the field, Reuther runs daily news and citation searches that report new technical press articles, company announcements, and industry presentations.

"It continues to surprise me how each year another five to 10 startups get funded and announced, and then release new AI accelerators," Reuther says. "One might think that the landscape is saturated enough, but then another batch of innovative accelerators is introduced."

In addition to summarizing the performance versus peak power of the current accelerators, each paper explores a new aspect of the field. For example, the paper published in 2022 investigated sources of performance increases, finding that they stem from smaller, denser transistor designs and the use of lower numerical precision (i.e., calculating fewer significant digits). The latest paper examined different architectural choices available, analyzing how the addition of certain components, such as more cores per processor or parallel performance, would change the system.

Reuther plans to continue the survey for the foreseeable future, stating that, in just the past few months, six new startups have announced their first AI accelerators.

"AI and the hardware it runs on are such hot topics, and it is important for Lincoln Laboratory to be an unbiased technical advisor for choosing and pursuing the right technologies," Reuther says. "Our AI accelerator surveys have helped many sponsors and government colleagues gain a better understanding of the AI accelerator landscape and make better research and acquisition decisions about them. This survey has also been very valuable to determine which GPUs we should consider for upcoming LLSC system purchases so it not only benefits our sponsors and mission programs, but also benefits all LLSC users."

The full set of papers and the accompanying datasets can be found here.