#Benchmark pc software#
#Benchmark pc free#

Using an STMicroelectronics chip with an ARM Cortex M7 processor core, Plumerai delivered COCO 14 results in 59.4 milliseconds of latency.

#Benchmark pc software#

On one common task, 'visual wake words,' which makes use of the data set known as COCO 14, 'common objects in context,' to test object recognition in images, the top score in terms of latency was taken by startup Plumerai, which creates its own software to train and deploy AI models on standard microprocessors. Andes's 'AndesCore' chips make use of the open-source RISC-V computer instruction set, which is competing with ARM and Intel to be an instruction set that can be freely modified for any kind of computing device.

In the MLPerf TinyML section, where benchmark tasks include such things as the latency in detecting a 'wake word' - the thing that activates a smart speaker or another AI assistant - eight vendors competed with novel processors, including computer chip designer Andes Technology. In the 'open' vision, where submitters can take liberties with neural network formation, which is dominated by Krai, the number of submissions with power measurements surged to 86% from 32%, said Kanter. 'Closed' refers to submissions that adhere strictly to the MLCommon's benchmark neural network configuration. 'This time around, the percentage of closed submissions with power measurement went from 15.7% to 17.6%, so, an increase, but we still have work to do there,' said Kanter. This time, Krai reported results for dozens of Raspberry Pi embedded computing devices. Krai reported many more combinations of chips this time where before it had reported only Nvidia's Jetson AGX Xavier accelerator.

#Benchmark pc free#

There were another 3,948 power measurements reported by Krai, the stealth-mode AI startup that always submits a large number of test results in the category 'Open Edge,' where submitters are free to use non-standard neural network approaches. This time around, there were 576 reported power results for inference in cloud data centers and in cloud 'edge' devices, across 30 different systems. As ZDNet reported in September, the number of submissions for power consumption had plummeted to just 350 submissions from 864 in the April report. For example, Intel and Hewlett Packard Enterprise, which both had multiple submissions to report in September, were absent from the latest report.Īcross different benchmark scores, said the MLCommons, results show as much as 3.3 times speed-up for computers running neural network tasks such as natural language processing, image recogntion, and speech recognition.Ī highlight of the report this time around is that more vendors submitted more results to measure the power consumption of their computer systems on AI tasks. Various companies that participate in the four-year-old effort may not show up from one report to the next.

That is up from 754 reported test results from 67 systems submitted by 13 submitters in the September version of the benchmark. 'It's hard to operate over a wide performance range, but that's actually something we've done very well.'įor example, the test for inference in cloud data centers, where the bulk of submissions are given, this time reported 926 distinct test results, across 84 systems, by 14 parties. The fastest performance on the benchmark ResNet-50 is a million times faster than the slowest system, he noted. 'This is a huge dynamic range,' said Kanter. Those results span a wide range of computing, from data centers down to what is known as 'TinyML,' running on devices such as embedded microchips that sip fractions of a watt of power. The MLCommons's executive director, David Kanter, lauded the record number of submissions, over 3,900. Those parties join frequent submitters Nvidia, Qualcomm, Dell, and Microsoft. Wednesday, the MLCommons, the industry consortium that oversees a popular test of machine learning performance, MLPerf, released its latest benchmark test report, showing new adherents including computer makers ASUS, H3C, and ZhejiangLab, a research institute formed by the Zhejiang province government in China, Zhejiang University and Chinese retail and AI giant Alibaba.