Apple claims that the iPhone 12’s A14 Bionic “challenges laptops” but...

Apple hosted its iPhone 12 launch event on Tuesday and announced more details on its latest Apple A14 Bionic SoC. The A14 SoC is at the heart of Apple’s newest iPad Air and all four versions of the new 5G-enabled iPhone 12. During the unveiling of the new iPhone 12 and the A14 Bionic chip, the company made some significant performance claims without qualifying or receiving them to back up. None of Apple’s claims are endorsed anywhere in the company’s hour-long presentation, or on the product pages, or in the press release. The total lack of evidence support is something that even Apple hasn’t done much in the past. Apple usually shows at least one bar chart that compares itself to the stated competition and mentions the performance measurement and the application tested. A simple benchmark bar chart would be at least what is expected in the silicon industry. I reached out to Apple’s analysts on Tuesday for details and got no response.

Question the performance of laptops?

Apple’s first unsubstantiated claim was that the Bionic “challenges the performance of laptops”. There were absolutely no details about the claim in the presentation or on the company’s website. Here’s what I would expect from Apple:

• Which smartphone?
• Which laptops?
• What were the configurations?
• What workload?
• Which benchmarks?

I appreciate the marketing weasel phrase “challenging” and not “striking”, but the question still arises as to what exactly was tested. Have you tested dirt cheap two-year-old Chromebooks with limited experience, or were the latest and greatest laptops with the latest and greatest processors and discrete graphics chosen? Which games were tested and which resolutions? Was the video transcoding done on the CPU or GPU? What was the video density? Have you selected benchmarks that only favor the iPhone? After about 30 years of benchmarking, 10 years for an OEM, 10 years for a processor, and 10 years as an analyst, I’ll tell you this is important.

CPU and GPU claims

Apple claimed that the A14 has both the fastest GPU and the fastest CPU by up to 50% compared to the competition. Flat out, no competitors named, benchmarks used, conditions explained. Nothing. We just have to take Apple’s word for it. In the CPU and GPU world, companies like Intel, AMD, Qualcomm, and NVIDIA would never make such claims without at least running a benchmark, displaying a graph, and then explaining at what settings. If you look at some of Intel and AMD’s performance claims from recent launch events, dozens of footnotes generate multiple pages explaining specific performance claims and how those claims were validated. Apple expects consumers to take their word for it and that reviewers will either trust them or validate those claims if they can figure out how. After all, Apple hasn’t told anyone how it got these numbers.

AI performance claims

In terms of AI, Apple claims that the A14 Bionic has a 16-core neural engine with an 80 percent increase in performance on certain workloads. Apple states that the new Neural Engine can perform 11 trillion operations per second, but does not communicate when it is done simultaneously or in total. Apple also claims that the CPU’s ML accelerators are 70% faster, but we don’t know what Apple is comparing to. For all we know, Apple could compare the A14 Bionic to the A12 or A11. There are no footnotes, no explanations, just significant percentages. We don’t know what workloads Apple is running, or whether it prefers Apple’s architecture, or how the chip stacks up in industry benchmarks like MLPerf. If you go to the Apple website, you cannot find any qualifications of Apple’s claims or how they were made. The technical data of the iPhone 12 only states that the A14 Bionic is equipped with 2^nd Generation Neural Engine.

Console games

Apple also likes to talk about console-quality games, but doesn’t qualify what that means or how it’s defined. When you consider where the Xbox Series X and PS5 are in terms of performance, I don’t think anyone believes Apple can compete with that. It would be helpful if Apple explained which console it was referring to, as consoles with the new SoC are getting a noticeable performance boost this year. Maybe Apple is comparing the A14 to the Switch, but this one has a 2015 NVIDIA SoC, and I’m not sure this makes a great comparison.

So what?

If Apple gets those performance numbers between 50 and 80% that it claims for the A14, I would like to know what applications it is using to get this performance. Realistically, there are performance improvements based on raw theoretical performance capabilities, but without actual benchmarks or applications running on them, it doesn’t really matter to users or developers. We see numbers like FLOPS all the time when we talk about the theoretical peak performance of an AMD or NVIDIA GPU. However, that number doesn’t account for actual real-world performance when you factor in the driver, operating system, API overhead, and how applications use that potential energy. Apple needs to be more transparent about performance claims like the rest of the industry and, based on those performance claims, explain what I can’t do with other phones that I can only do with the new A14 Bionic processor on the iPhone 12.

New Mac performance? Trust us

This alarming trend of Apple making claims with little to no evidence is significant to iPhone buyers, developers, and critically the new arm-based Macs. I expect the company will make many unsubstantiated claims when it announced the new Macs next month, as it did when it launched on Tuesday. The risk is much greater for developers in the new Mac case because developers and consumers are making such a big leap in confidence. More on that later.

I always liked Apple’s claim to “think differently,” but when it comes to undocumented and unfounded processor performance claims, I wish it were more open and direct.

Note: The authors and editors of Moor Insights & Strategy may have contributed to this article.

Disclosure: Like all research and analyst firms, Moor Insights & Strategy offers or has offered many high-tech companies in the industry, including 8 × 8, Advanced Micro Devices, Amazon, Applied Micro and ARM, fee-based research, analysis, advice or advice, Aruba Networks, AT&T, AWS, A-10 Strategies, Bitfusion, Blaize, Calix, Cisco Systems, Klare Software, Cloudera, Clumio, Cognitive Systems, CompuCom, Dell, Dell EMC, Dell Technologies, Diablo Technologies , digital optics, Dreamchain, Echelon, Ericsson, Extreme Networks, Flex, Foxconn, Frame, Fujitsu, Gen Z-Konsortium, adhesive networks, GlobalFoundries, Google (Nest-Revolve), Google Cloud, HP Inc., Hewlett Packard Enterprise, Honeywell, Huawei Technologies, IBM, Ion VR, Inseego, Intel, Interdigital, Jabil Circuit, Konica Minolta, Lattice Semiconductors, Lenovo, Linux Foundation, MapBox, Mavenir, Marseille Inc., Mayfair Equity, Meraki (Cisco), Mesophere, Microsoft, Mojo Networks, National Instruments, NetApp, Nightwatch, NOKIA (Alcatel-Lu Cent), Nortek, Novumind, NVIDIA, ON Semiconductor, ONUG, OpenStack Foundation, Oracle, Poly, Panasas, Peraso, Pexip, Pixelworks, Federdesign, Portworx, pure storage, Qualcomm, Rackspace, Rambus, Rayvolt E- Bikes, Red Hat, Residio, Samsung Electronics, SAP, SAS, Scale Computing, Schneider Electric, Silver Peak, SONY, Springpath, Spirent, Splunk, Sprint, Stratus Technologies, Symantec, Synaptik, Syniversum, Synopsys, Tanium, TE connectivity, TensTorrent, Tobii Technology, Twitter, Unity Technologies, UiPath, Verizon Communications, Vidyo, VMware, Wave Computing, Wellsmith, Xilinx, Zebra, Zededa, and Zoho which can be cited in this article.