AI performance soared 100 times in 6 years, and CPU overtook Apple and Intel？ Qualcomm’s strongest chip cheats announced.

Core East and West (WeChat official account: aichip001)
Author | Heart margin
Edit | Desert Shadow

Core things reported in Maui on October 25th (October 26th, Beijing time). Yesterday, Qualcomm launched the third generation Snapdragon 8, the flagship PC platform Snapdragon X Elite, the flagship audio platform S7/S7 Pro, and the cross-platform technology of snapdragon seam (Qualcomm threw out a three-piece set of the strongest chips! The mobile phone runs a large model with 10 billion parameters, and the PC chip attacks Apple.

During the keynote speech on the second day of the 2023 Snapdragon Summit held today, many senior executives from Qualcomm took turns to go into battle, and delivered dry goods with intensive technology for more than 3.5 hours, explaining in detail the core technologies behind the remarkable performance improvement of three new flagship chips.

Zhao Ming, CEO of Glory, announced on the spot that Glory Magic6 is equipped with 7 billion parameters of end-side AI large model technology, and announced that Glory will join the X Elite family in Snapdragon to design an Arm-based PC.

In order to demonstrate the generative AI strength of the third generation Snapdragon 8, Qualcomm demonstrated the terminal side running voice booking, voice dialogue AI virtual assistant, and AI literacy map at the Snapdragon Summit.

However, I accidentally overturned my car when demonstrating a Chinese conversation with the Hundred Rivers Intelligent Language Model-there was no response. It may be due to the control of the speech time, but the demonstration was skipped without restarting.

Qualcomm called Snapdragon X Elite the "most powerful, intelligent and efficient" processor in the same class of Windows processors. The PC equipped with Snapdragon X Elite is expected to be available in mid-2024.

Snapdragon X Elite adopts 4nm process technology, equipped with 5G baseband chips Snapdragon X65 and Fast Connect 7800 Wi-Fi/ Bluetooth chips, and adopts the most advanced CPU memory architecture design and LPDDR5X memory in Qualcomm, with a maximum memory bandwidth of 136GB/s and a total cache of 42MB. The low power consumption can support a PC charging tube for several days at a time.

Judging from several test data exposed by Qualcomm, Qualcomm said that Snapdragon X Elite would "change the rules of the game", and it was really not bragging.

At the same power consumption level, the peak CPU performance of Snapdragon X Elite is twice as high as that of Intel Core i7-13800H, a 14-core x86 processor. Under the same peak performance, its power consumption is 65% less than that of i7-13800H competitors.

Compared with single thread, under the same peak performance, the power consumption of single-threaded CPU of Snapdragon X Elite is 30% less than that of Apple M2 Max, an Arm processor, and 70% less than that of Intel Core i9-13980HX, an x86 processor.

Compared with multithreading, the competing object is M2, and its peak multithreading CPU performance is 50% higher than that of Apple M2 at the same power consumption level. At this point, it is estimated that the multithreading performance has not hit M2 Max.

The chip adopts customized integrated Qualcomm Oryon CPU, with 12 cores and 3 Cluster design. The maximum frequency is 3.8GHz, and the dual-core boost frequency is 4.3GHz. It is the first Arm architecture PC processor with a main frequency of 4.0GHz, which can support video transcoding, large data set analysis, code compilation and other functions, and provide long battery life.

Looking at the integrated Adreno GPU, the performance improvement is also considerable. In the test data, we can "punch" Intel Core i7-13800H and "kick" AMD Ryzen 9 9740HS.

Its GPU computing power can reach 4.6TFLOPS, and it supports up to 4K@120Hz internal display, HDR10, 3 UHD Ultra HD or dual 5K external displays.

Snapdragon X Elite also has the ability to run large models on the terminal side, and its AI processing ability is 4.5 times that of competing products. For example, the terminal chat assistant for Windows 11 PC (based on the 7 billion Llama 2 language model) can process 30 token per second.

For another example, when running the classic Wensheng graphic model Stable Diffusion 1.5, the image generation speed of Snapdragon X Elite is 2.7 times that of Intel competitors; When running the optimized version of Stable Diffusion, its image generation takes less than 1 second, and the speed is 20 times that of competing products.

UL Procyon AI inference benchmark test shows that the performance of Snapdragon X Elite is 10 times faster than that of x86 processors Intel Core i7-13800H and AMD Ryzen-9 7940HS, which enables users to enjoy a more seamless experience when using video calling functions such as background blur, noise reduction and AI filter.

This is mainly due to Qualcomm Hexagon NPU, which is responsible for accelerating AI computing. It has optimized the architecture of generative AI and introduced micro Tile reasoning technology, which has increased the processing speed of large matrices to 2.5 times and doubled the large shared memory. Hexagon NPU can achieve AI computing power of up to 45TOPS INT4, which is 100 times higher than that in 2017.

The upgraded Qualcomm AI engine has a performance of 75TOPS, and supports the production AI model with more than 13 billion parameters running locally on PC.

Snapdragon X Elite is the first PC processor integrated with a constant awareness ISP. Its built-in Qualcomm Sensing Hub is an always-on low-power AI subsystem. Compared with the previous generation products, its AI performance is improved by 2 times and its memory is improved by 50%.

Qualcomm Sensor Hub integrates a constant sensing ISP, two micro NPUs, a DSP and memory, which can keep sensitive data on the computer, enhance security, login experience and privacy, and has the ability to wake up devices in sleep mode.

Thanks to the software, Qualcomm AI software stack makes the development based on Snapdragon X Elite faster and easier.

In addition, Snapdragon X Elite supports fast connection between 5G and Wi-Fi 7, immersive lossless audio, more advanced camera ISP, and provides intelligent security functions, including Microsoft Pluton solution run by Qualcomm security processing unit, full memory encryption, Microsoft security core PC support, and zero trust protection for sensitive information such as biometric information.

According to Qualcomm, 94% of business leaders indicate that AI has improved their productivity, and about 50% of IT decision makers are prepared to change PC brands based on AI performance.

Then, Qualcomm exposed a picture of Xiaolong’s computing ecosystem.

The compilation speed of Snapdragon X Elite is 25% faster than that of 12-core Intel Core i7-1360P.

The following pictures show some leading PC OEM that Qualcomm has cooperated with.

Qualcomm called the third generation Snapdragon 8 "a master of terminal-side intelligence". The flagship mobile chip adopts 4nm process, and is equipped with the fastest device-side memory LPDDR5X(@4.8GHz), 5G baseband chip Snapdragon X75 with integrated AI tensor hardware accelerator, and Fast Connect 7800 Wi-Fi/ Bluetooth chip.

Compared with the previous generation, CPU performance is improved by 30% and energy efficiency is improved by 20%. This upgrade is related to the change of core configuration. Besides the highest frequency upgrade, it has one more performance core and one less energy efficiency core than the previous generation.

The Kryo CPU of the third generation Snapdragon 8 is based on 64-bit Arm architecture, and adopts "1+5+2" core configuration, including one main core Cortex-X4 with the highest frequency of 3.3GHz, five performance cores with the highest frequency of 3.2GHz, and two energy-efficient cores with the highest frequency of 2.3GHz.

In contrast, the second-generation Snapdragon 8 adopts a "1+4+3" core configuration, including a Cortex-X3 with the highest frequency of 3.36GHz, four performance cores with the highest frequency of 2.8GHz, and three energy efficiency cores with the highest frequency of 2.0GHz.

Achieving optimal performance benefits from the blessing of heterogeneous computing.

Different hardware combination algorithms can support different use cases such as audio/voice, voice assistant, photography/video, and generative AI.

1. Generative AI: Let the mobile phone run a large model with tens of billions of parameters.

Qualcomm demonstrated the third-generation Snapdragon 8 running AI functions such as voice booking, voice dialogue AI virtual assistant, and Stable Diffusion text map on the terminal side.

For generative AI, Snapdragon 8, the third generation, took the lead in supporting multi-modal general AI model, and now it has supported running large models with more than 10 billion parameters. Running a large language model can generate more than 20 Token per second, and it takes 0.6 seconds to generate an image with Stable Diffusion model.

Compared with the previous generation platform, the AI performance of Hexagon NPU is improved by 98% and the energy efficiency is improved by 40%, which is mainly achieved by optimizing the main frequency and improving the micro-Tile reasoning speed.

Hexagon NPU architecture is shown in the figure. From left to right, there is an accelerator-specific power rail, upgraded micro Tile reasoning, upgraded micro architecture, Tensor acceleration unit with higher bandwidth, Scalar and Vector processing unit with higher clock speed, and large shared memory with twice the bandwidth, which together support the large model reasoning on the terminal side.

Qualcomm AI engine consists of Hexagon NPU, Adreno GPU, sensor hub, Kryo CPU and memory.

Qualcomm sensor hub integrates two constant sensing ISPs, two micro NPUs, a DSP and memory, which can safely use data and make AI virtual assistant provide users with more personalized responses. Compared with the previous generation products, its AI performance is improved by 3.5 times and its memory is improved by 30%.

Compared with the cloud, the terminal can respond immediately when running the big language model without delay, which is helpful to improve the user experience.

Qualcomm AI software stack is fully optimized for the third generation Snapdragon 8 and Snapdragon X Elite, and has released more than 20 models, and added support for PyTorch ExecuTorch.

2, image: AI deeply participates in optimization, and it is a blockbuster with a shot.

The upgraded Hexagon NPU can not only run the generative AI model, but also cooperate with cognitive ISPs and sensors, so that the third-generation Snapdragon 8 can support better photography and image editing functions, including photo expansion with AI background, video object erasure, AI background generation and so on.

For example, when shooting a video with a duration of 30 seconds and a frame rate of 60fps, the ArcSoft video object erasure function can remove unwanted backgrounds or people, making video keying easy.

For another example, Qualcomm cooperated with Visionary AI to optimize night scene video shooting with AI, which can shoot clear video in dark environment.

The third generation Snapdragon 8 is the first mobile platform to introduce Dolby Vision HDR shooting technology, which can shoot a richer range of colors and shadows.

Last year, the cognitive ISP image processor pioneered by Snapdragon 8, the second generation, came into play again, including three 18-bit ISPs, which can achieve real-time image semantic segmentation of up to 12 layers.

By semantically segmenting different objects such as faces, skin, hair, eyes, buildings, trees, etc. in the picture, and then optimizing them according to the characteristics of each object, a more textured tuning effect can be achieved. Cognitive ISP is directly connected with Hexagon NPU, so it can optimize the shooting process in real time, and it is not a problem to complete the background replacement of Vloggers view.

There is also a Vlogger View function, which can put the video layers taken by the front and rear cameras into one view. Its improved image segmentation technology can delete the background from the selfie video, making you look like you are standing in front of the background taken by your rear camera.

Qualcomm wants to turn the camera into a powerful hand-held image sensor. In optimizing the depth of field of shooting video, depth and RGB sensors can capture depth, saving 95% power consumption with AI; The depth map of 1080p30 frame can be realized by matching with iToF sensor.

Qualcomm also cooperated with Samsung to create a photo experience optimized for Snapdragon. Using Samsung’s 200-megapixel image sensor, Samsung Zoom Anyplace zoom technology can take 4 times close-ups of moving objects without losing image quality.

In addition, Qualcomm and Sony launched two new HDR image processors last year, and now they are building a new DCG HDR image sensor (Dual Conversion Gain). The third-generation Snapdragon 8, which supports the DCG HDR image sensor for the first time, can take better photos and video shoot for computational photography.

The third generation Snapdragon 8 supports three video shots of HDR image sensors. Through targeted fine-tuning, it can seamlessly switch between any HDR modes, saving 60% energy consumption through background operation between sensors.

Some Qualcomm partners are optimizing sensors to monitor all kinds of information. For example, Prophesee’s event vision sensor can stop taking paste pictures when capturing dynamic pictures, stmicroelectronics’s smaller and more accurate smoke and air quality monitoring device, and two constant-sensing cameras support code scanning and face recognition.

TrinamiX cooperated with Qualcomm to launch the first mobile near-infrared spectrometer integrated in the reference design of smart phones, which can be used for the determination of molecular biomarkers, non-invasively monitoring skin health and analyzing food nutrition.

3, the game: support real-time light chasing, supporting the frame rate of 240fps.

The performance of Adreno GPU of the third generation Snapdragon 8 is improved by 25%, the energy efficiency is improved by 25%, and the optical tracking performance is improved by 40%.

The chip supports the new Snapdragon Elite Gaming feature, and is the first mobile platform in the world to support playing 240fps games. By adopting the frame generation algorithm of Adreno Frame Motion Engine 2.0(AEME 2.0), the frame rate is doubled while maintaining the same power consumption.

It supports 8K super-resolution with the help of Snapdragon game super-resolution technology, and it is also the first mobile processor that can run unreal engine 5 Lumen real-time global lighting system at full interactive frame rate, which makes the lighting and lighting effects of mobile games more vivid. Netease’s martial arts mobile game "Against the Water" will support these technologies.

Qualcomm’s Snapdragon Sound Snapdragon listening technology has the characteristics of almost no delay in playing game sounds, lossless and high-resolution music streaming, stable connection in complex communication environment, and support for Bluetooth classics and LE audio. This technology has passed the formal certification of Hi·Res Audio Wireless standard of Japan Audio Association.

Snapdragon’s listening technology has been applied to the latest third-generation Snapdragon 8 and Snapdragon X Elite. When it is used together with the terminals using the first-generation Qualcomm S7 or S7 Pro audio platform, users will enjoy a better audio experience.

The first-generation S7 and S7 Pro audio platforms newly launched by Qualcomm have improved the performance of DSP to three times that of the second-generation S5 of the previous generation platform, the AI performance to nearly 100 times that of the previous generation platform, the computing performance to six times that of the previous generation platform, and the storage performance to three times that of the previous generation platform. The first batch of terminals equipped with this audio platform will come out next year.

Its dedicated cores for audio management include the fourth generation Qualcomm Adaptive Active Noise Reduction (ANC), hearing loss compensation, transparency, human voice amplification, noise management and so on. When the earphone is loose, or the noise in the environment suddenly changes, ANC can automatically and seamlessly switch between noise reduction mode and transparent mode to improve the call effect in the open environment.

The audio platform is built with a dedicated Qualcomm micro NPU AI engine, multiple DSP cores and a sensor hub. The cooperation of end-side AI technology and audio technology can further optimize the sound experience on the device side with ultra-low power consumption without sacrificing battery life.

The main difference between the first generation S7 and S7 Pro lies in the scope and quality of communication connection.

The first generation S7 Pro (the architecture is shown below) is the first audio platform that supports Qualcomm Extended Personal Area Network (XPAN) technology and ultra-low-power Wi-Fi connection, supports Bluetooth 5.4 and Auracast to broadcast audio, and can seamlessly transition from Bluetooth to Wi-Fi to ensure stable connection.

XPAN can achieve a connection distance far beyond that achieved by Bluetooth only at present. When it is used with Snapdragon Listening Technology, even if your mobile phone is far away from the other side of a building, it can still maintain uninterrupted lossless audio connection between headphones and mobile phones.

The audio platform can provide 24-bit multi-channel lossless music streaming up to 192kHz via Wi-Fi, and can provide 10-hour unlimited lossless music streaming at 96kHz via Bluetooth and Wi-Fi without increasing power consumption.

Thanks to Seamless technology, headphones can be found in time and seamlessly switched to other terminals.

For example, when the user is using headphones to connect to the mobile phone to listen to music, then the laptop suddenly accesses a video phone, and the headphones can immediately switch to the connected computer, and then automatically switch back to the connected mobile phone after the call, so as to continue listening to music.

Metauniverse and space computing are also the direction of Qualcomm’s efforts.

At the end of September this year, Qualcomm launched two new space computing platforms-the second generation Snapdragon XR2 and the first generation Snapdragon AR1, to support the creation of a new generation of mixed reality (MR), virtual reality (VR) and augmented reality (AR) devices.

The first generation of Snapdragon AR1 has been shipped and used in the latest smart glasses products of Meta. Smart glasses equipped with AR1 can be used for taking photos and videos, and are assisted by powerful AI assistants. AI can be used to enhance perception, such as helping to read menus or translating conversations in real time.

LG, OPPO, NTT QonoQ Devices, Thunderbird Innovation and Sharp all used the first generation Snapdragon AR1. Qualcomm also cooperated with upstream and downstream manufacturers to build a smart glasses reference platform based on the first generation AR1 of Snapdragon, and there will be different designs for reference.

In the space computing platform, AI is also an important foundation. Qualcomm is also exploring the combination of smart phones and AR glasses, and running a generative AI application on the terminal side. The following figure shows the partner list of Qualcomm XR developer platform Snapdragon spaces.

At today’s Snapdragon Summit, Zhao Ming, CEO of Glory, shared the achievements of cooperation with Qualcomm on the spot, and showed the strength of Glory in folding screen mobile phones, running large models on the terminal side, and cross-device collaboration.

He first showed Magic V Purse, a folding screen mobile phone with a thickness of only 8.6mm and a weight of only 214g g. Through short gestures, consumers can use this folding screen mobile phone as two mobile phones, realizing "parallel space", logging in accounts in the first space and the second space respectively, playing the same game with two accounts, and even playing with themselves through dual screens.

Zhao Ming also talked about the progress of the terminal-side running model, and thought that it needed to balance enough resources, memory, computing power, battery life and data privacy protection, and ensure that the AI strategy recognized the current limitations and provided specific solutions. A video of children dancing and smiling was generated by using Wensheng video model on HONOR through natural language dialogue.

Zhao Ming announced that Glory Magic6 will support the terminal side to run a large model with 7 billion parameters. He said that this is the ability that few manufacturers can do.

Magic6 supports opening emails with eyes, and seamless connection across terminal devices (mobile phones, laptops, tablets, etc.) and operating systems (Windows, Android, etc.). For example, a laptop can directly retrieve pictures from a mobile phone photo album, a mobile phone camera can become a part of the built-in capabilities of the laptop, Android games can be directly moved from the mobile phone screen to the laptop screen and then played, and different devices can be interactively controlled with the same set of keyboards and mice.

Zhao Ming also announced that Glory will join the X Elite family in Snapdragon, design a PC based on Arm, and launch related products next year to deliver a high-quality user experience.

In the two-day keynote speech at the Snapdragon Summit, AI technology became the protagonist of all the latest chip product lines in Qualcomm. As Qualcomm President and CEO An Meng said, in the AI era, terminal-side generative AI is very important to create a powerful, fast, personalized, efficient, safe and highly optimized experience.

The prologue of the end-to-end generative AI revolution has just begun, and Qualcomm Snapdragon is in the front row. With the entry of mobile phones and PCs equipped with the third generation Snapdragon 8 and Snapdragon X Elite into thousands of households, it will be the general trend for more productive AI use cases to migrate to the edge side and terminal side.