qualcomm camera wp-8aj

Qualcomm Defines Premium-

Smartphone Cameras

By Aakash Jani Senior Analyst

August 2021

www.linleygroup.com

©2021 The Linley Group - 1 -

Qualcomm Defines Premium-Smartphone Cameras

By Aakash Jani, Senior Analyst, The Linley Group

The Snapdragon 888 and 888+ are the latest flagship processors from Qualcomm, offering a collection of industry-leading camera technologies that provide a lead over competitors on DXOmark’s camera and video benchmark. These processors enable feature-rich 4K HDR video and photography, AI-driven “3A” technology, and industry-leading hardware to allow one-inch camera sensors. Qualcomm sponsored this white paper, but the opinions and analysis are the author’s.

In recent user surveys conducted by Qualcomm, consumers ranked camera quality as the second-most important feature when buying a new premium smartphone ($1,000+); battery life was first, as Figure 1 shows. Additionally, 60% said a premium camera would determine their next purchase. As smartphones become more feature driven, OEMs are increasingly designing their products around tangible capabilities, such as camera quality.

Figure 1. Features determining consumer premium-smartphone purchases. Just behind battery life, camera quality is the second-most important factor when purchasing a smartphone. Additionally, new camera-driven capabilities, such as 8K resolution, help satisfy buyers’ desire for the latest technology and maximum overall performance. (Data source: Qualcomm)

Raw camera quality (megapixels or zoom magnification) is no longer enough. Consumers are basing their purchasing decisions on premium features in addition to a flagship camera. Weekend campers need quick auto focus for the chance encounter with local wildlife. At-home chefs require built-in bokeh to accentuate their plating. Future pro athletes need highlight reels for prospective colleges.

Qualcomm Defines Premium Smartphone-Cameras

©2021 The Linley Group, Inc. - 2 -

Smartphone cameras became the main driver for many customers’ decisions because of their imminent crossover with DSLR quality. They deliver double-digit increases in processing power year over year while continually updating features through software patches. Bulky DSLR cameras, by contrast, make smaller updates only about every two years. As a result, premium smartphones are already similar to mainstream DLSRs and are rapidly catching up to high-end prosumer models.

Qualcomm enables smartphone OEMs to deliver these desirable features by providing industry-leading hardware and software expertise to facilitate camera design. Its latest premium Snapdragon mobile chips, the 888 and 888+, combine a triple-core image processor with an industry-leading AI engine that carries a rating of 32 trillion INT8 operations per second (TOPS). The company built the camera platform to process 2.7 gigapixels per second, supporting 4K HDR video and photography.

Color and Resolution Technology

Qualcomm introduced 4K video to smartphones in 2013 with the Snapdragon 800, the first Android chip to support that resolution. Five years later, the company added 10-bit color, allowing phones to capture 4K HDR video in one billion colors. In last year’s Snapdragon 865, it continued to make incremental upgrades to its 4K capability, including 120fps slow-motion video capture, and it introduced 8K video capture (at 30fps).

The company delivered 10-bit color for video while competitors were still stuck at 8 bits, allowing Snapdragon to provide finer gradation between colors and tones. The difference between the two is most evident when shooting twilight landscapes: compared with 8-bit color, 10-bit color enables dark-blue and black regions to make smoother transitions from shadowy landscapes to the emerging sun.

Early in Snapdragon’s development, Qualcomm acknowledged the shortcomings of the JPEG format, which is limited to 8-bit color. The 855 was the first Snapdragon to provide hardware support for HEIF photo and video capture. HEIF photos were still limited to 8 bits, but HEVC video was 10 bits. The Snapdragon 888 ultimately became the first smartphone processor to use HEIF to take photos and HEVC to record videos in 10-bit format for extended color capabilities.

High-dynamic-range (HDR) photography has grown in popularity as more monitors and smartphone displays implement it. The technology captures differences in light with greater fidelity than standard definition range (SDR). Revisiting the twilight example, an HDR-enabled smartphone camera will capture interactions between light rays and clouds that an SDR-enabled camera would miss.

Each smartphone OEM has a unique approach to HDR, which produces differences in the types of lighting it can handle. Smartphone vendors either use spatial, such as qHDR, or temporally multiplexed, such as MFHDR, sensors. The former includes specific pixels for different exposures, and the final image is typically created at one-fourth of the full



sensor resolution. These sensors excel at reducing ghosting artifacts but are limited in dynamic range. MFHDR sensors capture the exposure frames at different times, and the resultant frames are stitched together to produce a final full-resolution image. This method offers strong dynamic range, but the time lag can produce ghosting, as Figure 2 shows. As a compromise, OEMs currently use (and will continue to use in the near future) sHDR sensors that offer a compromise between qHDR and MFHDR. To offer its OEMs flexibility, the Snapdragon image processor supports all of these methods.

Figure 2. HDR-sensor comparison. Relative to MFHDR, qHDR sensors reduce ghosting, but they have lower dynamic range; OEMs may therefore choose sHDR sensors as a compromise. Snapdragon image processors natively support all three of these types. (Images captured by Sony Semiconductor Solutions and processed by Qualcomm)

AI Enables 3A and More

Photographers can manually adjust the focus, exposure, and white balance of standard DSLR cameras in accordance with the subject to improve the quality of the shot. They can also remove blur from the foreground image by calibrating the camera’s diopter through the lens, optimizing focus. Exposure determines how bright or dark an image appears and is controlled by the aperture, shutter speed, and sensor integration time and gain. Overexposure leads to washed-out pictures, while underexposure produces grungy and murky pictures. White balance is closely tied with color temperature; it adds the opposite color to an image to rebalance the temperature. In daylight, the same photo with different white-balance settings can appear anywhere from gray to blue.



Qualcomm automated these three aspects of photography, using AI to increase photo quality. Its auto-exposure, auto-focus, and auto-white-balance (3A) effort began in 2014, employing dual LEDs for ambient-light matching and an auto-focus algorithm that prioritized face matching. Over the past seven years, the company hit 3A milestones by adding features such as scene-change awareness, motion-aware auto exposure, and IR/RGB-assisted auto white balance.

These improvements can turn a smartphone user from an amateur photographer to a near professional. Qualcomm trained many of them by studying people’s eye movements as they viewed different images. Instead of always focusing on the foreground, Snapdragon products can distinguish between aspects of a scene (e.g., mountains or people) and auto focus accordingly.

After talking with Snapdragon Insider and expert photographer Ryan Resatka, we learned about some important features that caught professionals’ eyes. Resatka was immediately enamored with the color profile in 888-powered devices. Color profiles act as a unique fingerprint for the chip vendor; the 888’s is cohesive and enables a device to capture authentic colors, requiring less postprocessing. Canon DSLR cameras set the standard for color profiles, but by implementing better color and white-balance technology, the 888 comes closer to that level.

For Resatka, color is the most important feature when choosing a camera: smartphone, or DSLR. Additionally, he found that the auto focus of the 888 platform was extraordinarily accurate and that the auto white balance pairs well with the HDR sensors. The device allowed him to capture detailed shots even when shooting straight into the sun. For professional photographers, a phone comes in handy when their main camera runs out of battery or breaks. Snapdragon-based models create RAW files that meet the high standards of these specialists while providing an easy-to-use interface for amateurs and prosumers.

Qualcomm integrates AI into more than just its 3A platform: it also applied its neural-network expertise to low-light photography and video. Software developer BlinkAI used Snapdragon’s sixth-generation AI Engine in the Xiaomi Mi 11 smartphone to capture video in as little as 0.1 lux, an Android first. As Figure 3 shows, the video reveals interactions between light and water, preserving both bright and dim areas.



Figure 3. Low-light video still. Through a partnership with BlinkAI, Qualcomm and Xiaomi released the first Android phone with low-light-video capabilities. (Image source: BlinkAI)

Snapdragon Leads on DXOmark

Smartphones are now the primary tool for amateur photography, so objective evaluations and rankings are important. DXOmark has become the leader in camera and display benchmarking with its comprehensive suite of tests. The company takes more than 1,600 photos and two hours of video on each smartphone model to quantify camera quality—a difficult task. The suite breaks down an overall score by individual features, such as exposure, color, and auto focus for both camera and video modes.

Serving in the Xiaomi Mi 11 Ultra, the Snapdragon 888 won first place on DXOmark’s benchmark list with a total score of 143, as Figure 4 shows. The Mi 11 Ultra beat other flagship phones, such as Apple’s iPhone 12 Pro Max (based on that company’s A14 processor), Huawei’s Mate 40 Pro+ (based on the HiSilicon Kirin 9000), and Samsung’s Galaxy S21 Ultra (based on the Samsung Exynos 2100). It ranked exceptionally well in the artifacts subcategory thanks to its staggered HDR technology, leading its competitors in both the photo and video categories. In other categories, the Xiaomi phone was on a par with the Mate 40 Pro+, as both scored similarly for better-known features such as focus, exposure, and texture.



Figure 4. Comparison of flagship-smartphone cameras. In overall performance as well as in the video and zoom categories, the Qualcomm processor leads the competition. For camera (still-shot) quality, it virtually ties HiSilicon’s Kirin 9000. Snapdragon’s high scores owe in part to its custom staggered HDR sensor technology. (Data source: DXOmark)

Qualcomm didn't reach the proverbial peak without assistance. It co-developed many smartphone technologies with OEM partners, enabling numerous Android firsts. Sporting a Snapdragon processor, Oppo’s Find X3 was the first Android phone to capture 10-bit HEIF photos, approaching RAW quality. In Xiaomi, Oppo, and OnePlus phones, Qualcomm technology allows recording of computational-HDR video at 4K resolution, another Android first.

Samsung and Xiaomi phones flexed Qualcomm’s tri-core image signal processor (ISP) to implement “director’s view,” which allows the user to preview images from three cameras simultaneously. While its triple-core ISP enables parallelism, it also delivers gigapixel throughput, supporting 108MP camera sensors. Although it scored well on the DXOmark suite, Qualcomm’s leadership camera technology also has features the benchmark doesn’t directly measure.

For example, Qualcomm worked with Leica and Sharp to debut a massive 1-inch image sensor, the largest to date, to help mimic DSLR lenses. Using the immense sensor, the Leica-branded phone takes in more light than competing phones and features a f/1.9 ultrawide lens. The sensor delivers 20MP still shots with superior dynamic range and noise reduction. Leica and Sharp employ the triple-core ISP to enable the photography company’s signature monochromatic user interface alongside computational bokeh and edge refinement.



Conclusion

Qualcomm began developing 4K technology more than eight years ago and delivered substantial upgrades for each product cycle, culminating in 108Mpixel photography and 8K video in the Snapdragon 865 as well as in the latest Snapdragon 888. This feat required color, file-format, and HDR-sensor optimizations to give customers a satisfying high-resolution HDR experience. Qualcomm changed the camera ecosystem by abandoning the popular JPEG format, adopting HEIC instead, and by investing heavily in various HDR sensor support instead of any single sensor. These efforts proved fruitful: they enhanced color/tone authenticity and reduced motion-based artifacts.

While improving the raw camera capabilities, the company pushed equally hard to enhance related features. Its 3A platform aims to simplify complex tasks such as exposure and white-balance optimization. As its AI capabilities advanced, Qualcomm also integrated neural networks to enable new features, such as low-light photography. DSLR-quality photos thus became easier to achieve, allowing more people to capture high-quality images.

Qualcomm’s long-term efforts and OEM partnerships earned it 5 of the top 10 spots on the DXOmark leaderboard. But by offering numerous camera features, the company’s processors enable phones to deliver more value than what the test suite captures, such as low-light video.

Each year, Qualcomm continues to innovate and lead in smartphone processor cameras. Snapdragon processors not only best the competition in DXOmark, but they also lap their predecessors. When it released the Snapdragon 888, the company delivered more than 20 industry firsts, many camera related. It will continue to define premium camera offerings as new Snapdragon processors enter the market.

Aakash Jani is a senior analyst at The Linley Group and a senior editor of Microprocessor Report. The Linley Group offers the most-comprehensive analysis of microprocessors and SoC design, going beyond business strategy to examine internal technology. Our in-depth articles cover topics that include embedded processors, mobile processors, server processors, AI accelerators, IoT processors, processor-IP cores, and Ethernet chips. For more information, access our website at www.linleygroup.com.

qualcomm camera wp-8aj

Documents