Forget about megapixels! Smartphone camera will whiten your teeth

Based on materials Android Authority

In the past couple of months, Qualcomm and MediaTek have announced their flagship processors.

new generation to be used intop smartphones in 2023. In fact, smartphones with these chipsets have already been announced or entered the market, for example, the OnePlus 11, the Xiaomi 13 series and the Vivo X90.

New processors provide increasedpower, hardware ray tracing and satellite communications. But in terms of camera capabilities, it looks like it will be another year without breakthroughs. MediaTek's and Qualcomm's high-end chips haven't changed much in terms of photo and video resolution or frame rates.

However, camera support is more than justimage resolution. Both companies are bringing quite a few subtle changes to mobile cameras, such as professional video technology, optimizations for 200-megapixel sensors, and native support for RGBW cameras. We are also seeing a trend towards the unification of hardware capabilities for artificial intelligence and image processing, and in 2023 this will allow at least one rather interesting feature to be implemented.

More detailed recognition

Qualcomm touts semantic segmentation inreal-time in Snapdragon 8 Gen 2. Semantic segmentation is the ability to identify certain objects and objects in a frame. This technology underlies many shooting modes. The camera software is able to identify certain scenes or people and then process the image accordingly.

Many smartphone brands use semanticsegmentation in portrait modes for one camera, while others use it for AI scene recognition (sunsets, landscapes, flowers, food). Some companies, such as Xiaomi and Google, even advertise the ability to completely replace the gray sky in your photo with blue.

Miscellaneous

Affiliate material

Reality and prospects of the IT professions market

What professions are the most popular and highly paid?

Saturday coffee #238

Pour a cup of invigorating Saturday coffee andcheck out the news of the week. Realme showed some interesting smartphones, new children's watches went on sale, a new minivan was brought to Russia, and Lego released a large set…

By Nissan to Karelia

In May of this year, Nissan announced the start of sales of the 2021 Nissan Qashqai and Nissan X-Trail crossovers in Russia. And in June we went to Karelia in updated crossovers…

Why flexible screens are doomed

Is this really the same revolution and the next step in the development of our pocket companions?

However, Qualcomm went even further.First, the company confirms that the latest version of its solution is fast enough for both real-time and video capture. The solution is also said to be able to identify certain elements such as teeth, hair, facial hair, lips, tissue, and more. And this can open up a number of interesting opportunities for those who like to shoot on a smartphone.

The most obvious of these is the creation of moreaccurate shots in portrait mode. Qualcomm's own video with ArcSoft demonstrates the ability to more accurately blur uneven backgrounds (see below) while keeping more complex subjects in focus.

However, the most interesting thing is that insmartphones with Snapdragon 8 Gen 2 may have more detailed and advanced beautification effects. Judd Heep, vice president of product management for cameras at Qualcomm, told Android Authority that the technology is initially focused on selfie cameras.

Selfie cameras already offer beautification options for blemish removal, skin smoothing, and shape correction, but this is just the tip of the iceberg in applying real-time semantic segmentation.

Theoretically, more complexeffects, such as teeth whitening. After all, the latest technology really allows you to recognize teeth in the frame. Heap agrees with this suggestion and explains that the company's partners can create their own semantic segmentation networks based on this technology to detect other objects.

"So yes, if you have a network thatrecognizes teeth really well, it can be passed to the image signal processor and it can discolor the teeth and turn them from yellow to white. Yes, it's quite possible."

Qualcomm is also touting the abilityrecognize hair, stating that this feature can be used to create more detailed images. It is theoretically possible that manufacturers implement gray hair removal or the ability to completely change hair color. Heep suggests that removing gray hair can be a challenge, especially if it's just a few gray hairs among many dark ones. He believes that a complete change in hair color is possible, although it may look unrealistic.

However, manufacturers will have to balance onThere is a fine line between providing beauty features that people demand and promoting distorted beauty standards. After all, we've seen a lot of dubious effects and filters over the years, such as facial thinning, nose reshaping, skin lightening, and eye widening.

However, the scope of advanced semanticimage segmentation is not limited to beautification. As shown in Qualcomm's video, this technology can also improve the display of clothing by adding sharpness to a jersey or jacket without affecting the rest of your body. The video even shows the ability to remove glare from glasses.

Qualcomm also confirmed that semanticreal-time segmentation can be programmed. Thus, companies can run different neural networks if they have other use cases for the technology.

Will the technology come to commercial devices?

Support for features like semantic segmentationreal-time is a marketing advantage for chipset makers, but will this technology really make its way into smartphones? Smartphone makers are not unanimous about the use of chipmaker camera features (eg unlimited slow motion at 960 fps, 120 fps burst mode on a 12 MP camera).

Heep confirmed that this feature is available out of the box for all brands of smartphones. “So there are no licensing fees, the manufacturer doesn't have to do anything else,” he explained.

In other words, it will not be easythe theoretical presence of the function, but what will appear in commercial devices in 2023. Therefore, we will follow the announcements of Samsung, Xiaomi, Oppo and other brands and see if real-time semantic segmentation will appear in their devices.

AI + image processing

Improvement of semantic segmentation fromQualcomm is possible thanks to its proprietary Hexagon Direct Link feature. Qualcomm effectively creates a link between the artificial intelligence chip and the image processor responsible for processing frames from the camera. MediaTek is following a similar path with the Dimensity 9200 chipset, claiming that it combines artificial intelligence and an image processor to better realize 8K/30fps and 4K/60fps with electronic stabilization. Meanwhile, Google's semi-custom Tensor chips in Pixel phones also use an AI processor that's fully integrated into the image processing process.

This combined approach, combining an AI chip andimage processors, from Qualcomm and MediaTek in particular, means the camera data can bypass comparatively slow RAM to provide real-time photo processing. Faster processing doesn't just mean less time in front of you with a frame processing message before a preview is available. This could potentially enable viewfinder previews in various modes, new photo modes, and new video features.

Qualcomm and/or MediaTek are already promising advancedcamera features in their latest chipsets, such as improved bokeh video, ultra-high video resolution, photo blur removal, and improved low-light results. But it's not hard to imagine future features such as more detailed and efficient augmented reality filters, Magic Eraser for movies, HDR burst mode for every shot, or multi-frame processing for full 50MP or 108MP shots.

Gadgets

As an advertisement

Robot vacuum cleaner 360 Botslab P7

Budget robot vacuum cleaner with voice control, the ability to build room maps and an operating time of up to 90 minutes, as well as wet cleaning.

360

Samsung OneUI 5.1 shell review - all the changes

Minor OneUI update with minor improvements. Some of them are very nice, some are copied from iOS.

All new Oppo INNO Day: competitor for Samsung Galaxy Flip

Two new smartphones, a breakthrough in wireless audio, and a wellness egg for the whole family.

Overview of the smart speaker "VK Capsule Neo" with disassembly

Inexpensive smart speaker with bright stickers included: a great gift for a child…

For the first time we got a taste of the futureopportunities when cameras bypassed traditional RAM in the 2017 Sony Xperia XZ Premium. This phone features a camera sensor with its own dedicated RAM, enabling for the first time ever 960 fps super slow motion video. So it's interesting to see what else is possible with the camera's much faster image processing.

Fortunately, this unified approach toAI hardware and image processor will not be exclusive to flagship devices. Heap confirmed that you can expect this feature to appear in mid-range chipsets at some point.

The future of smartphone cameras

Interestingly, both Qualcomm and MediaTek are coming tothe same conclusion about combining the hardware capabilities of artificial intelligence and image processing. And there is no doubt that this could be the basis for future smartphone camera developments. So while today's high-end chipsets don't make big headlines about camera features, they still bring important improvements.

This final step in image segmentationlooks especially interesting. Along with more accurate portrait mode, more detailed image processing, and better beautification, real-time semantic segmentation already provides some interesting features. It's still interesting to see what else smartphone makers will come up with with this mode and a more unified approach to AI and imaging hardware.