Computational Photography Victory

Greetings! We have already published a lot of materials on the topic of what is happening with the modern mass media.

photography.Is it possible to trust a smartphone to “draw” the Moon, a model’s face, and anything else, and then consider the result as a photograph. And I would like to make one prediction as an intermediate result: computational photography in the mass segment will defeat the traditional one. And this is inevitable, because there is not a single aspect in which its opposite could count on success, if we are not talking about highly specialized areas of application. A neural processor and its algorithms in a mobile device will always be more successful in the eyes of the consumer for three reasons at once - because they will be supported by biology, physics and psychology. And it will always be so, until some fundamentally different way of capturing what is happening appears. But only it will no longer be a photograph and generally not an image on a plane. So far, there is no reason why the standard attempt to give maximum accuracy will win over "finished" shots.

Samsung's official explanation of how they "paint" the Moon

Biological aspect

First, let's remember how it works in generalhuman vision. This is a biological stereoscopic optical system that allows you to perceive electromagnetic radiation of a certain spectrum, creating a sense of the position of objects in space. In fact, our vision is an illusion that exists in our head, but based on the interpretation of data received from the external environment. We will return to the illusory nature of perception in the section of psychology, but for now we will delve a little into the physical and chemical aspect. Photons hit the photoreceptor cells, causing a chemical reaction in the pigment, which is then converted back into an electronic impulse transmitted by a neural connection to the brain. In principle, our eye and the camera are arranged in a similar way, so in the case when we look at a photograph, we get some kind of recursion - we see an image of the image.

Image from Wikipedia.org

But there is also an important difference.A photograph is an image on a plane, while the very meaning of the existence of vision for a person is to perceive information in volume, to obtain a spatial picture of the world. Those. no photograph, no matter how carefully we think about the camera that will take it, and the screen that will reproduce it, will not give the same sensations of space that we get with our eyes. Actually, we can always feel the illusory volume in the picture until we start using additional devices that can deceive our vision, forcing us to see the picture not flat, but in its entirety. But these tricks lie not so much in the quality of the frame, but in the way it is shown to the viewer.

Remember when Google suggested turning any smartphone into VR glasses? And all because our vision cannot be deceived by simply showing an image on a plane.

Miscellaneous

Affiliate material

Reality and prospects of the IT professions market

What professions are the most popular and highly paid?

Saturday coffee #243

Pour a cup of invigorating Saturday coffee andcheck out the news of the week. Huawei showed new products, Oppo introduced new flagships, Anker started selling big batteries, and KINGKONG is coming to Russia…

Mitsubishi Pajero Sport test. Real SUV

At the beginning of 2022, according to Avtostat, the Mitsubishi Pajero Sport frame SUV became the brand's second most popular car in Russia, losing the lead only to its younger brother Outlander...

Review of the smartphone Infinix Note 12 G96 (X670)

A mid-budget smartphone with an AMOLED screen, a practical and interesting design, but not the newest hardware…

The fact is that not all cells of the eye are the same.This is not a matrix and not a film with a homogeneous structure, where each pixel and each particle of the photosensitive layer is completely similar to any neighboring ones. If you have not forgotten the school biology course, then remember the rods, cones and ganglion cells. Cones, which are 100 times less sensitive to light than rods, are responsible for color and perception of fast movement. Rods, on the other hand, are photoreceptors that can pick up even two or three photons of light, but are sensitive only to the emerald green part of the spectrum, so they are responsible for peripheral vision and the ability to see in low light conditions. At the same time, the distribution of photoreceptors in the retina is uneven, as well as the very number of sensitive cells and their proportions in the retina.

The sensitivity of the human eye to different colors. Non-ideal plot of the real scatter across the population.

Considering that the principles of lens focusingthe camera and the human eye are the same, and the mechanics differ only in the way the components are “manufactured”, the main difference comes in the “matrix”. The retina of the eye has nothing to do with the concept of uniformity and uniformity, while the matrix of the camera is an ordered and uniform structure. It doesn't make any sense to create something else. First, we do not yet have complete information about the functioning of all eye cells. Secondly, even if we were to get such details, everything would still come down to the question of what kind of vision should be taken as a standard (individual features give tens of percent of the difference).

It turns out that from the point of view of biology, anyphotography is an optical illusion and an illusion. It is currently not possible to create a mechanism for reproducing a person's spatial vision, but if it is created, it will no longer be what we used to call photography. The creation of a flat image that is absolutely reliable from the point of view of human vision is impossible or requires other means of demonstration.

Physical aspect

As for the physical component of the problem"traditional" photo, it needs to be divided into two aspects. The first is the general problem of digital photography, based on the very nature of light and the principles of building a photosensitive digital matrix. The second is the physical limits on the size of the device.

As much as we would like, it is impossible to createan ideal camera in which one photon will hit one pixel, and the quantum effect will be equal to 100%. We need pixels that will capture the maximum number of photons in order to reduce the amount of noise, which depends on the light level. Accordingly, we always need a matrix with a certain signal-to-noise ratio, but this ratio will never perfectly match the actual number of photons.

Get rid of error, called asphotons that did not knock out an electron in the photodiode, and randomly excited electrons due to heating of the matrix, will also not work in practice. Algorithms will always be used to compensate for these parameters, but never exact counting. If it is possible to create such a unit theoretically, then in practice it will be a laboratory installation, hardly at least roughly corresponding to the concepts of mobility. That is, a digital photograph itself will always require some processing with some error, and not at all ideally reproduce the data from the collected photons of light.

If we talk about practically achievablemeans of improving the quality of photography by purely physical methods, then we are left with two directions - improving the optical system and increasing the photosensitivity of the matrix. Both that and another rests against the same barrier - the physical sizes and power consumption. Suppose, in some not so distant future, the issue of energy consumption in mobile devices will be resolved. But to solve the issues of optics in the lens and the number of photons on the matrix without increasing their size will not work.

Visual ratio of the sizes of matrices. The bigger, the better.

There will always be a temptation to add a little moreor reduce, depending on the desired result. Moreover, depending on what you are looking for, you can calculate where the limit is, beyond which the change in size will already be inadequate to the resulting increase in characteristics or cause noticeable damage. 100-150 years ago, group shots were very fashionable, in which you could make out the facial features of each of the several hundred people in the frame. Now many perceive these pictures as a kind of mystery, because neither a film nor a modern digital camera can take such a picture. Simply because there are no cameras of this size in everyday life. Full frame nervously smokes aside when it comes to a meter by meter photographic plate. And the "pathetic" APS-C begins to look very impressive as soon as we impose it on the size of a smartphone.

The famous photograph of the Kexholm regiment.

A camera similar to the one required for this shot.

It turns out that from the point of view of physicstraditional digital photography is running in a circle, when the number of circles and the line can be determined arbitrarily, since you still won’t be able to reach the horizon, and both two-meter giants and dwarfs participate in the competition. And the latter are orders of magnitude more.

Psychological aspect

So, it turns out that the perfect photo,transmitting as it really is, without any conventions, turns out to be unattainable from the point of view of biology and physics. I will make a reservation once again that we are talking exclusively about the image on the plane and with the help of already known technologies. But since we have no way to act otherwise than by selecting some balance between different versions of what looks like the truth, then why not appeal to the historical experience gained over several thousand years of art development? That is, to give the consumer something that is fashionable and beautiful. And for the gradation of beauty and conformity to fashion, psychology is responsible to a greater extent.

Why do I say that in this caseis there an appeal to historical experience? Yes, because it has already happened. The art of the Roman Empire successfully made its way from a sculptural portrait depicting a person as he is, without embellishment, to medieval religious art, where the carnal component was completely uninteresting, so it was necessary to create an incorporeal image. Then, in the Renaissance, it was necessary to rediscover the laws of perspective and the principles of chiaroscuro. But before classical art reached realism, a new turn towards impressionism, post-impressionism, and further to styles followed, whose representatives directly appealed to what we see and how we perceive, again discarding the material, but now not for the sake of religious spirituality, but in search of the essence of beauty.

A visual "line of development" of portraits in Roman art.

And here it turned out that if we considerlarge enough groups of people, then the feeling of beauty may well be generalized. It can be chosen in the right proportion so that the majority say: "Yes, it's not like it really is, but it's beautiful and harmonious." In the digital age, it was this approach that ensured the rise of Instagram*. At a time when mobile photography was an outsider in terms of a purely technical component, since half of the frames required an obligatory explanation of what you wanted to capture in general, the service offered a set of filters that allow you to “make it beautiful”. And this process was automated as much as possible, the user only had to move the settings sliders a little, or even just select a ready-made preset.

The magical world of Instagram* filters for all occasions.

The key point was that the setsatisfied the vast majority and working with him was as simple as possible. Even the existing strict limitation on the mandatory squareness of the frame was not an accidental component of success. Perhaps you have not heard anything about the principles of composition, but the inability to simply take a finished frame and use it in its entirety made you intuitively discard the superfluous, shifting the focus to the main thing. And it was still your work, and not a magic button that turns an ordinary frame into something pleasing to the eye. Until there are frame enhancers based on artificial intelligence. Because why any individual settings at all, if all people follow approximately the same algorithms, wanting to get “beautiful”.

What a variety of ideas about beauty in social networks!

Gadgets

As an advertisement

Robot vacuum cleaner 360 Botslab P7

Budget robot vacuum cleaner with voice control, the ability to build room maps and an operating time of up to 90 minutes, as well as wet cleaning.

360

Overview of the compact flagship Samsung Galaxy S23 (SM‑S911B/DS)

Compact smartphone for calls and instant messengers. Excellent assembly, while similar to the iPhone 14 Pro in appearance, but completely different stuffing and character. The base model of the new line.

Which smartphone with a keyboard can you buy right now in 2023?

Which one you don't need to buy. Are smartphones with keyboards still on the market in 2023?

Huawei AX3 Pro Wi-Fi Router Impressions

I'm talking about a very fast router for home or office. With the disassembly of the device ...

If the vast majority day by dayphotographs the same set of subjects and has the same ideas about which frame will be the most beautiful, then what does the accuracy of transmission have to do with it? You need to look for a set of settings for the photomodule of a smartphone that gives the user a frame that he will then carry on the social network. And since we have a tool that allows us to analyze what kind of frames are posted on social networks and what kind of reaction they cause, the task becomes even easier. Even, it would seem, such an individual feature as faces is a convention. And it doesn't matter if it's divas from beauty blogs of the 21st century or paintings by the Little Dutch, mass art is not striking in variety. It doesn't require it at all.

Neural networks, computational photography is everythingreflection of human nature and the psychology of the masses. We take a certain real-life element and, through simple operations, bring it to a state of socially acceptable, discarding the negative aspects and strengthening the positive ones. Since the computing power of mobile device processors is sufficient to do this on the fly, here is the finished recipe for computational photography. And the further this computing power grows, and the neural networks improve in processing the experience previously accumulated by mankind, the more accurately the camera will guess what the user would like to see.

Instead of a conclusion

Computational photography will inevitably takedominant position in absolute terms. This is just as true as the fact that budget smartphones managed to defeat push-button phones, although by the standards of top-end devices, these are rare bastards that are not able to fully implement any function. And by the standards of classic phones, they have neither autonomy nor reliability. But push-button phones remained 3% of the market. Because more highly specialized gadgets are not available.

Neuroprocessed photographs forsocial networks will never be able to reach the same level as a shot taken by a high-class professional on Hasselblad. But not because his camera has some miraculous properties for the most accurate transmission of reality, but because the time and effort that this professional spent on first learning how to shoot, and then also processing the frame, 99.99 % of photographers will never spend on these skills and, accordingly, they will not receive his skills. And the photo artist will not need the help of computational photography to refine on the fly, because he will process the frame later.

No computational photography required andwhen it comes to fixing the state of objects. A forensic scientist or engineer doesn't need to be pretty, and they don't need composition. They need precision. However, if you look, for example, at what the camera settings of telescopes are, it turns out that this is also a computational photograph. It's just that the scenes and shooting conditions do not change, which means that the set of settings is fixed.

So yes, computational photography will win, but it won't destroy traditional photography.

*Meta Platforms, which includes the social networks Facebook and Instagram, has been recognized as an extremist organization and banned in the Russian Federation.