Sensor Size and Outcomes- The 'how good are smartphones?' debate.

Does the separate, dedicated camera have a place? Aren't smartphones just as good, or even better? I've met several people who think they are, and, to be honest, they have a point. But only 'up to a point'.

Smartphone cameras have come a very long way in two decades. Perhaps the iPhone introduction represents when they came of age. They pleased a lot of people who no longer felt the need for anything else for their photography. They have only improved further since. Then there's the ease and speed of sharing they bring. 

There's no denying the incredible capabilities of the latest camera phones. They represent their own 'ecosystem' with communications and processing of a very advanced nature. 

And yet...photojournalists and 'the press' still generally use DSLRs, or mirrorless cameras with DSLR-size sensors. So do professional event and wedding photographers. And they're used for many other purposes in the media, including smartphone advertising material. Why is this? Sentimentality, or just the need to turn up with gear meeting the expectations of professional customers? Do any technical virtues remain for the 'real' camera?

A DSLR allows an optical preview useable in almost any level of background light. It generally allows almost instantaneous image capture when the shutter is pressed. Mirrorless cameras are almost as responsive. Smartphones are usually a little slower. Opportunists and Paparazzi take note.

Smartphones generally produce punchy, if often not colour-realistic output, fitting the expectations of the generation raised on a certain set of production values and used to computer graphics, often based on heavily-modified colour vibrancy and saturation. A higher-performing camera calibrated as closely as possible to 'colour reality' can initially look a little flat in many situations, viewed straight out of the camera with no editing. Closer inspection will display less grain, higher resolution, and more tonal and colour nuance, but the initial punch of the phone image, especially on a small display, is often higher. Hence people often consider phone cameras to be 'better'. If this is how you approach most of your photography, they are better. Phones also often have effective in-phone dynamic range correction, and apply shadow and other light level correction. All this is automated, and therefore amenable to the average user. Again, a large sensor camera will allow light curve adjustment and exposure stacking to be achieved better, but it will take some effort and understanding. Editing software will likely be necessary for the very best and most flexible results. Because the dedicated camera is intended as a flexible tool, in-camera light curve adjustment is likely to be more minimal, often resulting in an out-of-camera image which is not optimal for purpose. Additionally, in favour of consistent and straightforward results from the phone, is the small absolute aperture. This results in high depth of field, meaning most of the shot is likely to be in adequate, if not brilliant, focus. A large aperture promises better results in terms of resolution and contrast, but focusing becomes much more of an issue. Another convenience plus point for the phone camera. 

Despite the very real progress, there are insurmountable technical barriers for phones as to how far you can go with the small sensors involved. What are they, and how do they restrict the photography outcome? And should you care? It all depends on what you want to do, and how much expenditure and effort you want to put in. 

If you don't need the technicalities, please jump to the conclusion.

The Physics and Engineering

What are the technical limitations of small sensors? Is bigger really better and why?

'Noise'-free, clean images

The surface area of the photosensor in a camera, whether stand-alone or part of a tablet or phone, determines certain aspects of the performance. The ability to capture a grain-free exposure at fast shutter speeds is a direct function of this variable. Yes, grain afflicts digital sensors in much the same way as it does film. Often, digital algorithms can remove a lot of it, but these also lose real detail and texture.

While using a lens of a certain focal ratio, say f/2.8, the total amount of light reaching the sensor in a given time, say 2 milliseconds or 1/500 sec, is directly proportional to the sensor area. This assumes everything in the optical path is suitably designed. An iPhone 11 sensor measures about 6 by 4 mm, giving an area of around 24 square mm. A full frame sensor has an area of about 35 times 23 mm, or 805 square mm. That's about 36 times greater. An APS-C sensor has about 14 times the iPhone area. Basically this means that full frame receives 36 times more light in 2 milliseconds with a f/2.8 lens. The signal-to-noise ratio will be correspondingly higher if both sensor signals are amplified up to provide the same signal level and exhibit similar electronic unwanted signal, known as 'noise'. In practice the noise mechanisms involved will make the situation somewhat better than 'area-proportional' for the phone, but generally total unwanted 'noise' at a certain exposure (light flux density times time) will scale at somewhere between the linear and area ratios of the sensors. Note that exposure is measured by flux density, and not by total light flux. A full frame sensor at ISO 100 will receive 2.6 times the light of an APS-C sensor at ISO 100, and 36 times more than the phone will at ISO 100. The ISO rating is therefore a little misleading with digital camera sensors. At a given ISO, the larger sensor will be 'quieter' with less grainy 'noise'.

A clear win for a clean, grain-free image with a FF or APS-C sensor, and it's down to physics. The FF has a slight edge compared to cropped, of typically just over one exposure stop. Both enjoy a substantial edge over the phone.

Now a phone can still give perfectly satisfactory results, especially in good light and when 'downsampled' to a small display. This latter process works to make noise considerably less of an issue, averaging it down and giving a cleaner looking image at small pixel counts. Most phones average a series of exposures in low light, aligning them and using powerful noise reduction. This can produce really impressive low-light results on the latest iPhones, for example. There may be little room for post-processing as things are already 'close-to-the edge' in terms of shadow noise. Despite the massive efforts made with in-phone image processing, there is still no doubt where the technical performance edge lies, exposure for exposure. There is going to be a lot more flexibility when it comes to post-processing with an image or series of exposures captured on a large sensor.

Resolution- the ability to discern detail

What about resolution? Here, the sensor resolution in megapixels may not necessarily be the governing factor. Optics is unavoidably constrained by an effect called the diffraction (Dawes) limit. This involves a possibly counter-intuitive result: a bigger diameter lens equals a sharper picture. For a perfectly-designed optical system at focus, resolution is proportional to the effective aperture. For a given camera and a particular lens, it's inversely proportional to the f-number. The resolution we are talking about here is the minimum distance between two points, at a certain distance, that the lens is able to resolve. Note that this is about how the lens projects light onto the sensor, rather than how well the sensor itself resolves that projected image. All this is because an optic, illuminated by a single point of light, results in a small disc of light at the sensor, called the Airy Disc. This disc gets progressively bigger as the aperture number increases, and the absolute aperture reduces, from say f/2.8 to f/22.  The Airy disc is a function of the wavelength of light, and represents the effect of diffraction caused by differential light travel time to the sensor via the edges of the aperture. You can't do anything about this at a given f-number.

Note the very significant caveat included in the above discussion. The optic must be near-perfectly designed and manufactured. If it isn't, stopping down will hide the defects better and give a sharper image. it does this by limiting the degradation caused by refractive errors in the lens elements.

We can see from the above that a larger sensor with the same number of pixels can potentially give a resolution advantage for a given f-stop. Thishappens when the sensor is not oversampling the image. Is this actually the case in practice? It depends on the relative size of the Airy diffraction disc and an individual sensor pixel site. For a perfectly-designed f/2.8 optic, the Airy disc at the sensor of a DSLR has a diameter of about 4 microns (4 thousandth of a mm). This actually turns out to be independent of focal length, for reasons we won't go into here. The pixel size of an EOS 1DX III, EOS 5D III or similar camera with full frame and around 20 Megapixels is about 6 microns. The pixel size and diffraction limit are well matched. For a 7D Mark I or II, the pixel size is pretty much exactly the same size as the airy disc. However the pixel size of an iPhone 11 sensor is 1.4 microns. We can see that small high pixel count sensors will begin to make unrealisable demands on the associated optics, even if these are perfectly manufactured. We will not be able to design an optic with sufficiently small f-number to take full advantage of the small sensor pixel count. 

Again, a larger sensor pays dividends with achievable resolution. The high pixel density of many phone and other sensors does still pay off, because it allows pixel shift image stabilisation with less loss of resolution. But optimum absolute resolution still goes to the larger sensor.

Depth of Field- creative freedom

This is partly down to fashion and preference, but prevailing trends for preferred depth of field effect mandate a large sensor. The best a small sensor can do is resort to image processing to attempt to simulate a small optical depth of field. A well-designed large sensor optic will provide very high sharpness and contrast in the area of correct focus, and again, the small sensor cannot match this. Software image sharpening can attempt to replicate this too, but the genuine information from the incoming light cannot be recovered.

Any imaging system has an external plane of focus. With a large sensor and aperture, this can be imagined as a fairly thin 'sheet' in front of the camera where the image is in good focus. This 'shallow depth of field' effect is considered highly desirable in many situations, especially portraiture. It is considered to give a contemporary cinematic quality to images; still or moving. Watch a recent movie and notice what is in focus in various types of scene. 

Conversely, a small aperture and/or small sensor generate a deep field with nearly everything in fairly good focus. The maximum resolution is lower, but that resolution is seen much more uniformly across the image where the subject matter is at different distances from the lens. Large field depth is often wanted, for example, in landscapes and other vistas with near and distant points of interest. It also places less burden on focusing systems or the person required to focus manually. Most cinematic footage, even today, is manually focused, with the result considered as an artistic part of the production.

For a given sensor size, depth of field increases with reducing effective aperture, or bigger f-number. An f1.2 lens gives a very shallow field, whereas an f4 lens will give more depth of field. But the F1.2 can be stopped down to f4 or further if desired. The f4 cannot be opened up. A wide lens is more flexible, creatively-speaking. You can choose the depth of field you want. Additionally, you can use a shorter exposure to freeze action if necessary.

That's different f-number lenses on the same sensor. What about the sensors? Here we find that the achievable shallow depth of field scales with sensor dimension, if the f-number is being held constant. A FF sensor will provide a shallower maximum DoF than APS-C, which will provide a far shallower DoF than a phone sensor. Note again that the FF setup can replicate everything the APS-C can do, if that's what you want, but the reverse is not true. The same and more so for the phone sensor; it will struggle to generate any shallow field effect without resorting to a software simulation. And the maximum point of sharpness will be sharper with the larger sensor. So the ability to generate focus snap, a blurred foreground and background, and an isolated sharply-focused subject is very substantially greater with the large sensor. But because the lens can be stopped down quite drastically, the large sensor can replicate the phone's large depth of field if that is what you want in a given imaging situation.

Creative freedom, even for a single lens choice, is much greater with the big sensor. Depth of field, shutter speed and ISO can be played off for a desired result. And note we cannot realistically compensate the smaller sensor by providing a wider maximum f-stop lens; the required f-number rapidly becomes unrealistically small.

Creative freedom in terms of depth of field goes very clearly to the larger sensor.

Conclusion

A large sensor, with an appropriate lens, allows cleaner images, faster exposures for freezing action, narrower depth of field and higher genuine resolution. The corresponding disadvantages for the phone camera can sometimes be mitigated to a degree in internal software, but the edge of flexibility will always go to the larger sensor.

 

Phone cameras now do an enormous amount of image processing, for example, 'stacking' images to reduce noise and user-motion blur. And software allows the simulation of a blurred background with attractive 'bokeh'. Generally these compensating tricks are inferior to the result obtained by getting things done optically in the first place. In some cases, the phone just can't satisfactorily approach what the dedicated camera and lens can. Photographing distant and low-light action, for example. Or freezing subject motion.

In addition, lens choice for a system such as Canon EF is huge, allowing specialised jobs to be done with specific optimised results.

Having said all that, for very many people, in very many common situations, a phone gives great results and huge convenience advantages. You can't reasonably argue with someone who enjoys their phone camera and sees no need to use anything else. As stated in the introduction, there are reasons the phone image may even be preferred, when comparing images straight out of each device with no further processing. The phone processing usually ensures a satisfactory or pleasing result to most users without any user intervention, whereas the camera user will often require much more flexible external software to 'tune' the image to taste.

The large-sensor camera can, at the end of the day, replicate the results the phone gives, although this may require out-of-camera editing. The phone cannot replicate everything the dedicated camera can do.

Light show rehearsal at Edinburgh Festival. Taken handheld, again with a EOS 5D III and 16-35mm F4. The lights were swivelling rapidly on the gimbals and a fast-ish exposure was necessary. Beyond IS0 3200 there is significant image degradation. Setting here is ISO 12800, the maximum non-extended value on the camera. External noise reduction was done with DxO PRIME set to 60% for luminance noise and 100% for chrominance noise. The white smudges are the collimated lights elsewhere in the city hitting the low cloud.

The DxO Photolab suite RAW file converter includes this high performance noise reduction tool. I've adjusted various sliders in the program to improve clarity and contrast.

The image retains good sharpness but some texture is lost in the noise reduction. It's a usable image for some purposes.

High ISO/low light performance of an 8-year-old Full Frame DSLR design. Things have moved on but not much since then, with the latest Full Frame cameras having a slight edge. The newest Canon, Sony and Nikon sensors have a further dynamic range/read noise advantage. This is a night time shot in Princes Street Edinburgh. EOS 5D III at IS0 3200 and F4. 1/30 sec handheld exposure with image stabilised 16mm lens. The result is quite clean and detailed even at full resolution.

All photographs and text are copyright Simon Packer Photography 2020