Marques Brownlee, also known as MKBHD, held its third annual 17-minute blind-camera test with 20 new smartphones. The phones were grouped together in a holder that hid the names of the phones while the public voted for their favorites.
First of all, it is important to note from the outset that this “test” is the furthest thing from scientific nature.
“It’s not a scientific test at all,” says Brownlee. “In fact, it’s a bit of the opposite of a scientific test.”
The purpose of this playoff style medium is not to objectively pretend that one camera is better than another, but rather serves as a good case study for what people think a photo is “good”. In the end, and after more than 10 million votes in total, Brownlee was able to draw some interesting conclusions from the information he gathered.
The concept of the test is simple: Brownlee set up a seed holder (the seeds were determined by his team, and in the end, honestly, the seeds didn’t seem to matter much) and paired each single letter test camera. Here’s the media as those who voted on the test saw it before the first images were released:
Those who would vote had no idea which phones were in turn-based competition, which was the point. Brownlee wanted to see how people would vote based solely on the images taken and nothing else. On each round, all smartphones would be placed in exactly the same position and photograph the exact same subject under the same circumstances. That way, the only differences in a photo’s appearance were based solely on how each smartphone is programmed to capture an image. Brownlee even went so far as not to tell the camera where to focus, leaving that to the smartphone, too.
All of the images Brownlee took may seem straightforward and unchallenging, but that was the point. The idea was to create scenes that could very easily occur in everyday life, but also to incorporate difficult aspects into each image that may not be immediately noticeable. In one image there could be a large mix of shadows and highlights that would test the dynamic range of each camera, while in another there could be a lot of competing textures and colors that would show how each sensor looks. ‘would adapt to the differences.
The photos were posted on Twitter and Instagram stories, each allowing polls. The first round used this photo:
Which photo is the best? A or B? (Survey)
– Marques Brownlee (@MKBHD) November 23, 2020
The second round used this image:
Which photo is the best? A or C? (Survey)
– Marques Brownlee (@MKBHD) November 24, 2020
The third round used this photo:
Which photo is the best? C or H? (Survey)
– Marques Brownlee (@MKBHD) November 25, 2020
And the final round used this photo:
Which photo is the best? H or K? (Survey)
– Marques Brownlee (@MKBHD) November 26, 2020
After the polls closed for all rounds, Brownlee revealed which cameras people voted on:
At the start of his video, Brownlee makes two important points: First, every year he’s taken this test, no camera brand has ever repeated a win. Second, the iPhone never made it out of the first round.
As you can see in the completed parenthesis with the new winner, both of these notes remained true after this year’s test:
So, naturally, the next question would be, “Is the Asus Zenfone 7 Pro the best smartphone camera?” The answer to this question is probably “well, not necessarily”. As he said at the start, this was not a scientific test and the results were not necessarily aimed at crowning a device with the best camera.
What he has done is indicate what the general voting population finds most appealing about a given image. Brownlee makes several interesting notes about the footage he and his team chose to take, and why he thinks the iPhone in particular continues to struggle in this competition.
“White balance has been a major and key new factor in our understanding of smartphone support this year,” he says. “And I would go so far as to say that it seems that was the reason the iPhone lost in the first round every time.”
White balance seems to be a big factor in what makes an image attractive, and Apple’s iPhones seem to lean more toward blue than other cameras on the market. What this does is make warmer tones – like her skin tone, as Brownlee points out – appear oddly tinted.
Brownlee says what he and his team deduced is that when the content and brightness are the same, people will choose a photo with slightly better color saturation. Now there is of course a limit to this, as oversaturation can make images look terrible, but proper white balance also has a direct impact on how the eye perceives the saturation of certain tones. The cooler the white balance, the more saturated the blues will be. The warmer the white balance, the more saturated warm colors like orange and red will appear.
So while the iPhone 12 Pro Max photo has a lot of detail and crispness (the iPhone was the “M” phone), it was the coolest color balance of the bunch and the enhanced exposure took it easy. blown sky in the background. This combination led to it losing once again to other smartphones that handled color and dynamic range better.
Which photo is the best? M or N? (Survey)
– Marques Brownlee (@MKBHD) November 23, 2020
So why do some phones always have such a cool tint for white balance? While his guess is as good as anyone since he’s not sure, the answer he offers could go back to the origins of photography and how even the film was conceived.
“In photos of people with lighter complexions, which most people do, it doesn’t affect the appearance of skin tone as much,” Brownlee speculates. “You can get away with this. And also, the blue sky will look bluer than it would be if you were warm. “
Even more interesting (or frustrating, depending on how you look at it), Brownlee has found that it’s not just what your smartphone captures, but how much the hosting service you choose for that photo matters as well. In the final showdown with the photo, the image of the two pumpkins was different between Twitter and Instagram, resulting in a pretty noticeable disparity in voting between the two platforms.
Watching Brownlee’s full conclusion and discussion of this year’s results is definitely worth it. While it is interesting to crown a single smartphone a winner is interesting, the philosophical discussion at the end is even more interesting. How the general public sees images and what elements of a photo they tend to value is useful information for any photographer.
To learn more about Marques Brownlee, be sure to subscribe to his YouTube channel.