Approve the Cookies
This website uses cookies to improve your user experience. By using this site, you agree to our use of cookies and our Privacy Policy.
OK
Index  •   • New posts  •   • RTAT  •   • 'Best of'  •   • Gallery  •   • Gear  •   • Reviews
Guest
New posts  •   • RTAT  •   • 'Best of'  •   • Gallery  •   • Gear  •   • Reviews
Register to forums    Log in

 
FORUMS Canon Cameras, Lenses & Accessories Canon EOS Digital Cameras 
Thread started 31 Oct 2014 (Friday) 23:42
Search threadPrev/next
sponsored links
(this ad will go away when you log in as a registered member)

How real and reliable are DXOmark scores?

 
CanonVsNikon
Senior Member
255 posts
Joined Jan 2013
     
Nov 03, 2014 11:38 |  #61
bannedPermanent ban

I'll be curious how a thread like this will play out once/if Canon pulls ahead in DR and their DXO marks are higher than competition. I'm guessing the tone of threads like these will be different.




  
  LOG IN TO REPLY
sponsored links
(this ad will go away when you log in as a registered member)
Charlie
Guess What! I'm Pregnant!
16,598 posts
Gallery: 8 photos
Likes: 6556
Joined Sep 2007
     
Nov 03, 2014 11:51 |  #62

CanonVsNikon wrote in post #17249611 (external link)
I'll be curious how a thread like this will play out once/if Canon pulls ahead in DR and their DXO marks are higher than competition. I'm guessing the tone of threads like these will be different.

Would love for that to happen, not really interested in switching to Nikon, but something like the D810 with a tilt/touch screen would be as much as I want from a camera. Hope canon comes up with something like that within this decade (probably not).


Sony A7siii/A7iii/ZV-1 - FE 24/1.4 - SY 24/2.8 - FE 35/2.8 - FE 50/1.8 - FE 85/1.8 - F 600/5.6 - CZ 100-300 - Tamron 17-28/2.8 - 28-75/2.8 - 28-200 RXD
Panasonic G9 - Laowa 7.5/2 - PL 15/1.7 - P 42.5/1.8 - OM 75/1.8 - PL 10-25/1.7 - P 12-32 - P 14-140

  
  LOG IN TO REPLY
AJSJones
Goldmember
Avatar
2,647 posts
Gallery: 6 photos
Likes: 92
Joined Dec 2001
Location: California
     
Nov 03, 2014 13:37 |  #63

Canon_Lover wrote in post #17249341 (external link)
People seriously believe Canon sensors are so bad that even touching the shadow slider will result in a ruined photo. People are even using patchwork systems with lens adapters on mirrorless cameras because of a fear that they can't do photography without extra DR. .

Really?
This is the kind of comment that gets the internet a bad reputation. Basically what you are saying is that "I think increased DR would be useful in some situations/for some styles of photography" can be translated into "All people who think better DR might be useful or who choose cameras with increased DR are completely stupid". Inane and arrogant, if you ask me.


My picture galleries (external link)

  
  LOG IN TO REPLY
speedync
Goldmember
1,532 posts
Gallery: 121 photos
Likes: 1090
Joined May 2011
Location: Geelong, Australia
     
Nov 03, 2014 15:27 |  #64

Hogloff wrote in post #17249359 (external link)
Depends what you shoot. If you shoot conditions that have a high dynamic range like say a rainforest with light shining through it...then the Sony sensor will show a difference in the end.

If you print large detailed photos, then the extra pixels of the A7R will show you a difference in the print.

If you just view images on your laptop, then just about any camera in the last 10 years will do.

Do you have your work displayed online anywhere? Can you give us a link to your efforts so we can see what you're talking/writing about? I am genuinely interested to see just how large the differences are in the finished shot




  
  LOG IN TO REPLY
Charlie
Guess What! I'm Pregnant!
16,598 posts
Gallery: 8 photos
Likes: 6556
Joined Sep 2007
     
Nov 03, 2014 16:20 |  #65

speedync wrote in post #17250095 (external link)
Do you have your work displayed online anywhere? Can you give us a link to your efforts so we can see what you're talking/writing about? I am genuinely interested to see just how large the differences are in the finished shot

The differences will not be gigantic, and will be dependent on the eye of the photographer.

I shoot for myself and that's all that really matters. If I can see improvement, then it may be worth it. My gear isnt exactly cheap, and that's because I am willing pay for small advantages. If a top notch sensor gives me a small advantage, then it'll be worthwhile for me to investigate.

I can say pretty confidently, that I'm a pretty good technical shooter, in that I can get exposure down, sharp, and steady shots. I'de rather not worry about gear when I'm out shooting. I dont want to worry about if my sensor can take a scene or not, or whether my tripod is level, ect. I shoot with filters, HDR, whatever I need to create an image that I see. As good as the 6D sensor is (and a noticeable improvement over the 5D2), it's not really the top of it's class. I want better IQ for a reasonable price. There's only so much DR can do, but the canon sensors are lagging so badly, it's pretty embarrassing. Do away with the AA filter already or maybe have a removable one to satisfy video shooters, I want the sharpest and crispiest photos I can get.

My 6D is now my action camera, and I'll eventually get a 5D3 for action and enjoy the joystick which I miss dearly, however, the downgrade in IQ definitely does NOT sit well. I do favor the canon's to do stitchwork though (DR a smaller issue if any with stitched images), I simply wont ever need 36mp stitched images.


Sony A7siii/A7iii/ZV-1 - FE 24/1.4 - SY 24/2.8 - FE 35/2.8 - FE 50/1.8 - FE 85/1.8 - F 600/5.6 - CZ 100-300 - Tamron 17-28/2.8 - 28-75/2.8 - 28-200 RXD
Panasonic G9 - Laowa 7.5/2 - PL 15/1.7 - P 42.5/1.8 - OM 75/1.8 - PL 10-25/1.7 - P 12-32 - P 14-140

  
  LOG IN TO REPLY
timbop
Goldmember
Avatar
2,980 posts
Gallery: 1 photo
Likes: 18
Joined Apr 2007
Location: Southern New Jersey, USA
     
Nov 03, 2014 16:45 |  #66

Hogloff wrote in post #17249293 (external link)
Well DXO just doesn't measure DR...have you ever taken the time to truly understand what they do?

The rest of your post...I don't understand. Yes great photographers existed 80 years ago. Do you think these great photographers just went into a camera store and randomly chose a camera, or do you think they researched the cameras that existed and picked one that met their needs?

DXO is just one of many research sites. It has great tests that you can use in your camera evaluations.

I haven't looked in a long time, but out of curiosity do they test AF accuracy and performance under various lighting and subject conditions? Also, do they check color accuracy under various light sources?


Current: 5DM3, 6D, 8mm fish, 24-105/4IS, 35/2IS, 70-200/2.8IS, 85/1.8, 100-400/IS v1, lensbaby composer with edge 80, 580's and AB800's
Formerly: 80D, 7D, 300D, 5D, 5DM2, 20D, 50D, 1DM2, 17-55IS, 24-70/2.8, 28-135IS, 40/2.8, 50/1.8, 50/1.4, 70-200/4IS, 70-300IS, 70-200/2.8, 100 macro, 400/5.6, tammy 17-50 and 28-75, sigma 50 macro & 100-300

  
  LOG IN TO REPLY
speedync
Goldmember
1,532 posts
Gallery: 121 photos
Likes: 1090
Joined May 2011
Location: Geelong, Australia
     
Nov 03, 2014 17:24 |  #67

Charlie wrote in post #17250187 (external link)
The differences will not be gigantic, and will be dependent on the eye of the photographer.

There's only so much DR can do, but the canon sensors are lagging so badly, it's pretty embarrassing.

Bit of a conflicting statement don't you think? I didn't ask for a spiel about what you are happy/not happy with, I simply asked to see some of his work. Out of genuine interest. Not to try & win some sort of weird/pointless internet argument




  
  LOG IN TO REPLY
offcamber
Senior Member
267 posts
Likes: 111
Joined Mar 2005
     
Nov 03, 2014 18:06 |  #68

DXO is a gimmick and their software is horrible to boot.


EOS-7D Mark II | EOS-50D | 1DX | 100F2.8L IS | 17-40F4L Sigma 50F1.4 | Canon 11-24F4| 24-70F2.8L | 40F2.8 | 85F1.8 | 70-200F2.8L IS II | 100-400F4.5L |300F4L IS | 580EXII | 550EX

  
  LOG IN TO REPLY
AJSJones
Goldmember
Avatar
2,647 posts
Gallery: 6 photos
Likes: 92
Joined Dec 2001
Location: California
     
Nov 03, 2014 18:10 |  #69

DxO repeatedly disclaim any relevance of their measurements to things they don't test. So anyone who thinks DxO scores should be used to "decide which camera/system to buy" is completely missing the point. From one section

All sensor scores reflect only the RAW sensor performance of a camera body. All measurements are performed on the RAW image file BEFORE demosaicing or other processing prior to final image delivery. DxOMark does not address such other important criteria as image signal processing, mechanical robustness, ease of use, flexibility, optics quality, value for money, etc. While RAW sensor performance is critically important, it is not the only factor that should be taken into consideration when choosing a digital camera.

timbop wrote in post #17250237 (external link)
I haven't looked in a long time, but out of curiosity do they test AF accuracy and performance under various lighting and subject conditions?

So, nope, AF accuracy is not something they test.

Here (external link) is where they present their testing protocols.

Here's (external link) an extensive discussion of the data they generated and presented for a Nikon D5000 and Canon EOS 500D comparison.

timbop wrote in post #17250237 (external link)
Also, do they check color accuracy under various light sources?

From that discussion,here's (external link) the information on color and the final text on the color analysis

Here we see that the Canon EOS 500D has very different color behavior than the Nikon D5000. The red channel is not selective enough, so the color matrix has to apply a stronger color processing to mix the different channels to obtain a satisfactory match with the sRGB. In other words, the Canon EOS 500D is a bit more color blind in comparison with the Nikon D5000. To compensate, a greater degree of color processing is applied, which increases noise.
This will impact the final RGB picture even if noise processing reduces the effect. Generally speaking, even with a good RAW converter, very low frequency colored noise remains on the final image.

To interpret that comment we ned to go deeper! For their take on "color accuracy" this comes from their section on measurements of color sensitivity (external link).

Color sensitivity is the number of reliably distinguishable colors up to noise. Roughly speaking, two colors are considered as distinguishable if their difference is larger than the noise. In this respect, color sensitivity is the generalization of color to the notion of tonal range.
We call Color Sensitivity the number derived from
{CS=mathematical equation} goes here:D
The above formula shows that the determinant of the noise covariance matrix is the volume of the incertitude ellipsoid, in which the difference of two colors is most likely due to noise. However, because digital images are encoded using integer values, the dimensions of the incertitude ellipsoid are quantized. Hence the integrand in the equation above can be seen as the density of distinguishable colors around the point (r, g, b), and takes quantization into account. The integral itself can then be interpreted as the total number of colors that can be distinguished by the sensor.
As with tonal range, color sensitivity is a number with no unit, so instead we consider log2(CS), which represents the number of bits necessary to encode all distinguishable color values.

Channel decomposition and color matrix
Assume that the three channels of a sensor have equal sensitivities (or, equivalently, that white balance has been applied). In practice, the primaries of the sRGB color space do not correspond with the primaries of the sensor. (See the explanation for Luther-Ives condition in the section SMI above.) For each sensor channel, the channel decomposition describes the linear combination of the primaries of the sRGB color space that best fits the sensor channel. Typically, the red sensor channel is expected to contain mostly red, a bit of green, and almost no blue. A decomposition of the red channel close to (1,0,0) shows that the red channel of the sensor is very “pure.”
The color matrix gives the coefficients of the linear combination of the sensor channels which have to be applied to compensate for the sensor channels’ lack of purity as compared to the sRGB primaries. As with the white balance scales and the relative sensitivities, the channel decomposition and the color matrix give exactly the same information in a complementary manner. The channel decomposition coefficient can be portrayed as a 3x3 matrix, which means that the color matrix is then simply the inverse of the channel decomposition coefficient matrix.
The channel decomposition is a low-level description of the sensor spectral response, while the color matrix is what engineers use to make the sensor react as though it had sRGB primaries.
A color matrix with large singular values yields a dramatic amplification of noise, and this is taken into account in the color sensitivity measurement.

We all agree that DxO Mark single number scores are crazy (Combined 0-60, mpg, luggage space and cupholder car index:D). Many have cited the usefulness of their data.

The technical expertise needed to understand the way they make their measurements and process their data is non-trivial (I understand some of it but there's a bunch I don't get either, so I'm not talking down to anyone:D) . If you don't have the expertise, feel free to ignore the results. If you don't understand (or simply ignore) the results, your comments on the data will be worthless. However, saying that "a lot of their output is not very helpful", is a valid comment:D


My picture galleries (external link)

  
  LOG IN TO REPLY
sega62
Senior Member
Avatar
756 posts
Likes: 8
Joined Aug 2011
     
Nov 03, 2014 19:33 |  #70

http://www.dxo.com/fr/​image-quality (external link)

From what i see on this page is a service to sell some gear to evaluate image quality just as good as in the real world, that is what they say anyway.
If you look at the image all the way up that page you see someone with a ton of Nikon lenses.

I would like Sony to come up with some device to check all these camera optic values and bring another side of the story, since Dxo is the only expertise.

Anyway, to me it looks like Dxo is pairing with Nikon on some level.....
So how can they be partial?




  
  LOG IN TO REPLY
AJSJones
Goldmember
Avatar
2,647 posts
Gallery: 6 photos
Likes: 92
Joined Dec 2001
Location: California
     
Nov 03, 2014 20:38 |  #71

sega62 wrote in post #17250519 (external link)
http://www.dxo.com/fr/​image-quality (external link)

From what i see on this page is a service to sell some gear to evaluate image quality just as good as in the real world, that is what they say anyway.
If you look at the image all the way up that page you see someone with a ton of Nikon lenses.

I would like Sony to come up with some device to check all these camera optic values and bring another side of the story, since Dxo is the only expertise.

Anyway, to me it looks like Dxo is pairing with Nikon on some level.....
So how can they be partial?

So unless you saw a lens from every manufacturer on that page, you would not be prepared to believe anything they say??? You could, as a cynical person, also interpret that page as "They must have to test a lot of Nikon lenses therefore they're so bad"?
Yes DxO manufactures lens and camera testing equipment and all you can do is look for something that you think suggests that any results they get on any lenses other than Nikon will be bad results? What's up with that! Not exactly an unbiased comment.

BTW why would Sony test equipment be any less "partial"?


My picture galleries (external link)

  
  LOG IN TO REPLY
sega62
Senior Member
Avatar
756 posts
Likes: 8
Joined Aug 2011
     
Nov 03, 2014 20:59 |  #72

Read between the lines




  
  LOG IN TO REPLY
AJSJones
Goldmember
Avatar
2,647 posts
Gallery: 6 photos
Likes: 92
Joined Dec 2001
Location: California
     
Nov 03, 2014 21:04 |  #73

One signature I read on a language forum is: I let my words speak for themselves. When you read between the lines all you get is silence. Anything you find there is of your own making.

Seems appropriate:D


My picture galleries (external link)

  
  LOG IN TO REPLY
timbop
Goldmember
Avatar
2,980 posts
Gallery: 1 photo
Likes: 18
Joined Apr 2007
Location: Southern New Jersey, USA
     
Nov 03, 2014 21:54 |  #74

AJSJones wrote in post #17250382 (external link)
DxO repeatedly disclaim any relevance of their measurements to things they don't test. So anyone who thinks DxO scores should be used to "decide which camera/system to buy" is completely missing the point. From one section
So, nope, AF accuracy is not something they test.

Here (external link) is where they present their testing protocols.

Here's (external link) an extensive discussion of the data they generated and presented for a Nikon D5000 and Canon EOS 500D comparison.
From that discussion,here's (external link) the information on color and the final text on the color analysis
To interpret that comment we ned to go deeper! For their take on "color accuracy" this comes from their section on measurements of color sensitivity (external link).


We all agree that DxO Mark single number scores are crazy (Combined 0-60, mpg, luggage space and cupholder car index:D). Many have cited the usefulness of their data.

The technical expertise needed to understand the way they make their measurements and process their data is non-trivial (I understand some of it but there's a bunch I don't get either, so I'm not talking down to anyone:D) . If you don't have the expertise, feel free to ignore the results. If you don't understand (or simply ignore) the results, your comments on the data will be worthless. However, saying that "a lot of their output is not very helpful", is a valid comment:D

So they don't really test color ACCURACY, unless I misread their attempt to confuse and befuddle anyone by displaying some intentionally important-looking formula paired with a bit of terminology from Isaac Newton. In the end, it seems they just compute the number of bits required to store calculated values for range - which isn't really the same thing as reproducing the colors that went into the front of the optics. But who can tell?


Current: 5DM3, 6D, 8mm fish, 24-105/4IS, 35/2IS, 70-200/2.8IS, 85/1.8, 100-400/IS v1, lensbaby composer with edge 80, 580's and AB800's
Formerly: 80D, 7D, 300D, 5D, 5DM2, 20D, 50D, 1DM2, 17-55IS, 24-70/2.8, 28-135IS, 40/2.8, 50/1.8, 50/1.4, 70-200/4IS, 70-300IS, 70-200/2.8, 100 macro, 400/5.6, tammy 17-50 and 28-75, sigma 50 macro & 100-300

  
  LOG IN TO REPLY
Charlie
Guess What! I'm Pregnant!
16,598 posts
Gallery: 8 photos
Likes: 6556
Joined Sep 2007
     
Nov 03, 2014 21:57 |  #75

timbop wrote in post #17250763 (external link)
So they don't really test color ACCURACY, unless I misread their attempt to confuse and befuddle anyone by displaying some intentionally important-looking formula paired with a bit of terminology from Isaac Newton. In the end, it seems they just compute the number of bits required to store calculated values for range - which isn't really the same thing as reproducing the colors that went into the front of the optics. But who can tell?

Color can be manipulated


Sony A7siii/A7iii/ZV-1 - FE 24/1.4 - SY 24/2.8 - FE 35/2.8 - FE 50/1.8 - FE 85/1.8 - F 600/5.6 - CZ 100-300 - Tamron 17-28/2.8 - 28-75/2.8 - 28-200 RXD
Panasonic G9 - Laowa 7.5/2 - PL 15/1.7 - P 42.5/1.8 - OM 75/1.8 - PL 10-25/1.7 - P 12-32 - P 14-140

  
  LOG IN TO REPLY
sponsored links
(this ad will go away when you log in as a registered member)

17,886 views & 0 likes for this thread
How real and reliable are DXOmark scores?
FORUMS Canon Cameras, Lenses & Accessories Canon EOS Digital Cameras 
AAA
x 1600
y 1600

Jump to forum...   •  Rules   •  Index   •  New posts   •  RTAT   •  'Best of'   •  Gallery   •  Gear   •  Reviews   •  Member list   •  Polls   •  Image rules   •  Search   •  Password reset

Not a member yet?
Register to forums
Registered members may log in to forums and access all the features: full search, image upload, follow forums, own gear list and ratings, likes, more forums, private messaging, thread follow, notifications, own gallery, all settings, view hosted photos, own reviews, see more and do more... and all is free. Don't be a stranger - register now and start posting!


COOKIES DISCLAIMER: This website uses cookies to improve your user experience. By using this site, you agree to our use of cookies and to our privacy policy.
Privacy policy and cookie usage info.


POWERED BY AMASS forum software 2.1forum software
version 2.1 /
code and design
by Pekka Saarinen ©
for photography-on-the.net

Latest registered member is mstaves
684 guests, 161 members online
Simultaneous users record so far is 15144, that happened on Nov 22, 2018

Photography-on-the.net Digital Photography Forums is the website for photographers and all who love great photos, camera and post processing techniques, gear talk, discussion and sharing. Professionals, hobbyists, newbies and those who don't even own a camera -- all are welcome regardless of skill, favourite brand, gear, gender or age. Registering and usage is free.