Approve the Cookies
This website uses cookies to improve your user experience. By using this site, you agree to our use of cookies and our Privacy Policy.
OK
Index  •   • New posts  •   • RTAT  •   • 'Best of'  •   • Gallery  •   • Gear  •   • Reviews
Guest
New posts  •   • RTAT  •   • 'Best of'  •   • Gallery  •   • Gear  •   • Reviews
Register to forums    Log in

 
FORUMS Canon Cameras, Lenses & Accessories Canon EOS Digital Cameras 
Thread started 31 Oct 2014 (Friday) 23:42
Search threadPrev/next
sponsored links
(this ad will go away when you log in as a registered member)

How real and reliable are DXOmark scores?

 
kcbrown
Cream of the Crop
Avatar
5,384 posts
Likes: 2
Joined Mar 2007
Location: Silicon Valley
     
Nov 04, 2014 07:16 |  #91

timbop wrote in post #17250852 (external link)
Yes, I get the concept of bayer sensors and RGB. I was thinking in terms of how quickly the wells reach saturation, and if that follows the intensity curve the way it should. I've always felt that the red channel in canon sensors clips a lot faster than green and blue; ie in warmer light I seem to get more of a red/pink hue than I should and under reddish light the highlughts clip pretty fast.

That would be interesting data for DxO to collect, admittedly. The problem is that it's not as simple as saying that the sensor saturates in, say, red, at a given light intensity level. The saturation level is likely to be a curve that is dependent upon light frequency.

Even so, that could be quite valuable to someone who is interested in how well a sensor records color information.


"There are some things that money can't buy, but they aren't Ls and aren't worth having" -- Shooter-boy
Canon: 2 x 7D, Sigma 17-50 f/2.8 OS, 55-250 IS, Sigma 8-16, 24-105L, Sigma 50/1.4, other assorted primes, and a 430EX.
Nikon: D750, D600, 24-85 VR, 50 f/1.8G, 85 f/1.8G, Tamron 24-70 VC, Tamron 70-300 VC.

  
  LOG IN TO REPLY
sponsored links
(this ad will go away when you log in as a registered member)
CanonVsNikon
Senior Member
255 posts
Joined Jan 2013
     
Nov 04, 2014 08:34 |  #92
bannedPermanent ban

I guess DXO is important enough to always ruffle feathers and seem to be the most active threads. Why are some people so against their scores? (don't answer, it's a rhetorical question)

Yes, you take the whole system into consideration. From lenses, AF speed, high ISO, low ISO, DR, build quality, warranty, etc.

DXO measures only one aspect of it, but it's a very important aspect. You HAVE to take this into consideration when looking for a camera system (and yes, whether you consider DR important to you is considering it)




  
  LOG IN TO REPLY
AJSJones
Goldmember
Avatar
2,647 posts
Gallery: 6 photos
Likes: 92
Joined Dec 2001
Location: California
     
Nov 04, 2014 11:26 |  #93

timbop wrote in post #17250763 (external link)
So they don't really test color ACCURACY, unless I misread their attempt to confuse and befuddle anyone by displaying some intentionally important-looking formula paired with a bit of terminology from Isaac Newton. In the end, it seems they just compute the number of bits required to store calculated values for range - which isn't really the same thing as reproducing the colors that went into the front of the optics. But who can tell?

Apparently what they say and how they describe the interpretation of their results is not comprehensible or useful to you. If you are befuddled by equations, that would explain it. Then you have no idea whether it's good or bad science and a conspiracy theory of "attempt to confuse" is all in your mind:D Go look in any scientific or engineering paper, for exampe about color theory, color matrices etc and you will find nothing but "confusion" (aka equations and complex concepts)! But it's from that R&D "confusion" that your camera comes:D

As I said, their results are not easily converted into easily digested snippets for lay people. If you read carefully, you find that they do not do any raw conversion or white balancing, because that's all to do with software and they only evaluate raw data. The do show that Canon's red channel is much more comtaminated by green and blue than that of the Nikon in that comparison - heck they even show spectral responses to demonstrate the problem. Unfortunately some folks are as befuddled by graphs as they are by equations. This is the old issue of DxO generating data and presenting it in a way that many people don't understand - so it is understandable that the utiity of the data gets dismissed by those who don't understand it. That's the theme of this thread, apparently. Some get the utility of the underlying data from the tests, others don't.

They make a reasonable attempt to describe what they mean by (and how they measure) color sensitivity and it's complicated. What you are asking for requires the manipulation of the raw data to white balance and that's not a property of the sensor. (White balancing amplifies noise in some channels more than others, so the red channel issue is partly a noise issue). Basically what they measure, as quoted in my post above, is how many different colours the sensor can reliably distinguish (like how many grayscale tones in B/W tonality). They represent it by the number of bits needed to encode that number. If you want to know the actual number of distinguishable colors the sensor can record, just take the number of bits they provide (X) and use it in 2^X.

If you are familiar with lighting you may have heard of the CRI - color rendition index -a measure of how many color squares on the standard test card are correctly rendered when illuminated by a lamp (street lights are notorious for misrepresenting colours). DxO's measurement is like that on steroids:) If you ask about "color accuracy" would it be better to get only 10,000 colors distinguished but perfectly, or 100,000 colors with some slightly off?

Edit: For highest accuracy at any location, shoot a Passport ColorChecker card to do the fine tuning after the raw conversion


My picture galleries (external link)

  
  LOG IN TO REPLY
gjl711
According to the lazy TF, My flatulence rates
Avatar
55,432 posts
Likes: 2391
Joined Aug 2006
Location: Deep in the heart of Texas
     
Nov 04, 2014 11:44 |  #94

AJSJones wrote in post #17251817 (external link)
... Unfortunately some folks are as befuddled by graphs as they are by equations. This is the old issue of DxO generating data and presenting it in a way that many people don't understand - so it is understandable that the utiity of the data gets dismissed by those who don't understand it. That's the theme of this thread, apparently. Some get the utility of the underlying data from the tests, others don't.....

Odd thing is that they do present it in a way anyone can understand. But when the results are negative to someone's point of view, the complex data is dragged out to show that the data must be wrong and DxO is just trying to confuse everyone.

One point though, DxO measurements are very specific and does not represent the entire package well. It's a data point to use in evaluating a camera, not an overall score of the entire camera itself.


Not sure why, but call me JJ.
I used to hate math but then I realised decimals have a point.
.
::Flickr:: (external link)
::Gear::

  
  LOG IN TO REPLY
AJSJones
Goldmember
Avatar
2,647 posts
Gallery: 6 photos
Likes: 92
Joined Dec 2001
Location: California
     
Nov 04, 2014 12:00 |  #95

gjl711 wrote in post #17251851 (external link)
Odd thing is that they do present it in a way anyone can understand. But when the results are negative to someone's point of view, the complex data is dragged out to show that the data must be wrong and DxO is just trying to confuse everyone.

I wish that were so but it's clear many simply don't relate to them. It's the same sort of person who says "I want to see real world images - no-one shoots test cards" - they can do their own subjective evaluation based on those images, but can't provide "objective" measures for others to use. Of course, there are also people who fit your description:(

gjl711 wrote in post #17251851 (external link)
One point though, DxO measurements are very specific and does not represent the entire package well. It's a data point to use in evaluating a camera, not an overall score of the entire camera itself.

Many have said that in this thread and the disclaimer is in many, many sections of the DxO site.


My picture galleries (external link)

  
  LOG IN TO REPLY
timbop
Goldmember
Avatar
2,980 posts
Gallery: 1 photo
Likes: 18
Joined Apr 2007
Location: Southern New Jersey, USA
     
Nov 04, 2014 12:32 |  #96

kcbrown, what drew me to have 2 5d3's, in order of descending importance:
1 - autofocus performance. otherwise, I would have kept my 5d2
2 - high ISO image quality. otherwise, I would not have replaced my 7d with a second 5d3
3 - speed of operation, controls, and handling. the 5d3's are the right blend of customizability, with the lone exception of only being able to assign the DOF preview button for onseshot <-> AI servo
4 - familiarity with canon UI, and canon's glass - otherwise I might have gone d700 6 years ago


Current: 5DM3, 6D, 8mm fish, 24-105/4IS, 35/2IS, 70-200/2.8IS, 85/1.8, 100-400/IS v1, lensbaby composer with edge 80, 580's and AB800's
Formerly: 80D, 7D, 300D, 5D, 5DM2, 20D, 50D, 1DM2, 17-55IS, 24-70/2.8, 28-135IS, 40/2.8, 50/1.8, 50/1.4, 70-200/4IS, 70-300IS, 70-200/2.8, 100 macro, 400/5.6, tammy 17-50 and 28-75, sigma 50 macro & 100-300

  
  LOG IN TO REPLY
AJSJones
Goldmember
Avatar
2,647 posts
Gallery: 6 photos
Likes: 92
Joined Dec 2001
Location: California
     
Nov 04, 2014 12:48 |  #97

timbop wrote in post #17251982 (external link)
kcbrown, what drew me to have 2 5d3's, in order of descending importance:
1 - autofocus performance. otherwise, I would have kept my 5d2
2 - high ISO image quality. otherwise, I would not have replaced my 7d with a second 5d3
3 - speed of operation, controls, and handling. the 5d3's are the right blend of customizability, with the lone exception of only being able to assign the DOF preview button for onseshot <-> AI servo
4 - familiarity with canon UI, and canon's glass - otherwise I might have gone d700 6 years ago

Not much of that list is addressed by DxO, so you can safely ignore the rest of their data:D. The only item in your list is high ISO IQ. That is, just coincidentally, where Canon leads Nikon in DR:D


My picture galleries (external link)

  
  LOG IN TO REPLY
jdizzle
Darth Noink
Avatar
69,419 posts
Likes: 56
Joined Aug 2006
Location: Harvesting Nano crystals
     
Nov 04, 2014 13:10 |  #98

David Arbogast wrote in post #17250944 (external link)
DXO scores are a huge deal here at POTN, as evidenced by explosive threads such as this one, because Canon sensors score lower. If DXO scored Canon sensors about the same or better than Sony sensors, it would be less of a discussion/debate.

I agree. If only Canon could make a sensor that would rival the Sony/Nikon.




  
  LOG IN TO REPLY
jdizzle
Darth Noink
Avatar
69,419 posts
Likes: 56
Joined Aug 2006
Location: Harvesting Nano crystals
     
Nov 04, 2014 13:14 |  #99

timbop wrote in post #17251982 (external link)
kcbrown, what drew me to have 2 5d3's, in order of descending importance:
1 - autofocus performance. otherwise, I would have kept my 5d2
2 - high ISO image quality. otherwise, I would not have replaced my 7d with a second 5d3
3 - speed of operation, controls, and handling. the 5d3's are the right blend of customizability, with the lone exception of only being able to assign the DOF preview button for onseshot <-> AI servo
4 - familiarity with canon UI, and canon's glass - otherwise I might have gone d700 6 years ago

I love the ease of use with Canon. There's no doubting that. I've been shooting Nikon for three years now and I've got a nice workflow with my cameras. Is it different? Slightly. I'm happy with my dual D750's, D4s, & D810 for wedding work. :)

I take DXO with a grain of salt.




  
  LOG IN TO REPLY
kcbrown
Cream of the Crop
Avatar
5,384 posts
Likes: 2
Joined Mar 2007
Location: Silicon Valley
     
Nov 04, 2014 14:04 |  #100

timbop wrote in post #17251982 (external link)
kcbrown, what drew me to have 2 5d3's, in order of descending importance:
1 - autofocus performance. otherwise, I would have kept my 5d2
2 - high ISO image quality. otherwise, I would not have replaced my 7d with a second 5d3

Aha, so you are concerned with the sensor performance, and it even ranks near the top! :)

DxO performs measurements that reveal the high ISO performance of the sensor, though I've seen nothing about them that will reveal or account for artifacts generated by the sensor. But as with all such data, you have to know how to interpret it. The signal to noise ratio graph, combined with the dynamic range graph, will tell you how well the sensor performs at high ISOs.

3 - speed of operation, controls, and handling. the 5d3's are the right blend of customizability, with the lone exception of only being able to assign the DOF preview button for onseshot <-> AI servo
4 - familiarity with canon UI, and canon's glass - otherwise I might have gone d700 6 years ago

I have to say that Canon does have an excellent user interface, quite a lot better than the Nikon. I've adapted to both easily enough, but it's easier to find things with the Canon. Their UI is more logically presented and organized, and that makes it a bit more "discoverable".


"There are some things that money can't buy, but they aren't Ls and aren't worth having" -- Shooter-boy
Canon: 2 x 7D, Sigma 17-50 f/2.8 OS, 55-250 IS, Sigma 8-16, 24-105L, Sigma 50/1.4, other assorted primes, and a 430EX.
Nikon: D750, D600, 24-85 VR, 50 f/1.8G, 85 f/1.8G, Tamron 24-70 VC, Tamron 70-300 VC.

  
  LOG IN TO REPLY
mfunnell
Senior Member
Avatar
375 posts
Likes: 11
Joined Jul 2005
Location: Sydney, Australia
     
Nov 04, 2014 16:27 |  #101

gjl711 wrote in post #17251851 (external link)
One point though, DxO measurements are very specific and does not represent the entire package well. It's a data point to use in evaluating a camera, not an overall score of the entire camera itself.

The other thing to note is that even restricted to analysis of sensor performance (something they always point out; though it's often ignored), the single 'headline' DxO number for sensor performance actually combines different aspects of sensor performance using a (last time I looked, undocumented) weighting scheme which may not reflect a photographer's personal weighting (eg. performance at base ISO may be more important to a landscape photographer; high ISO performance to a night sports photographer) and ignores other aspects of the sensor such as resolution and (to a certain extent) sensor size, which may be of considerble importance for someone's use. 'Drilling down' into the measures it is derived from is always indicated if you weight one or more of DxO's 'use case' numbers as more important than others.

And, of course and as you point out, that headline DxO number and even the subsidiary numbers it is derived from say nothing about any other aspect of the camera or camera system.

...Mike


Some digital cameras, some film cameras, some lenses & other kit.
Day-to-day photos on flickr (external link), some older stuff at dA (external link).

  
  LOG IN TO REPLY
timbop
Goldmember
Avatar
2,980 posts
Gallery: 1 photo
Likes: 18
Joined Apr 2007
Location: Southern New Jersey, USA
     
Nov 04, 2014 20:58 |  #102

kcbrown wrote in post #17252141 (external link)
Aha, so you are concerned with the sensor performance, and it even ranks near the top! :)

...

I NEVER said I don't care about image quality, and obviously that comes from the sensor. You're trying to twist my words; the performance I am most concerned about is high ISO noise not base ISO DR.


Current: 5DM3, 6D, 8mm fish, 24-105/4IS, 35/2IS, 70-200/2.8IS, 85/1.8, 100-400/IS v1, lensbaby composer with edge 80, 580's and AB800's
Formerly: 80D, 7D, 300D, 5D, 5DM2, 20D, 50D, 1DM2, 17-55IS, 24-70/2.8, 28-135IS, 40/2.8, 50/1.8, 50/1.4, 70-200/4IS, 70-300IS, 70-200/2.8, 100 macro, 400/5.6, tammy 17-50 and 28-75, sigma 50 macro & 100-300

  
  LOG IN TO REPLY
kcbrown
Cream of the Crop
Avatar
5,384 posts
Likes: 2
Joined Mar 2007
Location: Silicon Valley
     
Nov 04, 2014 22:18 |  #103

timbop wrote in post #17252809 (external link)
I NEVER said I don't care about image quality, and obviously that comes from the sensor. You're trying to twist my words; the performance I am most concerned about is high ISO noise not base ISO DR.

Understood. The point here, however, is that the fact that you are concerned about image quality means that at least some of DxO's measurements are relevant to you. The scores might not be (though, if you notice, the 5D3's high ISO score is right up there), but the measurements certainly are. And that includes the DR as measured at the ISOs you're interested in.


"There are some things that money can't buy, but they aren't Ls and aren't worth having" -- Shooter-boy
Canon: 2 x 7D, Sigma 17-50 f/2.8 OS, 55-250 IS, Sigma 8-16, 24-105L, Sigma 50/1.4, other assorted primes, and a 430EX.
Nikon: D750, D600, 24-85 VR, 50 f/1.8G, 85 f/1.8G, Tamron 24-70 VC, Tamron 70-300 VC.

  
  LOG IN TO REPLY
sponsored links
(this ad will go away when you log in as a registered member)

16,366 views & 0 likes for this thread
How real and reliable are DXOmark scores?
FORUMS Canon Cameras, Lenses & Accessories Canon EOS Digital Cameras 
AAA
x 1600
y 1600

Jump to forum...   •  Rules   •  Index   •  New posts   •  RTAT   •  'Best of'   •  Gallery   •  Gear   •  Reviews   •  Member list   •  Polls   •  Image rules   •  Search   •  Password reset

Not a member yet?
Register to forums
Registered members may log in to forums and access all the features: full search, image upload, follow forums, own gear list and ratings, likes, more forums, private messaging, thread follow, notifications, own gallery, all settings, view hosted photos, own reviews, see more and do more... and all is free. Don't be a stranger - register now and start posting!


COOKIES DISCLAIMER: This website uses cookies to improve your user experience. By using this site, you agree to our use of cookies and to our privacy policy.
Privacy policy and cookie usage info.


POWERED BY AMASS forum software 2.1forum software
version 2.1 /
code and design
by Pekka Saarinen ©
for photography-on-the.net

Latest registered member is Edenfieldr
966 guests, 367 members online
Simultaneous users record so far is 15144, that happened on Nov 22, 2018

Photography-on-the.net Digital Photography Forums is the website for photographers and all who love great photos, camera and post processing techniques, gear talk, discussion and sharing. Professionals, hobbyists, newbies and those who don't even own a camera -- all are welcome regardless of skill, favourite brand, gear, gender or age. Registering and usage is free.