Lightroom Benchmarks

I often see people asking for recommendations on this forum for computer hardware for lightroom use and I often wonder what the cost in upgraded hardware (say a i7 3770K vs i5 3570K) gets you in lightroom. There is a benchmark thread for Photoshop but I haven't seen anything for Lightroom. I know with the users on this forum we could put together a pretty good comparison of hardware. I haven't wanted to for some time but haven't had the time. Seems like we would need to compare a few main parts.

CPU:
- Cores (2 vs. 4 vs. 8 vs. 12)
- Clock Speed

I have tested clock speed but not cores on my rig. Clock speed does make a difference but not a lot. Cores should be easy, I can test 4 with my 3570K and then drop two and retest for example. Not sure if anyone is running a 6c 12t chip on here but would be interesting to see.

RAM:
- Amount (2GB vs. 4 GB vs. 8GB. vs. 16)
- Frequency (1333 vs. 1600 vs. 1866 vs. 2133 ect)

This should also be easy to test by removing RAM (we would need to keep pairs for dual channel setups) and re-testing. Frequency should also be easy as someone with 2133 RAM could run any speed under that with a few BIOS changes.

Disk:

- 5400RPM vs. 7200RPM vs. SSD

This one is a bit more complicated as I am not 100% sure how Lightroom uses files. For import I think it would effect where they are stored. However, after that I am not sure if they are ever accessed again from the original location. I would want to understand the cache usage and how the files are stored and used before making setups to test.

GPU:
Everything I have read says the GPU isn't touched by lighrtroom (except for display). Therefore, I don't see a need for any tests.

Benchmarks:

I think we would need to have everyone use the same RAW files to keep things consistent. Getting everyone those files is something I would have to figure out. I would be interested to see tests of maybe 50 RAWs run in a batch. The larger batch helps take out some of the timing errors.

Test 1: Render 1:1 previews
Render 1:1 previews for the 50 RAW photos.

Test 2: Processing
Process a batch of 50 RAWs with the same preset.

Test 3: Export
Export the 50 RAWs to disk.

I think that would cover it and give a good idea on the performance differences. Anyone have any comments on tests, setups, protocol on tests ect.? Maybe I should have left it at is anyone even interested in the benchmarks.

should be very interesting and very informative. cant wait for the results. if i know what i was doing with lightroom and had a good copy id run the same tests on mine. maybe in the future!

I'm very interested in this!!

Well got started on this a bit and ran into a few hiccups. My motherboard won't let me underclock my CPU. I may just be missing a setting but Ill have to do 3500 MHz to 4200 MHz for now. If its a linear relationship I may just be able to graph them and get a general idea of what below 3500 and above 4200 would look like.

Second issue was the processing. If I create a preset with a bunch of changes and then select the whole catalog. It applies it but doesn't actually process each photo. Have to figure out something to test that.

The 'import' benchmark was also clairfied to generating 1:1 previews. Import is almost instant, I generate all my 1:1 previews upon import. It takes some time and loads the processor pretty well.

Some preliminary results. I had results for the dropping cores too but my laptop took a dump and I lost all 4 tests.

Anyway on to the results.

Exporting to the SSD vs. HDD. The same. I was surprised by this, especially across frequencies but I have seen it before too when I ran some tests in the past. I may try putting the source images on the HDD and seeing if it effects the time to make 1:1 previews.

Changing CPU frequency. I see this come up quite often when people ask if they should get processor X or Y and the only real difference is frequency. It does make a difference but keep in mind that is over 50 photos. From 3500 to 4200 MHz there is a 18 second difference or 0.36 seconds per photo.

IMAGE LINK: http://s111.photobucke​t.com …ange_zps10170c6​2.jpg.html

IMAGE LINK: http://s111.photobucke​t.com …ange_zpsb21d2ac​b.jpg.html

IMAGE LINK: http://s111.photobucke​t.com …Data_zps1129fe6​8.jpg.html

A look at how Lightroom uses the CPU and RAM. Its a bit harder to see on the RAM due to the scale but the usage on both surge up and down as each photo is either Rendered for a 1:1 preview or exported to JPEG.

Never went over 2.5 GB of RAM usage for either operation. The big RAM usage that some report I think is when using some of the processing tools (brushes, gradients, ect.)

Rendering 1:1 Previews:

IMAGE LINK: http://s111.photobucke​t.com …sage_zps29c71e7​6.jpg.html

IMAGE LINK: http://s111.photobucke​t.com …sage_zps74465b5​4.jpg.html

A couple more quick tests before bed because I got curious. Tried putting differnt parts of the data on a RAM Disk. A RAM disk is extremely fast and is basically just a chunk of RAM setup by software to act like any other disk. Only issue was the Catalog couldn't be put on it as Lightroom saw it as a volatile drive (power goes off you lose your catalog) and I couldn't export to it due to permissions issues. Small gains using the RAM disk. Ill test with the HDD soon.

IMAGE LINK: http://s111.photobucke​t.com …ture_zpsdce0280​c.jpg.html

Tried to knock out the HDD tests this morning. Had some intresting results. I was getting in the 104 range for the 1:1 test at first and thought maybe splitting the load was helping speed things up slightly. Then I put all the tasks on the HDD and got the same result 104. So I re-ran the tests with everything on the SSD and got 104. I then re-checked the all SSD test and 104 again. Re-checked two of the HDD options and 104 again.

I honestly have no idea what changed overnight. I shutdown overnight and when I restarted I confirmed all the settings in CPU-z just to be safe. Nothing changed. I guess it goes to show that other things can effect the speed of the work as much as faster RAM or a few thousand MHz bump.

Anyway, updated results.

HTTP response: 404 | MIME changed to 'image/gif' | Byte size: ZERO | PHOTOBUCKET ERROR IMAGE

Most important (IMO) isn't how fast LR renders an export, but how responsive it is while editing images.
A small change there can feel like a big improvement.
But how to test that? Maybe change all "basic" sliders on an image a set amount, and measure the time until the image is updated? Then again, if you can reliably measure that time, LR would probably feel "too slow"

René Damkot wrote in post #16054876
Most important (IMO) isn't how fast LR renders an export, but how responsive it is while editing images.
A small change there can feel like a big improvement.
But how to test that? Maybe change all "basic" sliders on an image a set amount, and measure the time until the image is updated? Then again, if you can reliably measure that time, LR would probably feel "too slow"

Yeah that is the one that is being tricky. I use 50 images to get a good run time and take out timing errors but haven't figured out a way to batch process/edit yet. I would assume the results of editing would follow suit of the other tests but can only assume without a test to run.

The way I read it, LR would take it's sweet time no matter what you try

Amamba wrote in post #16060177
The way I read it, LR would take it's sweet time no matter what you try

Here are some tests I did awhile ago. You can see that moving from the laptop to the current desktop things sped up considerably.

Looking at the results and tests I have done so far I wouldn't expect a whole lot of change. I'm trying to find my 5400 RPM drive to test that as I think it would slow it down considerably. The changes made represent incremental upgrades in performance and it often takes a leap in hardware updates to really 'feel' and see changes.

IMAGE LINK: http://s111.photobucke​t.com …re-8_zpsb8dc009d.png.html

Silvrr, can you please upload one of your RAW files so that we're all on the same page?


In the meantime, here's a RAW that could serve as a interim standard.
It's from a Drebel XT (8mp). https://dl.dropboxuser​content.com/u/60092457​/Raw/1.CR2

Make a bunch of copies* - 50 is as good a number as any - and batch away!

(*With Windoze, you can right + drag = 'Copy Here' the file a short distance to an area within the same folder as the original. Now you have 2. Highlight and drag them to make 4, etc. The file names do become amusing: "Copy of copy of copy...." No idea how a Mac works, but there must be something similar. Right?)

This file is dead neutral, with no sharpening, NR, etc. assigned from DPP. The only different issue is the tungsten WB used to counteract the nasty light pollution.

FWIW, I'd be interested to see straight conversions using LR, DPP, and anything else, running on a variety of systems.

Geonerd wrote in post #16082972
Silvrr, can you please upload one of your RAW files so that we're all on the same page?


In the meantime, here's a RAW that could serve as a interim standard.
It's from a Drebel XT (8mp). https://dl.dropboxuser​content.com/u/60092457​/Raw/1.CR2

Make a bunch of copies* - 50 is as good a number as any - and batch away!

(*With Windoze, you can right + drag = 'Copy Here' the file a short distance to an area within the same folder as the original. Now you have 2. Highlight and drag them to make 4, etc. The file names do become amusing: "Copy of copy of copy...." No idea how a Mac works, but there must be something similar. Right?)

This file is dead neutral, with no sharpening, NR, etc. assigned from DPP. The only different issue is the tungsten WB used to counteract the nasty light pollution.

FWIW, I'd be interested to see straight conversions using LR, DPP, and anything else, running on a variety of systems.

It's 50 seperate files so uploading only one wouldn't give apples to apples. I have it on my list to try the tests with a few different sets of files to see how it effects the benchmarks. If its close (hoping it is) then anyone can use 50 RAW files of there own. I should have files for 10, 15 and 18 megapixels.

silvrr wrote in post #16083352
It's 50 seperate files so uploading only one wouldn't give apples to apples. I have it on my list to try the tests with a few different sets of files to see how it effects the benchmarks. If its close (hoping it is) then anyone can use 50 RAW files of there own. I should have files for 10, 15 and 18 megapixels.

Cool.