Approve the Cookies
This website uses cookies to improve your user experience. By using this site, you agree to our use of cookies and our Privacy Policy.
OK
Index  •   • New posts  •   • RTAT  •   • 'Best of'  •   • Gallery  •   • Gear  •   • Reviews
Guest
New posts  •   • RTAT  •   • 'Best of'  •   • Gallery  •   • Gear  •   • Reviews
Register to forums    Log in

 
FORUMS Photography Talk by Genre General Photography Talk 
Thread started 14 May 2015 (Thursday) 16:59
Search threadPrev/next
sponsored links
(this ad will go away when you log in as a registered member)

Photo Algorithm Makes Reflections Disappear

 
Tedder
Senior Member
Avatar
378 posts
Likes: 74
Joined Jan 2009
     
May 14, 2015 16:59 |  #1

"MIT Photo Algorithm Makes Window Reflections Disappear"

... Besides finding double reflections, the researchers' technique also involves looking for statistically anomalous pixels—those that don't seem to be a part of the overall image. The researchers fed an algorithm 50,000 images to teach it how to differentiate pixels that are meant to be there from pixels that aren't. When the "trained" algorithm divides an image into an 8x8 pixel square, it can reliably distinguish between reflection and overall image.



source: http://www.popularmech​anics.com …rithm-window-reflections/ (external link)


Tedder Stephenson's Flickr (external link)
Various Items (external link) Mineral Matters (external link) The Bench (external link) Tracks (external link) Cars and Stripes (external link) Behind the Wheel (external link) Shadows of Turning (external link) Circles of Confusion (external link) Waterous Disturbulations (external link)

  
  LOG IN TO REPLY
sponsored links
(this ad will go away when you log in as a registered member)
BigAl007
Cream of the Crop
Avatar
8,020 posts
Gallery: 543 photos
Best ofs: 1
Likes: 1628
Joined Dec 2010
Location: Repps cum Bastwick, Gt Yarmouth, Norfolk, UK.
     
May 14, 2015 20:33 |  #2

Well I wasn't terribly impressed with the after shot in the linked article. There was still a lot of the reflection left over. Actually the "removed" bit actually seemed better to me, almost as if it were designed to isolate and enhance the reflection rather than the non reflected image.

Alan


My Flickr (external link)
My new Aviation images blog site (external link)

  
  LOG IN TO REPLY
Tedder
THREAD ­ STARTER
Senior Member
Avatar
378 posts
Likes: 74
Joined Jan 2009
     
May 14, 2015 23:44 as a reply to  @ BigAl007's post |  #3

I agree that the results are not great. I do find the “trained algorithm” idea interesting, though. And this technology will probably have uses beyond removing reflections.

The article at MIT News (external link) contains more information and includes the following quotation:

“The ideas here can progress into routine photography, if the algorithm is further robustified and becomes part of toolboxes used in digital photography.”


I hope I live long enough to enjoy those robustified algorithms.


Tedder Stephenson's Flickr (external link)
Various Items (external link) Mineral Matters (external link) The Bench (external link) Tracks (external link) Cars and Stripes (external link) Behind the Wheel (external link) Shadows of Turning (external link) Circles of Confusion (external link) Waterous Disturbulations (external link)

  
  LOG IN TO REPLY
Trvlr323
Goldmember
Avatar
3,290 posts
Likes: 1090
Joined Apr 2007
     
May 14, 2015 23:54 |  #4

I agree with Alan. Someone should be a little concerned about their research funding on this one ;-)a


Sometimes not taking a photograph can be as problematic as taking one. - Alex Webb

  
  LOG IN TO REPLY
Kolor-Pikker
Goldmember
2,790 posts
Likes: 59
Joined Aug 2009
Location: Moscow
Post edited over 4 years ago by Kolor-Pikker. (3 edits in all)
     
May 15, 2015 05:27 |  #5

nqjudo wrote in post #17557002 (external link)
I agree with Alan. Someone should be a little concerned about their research funding on this one ;-)a

The theory is interesting, even if the current implementation is wonky, or perhaps even misdirected.

First of all this depends on the phase difference between the inner and outer reflections of the glass, but the image needs sufficiently high resolution for that difference to be detectible in the first place in a static image; I can't imagine the difference being more than a few pixels on a 20mp+ sensor and close up to the glass surface.

However, while deducing the origin of the reflection post-fact may be difficult, if they integrate this algorithm into a camera's software and have it dynamically track reflections as the camera is shifting in space, by the time the exposure is taken the camera might already have the reflection data figured out, possibly even as part of a double-exposure. This of course will only work on cameras that constantly read off the sensor, meaning point & shoots and some mirrorless systems, although I'm guessing these are the kinds of cameras most people tend to shoot through glass with in the first place.

Alternatively, the purpose of this algorithm may be more useful in other applications, like machine vision or something technical where reflections may be a problem. One thing I can think of is surveillance, since video cameras constantly read the sensor, they would be optimal candidates for reflection removal.


5DmkII | 24-70 f/2.8L II | Pentax 645Z | 55/2.8 SDM | 120/4 Macro | 150/2.8 IF
I acquired an expensive camera so I can hang out in forums, annoy wedding photographers during formals and look down on P&S users... all the while telling people it's the photographer, not the camera.

  
  LOG IN TO REPLY
BigAl007
Cream of the Crop
Avatar
8,020 posts
Gallery: 543 photos
Best ofs: 1
Likes: 1628
Joined Dec 2010
Location: Repps cum Bastwick, Gt Yarmouth, Norfolk, UK.
     
May 15, 2015 07:14 |  #6

Kolor-Pikker wrote in post #17557180 (external link)
Alternatively, the purpose of this algorithm may be more useful in other applications, like machine vision or something technical where reflections may be a problem. One thing I can think of is surveillance, since video cameras constantly read the sensor, they would be optimal candidates for reflection removal.


Even reading the initially linked article it seemed as if this was far more about the problems faced by machine vision systems, which often don't actually need to produce a good looking image but a reliable data stream to make decisions from. Even on that single example image, I am sure you would get a much more reliable edge detection on the outside image data once this algorithm had done it stuff, especially if you ran them on the RGB colour separations. Add in that the pulled off reflection image could also have the edge detect run on it, and compared to the results of the other version. Given that it seems to be able to detect the phase difference of the reflections on the two surfaces, it could even become possible to allow for the refractive effects of the glass that the system is looking through, possibly by pinging an IR laser pulse through it. You would then even be able to measure the thickness of the glass.

Alan


My Flickr (external link)
My new Aviation images blog site (external link)

  
  LOG IN TO REPLY
Trvlr323
Goldmember
Avatar
3,290 posts
Likes: 1090
Joined Apr 2007
     
May 15, 2015 08:44 |  #7

Kolor-Pikker wrote in post #17557180 (external link)
The theory is interesting, even if the current implementation is wonky, or perhaps even misdirected.

First of all this depends on the phase difference between the inner and outer reflections of the glass, but the image needs sufficiently high resolution for that difference to be detectible in the first place in a static image; I can't imagine the difference being more than a few pixels on a 20mp+ sensor and close up to the glass surface.

However, while deducing the origin of the reflection post-fact may be difficult, if they integrate this algorithm into a camera's software and have it dynamically track reflections as the camera is shifting in space, by the time the exposure is taken the camera might already have the reflection data figured out, possibly even as part of a double-exposure. This of course will only work on cameras that constantly read off the sensor, meaning point & shoots and some mirrorless systems, although I'm guessing these are the kinds of cameras most people tend to shoot through glass with in the first place.

Alternatively, the purpose of this algorithm may be more useful in other applications, like machine vision or something technical where reflections may be a problem. One thing I can think of is surveillance, since video cameras constantly read the sensor, they would be optimal candidates for reflection removal.


LOL. Well yes, of course. One can conclude many interesting directions, possibilities and uses from a given research or result. I agree with your comments. Winky emoticons however are useful for denoting some kind of playfulness within the context of the discussion. I thought it might be somewhat more lighthearted than using several paragraphs to state the obvious.


Sometimes not taking a photograph can be as problematic as taking one. - Alex Webb

  
  LOG IN TO REPLY
sponsored links
(this ad will go away when you log in as a registered member)

774 views & 0 likes for this thread
Photo Algorithm Makes Reflections Disappear
FORUMS Photography Talk by Genre General Photography Talk 
AAA
x 1600
y 1600

Jump to forum...   •  Rules   •  Index   •  New posts   •  RTAT   •  'Best of'   •  Gallery   •  Gear   •  Reviews   •  Member list   •  Polls   •  Image rules   •  Search   •  Password reset

Not a member yet?
Register to forums
Registered members may log in to forums and access all the features: full search, image upload, follow forums, own gear list and ratings, likes, more forums, private messaging, thread follow, notifications, own gallery, all settings, view hosted photos, own reviews, see more and do more... and all is free. Don't be a stranger - register now and start posting!


COOKIES DISCLAIMER: This website uses cookies to improve your user experience. By using this site, you agree to our use of cookies and to our privacy policy.
Privacy policy and cookie usage info.


POWERED BY AMASS forum software 2.1forum software
version 2.1 /
code and design
by Pekka Saarinen ©
for photography-on-the.net

Latest registered member is htam
1078 guests, 232 members online
Simultaneous users record so far is 15144, that happened on Nov 22, 2018

Photography-on-the.net Digital Photography Forums is the website for photographers and all who love great photos, camera and post processing techniques, gear talk, discussion and sharing. Professionals, hobbyists, newbies and those who don't even own a camera -- all are welcome regardless of skill, favourite brand, gear, gender or age. Registering and usage is free.