I'd say it's perfectly normal to respond differently to photos compared to a story, whether in book or movie form.
It makes sense -- with a story we are given a context and typically are led to care for a character or characters in some way, so that we would respond to, say, a death scene with not just, "wow, a death scene" but "oh no, Joe/Mary died!"
It's like comparing the work of the cinematographer to that of the director and the actors and the scriptwriter -- the cinematographer might do breathtaking work, which is what we as photographers would like to do -- but it's the story as a whole and the portrayal of that story that gives the whole emotional impact.
Think of watching a movie where at the beginning you see someone die. It may have a touching quality if it's well done, but aside from that, what emotion would you realistically expect? You know nothing about the person who died or what led to the death, so you don't have a context for a specific emotional response.
On the other hand, if you saw that scene at the end of the movie, and the character was someone you had come to care about, then a whole complex of emotions could be expected -- some kind of a tear jerk response!
That being said, there are some basic responses that good photography can elicit -- maybe a "WOW" response from a big print from a gorgeous landscape or a beautiful bird in flight, or a heart-rending response to someone suffering in some way, or a smile and a little twinge of the heart to seeing a young child in gleeful play or a mother caressing an infant, or two lovers in an embrace. As a photographer, those are the emotional responses I could aspire to.
Also, some photos have an understood context -- I can barely look at a scene of the Viet Nam War memorial in Washington DC without gettting teary-eyed, or a scene of the Kennedy assasination, or of the World Trade Center on 9/11. Without the context, though, what would my "emotional" response be? The emotion comes from the (his)tory, not from the image alone.