In my opinion, it depends. I agree with any word used, if only it doesn't carry the plot. If you say: 'The trebuchet propelled a boulder into the air.' There might be less commotion than when you say: 'Anna had always liked trebuchets.' (and then go on, not explaining what a trebuchet is (I know it's not a very good example)).
Now in the other cases:
From time to time I have to look up a word while reading. It does bother me. I curse myself for not knowing the word. But at least it teaches me something.
When I write in English, I don't think I'll be having problems with people not understanding me, but in Dutch they do seem to bother. But I still think I'm right, the conversation went like this:
-Reader: you use too much difficult and bombastic words
-Me: *rereads text* really?
-Reader: yeah, totally!
-Me: can you give me an example from my text?
-Reader: *gives some examples*
-Me: but those are normal words
*mutterIhopethisisn'tyourmothertongueforyoursakemu tter*
Like Ace said: shouldn't you be happy if a book learns you something, rather than annoyed?