Theon Greyjoy always irritated me. He had, for me, none of the power of some of the series' villains, nor the likeable qualities of the 'heroes'. He always struck me as a petty twerp with a cruel streak.
Sansa bugged me a good deal early on, too; but as time went on, and we saw her character develop somewhat, she grew on me. Now I tolerate her with mild curiosity - a big improvement, I assure you.
Catelyn Stark could grate a little, too. And her current...disposition...isn't helping any. She plays the role of wooden, put-upon martyr too well for my taste. Besides, she gave Jon a hard time, so she has a black mark next to her name for me.
I like characters with charisma; whether they are good, evil, or ambiguous. Characters with style and power. Conversely, characters with none of these qualities tend to irk me.