Most annoying to me, from the top of my head, is when you are trying to find a word to say, and your interlocutor finishes your sentence with a word which is sort of fits in, but not the one you had in mind.

I'm not sure I'm all that against interrupters, and I think I can handle them and can sort of be one myself, but it is surely exhausting.