Okay, crits first: 'heart flutters' is somewhat overused and could do with being more inventive. 'Exhausted by the irritation' doesn't sound right...it seems to trivialise the subject, in which case it's unlikely that it would exhaust you. 'Incongruent imposter' is bordering on being an act of poetry, of which I've been accused on occasion too. Also, the start of the last stanza is a bit detached from the end of stanza two. 'An enemy' just appears and doesn't make me instantly go 'oh right, gotcha'.
However, lines 3 and 4 are good, so are the first two lines of stanza 2 and the final line. 'Something which exists independently of my control' is maybe too philosophical although it works quite well.
Hope these suggestions were useful.

A good start with L1. "push you away for not sitting close"? - seems inconsistent. Are you sure it is irritation that exhausts you in L5? I like the idea of objectively watching the imposter. I've written about this too. But I think you can do better at describing this objective/highly personasl phenomenon.

On the contrary, I thought 'push you away for not sitting close' was extremely clear. He's speaking in contradictions. It don't get much more contradictory than that.

I didn't care for 'an enemy has permeated my mind', since in this line you're stating specifically what you succeeded in hinting at - quite well, in my opinion - throughout the entire poem. That aside, I thought it was a well-built and powerful poem; and I understood and identified with it, which is one of the trademarks of a good piece of work.

It works better with the edit, but 'irritation' still jars...