Gideon is well aware that he is nobody's lover, nor is he a simpleton. Being so reminded, as though he might have forgotten at some point, chafes. Especially when he feels he's done nothing wrong. If you're so insecure in your relationship, my actions will make little difference.