“If a person straightens himself like an arrow [i.e. he corrects his actions and perfects his character], then too he will certainly have some enemy or the other.” (Imaam Shaafi’ Rahmatullahi alaih)
Therefore, one should not be concerned about the comments and attitudes of others towards one. As long as one is following the Shariah of Allah and the Sunnah of Rasoolullah (Sallallahu alaihi wasallam) the criticism of the critics and enmity of enemies should be ignored. Their comments should be assigned to the trash can.