Quote from Side_salad
are you on multiplayer? if you are, then the endermen won't always attack you.
Quote from iSmile
It's only when you look at them from a far distance, when they attack you.
Otherwise they are just evil creatures with a sense to kill cuddly little animals.