Assume p=player, e=enemy, r=enemy rotation (eg 0-359) with 0 at +x and going towards +y
d=(cos(r), sin(r))
pd=(p.x-e.x, p.y-e.y)
pdn=pd/sqrt(pd.x*pd.x+pd.y*pd.y) OR pdn=normalize(pd) Direction to player, normalized
dot=pdn.x*d.x+pdn.y*d.y OR dot=dot(pdn,d)
1 = enemy is perfectly looking at player
0 = enemy is looking sideways from player
-1 = enemy is perfectly looking away from player