I think it might be because he's really good with a bow and arrow. I'm not that scared of him, but I'd rather not have an arrow in me if I can avoid it. Since he does know where I live. So I just have healthy concern about him.

Unless we're talking about a completely different green guy. Then some people have traumatizing memories associated with green things. I don't, but it's not completely unheard of.