Orthodox Jews And Baseball

If a person would stand outside certain shuls in Manhattan and watch people entering and leaving he might think that inside the shul they play baseball. The reason for this mistake is that many of the people are wearing baseball caps.

I have curiously observed this phenomenon for much time. It was explained to me that people don't want to wear their yarmulkes on the street. I am just curious. WHY NOT??? Shouldn't we be proud of the fact that we are Jewish?? America is the "land of the free and the home of the brave". We have nothing of which to be ashamed or afraid.

If there is a compelling reason to hide one's yarmulke of which I am not aware, I would love to hear.