A gentile goes to the ball game with his family. On the court or field there are scantily clad young woman whose job it is to "lead the cheers". The real reason they are there is because most sports fans are men and men like to look at beautiful young women. That is why Sports Illustrated has a "swimsuit issue" that has VERY LITTLE to do with swimsuits. This issue is [or was when I was a kid but something tells me that things haven't changed much...] by far the most popular issue of the year. Men like sports and those who are selling sports want the men to get the MAXIMUM entertainment and those young ladies enhance the experience [a cold beer and a man thinks he is in gan eden..]. So this goy looks at the cheerleaders. His wife knows he is looking and she understands. If she were that attractive she'd be a cheerleader herself. As long as he doesn't actually DO anything she is cool with him.
This is where the bracha of she-lo asani goy comes in. The Torah wants us to be 100 percent completely focused on our wives. Even a glance at another woman for the sake of pleasure is a form, albeit [I LOVE that word! 3 in one. "All be it"] subtle form of infidelity. Not grounds for divorce - if it were, the only married Jews left would be the blind ones - but a type of cheating on one's wife general society would consider completely within the bounds of acceptable and yet we consider a serious sin.
There is a halacha that when one is intimate with his wife it is assur to think of another woman. Why not? She won't know [if he has the sechel not to tell her...]. Well, first of all, on a mystical or subconscious level she DOES know. In addition, the Torah wants us to be COMPLETELY with our wives. A thought in another direction distances us from her. The Torah doesn't just want to govern our actions but our internal world as well. That is how we become pure.
When somebody gazes at another woman he is not only compromising his marital bond but is also USING another person with no concern for her as a PERSON but as an object of his pleasure. "Object" is the key word. A form, subtle once again, of a very serious infraction that we all believe to be wrong. See דברים כ"ב כ"ח - ouch. Yeah that one.
So looking at other women is a sin against one's wife, against the other woman and foremost a contamination of one's own neshama.
Note: I am male and am writing from a male perspective so all of the bnos yisrael reading this are invited to extract the appropriate lessons for themselves. I have already received one email from a bas aliya to that effect.