People like to say ישראל אע"פ שחטא ישראל הוא so we should love חילונים.
Someone pointed out that a ישראל שלא חטא is ALSO a ישראל so we should love frum Jews too, even if they don't think or live as we believe they should. What you and I have in common with people with whom we disagree is that we believe what we believe b/c of the way we were raised, the home we grew up in, the schools we went to, the types of friends we had etc. etc. just like them.
If we were raised in Crown Heights or Kiryas Yoel we wouldn't think exactly as we think today. So we are ALL products of our environment and the only One with a monopoly on absolute אמת is He whose seal is אמת.