In his book, The Rituals and Practices of a Jewish Life, Rabbi Kerry Olitzky makes the case for wearing the talit katan (the fringed undergarment worn my frum guys).

That would feel a little less hypocritical to me. This way I can get that reminder to be a good Jew without having people make erroneous assumptions about me all day.

Plus, wearing fringes is actually a commandment in the Torah, not just a custom.