The only time I had a need to wear something even slightly formal (i.e., something better than shorts and a t-shirt) was when I visited a university. So unless you know in advance that you'll be going somewhere formal, leave your good clothes at home.
In Canada, I almost always wear long pants. China was very hot and humid, so I bought a pair of shorts the second day I was there. Once I started wearing shorts, I noticed people looking at my legs with a look of curiosity (not admiration :-). I never figured out if the novelty was (a) hairy legs; (b) my white legs; (c) that I was wearing socks (most Chinese don't).