You think wrongly.

Wanking from time to time is fine, it helps you to release endorphins and make you feel better. The problem comes when you wank too much and you get addicted to those endorphins, everything becomes dull and boring.

Personally, I believe everyone should limit, it will sharpen your social skills and confidence.