Sometimes, I use sand-like rag to scrub my skin. Dead skins can clog up pores which can cause pimples. Whenever I do that, my skin feels a bit more smooth and clean.
I think it also depends on your genetic and where you live. Dry air or humid can affect your skin too.