It's actually recommended for configuration files and user-generated content (e.g. downloaded videos/images or created folders) to remain on the device after an application is removed. In the case of config files, this is so you do not have to restore all your settings if you re-install an application.

Using apt-get --purge remove <package> will remove config files etc, but user-generated content will remain.