No need to do an rm -rf (dangerous command anyway, to be used very sparingly) - a simple rm will do. If it is a symlink, it will delete the link, not the files it links to.