You can just use dpkg to remove a single package. Apt is a package manager so will remove the whole group including deps.

Probably a way to lock a package also, so will never be removed. Just google around.