Given how a speaker works (by moving a membrane to create sound waves), covering the hole would cause a definite loss in sound quality (if not block it out altogether).