I figured out that there is a list of certificates stored in /etc/secure/s/certman.common-ca
Sadly, I cannot simply remove the offending certificate as the file is signed. Guess I'll just have to settle for filing a bug report and let someone else fix it