Confirming here...

Have anyone tested to Delete the tracker Database and restart the daemon to delete the unused tags?

You can find it here: Link - I guess it could also work for this issue.

But I'm a little bit afraid to test it myself. I'm new at Linux and still learning...
So, have a crack that knows what he does tryed this out?

Edit:
Ah, thanx benny1967. Same Idea.