Probably it would be enough (or even better) to only have an option to vote interesting/relevant posts up.

If it's true that the whole rating system is only meant to help a machine decide what's relevant and what's not (in order to make it visible on the front page or base karma calculations on the numbe rof relevant posts...), telling the machine that a post is relevant should be all that's needed.
I don't see the need for "not relevant" if "not voting" serves the same purpose much easier.

The downside of the current system (heart/thumbs down) is that with controversial posts, it makes people vote for or against the opinion expressed by the author rather than saying anything about how relevant it is for the community.


Example:
I have to admit I could never ever get myself to voting positively for a posting that suggests jPhone-like changes to the Maemo-UI; if I find somebody voted this down before, I will happily click 'thumbs down', too, only to express my negative feelings. The more votes there are in general, the more people will feel this is some kind of poll and they want to express their opinion, too.
Votes shouldn't be about opinions, though, and they certainly aren't polls. In this example, even if I still don't vote for this post, others will and thereby mark it as relevant (which it is, not matter what I think about it).