Either the proximity sensor is malfunctioning or you are holding it in a peculiar manner such that the sensor is far enough from your face not to be triggered. The screen protector is probably not an issue, when I first got my N900 I got a screen protector without a cutout for the proximity sensor and that actually resulted in the sensor being triggered even when it wasn't up close to my face keeping the screen blank. There is an outside chance that this is a software issue in which case reflashing should cure it.