Okay. I just confirmed that this problem was caused my chosen language - which happened to be English-United States (en-US in headers). I changed my Device Language to Finnish (fi-FI in headers) and now I'm able to access Ovi Store!

Maybe everybody should install Hide User Agent and use fi-FI in headers (for language). This might be handy workaround for time being.

Update: en-UK should work too...?