From what I can see the fmradio application is available in one of the standard Nokia repositories, so you should find fmradio if you look in the application manager. As soon as it's installed you have to enable it through the applet selection function in the top-left menu of the Home screen. It'll look almost like the Internet radio when enabled.
Then remember to plug in the headset cable, it acts as an antenna. Without it you'll hear only static. If you click on the 'maintenance' icon (the wrench) in the applet you can search for and add stations, and also switch the sound from the headphones/earbuds to the N800 speakers.