The one big thing that stands in the way right now is that applications that want to draw directly to the framebuffer (like mplayer -vo fbdev) have no problem writing to /dev/fb0 but what they write never shows up on the screen.