2lardman: There was an idea about also using DSP for scaling (video controller does not have scaling support), there was also an attempt to do this http://www.mail-archive.com/maemo-de.../msg05345.html
Considering all this, seems like C55x DSP programming is a very hard task (without proper tools, documentation and examples), as it is a very unusual cpu. So I don't feel like doing any DSP programming in the near future, it seems to be more efficient to use what is already provided on the device (like MP3 decoding using DSP) and apparently developed by the people who are proficient with this. The default video plays performs scaling in some way, so probably the code for it exists. It would be best if Nokia could provide some high level API for using colorspace conversion and scaling: http://www.mail-archive.com/maemo-de.../msg04511.html

On the other hand, ARM core also has some enhanced DSP instructions and is supposed to be suitable for multimedia: http://arm.com/pdfs/DVI0035B_926_PO.pdf
Moreover, optimization of video decoding to use these armv5te instructions has been already started and there are some results already (in upstream ffmpeg SVN, one of the ffmpeg developers got Nokia 770 too ). The next build of mplayer for nokia 770 will also use these optimizations.