You could try using the proper Youtube.com site instead of the mobile phone version? The tablets do actually work with the full site, the tablet browser is far better than those on mobile phones.

Alternatively you could install Canola and watch YouTube through that, which is probably the easiest way to navigate the site, and it uses the proper Flash videos too.

I've had 3gpp files work when I transferred them from a phone, so the player should support it, but perhaps YouTube's mobile site uses some weird version of the format?