armv5te, armv6 or armv7 are most useful when having hand crafted optimizations for these instruction sets. The difference can be very significant, especially for multimedia. But the compiler itself is very dumb and can't generate fast code (at least this is true for gcc).

Some part of this speedup apparently comes from just not using thumb, the other part comes from using vfp. There is no penalty for switching arm/thumb modes, this is done at function call or return boundaries (you can call thumb functions from normal arm code, and arm code can be called from thumb). This switching does not have any overhead at all when compared to normal arm->arm functions calls.