One reason I'm thinking about is that there are quite complex rules on which details at which sizes are to be rendered, based on a zoom level. Your average SVG files don't provide those rules, and even if they did, rendering would become so complex that your phone would die of overheating / battery drain very fast.

Another way is to use OSM data (maybe in PBF format) to do the same, but I'm wondering how much study one has to invest into it, and how efficient would it be in the end. Wouldn't it be that in addition to quite big OSM datafiles there would need to be an even bigger index to efficiently search for needed data fragments?

Tiles are simple, on the other hand, and straightforward. You don't need to muck around with projections all the time, resorting instead to simple Cartesian arithmetic.

Maybe when the phones grow another gigabyte of RAM and a few more processing cores, these reasonings will be outdated.