maemo.org - Talk

maemo.org - Talk (https://talk.maemo.org/index.php)
-   Applications (https://talk.maemo.org/forumdisplay.php?f=41)
-   -   [SFOS] [Announce] Native offline maps: OSM Scout Server (https://talk.maemo.org/showthread.php?t=97823)

Fellfrosch 2018-05-15 17:05

Re: [Announce] Native offline maps: OSM Scout Server
 
The previously downloaded maps where already vector maps. There must be a different problem:
If you look inside the federal state Baden Württemberg, there you can download the whole federal state, size nearly 2 GB. But you also can download the further splitted areas which are all together 3,5 GB so it has something to do with the splitting. Maybe because the areas are overlapping.

rinigus 2018-05-15 19:32

Re: [Announce] Native offline maps: OSM Scout Server
 
Quote:

Originally Posted by Fellfrosch (Post 1544333)
Hi rinigus, How do I download larger areas? I had some older maps on my device and decided to download the new maps. But now I'm quite unsure. Before I had a package "Germany" that was nearly 7 GB. But now Germany is spitted up in different federal states and some of them are even splitted up further. All german maps together would be much more than 20 GB. And I would have to choose a lot of areas seperately. Is that by intend?

I have dropped full Germany and France imports. But *DON'T PANIC*, its still possible to download them and it shouldn't take much more space than before.

Germany and France resulted in huge Mapnik SQLite databases (4GB+) that, due to limitations of FAT FS on SDCards, resulted in periodic questions "why my Germany download doesn't work?". An additional bonus was that it reduced a time of imports by few hours, reduced HDD requirements of the servers as well.

For this convenience on my side, users that want full Germany and France have to get them by regions. So, in your case, you would have to select all subregions (except Berlin which is in some other region as well, if I understood correctly). For Bayern, for example, you could just choose it and there is no need to get all its subregions since they should be all included in it.

Now, the sizes displayed in the menus correspond to the case where it is assumed that you will download only that region. This is to prepare you for the worst-case scenario. Fortunately, the regions in Germany overlap and will share the same data - will be downloaded and stored only once. Let's go through main backends, so you'll get the idea:

* Address parser is given for full country. So, in case of Germany, its 200MB is accounted for in each of the regions

* Geocoder-NLP and OSM Scout are the only backends that don't overlap, but are cut by the border. [technically, there is a minimal overlap, but its not important]

* Mapbox GL and Valhalla are using tiles. If the tile is touching a region, its accounted for in it. Fortunately, tiles are downloaded and unpacked only once.

In your case, you would have to unsubscribe from Germany and subscribe to all its regions. For curiosity, would be good interesting to know current (old schema) and the new disk usage. Note that I pulled Valhalla tiles in somewhat bigger tiles, but I would expect it didn't change much Germany since Valhalla tiles over there were big already before.

pichlo 2018-05-15 19:34

Re: [Announce] Native offline maps: OSM Scout Server
 
Quote:

Originally Posted by Fellfrosch (Post 1544340)
The previously downloaded maps where already vector maps. There must be a different problem:
If you look inside the federal state Baden Württemberg, there you can download the whole federal state, size nearly 2 GB. But you also can download the further splitted areas which are all together 3,5 GB so it has something to do with the splitting. Maybe because the areas are overlapping.

That's exactly it. Overlapping. I had a similar problem. Downloading maps for Germany, I had to do that one state at a time. But then the total space taken on the memory card was much smaller than the grand total of all maps added together. When I say much I really mean MUCH. Like less than a half. I remember adding the last one, which alone was supposed to be about 1.5GB, added only about 200MB on the card.

Fellfrosch 2018-05-16 06:35

Re: [Announce] Native offline maps: OSM Scout Server
 
@rinigus thanx for your detailed explanations. I've downloaded all the regions, and am absolutely satisfied. The total space taken doesn't differ to much (less than 500 MB) from the Full Germany download. So everything is fine and I'm totally flashed again by your great work and the ongoing support!!!

pichlo 2018-05-16 12:46

Re: [Announce] Native offline maps: OSM Scout Server
 
Quote:

Originally Posted by Fellfrosch (Post 1544347)
The total space taken doesn't differ to much (less than 500 MB) from the Full Germany download.

I expect that to be simply because the maps have grown, not because of the fragmenting. I just updated all my maps (15 areas, including Germany already fragmented before the update) and the total space taken has increased by about 1 GB. I now have only 1 GB left free on a 16 GB SD card :)

Whilst on the topic, rinigus, would it be possible to update one map at a time? I did not see such an option, I had to update them all in one large go taking a few hours.

rinigus 2018-05-16 17:43

Re: [Announce] Native offline maps: OSM Scout Server
 
Quote:

Originally Posted by pichlo (Post 1544356)
Whilst on the topic, rinigus, would it be possible to update one map at a time? I did not see such an option, I had to update them all in one large go taking a few hours.

Not currently and not in my plans. In addition to work that is needed to make it happen, there are issues with the possible database format changes that would interfere with it. I am sure its possible to do, its just there are different priorities for me (geocoder, for example).

I presume it depends on requirements - whether to keep large regions on device or not. Notice, however, that these days we have two servers distributing the maps. So, if one is down, the second takes over. Since switching to two servers, I don't remember any downtime where the both were unavailable. Which makes it reasonable to consider downloading maps just few days before the trip. But that doesn't help if you have to travel constantly, though

pichlo 2018-05-16 21:19

Re: [Announce] Native offline maps: OSM Scout Server
 
Thanks, rinigus. It was just a suggestion, definitely not a priority.

One possible workaround for the user is quite simple. Unsubscribe from all but one, update, subscribe again one at a time.

rinigus 2018-05-17 05:37

Re: [Announce] Native offline maps: OSM Scout Server
 
Quote:

Originally Posted by pichlo (Post 1544373)
One possible workaround for the user is quite simple. Unsubscribe from all but one, update, subscribe again one at a time.

That's not going to work. As soon as you accept update, new map definitions are loaded and the old maps are considered out-of-date. So, when you start plugging them back by re-subscribing, you would be doing so for new map version. And you would have to download them.

While you are presented the maps by regions, some backends are imported in one go. For example, Valhalla needs maps from the same import to calculate the routes. Hence, we cannot mix old and new maps. And thinking of it, response to original request - looks to be impossible to update only selected regions if you want to use Valhalla.

pichlo 2018-05-17 06:41

Re: [Announce] Native offline maps: OSM Scout Server
 
Quote:

Originally Posted by rinigus (Post 1544379)
That's not going to work. As soon as you accept update, new map definitions are loaded and the old maps are considered out-of-date. So, when you start plugging them back by re-subscribing, you would be doing so for new map version. And you would have to download them.

I know. That was the point. Downloading them one at a time, rather than all 15 in one go.

I should have included one step, purging the cache: Unsubscribe - purge - update - subscribe (and download).

pichlo 2018-05-18 09:39

Re: [Announce] Native offline maps: OSM Scout Server
 
Quote:

Originally Posted by peterleinchen (Post 1542926)
Do you have a more simple way than liberapay (speak paypal) for donations?

I second that. I was just trying to send you a donation but you do not make it exactly easy :(


All times are GMT. The time now is 21:52.

vBulletin® Version 3.8.8