Re: [Announce] Native offline maps: OSM Scout Server
 

I rushed with the response, I think its much easier.

To "Select" some folder, you would have to go in FileManager widget into a parent folder, make a long press on the folder that you want to choose and then the "Select" will appear as a context menu.

For example, let's say we want to choose /home/nemo/Maps:

* You navigate to /home/nemo

* Long press on "Maps"

* Context menu "Select" appears, choose that

* Accept in Settings

* Maps storage is selected, Map Manager would be available on the main window.

Let me know if it helped.

Re: [Announce] Native offline maps: OSM Scout Server
 

Just tried the new version, selecting the storage folder and subscribing to a map works fine, but the download stops with an error after about 200-300 MB (I'm trying to download the germany map with geocoder data). Error is always "Connection Closed". My internet connection is stable without any interruption during the download.

Any idea what could be the problem?

Re: [Announce] Native offline maps: OSM Scout Server
 

Ahhh... got it! This is what I saw, and found very confusing, or maybe I'm just stupid. :confused:
Thanks!
.

Re: 0.7.0 release
 

Sure! I've talked a bit more with the NLP lab people and they are indeed fine with both hosting and generating data for OSM Scout Server as long as the data is hosted as static files that are batch-updated once in a while (every few days, etc.). As that condition is satisfied it should be possible to host at least ~100 GB, without transferred bandwidth limitations.

So now I need to know how can I "join" the OSM Scout Server data CDN with these resources. :) Are there some mirroring scripts I could add to cron or something similar ?

I can also help generate the data sets if needed, on one of the NLP compute machines, a fairly beefy system (220 GB RAM & 48 CPU core) running Fedora 24 with some 200-300 GB of local storage that could be used for the data generation run.

Re: [Announce] Native offline maps: OSM Scout Server
 

So far, the biggest map I have tried is Spain and that downloaded without any issues. However, in my place, it wasn't happy about mobile net. That's not surprising since we have a place/operator combination that resets connections quite frequently. Worked fine on WiFi.

One suspicion would be that maybe phone enters a deep sleep while downloading leading to "Connection closed". Namely, I have not used any keepalive and the both of my devices (N4 and OPX) worked just fine to my surprise. So, I didn't even look into keepalive locks. Maybe its not working for you.

Try to put your device on charging and download then.

On the technical part: many of the databases are split into several parts. As long as you see INFO messages on the main screen with "File downloaded" you are progressing even if after restart of the download the apparent size remains the same. So, you should be able to download by these batches... However if you are stuck on some bigger file (larger than 200-300 MB) then it will be trying to get the same file again.

Let me know if putting on charger helped. If that''s a problem, I'll add keepalive lock and we should be OK.

Re: [Announce] Native offline maps: OSM Scout Server
 

What you were selecting is actually a parent folder (".."). That's disabled to prevent users selecting /home/nemo instead of /home/nemo/Maps and later "cleaning up" their whole home dir :) [you have to authorize deleting, unless you do that will not happen]

Re: 0.7.0 release
 

Brilliant! I'll think how to do that and contact a bit later tonight

Re: [Announce] Native offline maps: OSM Scout Server
 

I tried with having the device charging(and device was on wifi for all tries so far), but the download stopped with the same error again (This time at about 400MB). Unfortunately the areas.dat for germany is bigger than that, so it doesn't get further than trying to download this file with every new start. Btw downloading the map for germany with osmscout worked fine last time I tried (same wifi, device not charging).

Re: [Announce] Native offline maps: OSM Scout Server
 

Not that it helps you - it worked for me when I downloaded Germany. I presume you mean the whole country, 4GB+ unpacked.

I presume you do have enough space to download full Germany.

Since you are on the charger already, I don't have to look into keepalive yet - it seems that this is not of a source of error.

Frankly, I don't know what to suggest. Its most probably something on CDN provider and I don't know how to debug it. I use S3-compatible service and, as far as I have seen in the documents, they do talk about large files and chopping them to pieces to ensure better redundancy. But how to use it, I have not really looked.

Maybe your connection is too fast/too slow and then some error on CDN provider backend is triggered?

In the case of areas.dat, you are downloading about 700MB bzip2'ed file. So, you are getting closer. Would you mind to try some part of Germany and later try to get full set again a bit later?

Let me know how it goes. We can always complain and ask for advice from the provider.

PS: OSM Scout uses @Karry's own server and has also different download implementation.

Re: 0.7.0 release
 

Now a proper reply: Thank you very much for an offer! It would be of great help and I hope that this service would not be abused. We have now some amount of time to figure it out while my current CDN is up.

As it is, I can process the planet reasonably fast on the system available to me. Its not as beefy, but made for serious calculations as well :) . Maybe later we could move data processing as well, but there is no need now for data processing. Unless they are interested on working on address parsing and NLP approach behind libpostal. Then we can discuss it separately.

At present, I use ~40GB for all datasets. I presume that during dataset version changes (version incompatibilities) we would need double of that space for a week or two. If the other backends would be added, the data requirement may increase, but let's see about that.

Datasets are served statically. In essence, we have a tree with different backends (osmscout, postal, geocoder-nlp) and a JSON file that describes them. By changing a line at https://github.com/rinigus/osmscout-...ded.json#L7112 , I could move all download requests to a new location (when you hit "Update list" in Map Manager you download that file by OSM Scout Server from github).

I presume we would keep conditions human and update once in a month?

So, it would boil down to the way we move the data between the servers. Or if its too complicated, we would just need cron jobs running. But that we could just discuss via PM/email. I'll get in touch with you tomorrow, maybe rather late though.