Re: [Announce] Native offline maps: OSM Scout Server
 

@XOleg, did you have problems with downloads using 0.7.0?

I am planning to approach planet chopping via polygons and when (if) I manage to set it up, it would be possible to add our own polygons into hierarchy. See for example POLY files at geofabric. As long as the number of polygons is not too large, I would be happy to add more with the preferences of early adopters high in priority.

I have in my mind something like @MartinK has done for modRana: https://github.com/M4rtinK/modrana-d...er/tools/polys . Except, in addition to poly files, I plan to have few others to describe relationship between libosmscout, libpostal, and geocoder-nlp datasets.

As soon as initial import is transferred to a new system, I will be able to get additional chopping as well. So, you would just have to wait a bit...

Re: [Announce] Native offline maps: OSM Scout Server
 

@rinigus I don't tried :-)
the download(Ural+Volga) by Wi-Fi is finished(from my last message).

One question. maybe it's possible to combine efforts of @rinigus, @MartinK, @karry to create one good maps/navigation app? It's not simple(to combine), but...

Re: [Announce] Native offline maps: OSM Scout Server
 

THNKS @rinigus For its genuine continuous development of this.
- would like to know if after that version 0.7.0 it not enable to update the maps manual anymore?
- would be a good idea to help a maps navigation app among all of us = )

Re: [Announce] Native offline maps: OSM Scout Server
 

Good! I will proceed with the update - just have to ensure that the timeout for connection would work and then get a new release out.

In reality, these efforts are combined already. Together with @Karry we work on the offline part of the maps/navigation app. We use the same backend linbrary - libosmscout - and improve it together with several others. Recently, we worked on different aspects with me focusing on search and @Karry just finished a great work on rendering speedup. Corresponding changelog is at https://github.com/Framstag/libosmscout/commits/master .

I would still have to work on search and make it faster and, hopefully, occupy less storage space. That would be my focus as soon as we sort out the distribution of the maps, with @MartinK offering a lot of help. Before that, @Karry actually implemented and showed how such distribution can be done for his OSM Scout.

Notice that we also incorporate developers that are not working/using Sailfish. Main developers of libosmscout and libpostal are working on Linux, so there are even more people involved.

While @Karry works on GUI of his application, OSM Scout Server relies on Poor Maps (@otsaloma) and/or modRana (@MartinK). The whole idea behind working on the server was to combine the efforts of different developers and bring parts developed for offline maps to the map applications that we had already. I think all in all there is a fast progress on the front of maps/navigation [wanted to continue using a word soon, but it should be probably banned in SFOS universe].

Yes, my idea is that, after 0.7, having self imported maps would be relatively hard (read impractical). This is due to difficulties on keeping dependencies between libosmscout/libpostal/geocoder-nlp and whatever fancy backend we would throw into the mix. I would suggest to try without self-imported maps for few releases and see if it is OK. If users would not like it after that, I would have to reconsider and make a simplish way to use custom imported maps.

Navigation app - help @MartinK and @otsaloma. OSM Scout Server does provide routes and Poor Maps / modRana can already show them. At present, Poor Maps is the best for navigation with its full navigation GUI during a drive. For me, the main missing part is rerouting in Poor Maps. After that, voice guidance. As far as I understood, rerouting code is already somewhere in modRana, but its not exposed yet. So, we have to be patient and help the devs as much as we could.

Re: [Announce] Native offline maps: OSM Scout Server
 

Just a quick feedback from my side: Using the linked test version I could succesfully download the Germany Map while being on my home wifi, so the improved download code seems to help for my case :)

Re: 0.7.1
 

I have just released 0.7.1.

This time, the main changes are updated translations. I would like to thank all the translators for the great job on updating the messages!

In addition, the downloading has been made more robust and should be able to recover from smaller hiccups. During my testing I've got once "Connection closed" but that might have been since I was in the area without coverage.

Please let me know if you have more issues with downloads. I hope that those as we had earlier with some of you in Germany/UK are now resolved.

The new code still does not have keepalive incorporated into it. I would prefer to keep the server without keepalive if possible. So far all my downloads have worked fine without being on charger suggesting that in this case keepalive is not required. I guess if we will not notice any major issues with downloads, I'll keep it as it is.

Re: 0.7.1
 

I will try to help with French, which I just requested on transifex. If you have time to add it to the list, I'll try to work out most of it this weekend.

I'll also take a look on the Linux-side code (not Sailfish for now), to see if I can help there or not.

Re: 0.7.1
 

Thanks for letting me know re transifex request - for some reason they notify me by email a day or two later. All accepted and should be ready.

Re Linux: Any help would be great! Few questions regarding it:

* Do you use server in Linux? If so, then I have to expose map downloading via command line options. Otherwise, its impossible to use it right now.

* On Linux side of things, there is a lot to be done on the backend libraries - libosmscout and geocoder. Its probably easier to start work on libosmscout since its a bigger community with different OS support and more help on the lists. geocoder though is a small code base, but it maybe not as well documented.


Additional note for users: @ade has been hit by a bug leading to OOM-killer on J1. Current suspicion is that the writing and uncompressing operations are slower that network download leading to accumulation of data in the buffers. Bug was exposed during libpostal dataset download which does contain a huge file. Issue was opened at https://github.com/rinigus/osmscout-server/issues/89 and I hope to resolve it within few days.

Re: 0.7.1
 

Thanks. I have already done a bunch of the translations. 100 remaining that I will try to complete before Sunday.


I intend to use it on Linux, yes. I have also installed it on my Jolla through you packages on OpenRepos, to have more context while doing the translations, and as a reference.
So far I am trying to see what the prerequisites are to build the different parts (libosmscout, osmscout-server, ...), and have a lot of thins to read to understand better what every part is doing. But I will try to finish the translation before spending more time on that front.

Ok. I assume the reference website is http://libosmscout.sourceforge.net/ with the mailing lists at https://sourceforge.net/p/libosmscou...t-development/ ?
Are there other channels I need to read ?

Thanks for your work and help by the way.

Re: 0.7.1
 

Assuming that we'll figure it out this weekend with @ade what's going on with memory during download, translations by the end of weekend would fit perfectly with the next point release.

Using the server on Linux - great! I used it earlier in development with modRana, but haven't tried lately. To make it possible, I'll have to fix https://github.com/rinigus/osmscout-server/issues/90 , but that shouldn't be too complicated. Let me fix OOM-killer bug first and finish small scripts to move downloads to @MartinK server. Then I should be able to get issue #90 resolved in a night or two.

Dependencies on Linux are actually much simpler. Assuming that you do have RAM (more than 2-3GB), you can skip all these country-specific libpostal datasets. You would need to install libpostal as at https://github.com/openvenues/libpostal , libosmscout, and sqlite.

To use with the server downloadable datasets, please use https://github.com/rinigus/libosmscout Sailfish branch, latest release. For fixing / development of libosmscout, please use https://github.com/Framstag/libosmscout. Sorry for a mess, but that way I can keep versions consistent (upstream does not provide releases). libosmscout can be also used without the server using their own QML/Qt application which is available at github. Note that the work that you would do on backend features of libosmscout (map drawing, routing, and several others) would be pulled into the server and OSM Scout by @Karry as well. So, you would have to see what annoys you most and find the easiest way to work on it. Ask/propose project(s) if you like. Note that the mailing list is reasonably active and we all try to help each other as much as we could.

Geocoder-nlp is available as a submodule for osmscout-server. So, that's easy dependency.

In Linux, all configuration is done via INI configuration file, under .config/osmscout-server . At present, translations are not exposed to Linux to avoid double work by translators. We should probably just modify build scripts to copy harbour-osmscout-*ts files to osmscout-*.ts files.

In Linux, if you run server on a proper server without GUI, you may prefer using Cairo drawing backend of libosmscout. For that, just uncomment https://github.com/rinigus/osmscout-...onsole.pro#L20 and comment out a line above. Its possible to use Qt for drawing on a server hardware as well, but you would have struggle a bit with some environment vars, if I remember correctly.

I hope I remembered all what is relevant to this case. If you have any trouble, just ask.

Re: [Announce] Native offline maps: OSM Scout Server
 

Quick note: we have just resolved issue of OOM-killer with @ade. I will publish a new release with this fix, unless someone will see degradation in their download stability. Note that downloads could be slower after it, but they should not be degraded in any major way.

Test build with the fix enabled:
https://mega.nz/#F!IQ0nmYZJ!RQgn0Lo13NwmtixZ-2am7w

I'll wait a day and maybe there will be new translations to incorporate into it as well.

@ade: thank you for fast testing and reporting a bug!

Re: [Announce] Native offline maps: OSM Scout Server
 

wonderful! :-)

@rinigus, I'll try to translate to Russian if necessary. I have an account for transifex but I don't understand how change a team.
Thanks in advance for help.

Re: [Announce] Native offline maps: OSM Scout Server
 

Its not necessary - but helpful, I presume. Assuming that you are XOleg @ transifex, I just invited you.

For me, this transifex is a bit mystery. But I am trying to navigate there with minimal reading and clicking the links rather randomly, I must confess. So far, it seems to be working.

Re: [Announce] Native offline maps: OSM Scout Server
 

Yes. it's working :-)

Re: [Announce] Native offline maps: OSM Scout Server
 

"Failed to download transliteration.dat.
Timeout"

Re: [Announce] Native offline maps: OSM Scout Server
 

Try again. If it persists then that's a problem. If it resumes and is able to download it on further tries, I think its OK. So far, the fixed problems were for cases where it was just impossible to download a huge file since connection was always dropped in the middle (Germany & UK) or server was crashing with OOM. Let's see if you could download on retry.

Edit: I can also increase timeout to 5 minutes and see if it helps. XOleg, should I try it? Do you get plenty of timeouts? At present, its 1 minute

Re: [Announce] Native offline maps: OSM Scout Server
 

I don't know :-) Phone works himself. I restart 1h ago, phone download maps(second district) for this moment.

P.S. I translate, but this not very quickly. Maps it's not my theme.

Re: [Announce] Native offline maps: OSM Scout Server
 

I'll look into the code again, maybe I missed something. As for translation - take your time. We can do as many point releases as needed. I don't think the next series 0.8.x would change strings too much, I plan to work later on backend features.

Re: [Announce] Native offline maps: OSM Scout Server
 

@rinigus Download is terminated without errors. 1h20' max after restart

Re: [Announce] Native offline maps: OSM Scout Server
 

Does it mean it was OK or program terminated?

Re: [Announce] Native offline maps: OSM Scout Server
 

Download is OK. Server continues to work.

Re: 0.7.1
 

Done for the French translation.
It is far from perfect, and I'm not happy with every translation, but it can be released as a first try.

If another French speaking fellow wants to take a look, this webpage can be really useful for the terms related to roads names : http://wiki.openstreetmap.org/wiki/F..._roads_tagging
There are lots of details in roads characterizations terms that are not that easy to understand at first sight without this reference.

There is no hurry in this.

Re: 0.7.1
 

Thank you for the translation, its a big work!

Re: 0.7.2
 

I have just published a new point release: 0.7.2

In addition to fixing a nasty bug imposed by fast download and slow writing to the media reported by @ade, this release introduces 4 more languages added by translators. While French and Russian are completely translated, there is also a great progress with Polish and Norwegian. So, I would like to thank the translators that step up to support these new languages in our list:

- @Sagittarii (@Zeta at TMO): French
- Oleg Artobolevsky @XOleg: Russian
- A @atlochowski: Polish
- Peer-Atle Motland @Pam: Norwegian

If you see something strange going on with the downloads (or some other functionality), let me know. I hope most of the issues with the downloads are fixed now, but there could be still some corner case lurking around.

We are working with @MartinK behind the scenes to get the distribution moved over to his server. Also I have to rearrange the scripts that are generating the maps, so we could easily add new regions and make updates automatic.

Re: [Announce] Native offline maps: OSM Scout Server
 

@rinigus I agree with @Zeta: "There are lots of details in roads characterizations terms that are not that easy to understand" and
"It is far from perfect, and I'm not happy with every translation, but it can be released as a first try."

@Zeta, thank for link :-)

Re: [Announce] Native offline maps: OSM Scout Server
 

Yes, its not very simple, sure! The link is added to Translation README at https://github.com/rinigus/osmscout-...ions/README.md and available via "Help pages" in Transifex project page

Re: data.modrana.org
 

To all users:

Together with @MartinK, we have just finished syncing the whole maps dataset into its new home - data.modrana.org. To use this new location, please open Map Manager and press

"Update lists"

button. This would load an updated list pointing to the new location.

In few days I am planning to stop the current distribution via CDN that I used for launching the service assuming that we don't run into any technical difficulties.

I think a former acknowledgment is appropriate, so you would all know who to thank for the service: Maps are hosted by Natural Language Processing Centre (https://nlp.fi.muni.cz/en/ , Faculty of Informatics, Masaryk University, Brno, Czech Republic) through modRana (http://modrana.org)

I'll work on automation of the updates and we can then later look into adding few other regions as well

Re: data.modrana.org
 

Seem to work perfect. I just downloaded two more countries.

Re: [Announce] Native offline maps: OSM Scout Server
 

@rinigus and @MartinK, many thanks for great job.

P.S. food for thought :-) maybe will be possible to use http://gis-lab.info/projects/osm_dump/ as source of maps for Russia? regions, not districts.

Re: [Announce] Native offline maps: OSM Scout Server
 

I am working on making it possible. Namely, chopping the regions by poly-files from planet.pbf. It would take some evenings to get the scripts done. After that, it would be possible for you to merge polygons with the names of the regions into the osmscout-server repository in github.

Re: [Announce] Native offline maps: OSM Scout Server
 

@XOleg, I have finished a first version of the scripts last night and it seems that all is going quite well.

You can insert the regions under

https://github.com/rinigus/osmscout-...port/hierarchy

as explained in https://github.com/rinigus/osmscout-...scripts/import README.

In short, you would have to fork my repo, make a patch, and submit it back. Even if you haven't worked with GitHub, its not too hard.

As you could see, its possible to add custom regions as well. I am preparing for the first update of the maps, but there will be more details regarding it a bit later

Re: [Announce] Native offline maps: OSM Scout Server
 

@rinigus thank for your very fast job, but... This is Chinese letter for me :-( I would prefer either download districts with new version of server or import regions for old version if there is no more simple solution.

Re: [Announce] Native offline maps: OSM Scout Server
 

:) sorry ,would have been complicated for me couple of month as well . So, let's get to the bottom of this .

Aim is to get regions prepared into the system so you and all others could download them into the phone using OSM Scout Server. In addition, the regions would be updated with new OSM data once in a while and follow OSM Scout Server versions.

To get maps to the phone, you need to know a little bit about how they are prepared. In short, I download current Planet PBF and process it to get PBF for each region that you see available in you're Map Manager. After getting region PBFs, I convert them into libosmscout and geocoder-nlp formats. All this is later uploaded into @MartinK server from where you download them.

Now all regions are described using simple tree of hierarchies that I linked earlier. Just try to go and browse into Russia. You would see districts, if I call them correctly. Each district has two files, name and poly. Name is simple: just the name of the region. Poly is a file that describes a line on the map surrounding that region. This file is used when regional PBFs are made and later imported.

So, to add a new region, you have to make a directory under corresponding district. The hierarchy has to be followed, otherwise we will very fart get into too slow imports and huge mess.

In case of Russia, the link that you provided for smaller regions also had Poly files. So, it's a matter of inserting them into corresponding folders and following our convention. I'll be happy if you or someone else would arrange it accordingly.

As far as I remember, you mentioned that you needed region which is between normal hierarchies. I think you could add that as well, just position it accordingly. For that, you will need to make own Poly file. See Barcelona for example of such simple region.

One question for Russian users : do you need full Russia map or are the regions fine? I am planning to remove full Russia import and just leave parts, as done for US. So, unless we have lots of users using full Russia map, there is no reason to keep it in this way .

Re: [Announce] Native offline maps: OSM Scout Server
 

@rinigus this is explanation https://en.wikipedia.org/wiki/Federa...icts_of_Russia

I'll try to read yet again, understand and make. Yes, two regions are interested for me, one from one district and second from other. IMHO: maybe it's simple to download full Russia map, but space and time for dowload are required... Full map(not for Russia only, Monaco is exception) is required maybe for football funs or for vacation on wheels via country...

Re: [Announce] Native offline maps: OSM Scout Server
 

Hi! There will be disruption of the service due to updates of the maps today. Its mainly caused by a small flaw in the design that is corrected already and will be released as a part of the next OSM Scout Server release.

After the maps are updated, you would have to "Update list" before you could download again. In general, if you see download errors related to missing file on a server, try to update the list in the manager and try to download again.

Re: [Announce] Native offline maps: OSM Scout Server
 

I have finished an update of the maps. Since the hierarchy was created a bit differently, few notes:

* Update the lists of countries before downloading anything. URL has changed to ensure that you do get a version that your app is expecting to get

* Some regions have different internal IDs (Russia, US/Regions, and possibly few others). To fix it, unsubscribe and then resubscribe again. For re-subscription, its better if you leave the country page and select it again for subscription in the dialogue covering all countries. Its a problem with the current update that should not be there in future, so let's live with this inconvenience.

* For all other regions, its sufficient to open a country page, press Unsubscribe and then Subscribe to initiate an update. Note that the corresponding country will be marked as an unavailable before you download a new version.

* All updates are done when you authorize the download.

* There are much more regions now with subdivisions in UK, Germany and few other countries. This is done by following Geofabrik.

* Full Russia, Canada (US, as before) are not provided. If you use them, let me know and we can try to put them back. I assumed that you cannot drive during one day through them.

In the next release, I plan to make updates easier by adding a corresponding dialogue. Then this manual Unsubscribe/Subscribe cycle would disappear.

Re: [Announce] Native offline maps: OSM Scout Server
 

Hi, I've upgraded from an older version without map management. Now OSM-Server doesn't work anymore. I have reinstalled it. I've made a new empty folder and chosen that. But after that I get the message:
Failed to open the database for tracking downloaded files.
Map Manager functionality would be disturbed.

I can't choose countries or anything to download and even my old Map database doesn't work.

Any advice?

Re: [Announce] Native offline maps: OSM Scout Server
 

That's new. It corresponds to the case where it cannot create or open SQLite database in the folder that you allocated. Its either some problem with the file permissions, my bug, Qt bug or combination of these.

Let's try to track it down.

* If you go to the allocated folder and try to create subfolder, does it work? Do you have permissions as user "nemo" to do it?

* Close OSM Scout Server, go to the allocated folder. Do you have "files.sqlite" in that folder? If you have it, delete this file and start application again.

* If it all fails, please try to allocate folder just in /home/nemo . Like /home/nemo/Maps.OSM

I assume you used some regular filename, without äöõ or some other fancy character in the path.

If we are still in trouble - let me know version of your SFOS, where did you try to allocate it. Please also open then new issue at github and paste the configuration file for OSM Scout Server from

/home/nemo/.config/harbour-osmscout-server/harbour-osmscout-server.conf

Please let me know how it went.

Re: [Announce] Native offline maps: OSM Scout Server
 

@rinigus It's OK for me. I saw "Timeout" after 10' of downloading accidentally, then I continue and download is finished after 30' from restart. 1GB for 2 districts.

I made:
- unsubscribe
- cleanup
- update list
- subscribe
- start download

Re: [Announce] Native offline maps: OSM Scout Server
 

I have added an error description as provided by Qt's SQL engine in the new build. You could get a build from

<LINK TAKEN DOWN SINCE THE ISSUE IS RESOLVED>

However, the error message in the case of read-only folder was rather bizarre "out of memory Error opening database". So, let's see, maybe you have better luck and error message in your case would be clearer.