Re: [Announce] Native offline maps: OSM Scout Server
 

Do you have a more simple way than liberapay (speak paypal) for donations?

Re: [Announce] Native offline maps: OSM Scout Server
 

thanks for asking! no, I don't. I like liberapay since its transparent - so all can learn from my experience. As for simplicity, I don't know much about it. I think it should be simple to go around the system by making payment a single time and just stop it. Some have done it before and its appreciated as much as anyone taking time and doing it.

Re: [Announce] Native offline maps: OSM Scout Server
 

Yes, sure.
But you/one will need another account on another system (and probably only via credit card)...

Re: [Announce] Native offline maps: OSM Scout Server
 

No, not really. You could just transfer from there to your bank account directly and there is no need to have anything in between.

Re: [Announce] Native offline maps: OSM Scout Server
 

@rinigus, AFAIU the OSM Scout Server Module: Fonts is currently only useful when providing Mapnik maps by OSM Scout Server.
As the Font Module is quite large (due to containing the Noto font), please clarify in its description on Openrepos (if so?):

• The Font Module benefits Mapnik users only.
• It is not used for displaying vector maps or OSM Scout bitmap maps.
• If it is a hard requirement for displaying Mapnik maps (i.e. no labels at all without it) or just enhancing the rendering of labels.

As I never used Mapnik offline maps in OSM Scout Server due to their huge size, I always wondered about aforementioned points.

P.S.: You may also consider altering the messages in OSM Scout Server, stating that the Font Module and / or Valhalla Module are "missing", as both are not needed for OSM Scout Server to work properly: I use libosmscout's maps and routing for a long time and they are working fine.

Re: [Announce] Native offline maps: OSM Scout Server
 

@olf, good point. There was a statement in Fonts module description, but I have added now a separate paragraph stressing it.

Fonts module is needed only if you use Mapnik backend.

Mapbox GL has the same fonts, but they are packaged differently, in Mapbox GL specific glyphs. These you download via OSM Scout Server and, in reality, they are larger than TTF fonts used by Mapnik. However, you probably store Mapbox GL data on SDCard and don't care much about it.

As for altering the messages - I'll take a look. I will link them to the selected profile. Opening an issue for it

Re: [Announce] Native offline maps: OSM Scout Server
 

Thinking about this - if the fonts module only contains the fonts and no executable code would it maybe make sense to turn it into a data pack that gets auto registered when users download the Mapnik rendering data ? IIRC there are already some global geographic datasets that serve a similar purpose.

This would automate one more thing, reduce number of packages users might need to install manually and possibly eliminate some out of space issues on devices with small rootfs volumes.

Re: [Announce] Native offline maps: OSM Scout Server
 

It depends on how we look into it.

These are Noto fonts which maybe attractive option for some other app to display full international characters set. If such interest will happen, we can repackage them accordingly.

Next, any investment of time into Mapnik maybe not so wise in long term. modRana is probably the only client which is using it and I would expect that, at some point, it will also switch to Mapbox GL. Which will leave us with the backend that nobody is using anymore. Use of the server on Linux is non-existent (probably only myself) and I don't see much reason for users to stick with Mapnik if suddenly all devices support Mapbox GL.

Moving over the data package is not trivial - not very difficult, but still would take time. It seems to me that this time can be spent better on other aspects of the map applications.

Re: [Announce] Native offline maps: OSM Scout Server
 

Actually, I plan to add Mapbox GL support in parallel with the current tile based widget. The main aim is to both enable early testing with fallback to the current widget & support for Jolla 1 class devices where Mapbox GL is not supposed to run very well.

But if some of the recent user feedback turns out to be correct (eq. Mapbox GL actually runs fine on Jolla 1 & similar) then this option might not be needed & modRana might be able to switch to Mapbox GL on Sailfish OS exclusively. Then indeed the Mapnik rendering might not longer be really needed.

Actually, I'm a Fedora user & packager (I maintain for example the Fedora pyotherside & pydbus packages) and have been thinking it might not be a bad idea to package the OSM Scout Server for Fedora to make it available to the broader Linux user community.

The features and APIs OSM Scout Server provides are more or less unrivaled by any other project and I can see many uses even on plain desktops/notebooks, such as offline POI search and map display. While desktop users are mostly online these days there might still be very valid usecases for using offline geodata, such as bad connectivity while traveling or wanting to guard ones privacy from online mapping vendors.

Also a bit subjective reason - I develop modRana primarily on desktop so it would be nice to have the same set of APIs available even during general development. Also modRana uses more and more of the awesome OSM Scout Server functionality, so having modRana on desktop as full featured as on Sailfish OS would be nice. :)

In any cases this is at this point still more or less a nice to have compared to Sailfish OS support, but I'm ready to help with package review & be a package maintainer if needed. :) There is also the possibility to use Copr, which is a Fedora service similar to PPAs/OBS to build OSM Scout Server packages rather than target inclusion as a "official" Fedora package from the start. :)

Re: [Announce] Native offline maps: OSM Scout Server
 

1.5.0 is out which should give better feedback to the users regarding missing plugins and data. There is no other new functionality in the server this time.

@MartinK: I'll probably have to port server's GUI to QtQuick then. This should be relatively simple and would allow using the server simply in Linux. Right now, console interface is rather un-intuitive and hard to use. Corresponding issue is opened, let's see when I can make it.

As soon as proper GUI is ready for Linux, would be great to package it for Fedora. Although, it will require packaging of Valhalla, probably.

Re: [Announce] Native offline maps: OSM Scout Server
 

I guess you can use Universal Components to keep Sailfish OS specific stuff (pull down menus, buttons, fast scroll listview, etc.) while being able to run the same UI codebase with QtQuick Controls 2 on desktop.

Thanks! You can definitely count me in for testing. :)

I've looked at the Sailfish OS Valhalla packaging and it looks pretty clean and not that Sailfish OS specific (even though I guess we could --enable-static=yes and use "normal" dynamic linking on Fedora). I'll try to build a package in Copr from the spec file and will report back my findings. :)

Re: [Announce] Native offline maps: OSM Scout Server
 

@rinigus does Valhalla support any routing options (fastest, shortest route) ?

Re: [Announce] Native offline maps: OSM Scout Server
 

@bomo, it does: https://github.com/valhalla/valhalla...costing-models

It has to be called by a map client. There are lots of options which maybe of interest, see the README. Since OSM Scout Server just forwards the URL as a part of Valhalla call, there are no adjustments needed on the server part - it should be ready.

Re: [Announce] Native offline maps: OSM Scout Server
 

Hi rinigus,

I'm trying to use OSM Scout Server with WhoGo Maps, tiles work fine, but I hava a little problem with route finding. Do both start and end points have to be on the same map?
Problem is map of Poland has one basic for whole country (selected in map selection) and detailed maps for each area (voivideship). As long as both start and endpoint are in the same area, routing works, if it's in different area (map) WhoGo Maps says no resoults found (although I can see in server mesages Valhalla has found it)

Re: [Announce] Native offline maps: OSM Scout Server
 

@MartinK: just read you have started on Fedora packaging, sounds great. There are probably few dependencies to go through. Note that I am using older libpostal, haven't worked on updating it yet.

Since its easier to test functionality on Desktop, I started porting GUI over to QtQuick 2. I hope to get it ready soon.

Re: [Announce] Native offline maps: OSM Scout Server
 

@rinigus do you know if valhalla could/will manage "offline traffic"?
thanks
/carlosgonz

Re: [Announce] Native offline maps: OSM Scout Server
 

The main issue is availability of the data. To my understanding, Mapzen started a project to collect such data, but got shut down. I think Mapbox (new home for Valhalla team) has an option to collect data for traffic on ios and android (not qt, if I understand correctly), so they may work on it too. But in Mapbox case, I didn't see any project that was supposed to make this data open. Although I may have missed it.

Re: [Announce] Native offline maps: OSM Scout Server
 

Well, the unique data needed is "speed-limit" of each street, with that could be knowing the traffic in whatever street; basically is comparing the speed current with speed of gps. As an examp: If I driving in a street where the speed-limit is 45mph but I driving 15mph for whatever reason then valhalla could suspecting a traffic based on data of "speed-limit/speed-gps/time-running-in-15mph" then new route could be generated. But this way perhaps need constant monitoring. : )
But even this feature could be implemented too without any of data, by just applying the "Logistic".
Thanks

Re: [Announce] Native offline maps: OSM Scout Server
 

Devil is in the details. You are welcome to search for "traffic open data" and read about it. Lots of challenges to make it work. For example, http://opentransportmap.info/ provides "average daily Traffic Volumes for the whole EU", but daytime is only for "Pilot Cities". Not much use of such data for routing in practice.

Re: [Announce] Native offline maps: OSM Scout Server
 

Are you rejecting?
First osmscout-server is opensource which you must accept shared ideas from any sources. There are diferent way to implement "offline traffic" not just following ideas of other devs as a example,I know this task is no easy,at the end I just sharing with you the idea, because I believe in your huge work that you already done, at the same time I am the first on started donating money for you work, of course I will keep this.

Re: [Announce] Native offline maps: OSM Scout Server
 

Your suggestion is very obscure and hardly generalizable. Traffic is inherently online and trying to hack vague data to an offline graph model would be a lot of trouble for little gain.

Re: [Announce] Native offline maps: OSM Scout Server
 

Hello otsaloma, thanks so much to come to clarify this.
I know you are professional in this area; Same to @rinigis. The problem is that you still do not understand my trick. I will explain more better I guess...

Re: [Announce] Native offline maps: OSM Scout Server
 

If we even forget about online/offline aspect, I am not aware of any data that is available on traffic conditions and covering large fractions of the world. By just monitoring your current speed and comparing it to the speed limit, I cannot make a choice of the future road since I don't have any data to base it on.

Actually, the best offline selector in this case is you - as a driver. You see traffic ahead and can decide to either stay or turn. If you turn, Valhalla will reroute accordingly. And again, you could alter your path as well.

Re: [Announce] Native offline maps: OSM Scout Server
 

Yeap rerouting is the solution of this too, you right. I just was thinking on funny feature, where whogo could be detect traffic by using some offline datas then whogo-maps could give an option to the driver if accept go to other way. Really Really Funny.
On my side I like offline datas. So once again thanks @Rinigus/@Otsaloma for supporting offline datas.

Re: [Announce] Native offline maps: OSM Scout Server
 

I have just merged Qt Quick 2 port of the server to the master branch. This allows to use the server on regular Linux with QML GUI. GUI is shaped in similar way to the GUI on SFOS, so, in theory, it should work on mobile Linux platforms as soon as they support QML QtQuick2. An obligatory screenshot is attached.

At present, there is a lot of code duplication in QML interfaces (Silica and QtQuick). Let's see, if we will get users and developer(s) on QtQuick side, I will organize it better by splitting more platform-dependent and independent code parts.

@MartinK, if you are packaging it for Fedora, maybe you could use qtquick version?

Re: [Announce] Native offline maps: OSM Scout Server
 

Sure, that's definitely the plan & thanks a lot for working on the QtQuick GUI! :)

At the moment I'm going "up" from the OSM Scout Server dependencies, and I'm currently trying to get prime_server, which is (as far as I can tell) a Valhalla dependency, to build on Fedora. I've hit some weird errors during compilation, but I'm suspecting it might have been some temporary Rawhide issue.

I'll give it another go tomorrow & will try to see if I'll get the same issues when I try the F27 and F28 mock targets.

BTW, once I manage to build the packages locally, I'll start putting them to this COPR repository:

https://fedoraproject.org/wiki/Releases/Rawhide

(COPR is basically the Fedora counterpart of OBS, with a significantly more sane and maintainable codebase than OBS has. :) )

Re: [Announce] Native offline maps: OSM Scout Server
 

@MartinK: if you have issues, check which repository was used at mine OBS project. Some dependencies were built, at least at some point, against the patched versions. It may have been to resolve SFOS case, but maybe something else.

prime_server is ZMQ-based server used in Valhalla and is needed, indeed.

As for OSM Scout Server itself, you may want to look at PRO file and suggest a better way to specify valhalla's and fonts path. Since you have more experience with packaging, I would wait for your advice on it.

Re: [Announce] Native offline maps: OSM Scout Server
 

New maps are out, Planet downloaded at the end of April. The main change in packaging of Valhalla. Now, I grouped the tiles somewhat differently and the planet is split into 1600 packages instead of 16000, as it was before.

The format is the same as before, so there are no incompatibilities with the current/bit older version of the server. All should work, as before.

For the end users, it would mean somewhat larger section of the Planet grabbed when downloading a country. While the neighboring territories will share the data, we end up with a bit more than we wanted to download.

For those who are involved in distribution of the maps, this suddenly makes it much easier to handle uploads/downloads. So, this change was made to make it a bit more sane syncing the servers.

Re: [Announce] Native offline maps: OSM Scout Server
 

Hi rinigus, How do I download larger areas? I had some older maps on my device and decided to download the new maps. But now I'm quite unsure. Before I had a package "Germany" that was nearly 7 GB. But now Germany is spitted up in different federal states and some of them are even splitted up further. All german maps together would be much more than 20 GB. And I would have to choose a lot of areas seperately. Is that by intend?

Re: [Announce] Native offline maps: OSM Scout Server
 

Regarding the size: remember that Rinigus has modified it so that now it can also serve vector data to application doing openGL (like WhoGo Maps, Laufhelden, etc.) and not only bitmap tiles.
That requires even more data on your sd card (see the new "Mapbox GL country-specific" data for each map).

Re: [Announce] Native offline maps: OSM Scout Server
 

The previously downloaded maps where already vector maps. There must be a different problem:
If you look inside the federal state Baden Württemberg, there you can download the whole federal state, size nearly 2 GB. But you also can download the further splitted areas which are all together 3,5 GB so it has something to do with the splitting. Maybe because the areas are overlapping.

Re: [Announce] Native offline maps: OSM Scout Server
 

I have dropped full Germany and France imports. But *DON'T PANIC*, its still possible to download them and it shouldn't take much more space than before.

Germany and France resulted in huge Mapnik SQLite databases (4GB+) that, due to limitations of FAT FS on SDCards, resulted in periodic questions "why my Germany download doesn't work?". An additional bonus was that it reduced a time of imports by few hours, reduced HDD requirements of the servers as well.

For this convenience on my side, users that want full Germany and France have to get them by regions. So, in your case, you would have to select all subregions (except Berlin which is in some other region as well, if I understood correctly). For Bayern, for example, you could just choose it and there is no need to get all its subregions since they should be all included in it.

Now, the sizes displayed in the menus correspond to the case where it is assumed that you will download only that region. This is to prepare you for the worst-case scenario. Fortunately, the regions in Germany overlap and will share the same data - will be downloaded and stored only once. Let's go through main backends, so you'll get the idea:

* Address parser is given for full country. So, in case of Germany, its 200MB is accounted for in each of the regions

* Geocoder-NLP and OSM Scout are the only backends that don't overlap, but are cut by the border. [technically, there is a minimal overlap, but its not important]

* Mapbox GL and Valhalla are using tiles. If the tile is touching a region, its accounted for in it. Fortunately, tiles are downloaded and unpacked only once.

In your case, you would have to unsubscribe from Germany and subscribe to all its regions. For curiosity, would be good interesting to know current (old schema) and the new disk usage. Note that I pulled Valhalla tiles in somewhat bigger tiles, but I would expect it didn't change much Germany since Valhalla tiles over there were big already before.

Re: [Announce] Native offline maps: OSM Scout Server
 

That's exactly it. Overlapping. I had a similar problem. Downloading maps for Germany, I had to do that one state at a time. But then the total space taken on the memory card was much smaller than the grand total of all maps added together. When I say much I really mean MUCH. Like less than a half. I remember adding the last one, which alone was supposed to be about 1.5GB, added only about 200MB on the card.

Re: [Announce] Native offline maps: OSM Scout Server
 

@rinigus thanx for your detailed explanations. I've downloaded all the regions, and am absolutely satisfied. The total space taken doesn't differ to much (less than 500 MB) from the Full Germany download. So everything is fine and I'm totally flashed again by your great work and the ongoing support!!!

Re: [Announce] Native offline maps: OSM Scout Server
 

I expect that to be simply because the maps have grown, not because of the fragmenting. I just updated all my maps (15 areas, including Germany already fragmented before the update) and the total space taken has increased by about 1 GB. I now have only 1 GB left free on a 16 GB SD card :)

Whilst on the topic, rinigus, would it be possible to update one map at a time? I did not see such an option, I had to update them all in one large go taking a few hours.

Re: [Announce] Native offline maps: OSM Scout Server
 

Not currently and not in my plans. In addition to work that is needed to make it happen, there are issues with the possible database format changes that would interfere with it. I am sure its possible to do, its just there are different priorities for me (geocoder, for example).

I presume it depends on requirements - whether to keep large regions on device or not. Notice, however, that these days we have two servers distributing the maps. So, if one is down, the second takes over. Since switching to two servers, I don't remember any downtime where the both were unavailable. Which makes it reasonable to consider downloading maps just few days before the trip. But that doesn't help if you have to travel constantly, though

Re: [Announce] Native offline maps: OSM Scout Server
 

Thanks, rinigus. It was just a suggestion, definitely not a priority.

One possible workaround for the user is quite simple. Unsubscribe from all but one, update, subscribe again one at a time.

Re: [Announce] Native offline maps: OSM Scout Server
 

That's not going to work. As soon as you accept update, new map definitions are loaded and the old maps are considered out-of-date. So, when you start plugging them back by re-subscribing, you would be doing so for new map version. And you would have to download them.

While you are presented the maps by regions, some backends are imported in one go. For example, Valhalla needs maps from the same import to calculate the routes. Hence, we cannot mix old and new maps. And thinking of it, response to original request - looks to be impossible to update only selected regions if you want to use Valhalla.

Re: [Announce] Native offline maps: OSM Scout Server
 

I know. That was the point. Downloading them one at a time, rather than all 15 in one go.

I should have included one step, purging the cache: Unsubscribe - purge - update - subscribe (and download).

Re: [Announce] Native offline maps: OSM Scout Server
 

I second that. I was just trying to send you a donation but you do not make it exactly easy :(