Notices

I would like to announce an offline maps solution: OSM Scout Server.

The server can be used as a drop-in replacement for online map services providing map tiles, search, and routing. As a result, together with the map client, a full offline solution is available for map search and navigation (car, bicycle, and walking).

The server supports:

• search via geocoder-nlp (https://github.com/rinigus/geocoder-nlp) that is based on libpostal (https://github.com/openvenues/libpostal);
• map rendering via mapnik (https://github.com/mapnik/mapnik);
• providing vector tiles for Mapbox GL-based applications;
• routing instructions via Valhalla (https://github.com/valhalla/valhalla);
• map rendering, search, and routing via libosmscout (http://libosmscout.sourceforge.net/).

License: GPL
Source code: https://github.com/rinigus/osmscout-server
User's guide: https://rinigus.github.io/osmscout-server
Packaging: OpenRepos https://openrepos.net/content/rinigus/osm-scout-server

Starting from 1.4.3, distribution via Jolla Store stopped until systemd linking is allowed (https://github.com/sailfishos/sdk-ha...tor/issues/102)

Translations: https://www.transifex.com/rinigus/osm-scout-server

At present, the server can be used to provide:

• map tiles for other applications;
• search for locations and free text search;
• search for POIs next to a reference area;
• calculate the route between two or more locations.

Starting from version 0.7, the server uses maps distributed via online distribution network. It is expected that the users would download, update, and, when needed, remove maps via server's GUI. This distribution model allows users to specify which components are required (libosmscout, geocoder-nlp, for example) and download only the required components.

To use the server, you have to start it and configure the client to access it. Recent versions of Poor Maps and modRana cover the full functionality of the server without any additional steps needed to be done by the user. In addition, an example configurations for Poor Maps and modRana are provided under "thirdparty" folder (see Github source tree).

The server can be used as Sailfish application or running in console on regular Linux. By default, it is configured to accept connections from localhost only, but you can change the configuration and serve it on LAN or even provide a globally accessible service, if you wish.

On Sailfish device, route calculations could take some time. As a reference, route calculation for shorter distances takes few seconds on a device (I use it on Nexus 4). For example, Stockholm -> Malmö (600 km) is about 15 seconds. Route calculation from Stockholm to Aarhus (950 km, via bridges) would take about 45 seconds. These are times that are visible for end user when using libosmscout backend (2016) - from starting route calculation request in Poor Maps till Poor Maps displayed the result.

For detailed up-to-date description and map import instructions, please see https://github.com/rinigus/osmscout-...ster/README.md

The server relies on the same library as an offline map viewing application OSM Scout by Lukáš Karas (@Karry). As the result, you could use the same maps when using the server or OSM Scout. However, at the current stage of development, the server is using the latest version of the upstream library without the compatibility with the older map formats, as done in OSM Scout. This may change in future, but at present the focus is on other aspects of the development.

Starting from 0.6.0 version, the server provides an additional geocoder that uses libpostal for normalization and parsing of the search requests.

When a new backend is added, such as with the release of 0.9.0, please ensure that you have subscription information updated. For that, in Map Manager, click on "Check for updates", decline updates, and unsubscribe/subscribe each territory that you have subscribed. After that, proceed with the downloads. This is not needed if you don't want to use the added backend(s).

Acknowledgments

The server is just a small part in large amount of work that have been done on mapping solutions by many other developers.

The underlying library was originally developed by Tim Teulings (@Framstag) who is the principal developer right now as well. Lukáš Karas (@Karry) has been working actively on the development of this library and Sailfish application OSM Scout. The server relies on their and other libosmscout developers code to provide the service.

On the other side, the server allows us to use excellent map applications developed over time for Sailfish and Linux. Already in the early development stage, I was helped by Poor Maps and modRana developers Osmo Salomaa (@otsaloma) and Martin Kolman (@M4rtinK), respectively. Thank you! While developing server, I could not cover all possible client applications and limited myself to these two. I am sure that if the server would become popular, the other plugins would be developed as well.

Changelog

Please see https://github.com/rinigus/osmscout-server/releases for changelog

I expect to ship the provider files in Poor Maps, so that if you have OSM Scout Server installed, it'll be listed in Poor Maps along rest of the providers. It just needs a bit of testing, so that release is maybe a week or so away.

Wouldn't it be easiest to just put the generated database files on a server for users to download? Bandwidth shouldn't be that expensive these days. It would mean writing conversion and upload scripts (preferrably in a separate repository) and having your app list what's on the server and download what the user chooses. Karry already has some kind of a download server.

Or are you looking to be able to import arbitrary smaller regions? Are full countries/states too large or slow?

Karry is working on it, indeed. He wrote some scripts that he uses to generate maps that are available on his server. Good news is that these maps are compatible with the server as well. From his server you could see the sizes of the files. Its a very important work that he does on it and I am looking forward to see where it would lead.

Unfortunately, I think that he used a bit older format of the map database, so right now you would still have to make the database yourself. Later, it would be possible to build the server using Karry's version of the library, so this problem would go away.

The second problem with pre-build maps is the cross-country routing. At present, libosmscout supports routing only within a map database. Whether the database covers 2, 3, or 1 countries does not matter. So, if you want to get from Sweden to Denmark, you would need a map covering the both countries. Since I'd like to get routing working as well, I see the need for letting user to decide the map region.

Rendering bigger maps are slower, sure. Fortunately, caching by Poor Maps and other applications would offset it a lot. There maybe parts in the library that would need to be optimized. But with the increased number of users, it should be easier to tackle the corner cases and fix them.

Your idea on getting lists of maps and downloading them is probably the way it is the easiest for the users and, as far as I know, Karry is planning to implement it for his application. I haven't thought about it too much yet and looked for bandwidth providers. My hope was that having a simple Import tool would at least postpone the need for a full-blown maps service. However, if someone wants to help Karry with the scripts and setting up the infrastructure, I am sure he will be very happy for all the help he can get!

rinigus

I had a chance to try your server. My observations:

• libosmscout is no masterpiece of cartographic rendering -- upsidedown labels, all kinds of things overlapping -- but I guess that was to be expected.
• There's a huge difference in rendering time between using a Mapzen metro extract for the Helsinki region and a Geofabrik extract for the whole Finland. So, your desire for users to be able to have databases of arbitrary regions instead of country extracts makes sense from a speed standpoint as well. With a Mapzen metro extract, the rendering time is surprisingly fast -- it's very usable!
• The large tile size seems fine. I doubt we need the smaller ones.
• The search seems very particular about the query format. e.g. "Erottaja" works, but "Erottaja, Helsinki" doesn't, "Tukholmankatu" works, but "Tukholmankatu 3" doesn't.
• The "admin_region" field in search results and thus the description field in Poor Maps is often empty. I can't conclude from the results when or why.
• The Nearby search seems to work fine, expect for the missing region data, which is a known problem. The raw OSM data is really bad for this though -- it's incomplete, outdated and very particular about how exactly POIs are tagged.

• libosmscout as a masterpiece: To my understanding, its constant optimization of speed vs quality. Label rendering is considered as an expensive part in tile rendering and there might be issues with it. If you have upside-down labels then note that this is unexpected and we should raise an issue in libosmscout developers list or github. An example tile would help (if you could provide server URL that would be of help too).
• difference in rendering times: Sometimes, there are corner cases that could make rendering slower than it should. For example, in Estonia, the rendering was majorly slowed down due to the handling of ferry lines with the ferry between Rostock and Helsinki increasing rendering time by order of magnitudes. This case was solved by @Karry. I don't know if you are hitting something similar. How long do you have to wait on the phone for a 1024x1024 tile to be rendered? There are also some messages printed in stdout by the library that are not displayed in Events of the server. I have converted Finland map and can check out on the phone if you would send me the zoom level and rough coordinates. Then we can see if there is something peculiar with Finland. But true, rendering is far from instant. That's were caching is helping a lot.
• Tile size: yes, large are probably the way to go.
• Search of Erottaja and admin_region: see below
• POI: that's maybe can be improved if I'll manage to find a way to get admin_regions for that. But yes, OSM data limitations are inherited by the server as well.

The search is implemented in two parts: search for location and search for any free text in the map. While the first is case insensitive, the free text search is case-sensitive.

The search results can be distinguished easily right now by presence (location) or absence (free text) of the admin_region field in the reply.

For location search, you have to add frequently an area where you search. When you search for Erottaja, you'll get the first response from the location search with

"admin_region": "Erottaja, Iitti, Päijät-Häme, Etelä-Suomi, Etelä-Suomi, Suomi"

The replies from the free text search find all objects that have the search string in object title. So, if you were searching for a bus stop in Helsinki then you get it as a part of free text search that can find an object with the name Erottaja, but its not recorded as "Erottaja, Helsinki" in that database. I don't know why the bus stop is not available in location search, maybe we should ask in libosmscout list.

As for Tukholmankatu, again all can be explained and the search can be guided. When you just use Tukholmankatu, location search is not finding anything since you haven't specified the area where you search. So, you get replies from the free text search. In free text, you have an object with this name which is a road. All buildings in that road are not a part of the database, unless they enter it as POI. To get an address, just add an area. So, the searches

Tukholmankatu 3, Helsinki

or even Tukholmankatu 3, Suomi would probably give what you want.

I am not sure how to improve this - whether to educate us as users or try to fix it in the library or server.

i have problem installing server. libosmscout depends on libprotobuf which i have newer version. could it be possible to update to newer version, or i just back down. tried to make symlink, didnt work.

You know what, wait a little bit. I have a suspicion that protobuf is not needed. As far as I know, its needed for import and I cannot see it in ldd of the server binary, which is a good sign. I'll check it out and will report back on it.

I doubt it's unexpected, since I see it all over the place: regular roads, forks, labels wrapping around a U-turn, etc. Couple examples below, center of the screen.

https://postimg.org/gallery/d4dc0wbm/

I'd say those Mapzen metro extract tiles render in about half the time of tiles from whole Finland data. And those tiles from whole Finland data are about what I'd expect, so it was not to report a problem, but a pleasant surprise when using a Mapzen metro extract. I don't have much use for offline maps myself, so I'll leave any possible debugging and timing for others.

I understand that it's a very difficult problem to parse and match search queries when people give them in many different formats and sometimes you might have partial matches, e.g. street found, but not the the full address, etc. But I also think that search should work with the same format queries as with online providers. Especially in an app like Poor Maps, which supports several providers and has a common history. No-one is going to read a long manual -- if normal search queries don't work, users will just conlude your software is broken.

Since it's a big problem, I wonder if there's a suitable external library you could use instead of trying to solve the problem on your own? For example, Mapzen has recently released Libpostal -- "a fast, multilingual, international street address parser trained on OpenStreetMap data".

https://mapzen.com/blog/inside-libpostal/

@kaari, please try again. I have changed RPM packaging script and dropped import library from the package. As a result, protobuf dependency is not needed. The new package for the library is uploaded to OpenRepos.

Please report back if it breaks anything. For those, who want to clean their devices:

* uninstall protobuf: pkcon remove protobuf . This should prompt for removal of the server and the libosmscout-qt.

* install the server again.

Everything is working on my device, but please report back and tell if anything is OK after this change. Sorry for this extra dependency, I seem to have left it by mistake.

In general, it looks similar to what I see. However, I haven't spotted upside-down labels as you did yet.


libpostal seems to be an interesting option. It looks that this is a preprocessor for the database query (see https://github.com/openvenues/libpos...not-a-geocoder ). Its an interesting option and certainly has to be tested. However, they do state that use on a mobile is "conceivable" and that its not designed with that in mind. I'll add libpostal to the todo (issues in github) and would take a look on it in future.

In case of Erottaja, Helsinki, libpostal would not help, I think. I'll have to check what is included into the search database as location and what is not. Hopefully, we can fix it.