Re: [Announce] Native offline maps: OSM Scout Server
 

Holy Moly, thanks for this really detailed explanations!

Of course it would be nice if maps would render faster, but anyway its so nice to have finally native offline maps!

Re: [Announce] Native offline maps: OSM Scout Server
 

Maybe it could be also related to the old bundled urrlib3 (issue 146) ? I've updated it in the master branch, but it is not (yet!) in any released version. So users packages from OpenRepos & Jolla Store still run with the old urrlib3 and might be getting those errors.

That should hopefully not be an issue soon as I plan to release updated modRana packages today or tomorrow. :)

Map tile scaling is unfortunately currently broken in modRana - and even if it worked, it is at present only meant to upscale current 256x256 pixel tiles - mainly to make text and small map features easier to read (at the cost of making them more blurry). The map scaling code (once fixed) should be relatively easy to adapt to supporting larger tile sizes - even though there might be some issues with overlays to overcome when some layers provide bigger tiles and others not.

You can also check the modRana debug logs: Go to Options -> Debug, enable the tile related debug logging & logging to a file and then restart modRana. On next start modRana should create a log file in ~/Documents/modrana_debug_logs/

This is something I have been wondering about - is that really needed ? Couldn't OSM Scout Server check incoming tile requests against the bounding boxes/shapefiles of multiple data packs and return data for the first match ? Or, depending on cost, just run the query on one pack after another until there is a hit ?

Some heuristic can be applied for this to run all further queries after a hit on the same data pack until there is another miss. Users could also enable just some packs in the OSM Scout Server UI to reduce the number of lookups.

Also a related idea - client side pack specification. OSM Scout Server clients could specify data pack name for the lookup. They could get the available pack names either from OSM Scout Server via a special query or by other means (listing all folders the OSM Scout Server data folder or even by downloading data packs themselves and pointing OSM scout to them).

Just a couple of ideas. :)

Yeah, that's certainly not an easy problem to solve. I wonder how the commercial/proprietary navigation systems handle that ? Or do they just have larger areas (whole Europe, etc.) so they don't have to care about it ?

BTW, a naive idea how inter pack routing might be done: :)

0. assuming two adjacent packs with a relatively straight border between them & some overlap + simple start-destination routing
1. draw a straight line from start to destination
2. note where it intersects the border
3. search for suitable (way) nodes around the intersection points in both map-packs
4. find a node near the border that is in both packs
5. compute two routes: start - shared border node, shared border node - destination
6. return combined route as the result

This is of course very simplified and would probably returns some pretty inefficient routes. It also does not handle complex border geometries, routes over multiple packs, non overlapping packs or having only packs that have no shared border at all. :)

Again - just a couple of ideas. :)

Re: [Announce] Native offline maps: OSM Scout Server
 

I already forgot about this bug, thank you for reminding about it! Yes, that's probably the reason behind the slower response. It got exposed when I switched from Qt-based HTTP server to microhttpd and is fixed in modRana's master branch. As far as I remember, the symptoms were very similar to what was described.

I am pretty confident that this is a reason for significant slow-down. We just have to wait 1-2 days to test it :)


Something like this has been implemented by Lukas (@Karry) in his OSM Scout application, if I understand correctly. There are some significant hurdles to overcome for server if we want to do that.

Server renders the map in as many threads as you have CPUs. For that, it actually shares database access. If we want to render in parallel, we may end up having the same number of databases open. I think this may end up with too much RAM consumed by the server. So, we could start writing in some logic and lock/unlock threads depending on the accessed databases. Its all possible, but not trivial.

Alternative is to limit to a single thread and render one tile in a time. This may slow down it too much taking into account that the mobile devices may start lowering CPU frequency and switching off some cores when the load is about 1.

We can return to it a bit later. Maybe Lukas will implement it that way that it will be exposed via libosmscout anyway :-). Ideally, it would be solved on the library level and all applications using the library would benefit from it (server included).

We can try to formulate it as an issue and see when and how to resolve it.

I was thinking at one point to add database and the stylesheet as a part of URL. Its possible - we just get the same problems as I was pointing out above.

I would suggest to get other issues solved first: better GUI, search, rendering, and routing come up into my mind on server side. Better routing support on the client part (voice commands?).

Routing between maps is possible. Many have solved it with what seems to be separate data packages: OSMAnd, MAPS.ME for example. From Valhalla's docs and few test runs, it seems that they also partition internally all the maps. So, its possible, just have to work on it. Corresponding issue is https://github.com/Framstag/libosmscout/issues/57

Keep them coming. Let's just hope that we can implement at least a fraction of them.

Re: [Announce] Native offline maps: OSM Scout Server
 

Yep, that definitely seems like the most sensible way. :)

A significant usability issue is also IMHO the constantly changing data format - if it was more stable there could be data repositories users could use instead of having to generate their own data packs. It might be even possible to download the data directly from the GUI (from the OSM Scout Server UI or via clients).

I might be even able to arrange something with the Masaryk University NLP lab which is already kindly hosting the 100+ GB of Monav offline routing data modRana can make use of (currently mainly on the N900).

But at the moment people might even have to re-generate their data files even due to am OSM Scout Server updates due to using a newer version of libosmscout.

Does it look like there might also be a solution for this issue ? I understand designing and supporting an unchanging binary data format is no easy task for libosmscout developers.

But at the same time I'm afraid the current unstable-data format can block many users from using OSM Scout Server & client apps - especially less advanced users who could make use of pre-generated map data.

Sounds promising! :)

Yeah, that's always the biggest problem! :D

Re: [Announce] Native offline maps: OSM Scout Server
 

The last time we had to regenerate the maps was due to the bug in map importer. As far as I remember, the format stayed the same - just data was imported wrongly. This bug was exposed during testing of the routing and I suspect that there maybe more bugs to come. Eventually, it will stabilize, we just have to accept some fluidity at this time.

To offset the map format changes I worked on the importers. Hopefully, it would be of help for users.

Its great that you have such a host - it could really become handy. Note that the maps do have to be regenerated relatively frequently. I presume that once a month should be a target. And then we would have to see how to split the maps - countries, regions, ... Maybe you should get in touch with Lukas. He was working on getting maps distributed via downloads and he may be interested in expanding on that part of the project.

But the supply of the maps to the users is, indeed, a major problem that we would have to resolve.

Re: [Announce] Native offline maps: OSM Scout Server
 

I for some reason though it was much more unstable, so it's good to hear it's not the case. :)

Sounds good. :)

I've tried the Windows importer a while ago and it seem to work fine - nice trick with drag-and-drop support on the import script. :)

That should be doable - I'm using one of the machines in the NLP compute cluster to generate the Monav data. It's a rather powerful computer (48 CPU cores & ~250 GB RAM) and can run a full update (download planet.osm.pbf, split to regional files, convert to Monav data, compress the results to tar.gz archives) in less than 24 hours. The conversion itself takes about 5-6 hours, the rest is "planet splitting".

BTW, the software used to generate the repository lives in the modrana-data-repository repository and it should not be that hard to extend it to generate also libosmscout-compatible data once the format is considered stable enough. :)

For the Monav data packs I've just used the same region definitions Geofabrik uses on their OSM data download page. They even have polygon files for each area - this is what I use to generate the regional *.osm.pbf files with osmosis from a fresh planet.osm.pbf file. No need to download all their PBF files every time the repository is updated. :)

Re: [Announce] Native offline maps: OSM Scout Server
 

It would be great if you could subscribe to https://lists.sourceforge.net/lists/...ut-development and introduce your options to distribute maps to the list. I'll mention this discussion over there as well. Lukas has been working on generating the maps for distribution and maybe you together could come up with some great solution. Through direct communication between you and Lukas we would ensure that your experience is not lost in translation :)

Re: [Announce] Native offline maps: OSM Scout Server
 

Hi all. I read this thread completely and I have to say thanks to all developers for your work and to all people who test our applications!

@rinigus you should consider to submit server to Coding Competition :-) There are some interesting prices like Jolla C. Other apps are welcome too!

To map distribution, I just recap what I send to libosmscout mailing list: server part for OSM Scout is ready, scripts are available here. I host it on Rasperry PI with 1 TiB disk dedicated for maps. It is dwarf to compare to your machine @MartinK :-) I will focus on client side next days/weeks. It will be available for osmscout server too off course. When everything will be ready, we can start working on map delivery network (MDN ?). What is university status to your server Martin. It is officially supported or it is tolerated "punk activity"?

Re: [Announce] Native offline maps: OSM Scout Server
 

Sounds good! We could try to create some simple osmscout data repo spec - for example how a JSON document describing available map packs looks like. Like this repos/repo mirrors would be independent from how they are created (your scripts, the modrana-data-repository generator, manual generation, etc.) and people will still be able to use them in a unified way from applications supporting the spec. :)

My bachelors & masters thesis were led by a supervisor from the NLP lab. We are still in touch after I've finished my studies and the current Monav data repository (and the whole modrana.org website) is hosted by the lab with his & labs consent. And I do credit them for this where appropriate this. :)

I will of course first ask them if they are OK with generating & hosting another data set, but I kinda expect it should not be an issue. In comparison to some of the corpus processing they do our data sets are very very small. :)

Re: [Announce] Native offline maps: OSM Scout Server
 

Map downloader is ready for merge to upstream library: https://github.com/Framstag/libosmscout/pull/185

Now, Silica UI...

Re: 0.5.0
 

@karry - great work, thank you very much! This is surely a step in the right direction!


I have just published a new release: 0.5.0

This release addresses the GUI shortcoming highlighted by @Feathers McGraw:

• its possible to specify units (Metric or Imperial). Since we use only km and mi., there is no need to separate british from american, as far as I understand
• to select the map (and few other settings), you are shown directory or file selection dialog. Should work for SD cards as well.

Unfortunately, my changes break the Spanish translation in few places. Maybe we will get an update a bit later and I'll publish new point releases.

If there are some other requirements with GUI, let me know. For me, its OK and I am planning to focus on some aspects of the library. There were suggestions by @otsaloma that were given early on and I'd like to see what I can improve.

Re: [Announce] Native offline maps: OSM Scout Server
 

@rinigus, I tested v.5 with Poor Maps.
- Great work, but it's not simple for ordinary user for this moment to import maps. NokiaMaps for N9 is good exemple for me.
- I need 2 districts - ural and volga. I haven't problem with ural-fed-district-latest.osm.pbf, but I don't see map after import from volga-fed-district-latest.osm.pbf(I configured path for server). And I see in folder 186.5MB instead 359.9MB for volga-fed-district-latest.osm.pbf. Or it's good?
- is it possible to navigate between 2 districts?

Re: [Announce] Native offline maps: OSM Scout Server
 

Thank you!

Sure, import of the maps can be improved. As you could see from yesterday's post by @karry, he is working on it :) . See also earlier discussion between @karry and @MartinK on the subject. We could expect that the devices would be able to download maps directly from some server, but there maybe some time before it will be up and running.

So, at present, an ordinary user would have to download maps to PC, drag-and-drop them on importer tool, and transfer imported folders to device. I think its reasonably simple and, taking into account that @karry's work, there is no point in making fancy GUI on PC side of things.

Re Ural and Volga: I tried to import Volga district and it leads to 328MB folder. When moved to device, all worked fine. So, I think something went wrong during import or while you copied to the device. Note that you should have got all the files that you have for Ural, maybe something is missing. If the importer finishes with such small import, please tell me what OS do you use, importer version, and maybe you could also provide Importer output? Otherwise its hard to debug the issue.

Routing between 2 maps is not possible (yet). What you can do is to join the regions that you are interested in. Use OSM-provided tools for that, as explained at https://github.com/rinigus/osmscout-server#maps . In particular, look at https://wiki.openstreetmap.org/wiki/...lel_Processing

You may consider to reduce the maps to the part that you actually use using osmconvert.

Re: [Announce] Native offline maps: OSM Scout Server
 

Btw, you can define more OSM (osm.pbf) files for one import. When they are overlapping, routing should be possible.

Re: [Announce] Native offline maps: OSM Scout Server
 

@karry - thanks for a tip, didn't know that! Note that the import script that is distributed with pre-compiled importer is not aware of that. So, I don't think it would work with it and the included import executable would have to be called directly.

Re: 0.5.0
 

That was quick! Thank you very much for making those changes.

Since you seem to be on a roll, I noticed another thing the other day, when I was searching for pubs near my current location from within Poor Maps. No pubs were returned in the results despite there being several nearby, but there were a few public transport platforms in there for some reason...not sure if poor maps is requesting the wrong POI type, or if it's a server side thing? The map package was the one I created using the script as we discussed previously in this thread.

Re: [Announce] Native offline maps: OSM Scout Server
 

@rinigus I understand there is no other way for this moment.

volga. It's not problem with copy from PC to phone. I use ubuntu 16.04 and last importer. I'll try to send Importer output, but how to do it? to get from terminal(it's long) and send after that.
Thanks for help.

P.S. It's not simple to replay by SfOS from phone :-)

Re: [Announce] Native offline maps: OSM Scout Server
 

Great, Linux makes it all easer for me. I would suggest to run importer again and see

du -sh volga-fed-district

For me it resulted in 328MB.

You could copy the output and paste it into https://ptpb.pw/f . For that, go to that web-site, copy your text into the large text box (central), set the expiry date (bottom right) for ~week from now, and press paste. As a result, you will get a light blue box with "status: created". Click on "created" and send me the URL via this forum. Then we can see whether import was fine.

Is it possible that you run out of storage? RAM is also an option, how much do you have in Ubuntu?

Re: 0.5.0
 

Thanks!

Its surely not Poor Maps issue. The problem could be either the wrong POI type or absence of the pubs nearby.

To see the available types, use the URL given https://github.com/rinigus/osmscout-...able-poi-types . For that, start the server and visit

http://localhost:8553/v1/poi_types

in the browser. [There is no better way right now]. When you look for nearby POIs, you are searching for objects which are within the given distance (with some precision) and are of the corresponding type. The type comparison is done by lowercasing the type you enter and assuming that you entered a substring of a type.

If the type was correct and pubs are still missing, try to do the same search in Poor Maps using some other search provider. You could also go to http://www.openstreetmap.org and see if you can find the pubs over there. Try to figure out what type they are and see if you could find them.

Let me know. If something is wrong and you see all these pubs in OSM, we would debug it further. Then I would need an example of a missing pub :)

Re: [Announce] Native offline maps: OSM Scout Server
 

@rinigus I'll do this slightly later.

RAM = 8 GB. I don't see warning "out of space".

Re: [Announce] Native offline maps: OSM Scout Server
 

- after du: "373M volga-fed-district"

- Importer output: https://ptpb.pw/Uyha , but it's only part. the beginning of the process is missing in terminal :-(

- 2 differencies today:
I deleted volga-fed-district.poly from folder
I enlarged free space from 4GB to 6GB.

- I see in Poor Maps ural-district as OSM_Scout_day_@1x and volga-district as OSM_Scout_day_@2x, OSM_Scout_night_@1x, OSM_Scout_night_@2x. And I don't see the difference after modification of path in server config.

Re: [Announce] Native offline maps: OSM Scout Server
 

@XOleg, all looks good with import - its finished. See "Import OK!" before deleting tmp files :)

Now regarding the jiggling with the maps, and that's important for all of us as users. Poor Maps and modRana do all they can to reduce the downloads. So, they cache the map tiles. Which is great, unless you are as @XOleg jumping from one map to another. In this case, when you change the map in the server, I suggest that you delete the cache in the client, close it for a good measure and reopen. That way new regions will be loaded.

There are ways to resolve it, but I am a bit on a fence to see what happens with the library. @Karry was working on support of multiple maps and maybe that's gonna be merged. At present, I am addressing other issues and let's see if we can get back to it.

@XOleg, please let me know if the cleaning of the cache helped.

On the side note: to celebrate opening of the store to the ported devices, I published OSM Scout Server in the store as well. You could choose your favorite provider now. I'll continue to publish it in OpenRepos as well.

Re: [Announce] Native offline maps: OSM Scout Server
 

@rinigus thanks for help. yes, cleaning of the cache helped.
hmm, Russia is not Monaco :-)

Re: 0.5.2
 

I have just published a new release: 0.5.2 followed by 0.5.3

The main change introduced in this release is addition of new translations - Czech by @Karry and Swedish by @eson57 (thank you very much!). I have updated also the coordinates returned on search of the street to ensure that they are on the street.

At present, I mainly work on improving the search engine. I am trying to adapt libpostal (thank you @otsaloma for suggestion!) for normalization and parsing of the search request. The results look very promising, but there are quite a few challenges with pushing it into mobile frame (RAM usage) and distribution of search databases. I will give an update when a bit more details are known.

Re: [Announce] Native offline maps: OSM Scout Server
 

OK, this is for getting feedback from the users and developers.

I have been working on integrating libpostal into the search functionality, as suggested by @otsaloma during his testing of OSM Scout Server. The development so far has been documented and discussed via mailing list https://sourceforge.net/p/libosmscou...ail.gmail.com/ . Below, is a summary of the thread.

Libpostal is a library that parses user input and splits the string into hierarchy: house number, street, city, ... In addition, it normalizes the strings to make it simple to use them for search. So, in theory, it would allow you to look for addresses without worrying about case, commas, accents and few other things. In practice, as much as I have tried, it works quite well. There are few cases were it fails and it would have to be addressed.

It turned out that there were no geocoders written so far around libpostal with an exception of integration it into Mapzen services (read: big iron). So, I wrote one: https://github.com/rinigus/geocoder-nlp . Its a small library that allows you to get coordinates and full transversal of an address/POI using a SQLite database.

When testing on mobile (OnePlus X), I was able to get search results for Germany (one of the worse cases due to huge OSM import) within 10s, usually within 5s (depends on how many levels you give). For that, RAM usage was <250MB RSS. So, I think it is possible to use this on mobile, especially taking into account quality of the results.

To make it mobile-worthy, I had to work around libpostal huge default memory requirements: 1GB RSS. This is due to the approach of libpostal - covering the planet in one go. So, it parses lots of languages and should be able to parse any address in the world. To reduce the requirements, I split the databases by country. Work is documented at https://github.com/openvenues/libpostal/issues/132 . In short, to make it work, custom databases are needed.

And here I come to the main issues with this approach:

1. Storage space requirements These come from libpostal databases and SQLite database for geocoding. Note that the numbers below are in addition to what is needed for libosmscout.

Libpostal requires language parser (I haven't split that, so it covers all with 690MB. In addition, country-specialized databases are

28M CZ
146M DE
10M DK
5.1M EE
9.4M FI
16M SE

So, if you want to cover FI and SE, you would need 690+9.4+16 MB.

In addition, SQLite database that is generated to get geocoding working. For Estonia, its 14MB (compare that with 69MB import for libosmscout). For Germany, its 1.1GB (compare it to 2.6 GB of libosmscout import). So, this is significant.

2. Database generation. Now, to generate libpostal databases, you would have to engage in machine learning. In short, that is not for an average user. It requires server-grade hardware, jiggling with 30GB files and significant CPU requirements. Good news is that these databases don't change frequently. So, its possible to make them using and extending scripts that I wrote already. Since its NLP in its pure form applied for postal addresses, probably Martin's servers in university could be considered? In short, to use this approach, distribution of the databases has to be provided.

That's all for now. I wonder, whether you as a user would be interested in such approach taking into account the storage requirements? I will surely be using it on my device, but if noone is interested ...

cheers,

rinigus

Re: [Announce] Native offline maps: OSM Scout Server
 

I will absolutely use this, and the storage requirements seem entirely reasonable to me. Thank you for your continued work on this!

Re: [Announce] Native offline maps: OSM Scout Server
 

The device requirements look about OK to me, and if they look about OK now, they'll be a complete non-issue in two or three years. Design for the future! Regarding databases, my impression is that server resources (both computation and storage + traffic) are quite cheap these days, and if the data doesn't change often, you only pay for computation time you need. So, if Martin can provide servers that's good, if not, it shouldn't be a problem either. You should be able to collect server fees as donations, or maybe charge for the app if that's one day possible. So, if the results are good, I'd say go for it.

I'm not likely to use offline maps myself though, with or without this change, as online is not a problem for me. But, if I did, I'd want to use something that works.

Re: [Announce] Native offline maps: OSM Scout Server
 

Out of interest, why not? When I generated a map for the whole of the UK it only took up 1GB of space.

Re: [Announce] Native offline maps: OSM Scout Server
 

In Finland unlimited data plans are the norm, so using online providers doesn't bring any cost. And online providers tend to be much better quality in terms of map rendering, search query parsing, navigations instructions, etc.

When travelling in Europe, I tend to just download Google map tiles and copy them to the device. After that, only searches and navigation cause data traffic, and those are just small JSON files, it's such little traffic that it's not a problem in Europe these days.

When travelling outside Europe, offline maps could be useful, but that's rare for me.

Re: [Announce] Native offline maps: OSM Scout Server
 

Thank you and @Feathers McGraw for positive support. As for servers, I guess you are right. Just don't want to engage right now into organizing financial aspects of it and would prefer to focus on making the server that works well.

Re: [Announce] Native offline maps: OSM Scout Server
 

for me offline it is much better that online,Both have advantages and disadvantages!!!
what about REroute in PoorMaps @otsaloma There is possibility to fix it?

Re: [Announce] Native offline maps: OSM Scout Server
 

Rerouting is planned, yes, and possible, in time.

Re: [Announce] Native offline maps: OSM Scout Server
 

perfect! @otsaloma

Re: [Announce] Native offline maps: OSM Scout Server
 

Just in case this is the code modRana uses to check if the current route is being followed:

https://github.com/M4rtinK/modrana/b..._turnByTurn.py

If should be fairly efficient as it works fine even on the N900 for fairly long roads. Feel free to (re)use. :)

Re: [Announce] Native offline maps: OSM Scout Server
 

This is announcement for a new release: 0.6.0

This release introduces libpostal (https://github.com/openvenues/libpostal ) based geocoder that is expected to significantly improve the search functionality of the server. To my knowledge, its the first time that libpostal has been used on mobile. For background and the requirements, see earlier post http://talk.maemo.org/showpost.php?p...5&postcount=65

In addition to libpostal-based geocoding, the server is compiled with the newer libosmscout version that addresses few drawing issues, to name the few changes. Also, Swedish and Spanish translations have been updated.

This release should be considered as a "technology preview" with the distribution of the required databases and simplification of selections planned to be addressed in future.

The new search functionality is disabled by default. To enable, you would first need to get few databases in addition to the regular libosmscout map databases. For distribution, I am using mega.nz for now:

http://tiny.cc/geocodernlpdata

I may have to update the corresponding link on mega from time to time, to get decryption in sync.

The required databases are listed below with the file location at the distributed folder:

* libpostal language parser: postal/postal-global.tar.bz2

* libpostal country-specific database: postal/countries/<SELECT THE NEEDED ONES>

* geocoder database: geocoder/<SELECT THE NEEDED ONES>

Country-specific databases are generated in accordance with http://download.geofabrik.de/ distribution. Exceptions were:

* Russia was added as a part of Europe;
* Kosovo is missing since I could not find any records from this country in libpostal training set.

Some countries are joined with the others. For example, Ireland and Northern Ireland are joined. In this exceptional case, I had to join Ireland and UK records to provide libpostal country-specific database.

All the archives (tar.bz2 or .bz2) have to be extracted before use. I suggest to download them to PC, extract and then move to the device.

As soon as you have all databases on device, enable new geocoder by

* Go to menu Geocoder

* Select "Use geocoder-nlp"

* Select location of libpostal language parser (extracted postal-global.tar.bz2)

* Select libpostal country-specific database (extracted postal/countries/AA.tar.bz2, for example)

* Select geocoder database (for example AAland.sqlite)

* Select the languages that you want to use for parsing. Don't leave "All languages selected", that may lead to too large memory requirements! At present, select languages used in OSM map for that region.

* Accept new settings and enjoy

Searches (while using search in Poor Maps) and routing-related searches would run then via new geocoder.

As every intelligence, AI of libpostal does make mistakes. To help you out in these cases, the server logs how was the search partitioned (house number, street, ...). As an additional parser, I added a primitive parser that constructs hierarchy by taking your input and splitting it into parts assuming that parts are split by comma. So, in these cases, use the following notation "house number, street, city, ..." (i.e. move from inner to outer admin region). To use primitive parser, enable it in Geocoder options. Primitive parser shows hierarchy with h-number ids in logs.

For map application developers: I have introduced a new version of search results available via v2, see https://github.com/rinigus/osmscout-...arch-version-2 . In addition to search results, it adds information on how the search string was parsed. As far as I have seen, something similar is expected or already done by Mapzen to give the feedback on libpostal parsing.

Re: 0.6.1
 

I have just released a new update: 0.6.1

This is a work of the translators who updated the Swedish and Spanish translations - thank you @carmenfdezb and @eson57!

Please note that this version uses exactly the same libosmscout library as 0.6.0. Due to the changes in libosmscout, there is a bug that can cause server crash when you use older maps with the border style (https://github.com/rinigus/osmscout-server/issues/61). @Karry has found the cause and its fixed upstream. However, since it would require generation of new maps, I decided that such rather rare case (use of borders style) does not require the update of underlying library. This update is planned for future major release.

Together with others, I have been working on improving import into libosmscout databases. We are testing one patch right now that should make location hierarchies more accurate on imports. As soon as its ready, I am planning to regenerate geocoder-nlp world databases (sqlite).

I am curious if anyone has tried geocoder-nlp/libpostal search functionality in 0.6.x? If you did, how where the results? So far there has been no feedback on this.

Re: [Announce] Native offline maps: OSM Scout Server
 

I did a quick test with the new libpostal/geocoder stuff, here are my thougts so far:


Search results are a lot more usefull, putting in <Street> <Number>, <ZIP-Code> <City> gives the correct result almost 100% of the time, without libpostal it often would find anything but the Address that was searched for

Time for an address search seems to depend a lot on the "comonness" of the street name:
- searching for an address with "Parkstraße", which is a very common street name in Germany, takes several Minutes to complete, even when full ZIP-Code and City is given
- In contrast a search with an unique (or uncommon) street name yields a result in under 5 seconds.
I don't know how the search code works, but it seems to me it searches for the Street Name first, maybe it would be more efficient to first search City / ZIP-Code and then the Street (just a wild guess, I have not looked at the code so far)

Since I just played around with the new functionality a bit, thats it for my first impressions. All tests where done on a JollaC with all the data on the SD-Card

Re: [Announce] Native offline maps: OSM Scout Server
 

Thank you very much for the profiling and an example!

I will use your case for optimization of the search. It does search first for the city (or county) and then moves over to within searches, but I should probably change SQL query slightly to apply these refinements earlier. I should be able to make it faster :)

Unfortunately, ZIP codes are not used in the search. So, there is no point in entering them. I don't know yet how to apply ZIP codes and it has been rather low priority for now. I may look into it a bit later (or anyone interested can look into implementing zip code searches as well).

Thanks again, its quite helpful to get your feedback!

Re: [Announce] Native offline maps: OSM Scout Server
 

@rinigus
access to http://tiny.cc is blocked in Russia :-(

Re: [Announce] Native offline maps: OSM Scout Server
 

@seiichiro0185, I made a new SQLite database that should speedup the searches. On OPX, the search time for Parkstraße 5 frankfurt dropped from 20+s to 1-2s. There is an additional index in the database for that which made it bigger. I will look into how to reduce the size of it.

Please test whether it works for you. Using the same server code, just get the new database from "testing/europe" subfolder in mega.nz share.

@XOleg, that's really strange. Current link is
https://mega.nz/#F!8EsxXRTT!arad2NJ9rEhpXXn-YXsF8w , but maybe mega.nz is blocked as well. I will work on database distribution soon and maybe we can find servers that are possible to connect to from Russia. Although, I cannot really check it out from here.