Notices

- after du: "373M volga-fed-district"

- Importer output: https://ptpb.pw/Uyha , but it's only part. the beginning of the process is missing in terminal :-(

- 2 differencies today:
I deleted volga-fed-district.poly from folder
I enlarged free space from 4GB to 6GB.

- I see in Poor Maps ural-district as OSM_Scout_day_@1x and volga-district as OSM_Scout_day_@2x, OSM_Scout_night_@1x, OSM_Scout_night_@2x. And I don't see the difference after modification of path in server config.

@XOleg, all looks good with import - its finished. See "Import OK!" before deleting tmp files

Now regarding the jiggling with the maps, and that's important for all of us as users. Poor Maps and modRana do all they can to reduce the downloads. So, they cache the map tiles. Which is great, unless you are as @XOleg jumping from one map to another. In this case, when you change the map in the server, I suggest that you delete the cache in the client, close it for a good measure and reopen. That way new regions will be loaded.

There are ways to resolve it, but I am a bit on a fence to see what happens with the library. @Karry was working on support of multiple maps and maybe that's gonna be merged. At present, I am addressing other issues and let's see if we can get back to it.

@XOleg, please let me know if the cleaning of the cache helped.

On the side note: to celebrate opening of the store to the ported devices, I published OSM Scout Server in the store as well. You could choose your favorite provider now. I'll continue to publish it in OpenRepos as well.

@rinigus thanks for help. yes, cleaning of the cache helped.
hmm, Russia is not Monaco :-)

I have just published a new release: 0.5.2 followed by 0.5.3

The main change introduced in this release is addition of new translations - Czech by @Karry and Swedish by @eson57 (thank you very much!). I have updated also the coordinates returned on search of the street to ensure that they are on the street.

At present, I mainly work on improving the search engine. I am trying to adapt libpostal (thank you @otsaloma for suggestion!) for normalization and parsing of the search request. The results look very promising, but there are quite a few challenges with pushing it into mobile frame (RAM usage) and distribution of search databases. I will give an update when a bit more details are known.

OK, this is for getting feedback from the users and developers.

I have been working on integrating libpostal into the search functionality, as suggested by @otsaloma during his testing of OSM Scout Server. The development so far has been documented and discussed via mailing list https://sourceforge.net/p/libosmscou...ail.gmail.com/ . Below, is a summary of the thread.

Libpostal is a library that parses user input and splits the string into hierarchy: house number, street, city, ... In addition, it normalizes the strings to make it simple to use them for search. So, in theory, it would allow you to look for addresses without worrying about case, commas, accents and few other things. In practice, as much as I have tried, it works quite well. There are few cases were it fails and it would have to be addressed.

It turned out that there were no geocoders written so far around libpostal with an exception of integration it into Mapzen services (read: big iron). So, I wrote one: https://github.com/rinigus/geocoder-nlp . Its a small library that allows you to get coordinates and full transversal of an address/POI using a SQLite database.

When testing on mobile (OnePlus X), I was able to get search results for Germany (one of the worse cases due to huge OSM import) within 10s, usually within 5s (depends on how many levels you give). For that, RAM usage was <250MB RSS. So, I think it is possible to use this on mobile, especially taking into account quality of the results.

To make it mobile-worthy, I had to work around libpostal huge default memory requirements: 1GB RSS. This is due to the approach of libpostal - covering the planet in one go. So, it parses lots of languages and should be able to parse any address in the world. To reduce the requirements, I split the databases by country. Work is documented at https://github.com/openvenues/libpostal/issues/132 . In short, to make it work, custom databases are needed.

And here I come to the main issues with this approach:

1. Storage space requirements These come from libpostal databases and SQLite database for geocoding. Note that the numbers below are in addition to what is needed for libosmscout.

Libpostal requires language parser (I haven't split that, so it covers all with 690MB. In addition, country-specialized databases are

28M CZ
146M DE
10M DK
5.1M EE
9.4M FI
16M SE

So, if you want to cover FI and SE, you would need 690+9.4+16 MB.

In addition, SQLite database that is generated to get geocoding working. For Estonia, its 14MB (compare that with 69MB import for libosmscout). For Germany, its 1.1GB (compare it to 2.6 GB of libosmscout import). So, this is significant.

2. Database generation. Now, to generate libpostal databases, you would have to engage in machine learning. In short, that is not for an average user. It requires server-grade hardware, jiggling with 30GB files and significant CPU requirements. Good news is that these databases don't change frequently. So, its possible to make them using and extending scripts that I wrote already. Since its NLP in its pure form applied for postal addresses, probably Martin's servers in university could be considered? In short, to use this approach, distribution of the databases has to be provided.

That's all for now. I wonder, whether you as a user would be interested in such approach taking into account the storage requirements? I will surely be using it on my device, but if noone is interested ...

cheers,

rinigus

I will absolutely use this, and the storage requirements seem entirely reasonable to me. Thank you for your continued work on this!

The device requirements look about OK to me, and if they look about OK now, they'll be a complete non-issue in two or three years. Design for the future! Regarding databases, my impression is that server resources (both computation and storage + traffic) are quite cheap these days, and if the data doesn't change often, you only pay for computation time you need. So, if Martin can provide servers that's good, if not, it shouldn't be a problem either. You should be able to collect server fees as donations, or maybe charge for the app if that's one day possible. So, if the results are good, I'd say go for it.

I'm not likely to use offline maps myself though, with or without this change, as online is not a problem for me. But, if I did, I'd want to use something that works.

Out of interest, why not? When I generated a map for the whole of the UK it only took up 1GB of space.

In Finland unlimited data plans are the norm, so using online providers doesn't bring any cost. And online providers tend to be much better quality in terms of map rendering, search query parsing, navigations instructions, etc.

When travelling in Europe, I tend to just download Google map tiles and copy them to the device. After that, only searches and navigation cause data traffic, and those are just small JSON files, it's such little traffic that it's not a problem in Europe these days.

When travelling outside Europe, offline maps could be useful, but that's rare for me.

Thank you and @Feathers McGraw for positive support. As for servers, I guess you are right. Just don't want to engage right now into organizing financial aspects of it and would prefer to focus on making the server that works well.