Re: [Announce] Native offline maps: OSM Scout Server
 

Hi rinigus, two questions, also related to search function:

• How do I have to enter a search string for OSM Scout Server?
  If I use WhoGo Maps (or soon Pure Maps) I normally use OpenCage or Photon. Both work quite OK. But the search with OSM Scout doesn't really work for me. For example if I want to search for "München" (Munic) I get a lot of results, but non of it is München itself but Münchner Straße, Münchenroth, Münchenroda...
  
• I was 3 weeks ago in England, visiting friends. They live quite rural, out of town. So they don't have a house number nor a street name. The only possibility to navigate to them, was using HERE WeGo (Android), because there you can enter the postcode. Is postcode search planned in future releases?

Re: [Announce] Native offline maps: OSM Scout Server
 

I second the postcode search requirement.

Regarding München, check out here :D
In short, form the search query as "<house number>,<street name>,<town>,<country", with fields omitted as necessary.
In your case, ",,München".

Re: [Announce] Native offline maps: OSM Scout Server
 

Re München: This has been an issue that was resolved probably few releases ago. It was resolved by sorting the results differently by geocoder-nlp and was raised earlier in this thread due to the similar issue in UK (see linked post by pichlo). When just tested, "München" gave a city as a first result. So, please try with the current version.

Re search string: when all works as it should, libpostal processes the addresses in the natural (for selected languages in OSM Scout Server) order of addresses. In Estonian, that would be street housenr, city, country (commas can be skipped). When it fails for one reason or another (shouldn't be frequent), you could set in OSM Scout Server settings for GeocoderNLP to use primitive parser which will use the same hierarchy sequence as suggested by
pichlo.

Re postcodes: sounds like its a popular request for UK. I have opened an issue regarding it, but haven't worked on it yet. In theory, its possible, since my import pipeline has that data available, at least in part. But I will have to think how to do that. Obviously, libpostal knows how to separate postal code from the address, but the search has to be done by my geocoder.

Re: [Announce] Native offline maps: OSM Scout Server
 

In the UK, postcodes are the way of giving directions. There is nearly a direct mapping between a postcode and geographic coordinates. There is a postcode for every street; long streets can have several postcodes. And of course important landmarks have their own postcodes too.

So I would not say they are "popular". More like "essential".

Re: [Announce] Native offline maps: OSM Scout Server
 

Can you specify how do you expect this to work. Do you just type in the postcode and get coordinates to drive to? Or do you specify post code and some additional information?

Having a mapping between postcodes -> coordinates is "simple". So, driving on the basis of postcodes only.

Having a mapping between postcodes and some extra data is harder. Not sure how to do that properly yet

Re: [Announce] Native offline maps: OSM Scout Server
 

Well in Here WeGo, I just put in the postcode. More wasn't necessary and as I told you my Frinds don't have a proper adress, no streetname, no house number, BUT a postcode.

Re: [Announce] Native offline maps: OSM Scout Server
 

@rinigus I have to excuse me. The problem with the search string was my mistake. On the last Map update, there must have gone some things wrong, therefore some parts were not downloded completely. After downloading them anew all works as expected.

Re: [Announce] Native offline maps: OSM Scout Server
 

GUI software programming can be FUN, especially if it has to account for quirks of both mobile and desktop platforms. :)

If you do decide to port Poor Maps to also run desktop Linux and other mobile platforms I have some experience with that from modRana development (8+ years :D). Also if you decide to use Universal Components for common multi-platform QML GUI codebase I'm standing by to answer any questions & fix/improve the API (time permitting). :)

That's definitely understandable - now after the "analysis" is done, it's not looking that complicated anymore. :) Still there is quite a bit to do in othter places in modRana as well (finally use proper TTS & improve navigation mode, use the new icon & branding recently contributed for modRana, add support for MApBox GL native widget, typed POI search, etc.).

Any other takes for a modRana coding adventure ? ;-)

Yep, that has been my idea as well.

I'm thinking about a map screen button that's shown when a road is being followed as the primary entry point for this feature. That should help with discoverability & make the feature available when most needed.

Yep, that could be the secondary access point after the on map button.

Yeah, I really need to handle that as well - sorry for not acting on your issue/PR for so long. :P

Yeah, that looks doable. I'm starting to think it might be good eventually rethink how some of these screens work but that can wait before the more pressing issues are implemented and fixed.

Re: [Announce] Native offline maps: OSM Scout Server
 

Postcode only! I don't know anyone in the UK who puts more than just the postcode into a satnav, like Pichlo said the mapping between postcode and coordinates is quite precise.

Let's say you are navigating to a house, 99% of the time the postcode would take you within 100m of the house and you'd just be looking at house numbers from that point. For something like a supermarket, the postcode would probably take you to the centre of the car park.

We don't tend to have long residential roads with 100s of houses either, typically the number of houses on a street would be less than 50.

Re: [Announce] Native offline maps: OSM Scout Server
 

Here is a link to UK postcodes: https://postcodeaddressfile.co.uk/

Re: [Announce] Native offline maps: OSM Scout Server
 

OK, that's sounds easier than I thought. Its in the pipeline, but would take some time before I'll get there.

Re: 1.8.0
 

Bumping an old comment :

On the other hand, 500G microSD cards are now a thing, meaning that you could certainly store half the planet on a single card, and could store a couple of countries easily on 128G cards.

Worth testing ?

The data distribution is going to be a tiny bit more challenging, though.

Re: 1.8.0
 

I have opened issue for the server on support for elevation data. When Valhalla team will get bit less busy with the work on 3.0 (there are few of mine issues opened already), I'll try to discuss with them on how to reduce storage footprint for this data. So, for now, let's wait a bit

Re: [Announce] Native offline maps: OSM Scout Server
 

i have two question for rinigus:

1) is it possible to download maps with PC lan connection and then transfer them on the microsd?

2)Does the server work the same if I put all the maps in the microsd instead of the internal memory?

Re: [Announce] Native offline maps: OSM Scout Server
 

1: yes, but you would have to install Qt version of the server on your Linux PC. That includes handling dependencies and compilation. So far, I don't know anyone except me who has done it . Recently, I had one user trying, but I don't know how it went (see issue at github). Which may relate to the lack of install instructions, though. After the download, you'll have to rsync data to the phone.

2. Yes, it does work "the same". There is some slow down, but that would depend on your hardware, I presume. I am using it on SD Card and it works very well.

As for downloading larger regions, you could just position the phone on charger and use WIFI, if needed

Re: [Announce] Native offline maps: OSM Scout Server
 

definitely, i will use wifi, since the other option is too complicated for me
Thanks foe your quick reply

Re: [Announce] Native offline maps: OSM Scout Server
 

tested auto_shorter feature enabled in 1.10.5 OSMSserver, work nice. very handy this option.
thanks @rinigus the-faster-dev.
test of route
shorter: 28km 30m
fast:::::: 35km 27m

Re: [Announce] Native offline maps: OSM Scout Server
 

auto_shorter was there already for Valhalla router. I just added a small check for libosmscout, so it wouldn't choke on it.

But this version of the server adds support for motorcycle and scooter. Again , work on Valhalla developers side . This should work with the next planet import.

Re: [Announce] Native offline maps: OSM Scout Server
 

############################################
###########################################
##########################################
##################post###################
#######moved#to#pure-maps#thread########
#####################################
###################################
#################################

Re: [Announce] Native offline maps: OSM Scout Server
 

wrong thread?

Re: [Announce] Native offline maps: OSM Scout Server
 

yes you right. apologize. m o v i n g . . .

Re: [Announce] Native offline maps: OSM Scout Server
 

Talking about routing preferences, most commercial solutions, among options like "prefer shortest", "prefer fastest", have options to "avoid toll roads" and "avoid motorways". What I am really missing in all of them is an option to "prefer major riads". So I often end up following the "fastest route" that takes me down some single-track, unpaved country lane, even though I remember from the times before GPS that there is a simpler route along major roads.

Is there such an option in OSM?

Re: [Announce] Native offline maps: OSM Scout Server
 

Nice option that would be.
But even nicer I do find the option between 'fastest' and 'shortest' maybe called 'optimal'?
Some car navigation systems offer this and calculate the route fast but keep in mind short-cuts that are shorter (but take more time).
So gor carlos example
fast 35km 27min
short 30km 30min
I would not drive 5km extra for saving 3 min.
But maybe there is route in between like
optimal 32km 29min?

--edit
have used the car mavi today and the option was
called 'efficient route' not optimal
it tries to save kilometers but still going fast

Re: [Announce] Native offline maps: OSM Scout Server
 

1) some countries(US) saving km is good to have the car in good value. example: if i wanna sell my car with many km, then my car is considered an old car with poor value.
2) fastest-route: in many countries with high traffic is not good, to much time wasted, because mostly people always use faster route.
then shortest-route is good option.
optimal-route could be nice option with online-traffic.

Re: [Announce] Native offline maps: OSM Scout Server
 

New maps are out there. Now, with the latest OSM Scout Server, motorcycle and motor scooter are supported by Valhalla router and map data.

Re: [Announce] Native offline maps: OSM Scout Server
 

Personal experience (using OSM Scout server with WhoGo/PureMaps) while driving during vacation (in Madeira. A *volcanic* Portuguese island, thus very steep) :

Yup, it works. When asking for "fastest route", it actually works this way.
Mostly because OpenStreetMaps has a very well filled databased with all necessary metadata tagged in.
So every path has the correct speed limitation in, and is correctly define (barely driveable unpaved country lane are correctly tagged so).
Thus Valhalla had enough data to determine it's better to stay on the main road and to take this kind of counterproductive useless short-cuts.

(At the same time, an up-to-date Tomtom, was having trouble, mostly because in such remote place, it only has edge between nodes on the map without any useful information, and thus barely driveable path seemed shorter on the map).


Now the big question : the FM radio drivers that currently exist on some smartphone :
- are they able to pickup FM RDS for TMC ?
- are they hackable to get DAB's TMC ? (I would suspect, at the cost of lots of CPU, if they work anything like desktop USB receivers, where "DAB"-mode is basically streaming the raw radio data for the CPU to decode in software).

Are there any good USB (optionally: -on-the-Go or Host) FM/DAB TMC receiver ? (like, e.g., the ones used by some Tomtoms and Garmins ?)
- Hum, that makes me think: Is should test what the Xperia USB-OTH is seeing when I plug in the Tomtom microUSB RDS receiver.

Are there any good online service for traffic informations ?

Yet another option would be the various FM USB sticks used on PC for software radio. (But guaranteed to make a big CPU hit).

Re: [Announce] Native offline maps: OSM Scout Server
 

Looks like the last import broke Mapnik's maps, with buildings missing. No idea how I missed it.All other backends should be fine.

Re: [Announce] Native offline maps: OSM Scout Server
 

Bug was induced by an update of GDAL library and tools used in Mapnik import. I reverted the update and reimported the planet. Import is uploaded now and, if all goes as planned, should up and running on @MartinK server tomorrow.

Re: [Announce] Native offline maps: OSM Scout Server
 

I have released a smaller OSM Scout Server update (1.11.0). As usual, translations have been updated, thank you for all involved!

There are two new features in geocoder.

First, as asked for by users of J1, it is possible now to disable libpostal parsing of the addresses. For that, go to Settings/GeocoderNLP and switch off "Use libpostal parser". Don't forget to keep "Use primitive parser" and keep in mind that, for primitive parser, you have to describe addresses as a hierarchy going from inner (house number) to outer layers (country). Its up to you and particular case to see which part of the address is needed, just keep the right order. I hope it will speed up things on J1. [it still uses libpostal, but for normalization only].

Second, you could search now for nearby objects of any type and any name (earlier, you had to specify at least one of them). In Pure/WhoGo Maps and modRana, for type, choose or type "Any" and keep the name blank. This allows you to check for any features on the map in the nearby of the reference point. Just keep radius sane since all objects within it will be loaded into RAM while performing the search.

Re: [Announce] Native offline maps: OSM Scout Server
 

@rinigus
this is a very special beg (so please feel free to ignore)

I have an exfat formatted SD card (I know, I know ;)) and (now) around 20G of maps data.
It works quite well with exfat-fuse drivers.

But after having increased the size of datasets from only Germany to Europe I experience long slowdowns at start and also crashes.

It is always for gz files of valhalla.
As I have seen lots of CPU (more than 100%) for harbour-osm-scout-server and mount.exfat I assume timeout of boost::filesystem actions.
If it is really time out would it be possible to extend that for me?
I know boost as a C++ library and found occurences only in mapnik*.so library. So maybe it is possible to check?

And as I fiddled with terminal I got also some other warnings of 'binding loop detected). Please see the screenshot (fingerterm sometimes kind of 'freezes' and I could not copy/paste to text).


--edit
this is triggered by opening the map manager:

Re: [Announce] Native offline maps: OSM Scout Server
 

I don't have whole Europe on my SD Card, so maybe its some scaling issue. Also, I use ext4 as a FS for SDCard.

Boost is used by Valhalla which is linked into the main program now. From the error messages, I don't see any timeouts, but some interrupted call. Why was it interrupted, I don't know.

Valhalla lib is in major rewrite right now by its devs. As a part of it, they plan to get rid of many dependencies, boost::filesystem included. Maybe that will help.

As for now, I suggest you to look realistically what do you want to use in the next 2 months. I suspect its not full EU. 2 months are from the fact that I am trying to update the maps once in 2 month period.

Re: [Announce] Native offline maps: OSM Scout Server
 

okay.

It was just a guess as I felt also slowdowns with bigger file operations. And could not think of any other reason as else exfat is stable.

I was used (times of Nokia maps) to have whole world on device. But at that times is was around 11G. Now with OpenMaps just Germany only is a few GBs. And I like to have a few countries I need to travel to always on device.

But I will (need to learn to) live with it ;)
Anyway, a real big thank you :)

Re: [Announce] Native offline maps: OSM Scout Server
 

Well, you could try ext4 as a FS. I don't know how efficient is the caching on Fuse-based FS and I would expect Valhalla to use it properly. Which makes a great use of free RAM, if its available.

When having many datasets, it could also slow down search for POIs and addresses, unless you choose to restrict search to one country / territory only.

Re: [Announce] Native offline maps: OSM Scout Server
 

Peter, have you tried keeping all maps but switching off using them all at once?
OSM Scout Server -> Settings -> Geocoder-NLP -> Search all available maps
When switched off, it enables a drop-down menu at the top of main OSM Scout Server's screen to select your currently used map.
Not sure if it makes any difference, though. The option's name suggests that it only applies for searching and I am sure rinivus would have mentioned it if he believed it was worth it, but... the map selection in the main screen does not mention any ifs or buts so I would try it :)

Re: [Announce] Native offline maps: OSM Scout Server
 

It applies to the geocoder only. Other backends (Valhalla and Mapbox ) will be still serving all datasets. Although, I would expect that Valhalla is influenced more than others due to its use of large number of files

Re: [Announce] Native offline maps: OSM Scout Server
 

Yep, unfortunately it seems only (or mostly) valhalla is affected.
I tried to set the search to one specific map only. But start time of scout server is a few minutes....

Re: [Announce] Native offline maps: OSM Scout Server
 

In the next few days, there will be an update in OSM Scout Server and its geocoder databases. This update will introduce some changes in the geocoder database format. As a result, it will be impossible to use new server with the old geocoder database and the old server with the new maps.

So, if you need new maps within a week or so, get them now. I will update the server at OpenRepos only after the new format has been pushed to the distribution servers. So, there maybe 12-hours gap when you cannot get new maps.

At present the changes are tested, consider it as an early warning.

Re: [Announce] Native offline maps: OSM Scout Server
 

New release is out: 1.12.0

In this release, long waited postal code support was introduced. As a result, geocoder database format has been extended and you have to update the datasets. Press "Check for updates" and proceed with the download. If datasets were up to date, only geocoder-nlp database will be updated.

Postal codes are imported from OSM and, for UK, from https://www.freemaptools.com/. Extra postal codes were reversed geocoded on import to figure out admin hierarchy of them (which part of UK they come from).

When searching with postal codes, you could either specify the code alone or together with some part of hierarchy. When postal code is specified together with address parts, its searching for records that have both - postal codes AND address - components in the record.

Postal codes should be detected by libpostal automatically. For some UK tests that I did, I could see that it was not working. This could be due to older training dataset or its incompleteness from the beginning. Unfortunately, libpostal development has been slow these days, but maybe, together with other users, we can fix few bugs and figure out what to do with the training data.

Primitive parser supports entry of postal codes using prefix "post:" . NB! IT'S CaSe SenSITIVE. So, for example:

post: AA BB CC , city

would be parsed with city as a part of admin and AA BB CC as a postal code. For primitive parser, postal codes can be entered anywhere in the string, just keep that part separated by commas. Documentation at https://rinigus.github.io/osmscout-server/en/ covers this as well.

In addition to postal codes, geocoder now returns phone number and website, if these data were at OSM. Pure Maps will be returning it in the next release, modRana has to be patched. All API is backward compatible, so it works already, just these components are not shown to the users.

I have optimized a bit further loading of libpostal for users of primitive parser only. Would be interesting to see, if @pichlo or others on J1 would get search results faster.

Translations have been updated - great work by all translators, thank you!

Re: [Announce] Native offline maps: OSM Scout Server
 

As mentioned yesterday, I published new Valhalla's datasets with public transport data. These are available now as an update of Valhalla's datasets via Map Manager of OSM Scout Server. The server does support it already, so no extra updates of the software is needed. Pure Maps supports public transport routing from the version 1.5.1. To add support for modRana, the costing option has to be added (see Pure Maps code for reference).

The data originates from https://transit.land/. Its a pain to download, took about 2 weeks of running one of Valhalla's programs - all due to slow server on their side. This project is one of the projects by Mapzen and, as far as I understand, is continued by volunteers. Hence the slow data feed, I presume.

Public transit is also not the priority of Valhalla's team at present, according to one of the issues in Valhalla's repo.

So, don't expect miracles, but you are welcome to test. We can still discuss issues over here and file them under Valhalla's repos. Its my first import with these data and its possible that something went wrong while downloading. So, some of the issues could be from the data import too.

From testing in Tallinn, it looks to be using busses only and ignoring other means of transport. But maybe its better in other places.

Enjoy!

Re: [Announce] Native offline maps: OSM Scout Server
 

Testing navigating with public transport here i Oslo. It always results i logg error: Exception in Valhalla: Cannot reach destination - too far from a transit stop. Just for you information. I don't expect you to prioritize this. It seems you work very hard too solve other issues.:)