Notices

I recently had another problem, finding "gas station" POIs in PoorMap, using OSM Scout Server as data source.

The problem is, that in OSM data "gas stations" have the tag name "amenity : fuel" so the search text "gas station" is not matching.

More details at: https://github.com/otsaloma/poor-maps/issues/59

@rinigus:
Do you think, this is a problem which could be and should be fixed by OSM Scout Server (or Geocoder-NLP)? If yes, I would create a separate Github issue.

Sure, its not a problem of Poor Maps, since Poor Maps just asks for data from data provider. In this case, it will run down to geocoder-nlp - please add the issue there.

Just a short update - I am preparing an update to the maps. At present, the last component - Mapbox GL tiles - is generated and when ready I will distribute them. This would bring the data to the level of OSM just before Christmas.

With the addition of Mapbox GL the imports do take significantly longer and I probably would have to optimize it further. Let's see after that whether once a month reimports are feasible or we have to move to once in two months schedule.

I have reworked the styles for Mapbox GL and, if there is interest, could try to look into how to make editing of them easier. That's in case if some of you would like to participate in making the styles.

As for timeline, the map import will probably take few nights, maybe a week.

All maps are imported and I am going to upload the maps this weekend. Since Mapbox GL tile schema has changed, I changed the version of the tiles as well. The new Mapbox GL maps will be compatible with the next version of OSM Scout Server (planned to be released this weekend as well), not the current one.

So, there could be some disturbance in the service this weekend due to non-perfect sync with software and data publishing.

New version of OSM Scout Server (1.4.0) and updated maps are out.

For those using Jolla Harbour, you can start using new maps only after Jolla Harbour version is updated as well.

Main work was done not on the server side itself, but on optimization of Mapbox GL tiles import, adjustments of the tile schema, and generation of map styles. On server side, I mainly updated documentation by adding sections on running the server and what does automatic activation means in terms of file generations at the home directory (asked during Jolla Harbour review). Thanks to the translators for updated strings, I included the version available this morning.

Now to give you an idea of what's going on with the implementation and import of the tiles, I describe it a bit below. While we use Mapbox GL Native as a library, lots of work done by Mapbox on styling is not available for use in external projects (when you want to use it independently from Mapbox service). Which is fine since they do have to get income stream as well, but requires for our project to make new styles and generate tiles. Fortunately, OpenMapTiles project has been working on tiles import and schema, but there are several problems with them which I have been fixing during several months:

* Very long time required to import - 20+ days (24h) of server time
* Problems with POI data (icons were frequently missing and its hard to debug)
* Several other problems as well, some of them I don't even recall now.

Now the import time was a major issue. Import works by first pushing all data from PBF (Planet.PBF) to PostGIS database and later generation of tiles via Mapnik and tilelive-copy. Pushing data into PostGIS takes about 8 hours of intensive CPU use and later ~12+ hours by some threads to reorganize the database. As such, this part is not a problem and has rather reasonable timeframe. Now, when I used OpenMapTiles scripts, there about 20 days of import were estimated.

By default, as usually used, the tilelive-copy (main import command generating tiles) doesn't use all available CPUs for several reasons. Since for OSM Scout Server we need tiles split into smaller databases covering parts of the World, it was simple to make parallel imports and write Makefiles that pushed the server to full usage.

Next, I had to optimize queries, pre-generate as many of them as possible to avoid repetition induced by the same queries run for tiles at different zoom levels, and PostGIS data storage layout.

All these optimizations resulted in dropping tile generation time (after PostGIS import) to about 4*24h, probably a bit less. I expect to get it somewhat faster after realizing that probably 1-2 days were slowed down significantly by huge ice polygons in Alaska, Canada, and Greenland. One way to reduce the polygons would be to run as many imports as possible leading to the increase in overall ambient temperature in these regions, but a faster way would be to split the polygins into the smaller ones. Splitting of the polygons will be tested the next time. My scripts are available at https://github.com/rinigus/mapbox-gl-importer .


In addition to imports, the styles have been reworked. For Mapbox GL, styles are described using a single JSON file which defines all the layers one-by-one. There are nice GUI utilities to work with the style file, but they all suffer from rather large amount of repetitive work. Namely, many layers use the same colors, similar logic, and so on. As a result, there is a lot of point-and-click or emacs/vi replace if you want to make different colorscheme (light and dark), for example.

I have setup a project for the styles used by OSM Scout Server that tries to address some of the repetitive work in describing the styles and modifying them: https://github.com/rinigus/mapbox-gl-styles . The styles are described by collection of layers (https://github.com/rinigus/mapbox-gl...les/src/layers) that are used by all styles, style-specific variables (example for https://github.com/rinigus/mapbox-gl...variables.less) allowing to use LESS color transformations, and the list of layers used by specific style. Thus, for example, style with English names is made by trivial modification of variables file (but that's easy to do with emacs/vi as well by running replace on JSON) and dark style is made by changing colors only.

In future, when Mapbox GL Native will start supporting expressions better, it will be possible to describe style layers with less repetitions than its done now, but that would take some time till Mapbox GL Native will get it implemented.

If you wish to edit styles for OSM Scout Server, let me know. On significant interest, I will try to address the simplicity of visual feedback and providing tiles. At present, I am using an editor, OSM Scout Server on Linux PC, and Mapbox GL QML widget based viewer on Linux. Its surely possible to replace the viewer with web-based one (Mapbox GL JS) and probably make it easier to get tiles for testing.

At present, I included osmbright and osmbright-en as OSM Scout Server styles, with the dark styles expected in the future releases. Plan is to get to parity with Mapnik styles.

Hmm, that turned out to be rather long description of what has been done. Sorry

Enjoy using the server and updated maps.

At first, thanks again for your continued work on osmscout server and the surrounding infrastructure.

I updated to the latest 1.4.0 and downloaded the new maps for germany on my tablet. Unfortunately I have a Problem using it together with Poor Maps GL. In higher zoom levels all text on the maps (like streetnames and house numbers) and POI icons are somehow missing. with 1.3.0 and fitting maps it was working fine.

Does anyone else see this problem or did I miss something?

same issues, me too. xperia x 5121

Just got reports regarding Spain (https://github.com/rinigus/osmscout-server/issues/233). I'll try to download Spain/Germany and see if its the same for me.

However, try to close Poor Maps GL and delete its cache via

and try again. I suspect that maybe caching could influence it. Please let me know if it helped.

Edit Just tried with Barcelona and all seems to be fine. Just keep zooming in and you should get street names. Although, most probably cache cleaning is needed for it (I was doing it routinely)

For Spain it just got confirmed - clearing cache helped and all works as expected.

So, please remove your application cache when Mapbox GL schema is changed. Right now it applies to Poor Maps GL and Laufhelden

Clearing the Poor Maps GL cache helped. It's now working as expected.