Simple speed comparison between Pure Maps 1.19.0 and Poor Maps 0.34.3, both with OSM Scout Server 1.14.1 (all three installed from OpenRepos), OSMscout 1.9 (installed from the Jolla Store), OSMand 3.3.7 and Maps(.me) 9.0.8-2 (both installed from F-Droid) in offline mode (mind to explicitly set that in Pure Maps!).
Note that the map data for all apps are on encrypted SD-card, thus slower and making the apps more I/O-limited (WRT read bandwidth, but also access latency) than on unencrypted SD-card or internal eMMC (which is much faster than an unencrypted SD-card).

Test procedure:
1. Set OSM Scout Server's settings, if it is involved in the test.
2. Clean app caches per Mashka (from OpenRepos) rsp. Cache Cleaner (from F-Droid), except for the "warm" start-up time tests.
3. Restart GUI ("lipstick"; this also shuts down an automatically started OSM Scout Server running in the background and restarts AlienDalvik) rsp. only AlienDalvik (for Android apps), except for the "warm" start-up time tests.
4. Wait 40 seconds after the desktop ("home screen") is fully displayed rsp. restarting AlienDalvik was triggered (better a full minute on Jolla 1 phones).
5. Start GPSinfo and wait until a stable GPS fix (location icon in statusbar permanently lit and > 6 satellites "in use") is acquired, wait for 10 more seconds and let GPSinfo run throughout the test (alongside AlienDalvik which is started on boot-up).
6. For address lookup and routing speed tests, wait 30 seconds after the map is properly displayed, before initiating an address rsp. route search. A short (~ 12 km) and medium (~ 59 km) route were used in car mode.

The tests were carried out on an Xperia X with SFOS 3.0.2 and two Jolla 1 with SFOS 2.2.1, GPSinfo refresh rate set to 1 s, sufficiently free (and "balanced" for BTRFS) space on mass storage (i.e., partitions on eMMC and SD-card), at least three times each (outliers dropped and results averaged) with the primitive stopwatch function of the SailfishOS' Clock app (on a different device). A single offline map provider was used (no overlays etc.). When trying to roughly reproduce these results, mind to have "Allow application background services to start on bootup" switched off (the default) for all Android apps in the SFOS Settings app -> Apps.
No data for Maps on the Xperia X and "Jolla 1 new", because any map download fails consistently.

Note that the three phones are configured slightly different:
- The Xperia X uses couple of multi-Gigabyte country maps (for all apps, except OSM Scout Server using libosmscout, which technically uses only a single map), five languages for address parsing, plus EXT4 as filesystem and AES-128 with XTS for encrypting a Sandisk ultra 128 GB from 2017.
- The "Jolla 1 new" also uses couple of multi-Gigabyte country maps (except OSM Scout Server using libosmscout), five languages for address parsing, plus EXT4 as filesystem but AES-128 with CBC-ESSIV:SHA256 (specifically slows random accesses more than XTS) for encrypting another Sandisk ultra 128 GB from 2017.
- The "Jolla 1 old" uses a single multi-Gigabyte country map (except for Maps, for which only a few smaller maps were downloaded), one language for address parsing, plus BTRFS (slower than EXT4) as filesystem and AES-128 with XTS for encrypting a Sandisk ultra 32 GB from 2017.

Also note that most outliers were too fast, often due to missing to clean a cache, to restart something (e.g. OSM Scout Server, AlienDalvik) or to assure only offline sources were used.
Outliers, which were too slow, were mostly caused by background services of Android apps (missed to disable them all to be started on boot-up or having run Android apps since last reboot) or by not keeping aforementioned setting times (in steps 4., 5. and 6., above).

All values are in seconds; due to manually triggering the stopwatch and app starts, etc. there is at least two seconds variability (i.e., ± 2 s).


Start-up times on the Xperia X ("cold" / "warm" caches for Android apps and OSMscout rsp. "cold" / "cold" & OSM Scout Server still running / "warm" & running):
- Pure Maps: 9, 8, 7
- Poor Maps (libosmscout): 6, 5, 4
- OSMscout: 7, 6
- OSMand: 30, 25

Start-up times on the "Jolla 1 new":
- Pure Maps: 18, 9, 8
- Poor Maps (libosmscout): 18, 7, 4
- OSMscout: 45, 38
- OSMand: 36, 34

Start-up times on the "Jolla 1 old":
- Pure Maps: 20, 14, 12
- Poor Maps (libosmscout): 22, 12, 8
- OSMscout: 12, 8
- OSMand: 65, 50
- Maps: 40, 30

Note that all Android apps have a disadvantage under the Android 4 runtimes, because they have to use Davlik (ART was introduced with Android 5) etc.


Address lookup speed on the Xperia X ("cold" caches, OSM Scout Server running, address simply "Vorm Baum 6" pasted into the search field):
- Pure Maps (geocoder-nlp; results are only from a single, wrong country!): 14
- Pure Maps (libosmscout; mediocre results): 3
- Poor Maps (geocoder-nlp; same results from a single, wrong country!): 8
- Poor Maps (libosmscout; mediocre results): 2
- OSMscout (first result / all hits; few, but good results): 1, 2
- OSMand (first result / all hits, good results): 9, 23

Address lookup speed on the "Jolla 1 new":
- Pure Maps (geocoder-nlp; single, perfect result): 3 (first test series; single, mediocre result!?!: 15)
- Pure Maps (libosmscout): 2
- Poor Maps (geocoder-nlp; same results from a single, wrong country!): 126
- Poor Maps (geocoder-nlp; a different address, single, correct result): 60
- Poor Maps (libosmscout): 4
- OSMscout (no result!): 23
- OSMand (first result / all hits): 22, 175

Address lookup speed on the "Jolla 1 old":
- Pure Maps (geocoder-nlp; good results): 40
- Pure Maps (libosmscout): 6
- Poor Maps (geocoder-nlp; good results): 45
- Poor Maps (libosmscout): 5
- OSMscout (first result / all hits): 2, 3
- OSMand (first result / all hits): 12, 30
- Maps (first result / all hits): 30, 35

Observations:
- Some apps are much faster on subsequent searches for similar search terms, seemingly due to different caching strategies: While some apps seem to evaluate the the search history (needs location to be saved together with the search term, there), OSMand seems to cache former search suggestions (with their location) and to evaluate them first (or in parallel to querying the geocoder) on subsequent searches.
- Furthermore the usability of the "search while typing" function and the quality of the results do vary (e.g. their order/ "how far up is the intended location?").
- The search issue with geocoder-nlp on the Xperia appeared while using its settings "Search all available maps", "Use libpostal parser", "Use primitive parser", five languages selected and eleven maps in use. The "Jolla 1 new" uses exactly the same settings, languages and seven maps, while "Jolla 1 old" uses the same settings but only a single language and map, and both do not show this bug (but PoorMaps does!?!).


Routing speed on the Xperia X (~ 12 km / ~ 59 km):
- Pure Maps (Valhalla): 2, 7
- Pure Maps (libosmscout): 4, 14
- OSMand (also provides height profile): 26, 28

Routing speed on the "Jolla 1 new" (~ 12 km / ~ 59 km; numbers in brackets are for redrawing the map):
- Pure Maps (Valhalla): 2 (+ 1-2), 3 (+ 1-2)
- Pure Maps (libosmscout): 4 (+ 1-2), 5 (+ 1-2)
- Poor Maps (Valhalla; without address lookup): 26 (+ 2-5), 30 (+ 2-5)
- Poor Maps (libosmscout; without address lookup): 9 (+ 2-5), 21 (+ 2-5)
- OSMand: 84 (+ 3-7), 115 (+ 3-7)

Routing speed on the "Jolla 1 old" (~ 12 km / ~ 59 km, only a single map):
- Pure Maps (Valhalla): 26, 42
- Pure Maps (libosmscout): 13, 25
- OSMand: 22 (big variation), 125
- Maps: 8 (plus for a preceding map redraw: + 14), 20 (+ 8)

Observations:
- One cannot set OSM Scout Server as routing engine in Poor Maps anymore, but routing with it works, if it has been set before; hence no results for Poor Maps on the Xperia X and "Jolla 1 old".
- Routing with OSMscout 1.9 failed for me on all devices (with a symlinked data directory), emitting "Can't open routing service", but worked before (tested on the Jolla 1s a long time ago).
- The test series with the "Jolla 1 new" were carried out many weeks later, but with exactly the same software versions; still Pure Maps (and maybe also Poor Maps) seemed to behave a bit differently than the "Jolla 1 old"; a forgotten "Mixed mode" setting (instead of "Offline mode") in Pure Maps might partially explain this.


P.S.: Performing benchmarking manually in a reproducible manner turned out to take more proper preparation and was much more tedious than expected, but ultimately finished yesterday.