GPS tends to suffer from reflections, etc., which cause apparent displacements. Mapping apps which assume you're on a road can generally account for thes little hiccups.

Regarding altitude, once you have enough satellites altitude should be reasonably accurate (though not as accurate as the horizontal positioning due to the satellite orbits and the geometry of it all). Though note that the altitude is probably given with respect to a nominal geodetic/ellipsoidal shape rather than the actual sea level.