Not just scale; also type of data.

If you're using Google Latitude, or Google Maps to navigate regularly, then if you're one of those people who regularly attend, say, anti-war protests, Google knows.

If you're using those tools, Google also knows if you regularly stop off at a particular brothel on your way home from the office. How often, and for how long.

Does it matter?

Hard to say.

The data is recorded. We don't know what future use of current data will be.