Werner Muensterberger, a psychiatrist who wrote about collecting and collectors, noticed that those fascinated by maps often came from broken homes or moved around a great deal as children. It makes an intuitive sense, that such children would be attracted to the stable world that maps seem to present. I love maps, I came from a broken home, and while we did not move around much, I did go to three separate first grades (three houses, three towns, three states) in the year of my parents' divorce. For this purpose, it is important to note that the pleasure I get from maps feels very much like what is described.
I learned about this observation from Ken Jennings' (all-time Jeopardy! champ guy. You've heard of him) book Maphead, which covers many things map-related, and covers them well: geography bees, geocaching, history of maps, wayfinding. Recommended.