We pulled the beginning of behaviorally modern humans from over 2 million years ago down to something like 50,000 - perhaps even 10,000 years ago. But there are other objections to the Genesis account. It narrows the founding population to two people, Adam and Eve, for example. Sticklers will not there is something a little fuzzy around the edges even of that, as the story of Cain and Abel, their immediate descendants, pretty clearly suggests there are other people around. But two is the traditional number, so lets stay with that.
What is the National Geographic estimate on the other end? How many people were in our ancestral pool? As a purely scientific matter, that depends entirely on when you take the snapshot. But as an impression created to influence our perception, the number seems to be quite large. There were Neanderthals all over Europe, and the hominid population of Africa has high-range estimates of over 400,000 creatures. Let's not push what we don't know too far, however. These archaic humans lived in small hunter-gatherer bands, each covering considerable territory. Let us not task National Geographic with defending such a high number. Let's cut it in half, then half again: 100,000. That is comfortably in the range set by Sarah Tishkoff of University of Maryland (and Watson, and Macauley).
Well first off, we can chuck those Neanderthals right out anyway. Fascinating creatures who dwelt over a wide area, they have contributed precisely no genes to our DNA. None of them were ancestors of ours. So what is the last of the snapshots of population where we can find a tribe who supplied all our subsequent genes (exclusive of later mutations, of course)? How many people in that group?
The number has been trending downward for decades. When I was in school it was well into the thousands, in the 90's the first suggestions came it might be under a thousand. A few good estimates put it at 200-400 now.
The lowest, quite recent estimate is 160 souls. This may seem an impossibly small number to those raised on images of widely scattered tribes hunting saber-tooth tigers, but there are three reasons why it may be enough. First, it seems to be just enough mathematically if we are tracing back DNA. Second, if the behaviorally modern humans did have some sudden and serious advantage, they would pretty quickly out-compete all surrounding tribes.
Third, and to me most interesting, involves imagining the probable route of expansion out of Africa around 50,000 years ago. Between what is now Djibouti and Yemen is the Mandeb Strait, the narrow connector between The Gulf of Aden and the Red Sea. Now 17 miles wide, it was much narrower 50,000 years ago, and was likely dotted with islands. Once boats become common, we imagine hordes of people pouring out of Africa whenever there are population pressures. 160 people seems way too small.
But imagine what must have actually happened. The coastal peoples would have fished, navigated, gone to islands on a temporary basis, and perhaps even gone at regular times of the year. But once one tribe had succeeded in settling on the Yemen side permanently, exploiting the limited resources there, they would be an enormous obstacle to any further tribes establishing themselves. Within a generation, they would know the tricks - the poisons, the dangers, the sources of water and food. And they would be able to defend the area far better than another tribe could attack it. As they came under population pressures themselves, they would gradually move farther east along the coast (not north along the Red Sea, as it is even more of a desert there). In only a century, the territory of this tribe would extend so far along the coast as to prove an insuperable obstacle for anyone trying to leave Africa to find a place to settle. It wouldn't look like much today, only a few miles, but it would be more than enough to discourage leapfrogging past them.
That tribe settled the rest of the world.