The leadoff hitters have been poor for the Red Sox this year, resulting in JD Drew leading off recently - a player one would not ordinarily think of in that spot.
I have no idea how that will work out, but who we think would "ordinarily" be in that spot is a function of the received wisdom of generations of baseball managers. They are mostly wrong. Before statisticians started working their slow changes on the game, the prototypical leadoff hitter was a middle infielder who had a moderately high batting average and stole bases. Baseball announcers were positively giddy about .270 hitters who danced around and "made things happen." Everyone seemed absolutely sure that hopping around disrupted a pitcher's attention so much that such players were worth their weight in gold. Luis Aparicio comes to mind. His year-by-year statistics are here, and reveal he should never have been anywhere near the leadoff spot, even in a pitchers era.
On Base Percentage has grudgingly become acknowledged as a more useful measure than Batting Average for leadoff and second hitters. Slugging Percentage is even more gradually becoming recognised as a better measure of power than home runs. The foundation of filling out a lineup card is, in fact, fairly straightforward. There are two numbers you need to know: On-Base Percentage and Slugging Percentage. The higher the player's OBP, the closer to the top of the order he goes. The higher his SP, the closer to the middle of the order he goes. As the middle of the order is between the 4th and 5th spot you have some flexibility there.
Because the high OBP's are sometimes also the high SP's, the player with the two highest added together should usually end up in the 3rd spot, with the compromises between OBP and SP spreading out from there.
Baseball traditionalists start going into cardiac arrest here. They say it's oversimplified, and there are a dozen other factors that have to be taken into consideration in deciding where a batter should hit. And that's true, as far as it goes. But the foundation on which you build should be OBP up, SP middle, not base-stealer up, guy-who-makes-contact second. You can illustrate it in half-a-dozen ways, via computer simulation, box score retrospectives, or experiment.
Watching baseball rather than studying it gives several false impressions. Stolen bases look very exciting, so we overestimate their value. Actually, they aren't worth that much. A player must succeed 70% of the time just to break even stealing bases (slightly less in pitchers parks, slightly more in hitters parks). This is because getting caught stealing is so enormously expensive that you have to steal a lot to make up for it. On a caught stealing, your team loses both a baserunner and an out. On a successful steal, the team only gains a base. Statistically, every 10 stolen bases over that 70% mark equals one additional run for your team. Not much.
10 walks, on the other hand, is worth almost 3 runs; a base-on-balls doesn't look very exciting, and so is undervalued. Luis Aparicio, above, drew very few walks and had a very good but not great base-stealing percentage. I am told his glove really was that good, so that going to the Hall of Fame isn't a joke. But for his bat - it's a joke. Phil Rizzuto? He's essentially Orlando Cabrera.
The whole concept of "rattling the pitcher" has some minor value, but it is much more apparent at lower levels of baseball. Little League pitchers definitely get rattled; Highschool pitchers might get unnerved. Greg Maddux, not so much. If you really like that sort of baseball strategy, you might also try yelling "Hey, batteh-batteh-batteh SWING batteh" at your next MLB game.
All those wonderful extras baseball managers are supposed to think of - not putting your slowest guy just before your fastest in the order, or making a guy lead off so he thinks about making contact rather than hitting homers, or that old favorite of hitting behind the runner - yeah, fine. Knock yourself out. But putting the players in order of OBP brings more batters to the plate over the course of a game.