Lumosity

Because some people at work are aware of my background and interests, I am sometimes asked about related subjects...

Hahahaha!  That is a complete lie.  The truth is that whenever a related subject comes up I insert myself in to the conversation and make pronouncements.  The people who bring up these topics are social workers, who have no scientific training and will fall for anything.  Not that they are any worse than anyone else in that.  People who do research seem to have trouble with the conceptual framework of what is evidence and what is trash, and they supposedly do have scientific training.

Thus, they don't want to hear that the idea they read about, or the speaker at the conference, or the program they are spending money on lacks any scientific support.  They prefer to be calmed by sciencey things the speaker or the book says.

Lumosity was mentioned this week, with the assurance that there was considerable scientific backing for this program of brain-improvement via brain exercises.  I will spare you the details of what was claimed versus what the website actually says, because I am uncertain how much of that is the result of the company's marketing and how much is my coworkers wishful thinking.  But let's take a look at what is there, and how much credence we will give to it.

First, it could be true.  This could turn out to be great, and the theory that certain types of brain exercise do generalise even though most don't is plausible.  We are indeed entering an era where IF it is possible to increase one's intelligence, we might be seeing the intriguing beginnings of it now.  I have nothing against these folks.  They may be right.

They are selling a product.  The first few sessions on the web are free, then you have to pay.  This does not in any way disprove their theory.  In fact, I consider it similar (or even superior?) to the supposedly disinterested parties who claim to be selflessly committed to an idea.  Those often have a worldview to protect, a need to appear noble, or a livelihood based on speaking fees they aren't counting as a similar financial interest.  If you are Dr. Horsefeather's kid, you might devote your life to defending Horsefeather's theories, even enduring poverty and ridicule, for reasons of your own.

Yet the fact that they stand to benefit should make us alert to a few things.  They cite independent research at reputable institutions as support.  Even if all of these studies were very strong, remember that they have no obligation to report those studies which should no result, or even harm from their product.  They put a half-dozen up on the page.  There could be twenty studies to the null hypothesis which you don't see. Or there could be none.  You don't know.

Next, you likely only partially know how research is done in this particular niche and what the standards are.  Plausibility is only one tool, and general reasoning only one more, in evaluating the claims.  Those ain't nuttin' but you may be missing something that a similar researcher would no is significant.  Certain usual measurements may not be mentioned, or problems getting a control group glossed over.

Further, people want this to be true.  Everyone would like to think that there are ways of getting smarter and you're one of the first one's to find it! One of the studies linked to sets out the problems and disappointments of this area of research over the decades quite well in the first four paragraphs.  (That study, BTW, is the best piece of evidence they show, to my thinking.)

This brings us to some items that are pretty strongly in their favor.  No, not the testimonials, or articles in the Best Newspapers, but the people they have been able to interest in this.  That they have gotten researchers from Stanford and Columbia and a few other colleges to be interested enough to sit on boards or attempt to research the usefulness is something in itself.  Heck, that they even seek research rather than avoiding it is something.

Yet we have to go back to some limitations of the studies which you might not notice if you weren't skeptical:  they are focused on Luminosity training for damaged individuals recovering or still developing,* there is not much about how permanent the gains are, and the results, though significant, are fairly modest.

*Which is great, and in fact more important than making me clever, but not the same thing that they are advertising.