There is an essay over at Quillette about the ACT. The author has been a university professor in both biology and psychology for decades, and has worked at an ACT prep center as a tutor, usually with individual students, often nonstandard. He has taken the ACT himself numerous times. It sounds ideal. Yet what it shows most clearly is how thoroughly assumptions and preconceptions affect the evidence before out eyes.
Students who take the ACT are asked to racially self-identify (not all do). These are the average composite scores for the five largest racial categories, followed (in parentheses) by the composite scores for those students who took four or more years of English and three or more years each of math, social studies, and natural sciences: Black = 16.3 (17.9), Hispanic/Latino = 18.3 (20.3), White = 21.7 (23.3), Asian = 24.9 (26.7), Two or More Races = 20.6 (22.5), and No Response = 19.2 (23.7). Clearly, taking a more rigorous high school curriculum helps everyone. (Italics mine. I stopped reading after that.)
No, you fool, how often do we have to go over this? The people who took the more rigorous high school curriculum were already smarter. They were already going to do better on the ACT. How much better? We don't know from this data. This data tells us precisely nothing about it. We do know from other sources how much prep classes help (Answer, it depends on what you are measuring, but it is mixed. Basically, if you are getting your first shot at a kid who has not figured out until recently that these tests are important, you can get some increase, teaching them the things that the competitive [whether against other children or against the test] children figured out years ago, usually gradually. Everyone else, they will show a natural improvement by being a year older.)
If that seems odd, remember that actual abstract reasoning does not start until around age 13, even in the brightest students. Children can follow logical reasoning and give some of it back before then, but don't start doing it on their own until then, and often unevenly. The brightest students may actually have the dumbest ideas, as they spread their wings earlier. So taking standardised tests at 16, 17, 18 is more different than one might think, Memorised vocabulary and what math or science has been taken can be force-fed, but doing analogies or figuring out what type of math is going to bring you to a solution involves real thinking. SAT/ACT prep courses make their basic living on the natural increase in scores for students that have three years of abstract thinking versus two under their belt. The classes take credit for that and convince the parents that the children did better mostly because of the expensive course they signed up for.
And as I said the good student who treated testing as a sideline her whole career who now sees that her PSAT projections aren't going to get her into the schools she wants is suddenly motivated to understand what is this test and how do I get more points. They can crash course and grab a goodly gain.
Once.
Which is also great word-of-mouth for the prep course. If they do good work with learning-disabled or decently intelligent but basically clueless students, I am glad to hear it. But be careful before shelling out big money for these. They work great in limited circumstances.
Yet here we are, with a clear expert making assertions that fail the basic logic tests that actually are part of some of the standard testing questions. And BTW, how is this person not getting perfects scores on these tests at this point? Are you kidding me? Only the occasional slip-up should be keeping them off the top scores now.
This person knows much more about the tests than I do at this point, much, much more. Yet he does not know some deeply important overall information about the tests. I think this is an example for a dozen other varieties of expert. The Russia team of the CIA in the 1980s knew incomparably more than I did about the USSR. If I had challenged one at a gathering (my parents did know one well who had retired to Wolfeboro, but no, I did not challenge him) I would have been quickly humiliated. Yet they had it badly wrong, not because they were stupid, or didn't work hard, or had bad training or even that they didn't think hard about the deep puzzles. They were wrong because of their assumptions, a hundred unquestioned assumptions they mostly shared.
I'm not sure it works that much better when experts disagree. I got to see mental health up close, and there was some tendency for the various schools of thought to double and triple down on pet ideas. I'm not sure experts changed their minds much. "Well, I was always taught that the first thing you ask a borderline is..." which carried the loaded implication that this is elementary, basic knowledge you fool, you fool. For years I said that the people who had experience and those who had education made equal, but different errors, and did not listen to each other well. But I just made that up from sitting at tables of people jockeying for position and watching what decisions got made. I guess it is still a good first approximation, but I like that summary less than I did in 1992.
The difficulty is that unlearning is much more difficult than learning. I don't know how we would teach children unlearning in any way that was not merely comical and artificial, having no bearing on future understanding.
7 comments:
"I don't know how we would teach children unlearning in any way that was not merely comical and artificial, having no bearing on future understanding."
Artificial is probably better than nothing. Detective stories, maybe? Some well-done ones have a character seem completely different when you know whodunit. A little history of scientists?
Now that I think of it: is there a personality type that finds it easier to "repent" of a mistake or ill-taught "fact"? mild-confirming-evidence.html
One would hope that researchers would be more open to walking back things they've erred on, and I have seen it, though not much outside their specialty. Inside the specialty--most of what I've seen has been collaboration work, and if you're obviously wrong...
Dorigo describes it well: https://www.science20.com/quantum_diaries_survivor/scientists_ye_pedantic_jerks
You might find his Revenge of the slimeballs fun.
Slightly off-topic, but my grandson got very frustrated with standardized testing last year when his school instituted the policy that all students would remain in the room until everyone had completed the test. It's not that he's slow, but rather meticulous and likes to check his answers. He did well until that policy when he felt pressured to not hold anyone up. Though all the answers are entered into their computers, they are given scratch paper. What he was doing was writing out all the answers (including essays) on paper, checking them over, and then entering them. I don't know the reason for that policy, but I'm pleased with the fact that it was reviewed and dropped.
More on-topic - my best advice to parents is to make sure the student gets a good night's sleep before the test, is fed, and not rushed. Second-best is that practice tests are good because they familiarize the student with the way the tests are constructed. That alone might be worth a prep course.
Technically, if the testing center adds in rules, it is not administering the test properly. So making everyone wait might be forbidden for that reason. Though I imagine schools will do it anyway just because. Their needs are paramount, doncha know.
As for spam, sometimes things get put in the spam folder wrongly, so I check it about once a week, and get to see the collection in a pure form. A few commenters, such as HMS Defiant, always get put there no matter how many times I mark it "not spam."
Is Quillette going to be bad for blood pressure? https://quillette.com/2022/07/18/cognitive-distortions/
The article's finding about Wikipedia isn't a surprise...
"
The study’s authors conclude that “[b]ecause our work goes beyond correlation to establish causation, we can conclude that Wikipedia doesn’t just reflect the state of the scientific literature, it helps shape it."
"
The life of a professional tutor is like Bill Murray's role in Groundhog Day. There is really no excuse for not being able to ace the test after your nth attempt. After a while, all the questions are the same.
The saving grace of being a tutor -- the only thing that breaks the loop -- is the joy of connecting with the students. Good or bad, and I'm mostly in my Father Flanagan mode here, each is a vitally interesting person.
You have to meet them where they start. Speaking in terms of standardized tests, weaker students have to learn to leverage simple rules and tactics to compensate for lack of subject knowledge. In the ACT for example, this might mean applying the "shorter is better" rule in English (70% correct) and spending 50 minutes out of 60 minutes in math doing problems 1-40, and only 10 minutes guessing on the remaining 20 questions. (The questions go easy to medium to hard and all count 1 point each with no penalty for guessing, so maximize the easy ones.) These strategies can usually increase an ACT score by 3 points (say 18 to 21) or the SAT by 150 points (850 to 1000) with a lot of work. Doable, but it takes tenacity. And you have to strike while the iron is hot.
Kids one standard deviation above the average (25 ACT, 1200 SAT) usually have the knowledge but often lack awareness of how the game is played. Once they grasp the formulaic nature of the questions (e.g., pedestrian rules of punctuation as opposed to elevated literary style), they too can often increase their scores by 5 / 200 points. They are in the sweet spot for the greatest percentage of improvement.
The two standard deviation high achievers often struggle to achieve. They start at 31 in the ACT or 1400 points SAT. It's hard to get above 34 / 1500. The higher you go, the harder it is to eke out improvements. They have great metacognition but need to work on consistency.
I am biased, of course, but I think that individual tutoring is far superior to class instruction. I saw the effects before I had ever heard of Bloom's two sigma problem.
I would estimate that 5-10% of the time a bright kid faithfully puts in the work and shows no improvement after repeated attempts. Frustrating, unfair, and mysterious.
It is true that over the last twenty years, kids read a lot less than previous generations, for all the obvious reasons. And even those who are so inclined lack the leisure to read for pleasure when they're taking 5 AP courses and loading up the resume. When I come across the rare students who read on their own, it's wonderful to watch them effortlessly dispose the verbal sections. They have a huge cognitive reserve, kind of like the aerobic capacity of Kenyans who win the Boston Marathon after a childhood of running 6 miles to and from school every day.
@ Galen - well put summary
Post a Comment