SAT Test Preppery

You may recall that one of the awesome parts of David Coleman’s shiny new SAT test was going to be how impervious it was to any kind of test prep . Coleman’s singular vision (because in Coleman’s world, he’s the only one with a vision) was a new SAT that would fix inequality in America . Because being able to afford expensive test prep would no longer matter.

Well, okay. Maybe it will matter a little.

After a few years of declaring that your Jedi test prep mind powers won’t work on the SAT, the College Board (which is totally a for-profit business and not some sort of service-oriented board of college representatives) has announced that test prep from their business partners at Khan Academy totally works ! Totally! As proven by their own in-house self-produced research.

Just a couple of thoughts.

1) In house self-produced research conducted by a business to show how well its product works is... dubious? How about some tobacco institute research showing that smoking is healthy. Also, why did the College Board only “study” the results of their own business partner without looking at any other test prep vendors.

2) Correlation and causation, for the love of God! College Board’s research shows a correlation between high scores and using Khan. That’s it. Was Khan just part of other strategies employed by the kinds of students who are so desperate to shine on the SAT that they do all the things, and are likely to score well anyway because they are smart, committed, and from the right background.

Here look at some of these charts of spurious correlations while I calm down. Meet you in the next paragraph.

Bob Schaeffer (Fairtest) in an email to Valerie Strauss at the Washington Post pretty well captures the dissonance of this moment in SAT history:

The College Board’s admission that SAT coaching can boost scores significantly once again demonstrates the hypocrisy of the testing industry. After six decades of aggressively claiming that SAT prep courses do not have a  major impact, the College Board has suddenly reversed its position. Of course, the program they now assert can make a big difference is the only one the College Board partners with. Not surprisingly, they did not study the offerings from any test-prep firm, many of which advertise even larger score gains.

But the College Board has apparently pushed back by reaching out to other media outlets. Famous Pretend Smart Guy Thomas Friedman let fly an eruption of smartitude in the New York Times ostensibly about owning your own future, talking about how we must all be limber and adaptable and flexible in the future, because it’s nobody’s fault but ours if we’re not ready for the future, which somehow led him to this non-sequitor:

Some are up for that: some not; and many want to but don’t know how, which is why the College Board has reshaped the PSAT and SAT exams to encourage lifelong learning.

1) You might think that Friedman goobered up by adding “exam” when the T in SAT already means test, but as it turns out, as of 1997, SAT doesn’t stand for anything. Make of that whatever you metaphorically wish.

2) How in the name of anything does the SAT encourage lifelong learning?

Here’s how Coleman explained the officially sanctioned test preppery to Friedman:

Students who took advantage of their PSAT results to launch their own free personalized improvement practice through Khan Academy advanced dramatically: 20 hours of practice was associated with an average 115-point increase from the PSAT to the SAT double the average gain among students who did not... Our aim is to transform the SAT into an invitation for students to own their future.”

The language suggests that Friedman concocted the whole piece just so he could plug Coleman’s Cash Cow. Because you may own your own future, but the College Board would like you to pay them for a chunk of it. And “own your own future” sounds uncomfortably like the current rhetoric that says, “Your health care, education, safety, housing and general well being are your own problem, so don’t come crying to anyone else when they take a bad turn.”

Understand I am a huge fan of internet-enhanced intelligence. The moment a question pops into my brain, I have pulled up a search and tracked down the answer. But I cannot imagine that living my whole professional or personal life on a foundation of reactive intelligence, of “flexibility” that is really just a tech-enhanced response specific to whatever problem I have been posed that can’t be good. Particularly when the problem is a standardized test with a shallow measure of isolated tasks. “Own your own future” makes a nice piece of PR fluffery, but it hardly seems like a real description of what the SAT can do. Particularly not if just twenty minutes with a canned prep program, a piece of software that literally teaches to the test, is the secret to doing well.