New and Improved

Each semester I spend a day in my Public Speaking class talking about critical thinking through the lens of advertising claims.  I point out that it is against the law for advertisers to lie, but that they are allowed to make claims that lead us to make incorrect assumptions. 

What advertisers don’t say may be far more meaningful and significant than what they do say.  Back in the 1970s the now-defunct National Airlines was a leading carrier to Florida, and ran ads touting the fact that no other airline flew to Miami/Fort Lauderdale for less than National.  What they didn’t say was that all fares were the same, because the Federal government regulated airline fares. When Folgers coffee claims that it’s “mountain grown,” what isn’t said is that all coffee is in some sense mountain grown (Arabica beans are grown at higher elevations than Robusta beans).  When products like shampoo and toothpaste highlight some special ingredient, what they fail to mention is that their competitors have the same ingredient.

Evaluating advertising claims requires knowing the right question to ask.  When Trident chewing gum reports that 4 out of 5 dentists surveyed recommend sugarless gum (for their patients who chew gum), the logical question is “What would I expect the results to be?”  I would expect 100% of dentists to prefer sugarless, making the statistic not that impressive.  When an actor washing his hair with ten times as much shampoo as needed claims to know that Denorex is working because it tingles, the relevant question is “Do you need your shampoo to tingle?”  When products are “new and improved,” the question becomes, "Is that really an acknowledgement that the old product was flawed?"

I thought about that upon learning a couple of weeks ago that there will be a “new and improved” SAT beginning in 2016.  Within minutes of the announcement, I was being asked what I thought by parents and colleagues, and my answer is that I’m not sure.  I like a lot of what I’ve heard.  I like the idea that the test might be more closely aligned with what students study in high school.  I like abandoning the 2400 scale, because it is still odd every time I hear someone talk about having gotten a 1950 on the test. I like the fact that the SAT will become less a test of stamina now that the Writing section will be optional, and being able to guess your way to success without penalty seems to reflect some back corner of the American Dream.  But I am also by nature a skeptic, especially when it comes to the College Board, and there are still a number of questions to be answered.

Foremost among those questions is whether the changes are motivated by philosophy or economics.  I would like to believe that the changes were in response to the thoughtful work done by the NACAC Commission on Testing, but that seems unlikely given how quickly the College Board was to dismiss and ignore the Commission report.

There is an ongoing internal battle for the soul of the College Board—membership organization or corporate entity, .org or .com?  In recent years the corporate forces seem to be winning the war. 

College Board meetings too often feel like infomercials for College Board products, and when I attended the business meeting at a National Forum several years ago, the treasurer reported a “non-profit” of $95 million, a figure that many businesses would envy.  That wealth, much of it generated by SAT fees, allows the College Board to do a lot of good, but it also leads to suspicion that most College Board decisions are based on how they impact the bottom line.   Is the new test an attempt to better measure the skills and knowledge students need in the 21^st century, a reaction to the fact that the SAT has lost market share to the ACT, or a strategic first step to position the College Board to become the primary provider of assessments of the new Common Core standards (partly written and developed by new College Board President David Coleman)?

Those questions are particularly relevant given the fact that the “new and improved” SAT reverses the miracle ingredient from the last iteration of the SAT, the Writing section.  At the time, the addition of the Writing section seemed designed to keep the University of California system as clients, and almost immediately critics such as MIT’s Les Perelman argued that the 25-minute essay and prompts were lousy measures of a student’s ability to write, especially when the scoring rubric did not penalize a student for writing that the War of 1812 started in 1944.

The other significant piece to the recent announcement is the College Board’s collaboration with Khan Academy to produce free test prep materials.  That seems like a good attempt to reestablish the SAT as a tool for college access rather than a test that measures and rewards privilege.  I have always been a skeptic when it comes to the test-prep industry, believing that it is one of the marketing success stories of our time and that the benefits of test prep are far more modest than generally assumed (but that may be the naïve dinosaur within me speaking), and it bothered me that several media reports about the new SAT took for granted that scoring well on the SAT is purely a function of test prep rather than economic advantage.  Whatever the reality, it hurts the credibility of the College Board and the college admissions profession if it is possible, or perceived to be possible, to game the test and the admissions process.

That leads to the final question.  What does the SAT measure, and is it measuring the right things?  It was originally designed to predict freshman year performance in conjunction with high school grades.  Grade inflation makes such a tool necessary, but studies such as that recently done by former Bates Dean of Admission Bill Hiss on the long-term impact of test optional policies raise questions about how much added predictive value SAT scores provide.   More importantly, neither the SAT, ACT, or any other test adequately measure personal qualities such as motivation, work ethic, and “grit” that may be the best predictors of success.  A test that could measure those would truly qualify as “new and improved.”