A PISA for Higher Education: The OECD Disruption

In a deliberate and thoughtful article in The Upshot (a New York Times blog) Kevin Carey has once and for all defined the costs and consequences associated with the way we “rank” colleges in the United States and globally. Using first-time data from new OECD research called the Program for the International Assessment of Adult Competencies (PIAAC), Carey builds a careful analysis that exposes a glaring and dangerous problem with the way we determine quality in higher education.

Carey’s main point is that unlike K-12, where we are compared internationally on what our students actually know and are able to do (PISA Report from OECD), in higher education we rank colleges and universities based on perceived status. Whether it is the US News and World Report rankings, or President Obama himself in his State of the Union message, the reference to US higher education as “the best in the world” refers to our top institutions, not the entire enterprise. Carey goes on to point out that our relatively poor performance with regards to adult learners (up to age 29) has dire economic and social implications as we look towards an increasingly competitive global marketplace in the years ahead.

He is quite right. But this OECD disruption underscores another big problem as well. Yes, like the fearsome Wizard of Oz being unmasked as the little man behind the curtain, we are brought down to earth with this data. But stopping there brings up the next issue: the OECD research exposes the fact that we do not currently have good data or appropriate comparisons among institutional types and sectors that fully represent the diversity within our student populations and the extent to which we succeed with them.

The proposals for college rankings that we have seen either perpetuate the old “Wizard of Oz” approach or fail to assess college and learner performance by institutional category or learners’ risk factors to achieve an “apples to apples” comparison. In fact, they penalize institutions that serve marginalized and low-income learners — the very people with which we need to have more success.

That failure, in turn, leads to the deeper and more significant inability to know how effective our institutions are and what it will take to improve their effectiveness with the students they are electing to serve.

Instead of a reasoned discussion about improvement, we continue to be treated to a spectacle that more closely approximates a mud-wrestling match with multiple teams. Thank you OECD and Kevin Carey for framing the reasons why the discussion about quality in American higher education needs to be re-framed.