Stars by Which To Navigate?

View Best in Class

Assumptions

This project, and those to follow, have operated under some key assumptions that merit discussion. They include 1) whether it is fair to judge all of the content-and-test frameworks included in this report as “standards”; 2) whether it’s appropriate to demand that standards be “evidence based”; and 3) why the call for “fewer, clearer, higher standards” is more complicated than it seems. We discuss each below.

Is it fair to judge Content-and-test frameworks as Standards?

Most of the documents reviewed in this report do not claim to be content standards per se (with the exception of CCSSI). They are “frameworks,” “guidance,” even “testing specifications.” But they stop short of representing themselves as standards. Is it legitimate, then, to appraise them as if they were standards?

Frankly, the lines between what standards claim to be, what they are, and how they’re used are often blurred. The PISA framework, for instance, has as its objective to test “not so much in terms of mastery of the school curriculum, but in terms of important knowledge and skills needed in adult life.” Yet it goes on to “define the content that students need to acquire, the processes that need to be performed and the contexts in which knowledge and skills are applied.” In other words, though it initially claims not to incorporate content standards, PISA later offers up the very content it professes to exclude, which now serves as an “implicit” (if not very good) set of standards.

In this way, the PISA framework has the same intention and performs essentially the same function as academic standards, and can reasonably be evaluated by the same criteria. Further, a number of American educators, policy makers, and standard setters look to PISA as a benchmark for what they should require and/or expect of American schools. (We think it’s no coincidence that the head of PISA has just been appointed to the CCSSI “validation committee,” along with his boss at the OECD.)

Much the same can be said for NAEP and TIMSS and the roles they play in American education generally and in standards setting and benchmarking particularly. Since all these frameworks get used for these purposes, it is important to appraise their content. Hence, we use this opportunity to evaluate the NAEP, PISA, TIMSS, and Common Core reading/writing/ communications and math frameworks as content (as well as skill) standards. Further, it’s long been observed that “what gets tested is what gets taught” so it’s more than reasonable to review assessment items (which we’ve done here when there were enough of them) and assessment frameworks in light of their likely impact at the classroom level.

That said, in the case of English language arts, reviewers confined their appraisals to the elements of ELA that the frameworks actually addressed. For instance, the Common Core standards are confined to reading, writing, and communications and do not attempt to encompass the full “content” of English language arts (e.g., literature and documents). Indeed, the drafters stipulate that their standards need to be accompanied by rich content-based curricula. Therefore, the draft Common Core standards are appraised as reading, writing, and communications standards, not ELA standards. The NAEP Reading and NAEP Writing Frameworks purport to be just what their names suggest—frameworks by which to design assessments of reading and writing performance—and are evaluated accordingly. Here, as in CCSSI, it will be very important for state officials to understand those limitations and to appreciate what else may be needed to generate complete standards and curricula for English language arts.

Is it appropriate to demand that Standards be “evidence based”?

Our second assumption is that expert judgment is a vital component of appraising academic standards. We respect the yearning for all standards to be “evidence-based” and the criteria used to assess standards in this report reflect that impulse. When real evidence is available, we ought to use it; ACT, for one, and researchers like William Schmidt at Michigan State have done much to expand the evidentiary base for standards development. Examining readiness for college via actual college work samples is also promising. But real evidence to date is quite limited. Perhaps that’s why CCSSI drafters cast such a wide evidentiary net.

Yet merely finding and naming a country that has similar content in its standards or locating similar content in another test or framework (e.g., the ACT or American Diploma Project) is not compelling evidence. In fact, most of these determinations were originally derived from expert opinion, preference, or survey results. They were not an attempt to “validate” the standard per se, inasmuch as that means that the standard is necessary to accomplish a higher standard or end goal. In other words, much of the cited evidence in the common standards is suggestive, not dispositive. Frankly, we’re not that optimistic that all academic standards can or must be “validated,” partly because we don’t define a good education strictly in terms of college-and-career readiness, and partly because true validation studies are hard to do well. (We observe, too, that the recently-named “Validation Committee” is light on card-carrying mathematicians and literature experts.)

Moreover, strict adherence to “evidence” may belie old fashioned common sense. Certainly the focus on “college and career readiness” is well-intended and legitimate as far as it goes. Students must be able to deal with the academic and occupational challenges that lie ahead. But young Americans must also be equipped to contribute in our democracy as citizen-participants.

In her 1987 seminal text, Democratic Education, Amy Gutmann wrote: “The democratic interpretation of equal education opportunity requires that all educable children learn enough to participate effectively in the democratic process.” An education, then, is not only about preparation for what follows immediately after high school, namely college and/or job, but also about preparation for citizenship, community life, and cultural participation. If we define the purpose of education in narrow, utilitarian terms, and then build standards atop that definition, we could end up with a population that is highly skilled yet ignorant, self-absorbed, amoral, and uncultured.

Why is the Call for “fewer, clearer, higher Standards” more Complicated than it Seems?

There’s been much talk about the need for “fewer, higher, and clearer” standards. At a glance, that’s a reasonable-sounding call to action. Among the failings of the “voluntary national standards” projects of the early 1990’s was their creators’ tendency to put into them everything but the kitchen sink, resulting in whopping fat volumes that nobody could realistically expect to use in actual schools. Yet “fewer” does not necessarily lead to clearer or higher; one might have a few standards that are vague and lack depth. Further, we’re inclined to believe that fewer, clearer, and higher standards work better in some disciplines than others. Math, for instance, is a single, cumulative discipline, quite unlike, say, English language arts, history, and science. A mathematics standard like “Compare and order whole numbers” is clear and explicit while a reading standard such as “Read and analyze both literary and non-literary texts” begs for clarification and explication. Perhaps it means that students need to evaluate the development of literary elements, or the effectiveness of rhetorical techniques, or the manipulation of stylistic devices? Perhaps they should describe the truth and/or validity of an argument or recognize and explain the presence of fallacious reasoning? Either way, it’s going to take more detail to transform that lofty but nebulous standard into an explicit one that’s actionable in the curriculum and the classroom.

• Tweet
• A A A RESIZE
• PRINT