Stars by Which To Navigate?View Best in Class
This project, and those to follow, have operated under some key assumptions that merit discussion. They include 1) whether it is fair to judge all of the content-and-test frameworks included in this report as â€œstandardsâ€; 2) whether itâ€™s appropriate to demand that standards be â€œevidence basedâ€; and 3) why the call for â€œfewer, clearer, higher standardsâ€ is more complicated than it seems. We discuss each below.
Is it fair to judge Content-and-test frameworks as Standards?
Most of the documents reviewed in this report do not claim to be content standards per se (with the exception of CCSSI). They are â€œframeworks,â€ â€œguidance,â€ even â€œtesting specifications.â€ But they stop short of representing themselves as standards. Is it legitimate, then, to appraise them as if they were standards?
Frankly, the lines between what standards claim to be, what they are, and how theyâ€™re used are often blurred. The PISA framework, for instance, has as its objective to test â€œnot so much in terms of mastery of the school curriculum, but in terms of important knowledge and skills needed in adult life.â€ Yet it goes on to â€œdefine the content that students need to acquire, the processes that need to be performed and the contexts in which knowledge and skills are applied.â€ In other words, though it initially claims not to incorporate content standards, PISA later offers up the very content it professes to exclude, which now serves as an â€œimplicitâ€ (if not very good) set of standards.
In this way, the PISA framework has the same intention and performs essentially the same function as academic standards, and can reasonably be evaluated by the same criteria. Further, a number of American educators, policy makers, and standard setters look to PISA as a benchmark for what they should require and/or expect of American schools. (We think itâ€™s no coincidence that the head of PISA has just been appointed to the CCSSI â€œvalidation committee,â€ along with his boss at the OECD.)
Much the same can be said for NAEP and TIMSS and the roles they play in American education generally and in standards setting and benchmarking particularly. Since all these frameworks get used for these purposes, it is important to appraise their content. Hence, we use this opportunity to evaluate the NAEP, PISA, TIMSS, and Common Core reading/writing/ communications and math frameworks as content (as well as skill) standards. Further, itâ€™s long been observed that â€œwhat gets tested is what gets taughtâ€ so itâ€™s more than reasonable to review assessment items (which weâ€™ve done here when there were enough of them) and assessment frameworks in light of their likely impact at the classroom level.
That said, in the case of English language arts, reviewers confined their appraisals to the elements of ELA that the frameworks actually addressed. For instance, the Common Core standards are confined to reading, writing, and communications and do not attempt to encompass the full â€œcontentâ€ of English language arts (e.g., literature and documents). Indeed, the drafters stipulate that their standards need to be accompanied by rich content-based curricula. Therefore, the draft Common Core standards are appraised as reading, writing, and communications standards, not ELA standards. The NAEP Reading and NAEP Writing Frameworks purport to be just what their names suggestâ€”frameworks by which to design assessments of reading and writing performanceâ€”and are evaluated accordingly. Here, as in CCSSI, it will be very important for state officials to understand those limitations and to appreciate what else may be needed to generate complete standards and curricula for English language arts.
Is it appropriate to demand that Standards be â€œevidence basedâ€?
Our second assumption is that expert judgment is a vital component of appraising academic standards. We respect the yearning for all standards to be â€œevidence-basedâ€ and the criteria used to assess standards in this report reflect that impulse. When real evidence is available, we ought to use it; ACT, for one, and researchers like William Schmidt at Michigan State have done much to expand the evidentiary base for standards development. Examining readiness for college via actual college work samples is also promising. But real evidence to date is quite limited. Perhaps thatâ€™s why CCSSI drafters cast such a wide evidentiary net.
Yet merely finding and naming a country that has similar content in its standards or locating similar content in another test or framework (e.g., the ACT or American Diploma Project) is not compelling evidence. In fact, most of these determinations were originally derived from expert opinion, preference, or survey results. They were not an attempt to â€œvalidateâ€ the standard per se, inasmuch as that means that the standard is necessary to accomplish a higher standard or end goal. In other words, much of the cited evidence in the common standards is suggestive, not dispositive. Frankly, weâ€™re not that optimistic that all academic standards can or must be â€œvalidated,â€ partly because we donâ€™t define a good education strictly in terms of college-and-career readiness, and partly because true validation studies are hard to do well. (We observe, too, that the recently-named â€œValidation Committeeâ€ is light on card-carrying mathematicians and literature experts.)
Moreover, strict adherence to â€œevidenceâ€ may belie old fashioned common sense. Certainly the focus on â€œcollege and career readinessâ€ is well-intended and legitimate as far as it goes. Students must be able to deal with the academic and occupational challenges that lie ahead. But young Americans must also be equipped to contribute in our democracy as citizen-participants.
In her 1987 seminal text, Democratic Education, Amy Gutmann wrote: â€œThe democratic interpretation of equal education opportunity requires that all educable children learn enough to participate effectively in the democratic process.â€ An education, then, is not only about preparation for what follows immediately after high school, namely college and/or job, but also about preparation for citizenship, community life, and cultural participation. If we define the purpose of education in narrow, utilitarian terms, and then build standards atop that definition, we could end up with a population that is highly skilled yet ignorant, self-absorbed, amoral, and uncultured.
Why is the Call for â€œfewer, clearer, higher Standardsâ€ more Complicated than it Seems?
Thereâ€™s been much talk about the need for â€œfewer, higher, and clearerâ€ standards. At a glance, thatâ€™s a reasonable-sounding call to action. Among the failings of the â€œvoluntary national standardsâ€ projects of the early 1990â€™s was their creatorsâ€™ tendency to put into them everything but the kitchen sink, resulting in whopping fat volumes that nobody could realistically expect to use in actual schools. Yet â€œfewerâ€ does not necessarily lead to clearer or higher; one might have a few standards that are vague and lack depth. Further, weâ€™re inclined to believe that fewer, clearer, and higher standards work better in some disciplines than others. Math, for instance, is a single, cumulative discipline, quite unlike, say, English language arts, history, and science. A mathematics standard like â€œCompare and order whole numbersâ€ is clear and explicit while a reading standard such as â€œRead and analyze both literary and non-literary textsâ€ begs for clarification and explication. Perhaps it means that students need to evaluate the development of literary elements, or the effectiveness of rhetorical techniques, or the manipulation of stylistic devices? Perhaps they should describe the truth and/or validity of an argument or recognize and explain the presence of fallacious reasoning? Either way, itâ€™s going to take more detail to transform that lofty but nebulous standard into an explicit one thatâ€™s actionable in the curriculum and the classroom.
Read moreABOUT THIS REPORT
Our review ofthe NAEP Framework
Our review ofState Standards