NAEP Framework

View Best in Class

Our review of the NAEP Framework

This report is part of our "Stars by Which to Navigate?" report.
REVIEW BY SUBJECT:

Introduction

The National Assessment of Educational Progress (NAEP) is known as “the nation’s report card.” NAEP mathematics exams are administered at grades four, eight, and twelve. The Mathematics Framework is seventy-four pages long and includes an overview of the exam, a description of the test design, sample exam items, and item formats, as well as a detailed list of content items—in essence, standards—for grades four, eight, and twelve. The standards are organized by strand (such as Measurement and Geometry), with grade levels placed side-by-side to show how they should develop from fourth to eighth to twelfth grade. Breakdowns are given of the percentage of exam items for each strand. Sample exam items appear throughout the framework to illustrate some of the standards and clarify aspects of the test design.

Review

The NAEP mathematics framework is evaluated on two dimensions: “content and rigor” and “clarity and specificity.” This review addresses these dimensions in several sections below. First, the framework’s content is compared with the content that should be covered by mathematics tests for fourth, eighth and twelfth graders. (See “Math Content-Specific Criteria,” page 11.) Separate sections describe the content covered by the NAEP framework for each grade level, as well as what is missing from and problematic about it. The next section sums up the content and rigor of the NAEP framework across grade levels. Then the review appraises the clarity and specificity of the NAEP framework as a whole, and the last section offers an overall summary and final grade.
In brief, most of the important content that we would expect students to learn is covered by the NAEP mathematics standards. But since the framework includes nearly 300 standards, it is impossible to distinguish the important standards from the less important ones. (For high school algebra, for instance, the heart of algebra—representing the abstract with symbols—is inappropriately deemphasized, while equal status is given to several non-symbolic approaches.) In addition, some important content is missing, and for other content that is included, inadequate detail is provided.

A majority of the standards also have minor or major problems with clarity—meaning they are awkward, verbose or imprecise, or they contain meaningless phrases or verbs that do not make clear mathematical sense.

Fourth Grade

Content Covered

Whole number arithmetic (and problem solving using whole number arithmetic) should make up the bulk of learning in mathematics in grades one through four. In the NAEP mathematics framework, there are standards that cover whole number arithmetic and its application to problem solving.

The measurement of lengths in inches and meters, and the computation of perimeters and areas of rectangles, are also covered, as is problem solving.Additionally, simple fraction and decimal comparison, including adding fractions with like denominators, is covered. Commutativity, associativity, and distributivity are presumably covered under the standard “Apply basic properties of operations,” and the inverse nature of addition and subtraction, and of multiplication and division, may also be covered by this standard, though there is no specific reference to these properties.

Estimation is covered and the basic vocabulary for geometry is introduced. Standards for elementary data analysis, statistics, and probability are presented shortly thereafter.

Content missing

There is no mention of single-digit addition and multiplication facts and the corresponding subtraction and division facts. While sample problems are given that use some of these facts, students need not know the more complicated single-digit facts in order to solve them. There is no explicit reference to the inverse nature of addition and subtraction or multiplication and division. Also, there is no mention of how precise measurements should be.

Eighth Grade

Content Covered

Mathematics in grades five through eight should thoroughly cover the arithmetic of rational numbers, including decimals, and also cover rates, ratios, proportions, and percentages. These represent the core content on which students should spend the most time. Arithmetic is covered by the NAEP framework under the standard that says, “Perform computations with rational numbers”; there are also a couple of supporting standards. Other standards cover rates, ratios, proportions, and percentages. Similar triangles, perimeters, volumes, circles, applications of the Pythagorean Theorem, and an introduction to the coordinate system are all present in the geometry section. Problem solving is included with this geometric content.

Additionally, basic data analysis, statistics, and probability are well covered. Absolute value and basic linear equations are also included.

Content missing

Ratios are covered inadequately by the vague standard “Use ratios to describe problem situations.” Problem solving with ratios is altogether missing, and there is no mention of the angles associated with a line cutting across parallel lines (e.g., vertical, corresponding, alternate interior/exterior angles, etc.).

Content problems

It is not unusual for frameworks or standards documents to include process or mathematical reasoning standards—standards that are not about math content, but about how to think mathematically. NAEP has such standards at the end of each content strand and they are often completely subjective. Nearly anything could be used to test them. For example, one standard reads, “Explain or justify a mathematical concept or relationship.” This nebulous admonition spans all of mathematics from kindergarten to the cutting edge of research.

There are also some mathematical misunderstandings, such as this one:

Interpret “=” as an equivalence between two expressions and use this interpretation to solve problems.

The language of this standard suggests that there are a variety of possible meanings for “=.” Yet the “=” sign is not up for interpretation—it actually means something.

Some of the many standards listed under data analysis, statistics, and probability (DASP) are descriptive or qualitative standards, not mathematical standards. (See “How much DASP do students need?” on page 12 for more about non-mathematical content in DASP standards.)
Consider this first example:

Given a graph or a set of data, determine whether information is represented effectively and appropriately.

This is not really a mathematics standard, as the answer is based on a value judgment rather than any mathematical foundation. Another standard reads as follows:

Visually choose the line that best fits given a scatterplot and informally explain the meaning of the line.

This is not mathematics, and a line chosen to fit a scatterplot in this manner is mathematically meaningless. Nor it is clear what is meant by “informally explain.”

Twelfth Grade

Content Covered

Most of the content covered by the grade twelve framework breaks up nicely into geometry and algebra. Proofs of traditional Euclidean theorems are generally included along with definitions and the necessary structure of geometry in the geometry section.

The core material requires a thorough understanding of linear and quadratic equations and functions. Both linear and quadratic equations come in multiple forms, and being able to algebraically (i.e., symbolically) translate between the various forms is important.

Most of the linear material is covered by the eighth-grade standards, but there is some new material in this standard:

Create and translate between different representations of algebraic expressions, equations, and inequalities (e.g., linear, quadratic, exponential, or trigonometric) using symbols, graphs, tables, diagrams, or written descriptions.

This would also seem to cover much of quadratics. Eight standards in all mention quadratics, including standards covering inequalities. An important part of the study of quadratic functions is understanding that the graph is symmetrical and learning to locate the maximum or minimum of the function. This opens up a whole new world of problems that can be solved and requires solid technical work that allows a student to transform the standard form of a quadratic function into its vertex form. The closest standard to this (albeit one with inadequate detail) reads as follows:

Recognize and analyze the general forms of linear, quadratic, rational, exponential, or trigonometric functions.

Roots and exponents are already covered in the eighth-grade framework. Logarithms, exponentials, and basic trigonometric functions are all covered. Probability and statistics, including combinations and permutations, are more than adequately covered.

Content missing

Among the content missing are geometric constructions. Explicit reference to the various forms of linear and quadratic equations, and translations between them, are also missing. Despite there being 120 standards for twelfth-grade mathematics (eight involving quadratics), significant details are missing, such as completing the square, the symmetry of the graph, and finding the maximum or minimum.

Content problems

The power of algebra is in its abstraction: the ability to reduce problems to symbols and solve them by manipulating the symbols. There are twenty-seven standards in the algebra strand for twelfth grade; however, algebra as abstraction, with the use of symbols, is not the emphasis. Consider again this twelfth-grade standard:

Create and translate between different representations of algebraic expressions, equations, and inequalities (e.g., linear, quadratic, exponential, or trigonometric) using symbols, graphs, tables, diagrams, or written descriptions.

The objects under discussion are symbolic: algebraic expressions, equations, and inequalities. Yet the symbolic approach to algebra is only one of five approaches (along with graphs, tables, etc). This de-emphasis of the symbolic misrepresents the nature of algebra.

Content and Rigor Conclusion

The content covered by the NAEP standards at the fourth-, eighth-, and twelfth-grade levels would normally earn the standards a score of six; coverage is good, though there are some missing items. There are significant flaws in both the eighth-grade andtwelfth-grade standards, but the primary problem with the NAEP framework is its failure to set priorities. With nearly 300 standard-style items, all presented as equal, there is no way for the reader or user to determine which are the most important. Minor standards appear as equal to major ones. This is particularly evident in the algebra strand, where the words and phrases tables, diagrams, graphs, and written descriptions appear fifty-seven times while the more important focus for algebra, the symbolic, only appears thirteen times. Secondary content overshadows primary content simply because the secondary content appears so much more often. According to the “Common Grading Metric” (see page 16), a set of standards cannot score above a five for content if they “do not distinguish between more important and less important content and skills.” Consequently, a score of five is indicated.

Clarity and Specificity

The structure of the NAEP framework is acceptable. It shows the progression of content from fourth through twelfth grade. It is organized by content strand. However, it has a number of problems with clarity and specificity, most pertaining to non-mathematical verbs and unclear language.

non-mathematicaL verbs

The NAEP framework is expressly designed to prepare students for the NAEP math exams, yet words like identify, interpret, represent, and recognize occur well over 100 times. These are not words that would be used in a math problem; indeed, examples from the NAEP exam never start with a request for the student to identify, interpret, represent, or recognize. These words are broadly misused in the standards. On the other hand, mathematical words such as compute and algorithm never appear and calculate occurs only twice.

unclear Language

A majority of the standards include awkward, verbose or imprecise, or meaningless phrases. For example, “Use ratios to describe problem situations” instead of “Use ratios to solve problems.” A grade four standard reads, “Explore properties of paths between points.” What this means for the fourth grade is not made clear. A twelfth-grade standard says, “Evaluate algebraic expressions including polynomials and rational expressions.”What does“evaluate x2”mean?One can evaluate the function at a given value for x, but “evaluate a polynomial” makes no sense mathematically. While the problems with unclear language in individual standards can be minor, the general effect is cumulative and quite negative.

Clarity and Specificity Conclusion

Although the standards offer some guidance about the content to be covered by the assessments, the lack of clear priorities is a serious problem for the standards. Students and teachers will not be able to determine which content is important. What’s more, the language used in a majority of the standards makes reading them difficult. The fact that the standards do not provide clear guidance to users results in a score of one for clarity and specificity.

Summary and Grade

The NAEP mathematics standards contain most of the important content we would expect students to learn, but it is hidden within the excessive number of standards (nearly 300). Less important material, such as data analysis, statistics, and probability, receives greater emphasis than more important material like symbolic algebra. Throughout the document, important content standards are bestowed the same status as secondary standards. Often standards containing crucial content are written in a way that also includes minor content, which makes it hard to identify core content.

Over half of the standards would benefit from either major or minor rewriting. Many standards include unnecessary verbiage, mathematically meaningless language, or just a lack of focus. The other major problem—the lack of clear guidance about which content is more important and which is less—is not easy to correct, especially if such a large number of standards are to be retained.

Math
Grade

C

  • Clarity and Specificity: 1/3
  • Content and Rigor: 5/7
  • Total Score: 6/10

Our Review OfNAEP Framework

Read moreABOUT THIS REPORT

DOCUMENTS REVIEWED

Our review ofState Standards