Rating Algebra Textbooks Paper presented at the annual meeting of the National Council of Teachers of Mathematics, Chicago, April 15, 2000.   Gerald Kulm Curtis D. Robert Professor Texas A&M University

This paper is based on work at AAAS/Project 2061, funded by a grant from the Carnegie Corporation of New York. The contributions of my AAAS colleagues Kathleen Morris and Laura Grier are gratefully acknowledged.  All opinions and conclusions contained in the paper, however, are solely the author’s and not those of the funder, AAAS, or other persons.   

Algebra has become a central focus of reform in the mathematics curriculum partly because it has often been viewed as a gate keeper for the further study of mathematics (Phillips & Lappan, 1998).  One notable and sometimes controversial outcome of mathematics education reform has been a move by many states and school districts to require algebra of all students.  This move presents a challenge to mathematics educators both in defining what we mean by “algebra” and in considering how algebra can be taught meaningfully and sensibly to a broader population of students.

According to Kaput (1999), the traditional image of algebra is about the rules of manipulating symbols, simplifying expressions, solving equations, and artificial application problems.  He goes on to recommend that new algebra that would be appropriate for all students and allow for learning with understanding, would have the following characteristics:

·        Begin early, building on informal knowledge

·        Integrate algebra learning with the learning of other subjects

·        Build on students’ natural linguistic and cognitive abilities

·        Encourage active learning

Another indicator of a change in the view of what algebra is, as well as expectations for what all students should learn can be found in NCTM Standards 2000.  The original math standards document (NCTM, 1989) provided somewhat general statements about algebra, mainly focusing on the high school grade levels.  The new Principles and Standards (NCTM, 2000) includes algebra at each of the four grade ranges and expands algebra to include patterns and functions, symbolic forms, and mathematical models.  Specific and detailed statements of expected student learning are stated for each of these three areas.  We can see these features by comparing statements in the two documents:

NCTM Standards (1989)

·        Apply algebraic methods to solve a variety of real-world and mathematical problems. (Standard 9-6, grades 5-8, #6)

NCTM Standards 2000 (Draft, October 1998)

·        Represent and investigate how a change in one variable relates to the change in a second variable. (Standard 2, grades 3-5)

·        Use symbolic algebra to represent situations and solve problems, especially those that involve linear relationships. (Standard 2, grades 6-8)

·        Represent situations that involve variable quantities with expressions, equations, inequalities, and systems of equations using a variety of equivalent forms. (Standard 2, grades 9-12)

The challenge of teaching a new kind of algebra with understanding for all students requires a careful look at the tools that mathematics teachers use; that is, the algebra textbooks.  How well are algebra textbooks aligned with the content of Standards 2000?  How well do textbooks reflect the kind of instruction that is necessary if all students are to learn and understand algebra?  The analysis and rating of current algebra textbooks, carried out by Project 2061 of the American Association for the Advancement of Science was aimed at answering these questions.

The Project 2061 Analysis

The analysis and rating of textbooks is an intensive effort that requires trained analysts in order to achieve consistency in ratings.  The process also requires careful selection of mathematics content for checking alignment, and close attention to important instructional quality criteria.  The following steps took place in the analysis of algebra textbooks during the summer and fall of 1999.

Textbook Selection

The first step was to select the textbooks.  Since the analysis is time-consuming and expensive, we were able to select only 12 algebra textbooks.  It was decided to select several of the “best selling” textbooks that are widely adopted and used in schools, a few recently published series whose development had been funded by NSF, and one or two books that were unique in their use of technology and applications.  With these criteria in mind, the following textbooks were selected for review, listed in alphabetical order: 

Algebra Learning Goals

The procedure examines content coverage in detail for a selected set of mathematics learning goals or standards.  Algebra goals from the 1998 draft of NCTM Principles and Standards for School Mathematics were selected, then reviewed by mathematicians and mathematics educators for their importance and priority in first year algebra.  We wanted to select algebra content that ought to be addressed by most algebra books, while also reflecting recent ideas about algebra for all students.  The following learning goals were selected as criteria:

Each textbook was examined in detail, noting the page number and description of all activities, explanations, examples, exercises, assessments, teacher notes or any other material that aligned substantively with any of the ideas or skills that are contained in each of these standards.  These lists of “sightings” of the standards ideas were placed in a database for the use by analysts who rated the instructional quality.  It is important to note that instructional quality was rated only for the content that specifically addressed one of these standards.

Rating Instructional Quality

The criteria for judging the instructional quality of mathematics textbooks uses a set of 24 criteria grouped into several key categories.  The criteria are drawn from the results of research and best practice in mathematics teaching and learning (see Appendix B in AAAS, 2000). The following table lists the instructional criteria that were applied to obtain the ratings.

The procedure for rating instructional quality has been developed, refined, and applied by Project 2061 over several years, achieving a high level of rater consistency (cf. Kulm, 1999; Kulm, Roseman, & Treistman, 1999; Bush, Kulm, & Surati, 2000).  In the algebra textbook analysis, we applied the same methods used in the analysis of middle grades mathematics textbooks.  That is, trained independent teams of high school mathematics teachers, mathematics educators, and mathematicians rated the textbooks on each of the 24 instructional criteria.  The teams met to reconcile the ratings, producing high levels of agreement.  Project 2061 staff used the ratings, justifications, and comments by the teams to produce summaries and highlights of the ratings for each textbook.

Results of the Analysis

The data from the analysts were examined and summarized both with respect to how well the texts addressed the three sets of algebraic ideas and on their instructional quality as represented by the six categories. 

Content Results

Content results are limited in this analysis to the three sets of ideas related to Representing and Modeling Functions, Representing Variable Quantities, and Operating with Symbols and Equations.  For these three standards-based ideas sets, all of the textbooks were reasonably well aligned in their content, providing appropriate coverage of the topics and ideas.

For Representing and Modeling Functions, the sightings of algebra activities revealed that a majority of the textbooks provide a sufficient number, variety, and depth of activities.  Some books used functions as a central theme throughout; others used equations to represent and model quantitative situations, introducing function concepts later.  In both cases, there were many opportunities to represent, translate among verbal rules, data, graphs and equations, and to interpret representations and model.  There were variations in the types of functions that were introduced and developed.  Some texts included recursive functions, and others such as step functions or probability functions.  All of the books do cover linear, quadratic, and exponential functions.  Some books explicitly mention and develop the concept of “modeling.”  Most of them, however, use the ideas of modeling without making it explicit or clear what the modeling process involves.

In their coverage of Representing Variable Quantities, all of the books use verbal rules, graphs, and symbolic expressions to represent a variety of applications and quantitative situations.  A great deal of coverage is devoted to translating among these representations and interpreting their meanings.  All of the books include linear and quadratic equations, inequalities, and systems of equations to represent situations and problems.

The books also provide appropriate coverage and practice on Operating with Symbols and Equations.  Some books introduce expressions, then linear equations, followed by more complex equations and polynomials; others introduce linear and nonlinear equations early in the context of applications.  The integrated materials use problem situations to motivate, present, and provide practice with these operations, whereas the more traditional texts demonstrate the procedures and then provide examples and practice.

Instructional Quality Results

Instructional quality ratings were determined by averaging each team’s score on each of the 24 criteria.  A 5 point scale (Poor, Fair, Satisfactory, Very Good, Excellent) is used to reflect the textbook’s strength for each of the criteria. 

Table 1. Summary of Instructional Analysis of Algebra Textbooks

The textbooks are strongest in engaging students in mathematics; all 12 books were rated at least satisfactory in this category and a few were rated very good.  In all of the books, a variety of contexts and applications are used that provide first-hand or meaningful experiences with the algebra standards ideas.  All but one or two of the books develop mathematical ideas in a satisfactory manner, justifying ideas, introducing and representing terms and demonstrating procedures, and providing appropriate practice.  For Category VI, assessment, results are mixed.  Most of the textbooks align assessment with standards and use applications in their assessments of student progress.  None of the books, however, use assessment results satisfactorily to promote better learning.

All of the algebra textbooks have some serious deficiencies.  Only a few of the books provide a sense of purpose either in introducing chapters and units or in conveying to students the purpose of a lesson.  All textbooks are weak in Category II, failing to build on student ideas about mathematics.  They do not provide or point to prerequisite knowledge, they seldom alert the teacher to misconceptions, or assist the teacher in dealing with conceptual difficulties that students might have.  Only two of the textbooks do a satisfactory job in Category V, promoting student thinking and reasoning about ideas and procedures in algebra.  Students are seldom asked to explain their reasoning or to think and reflect about what they have learned, connecting it with previously learned ideas.

Instructional Quality in Specific Content Strands

There are variations in instructional quality across the three sets of algebra ideas.  Overall, the textbooks did the best job of addressing the ideas of Representing and Modeling Functions.  Nine of the 12 books reached overall satisfactory average ratings on this content.  Five of the books were rated satisfactory for their instruction on the ideas of Operating with Symbols and Equations.  However, five books had very low ratings on instructional quality in this content area.  The poorest ratings overall were on the content of Representing Variable Quantities.  Here, only one books received satisfactory ratings, while the others had many instructional deficiencies. On the Representing Variable Quantities, the poor ratings for the books were primarily due to taking a mainly procedural approach, neglecting to build on student ideas and encourage students to think about and explain representations they learn and use.

Summary

All of the books are reasonably aligned with the standards-based ideas that were used in the analysis.  They developed the ideas of functions, representation of variable quantities, and operating with equations with appropriate focus and depth.  However, only seven of the twelve texts received overall satisfactory ratings on instructional quality.  The most serious deficiencies, even among some of the higher rated books, was a failure to build on students’ ideas in mathematics and encourage students to think and reason about what they learn.  Overall, the texts do a good job of engaging students in interesting problems and presenting algebra through demonstrating procedures and providing practice.

How well do these textbooks support the reform ideas suggested by Kaput (1999)?  Not very well.  They do encourage active learning.  However, few of them appear to build on students’ informal knowledge.  Although there are many applications of quantitative situations, there is little evidence that the books integrate algebra learning with the learning of other subjects.  Finally the lack of opportunity for student thinking and reasoning illustrates a failure to build on students’ natural linguistic and cognitive abilities.  The bottom line is that although some of these algebra textbooks have good potential for helping students learn, there is a great deal of room for improvement.

References

American Association for the Advancement of Science. (2000).  Middle grades mathematics textbooks: A benchmarks-based evaluation. Washington, DC: American Association for the Advancement of Science.

Bush, W., Kulm, G., & Surati, D.  (2000).  Preparing teachers for mathematics textbook selection.  Journal of Staff Development, 21(2), 34-38.

Kaput, J. (1999).  Teaching and learning a new algebra.  In E. Fennema and T. Romberg (Eds.),  Mathematics classrooms that promote understanding.  Mahwah, NJ: Lawrence Erlbaum.

Kulm, G. (1999).  Making sure your mathematics curriculum meets standards.  Mathematics Teaching in the Middle School, 4(8), 536-541.

Kulm, G., Roseman, J. E., and Treistman, M. (1999). A benchmarks-based approach to textbook evaluation. Science Books & Films, 35(4), 147-153.

National Council of Teachers of Mathematics. (1989).  Curriculum and evaluation standards for school mathematics.  Reston, VA: Author.

National Council of Teachers of Mathematics. (1998).  Principles and standards for school mathematics: Discussion draft. Reston, VA: Author.

Phillips, E. & Lappan, G. (1998). Algebra: The first gate. In L. Leutzinger (Ed.), Mathematics in the Middle, 10-19.