Research

NMSA RESEARCH SUMMARY #20 WHAT WORKS TO IMPROVE STUDENT ACHIEVEMENT
By Susan Trimble

The Context/Problem

Calls for accountability and improved test scores heightened by the provisions of No Child Left Behind have led middle level educators to seek ways to increase student achievement with new vigor (No Child Left Behind, 2002). There is, however, no concise, quick fix answer or series of specific activities that will result in increased student achievement. The process is complex for several reasons. In the larger context, societal attitudes and political pressures influence sustainability (Oakes, Quartz, Ryan, & Lipton, 2000). Within the school, multiple factors interact to influence the process of learning, i.e., a combination of different learners, past experiences, processes, and stakeholders.

Since the early 1990’s various approaches to increasing student achievement have emerged to address the issue of increased accountability. Each approach provides some measure of success, but often entails harmful consequences or stumbles upon unforeseen complications.

* Teaching-to-the-test is one approach that may yield short-term student gains but lacks long-term sustainability. Such practices limit the range and depth of learning activities that are needed to cultivate the full education of young adolescents. Lewis, for instance, noted in a short commentary that the test-based accountability is “undermining good policies…and narrows the whole education enterprise” (Lewis, 2002, p. 179).
* The legislative approach mandates competencies, benchmarks, and consequences. The measures provide guidelines or goals but do not offer suggestions to meet these demands. They have the subtle yet detrimental effect of withdrawing the judgment capacity of individuals closest to the students (Meier, 2002).
* Alternative types of schooling have been suggested as a means of promoting increased learning. For example, charter schools have increased in number, but report only academic gains similar to their districts (Horn & Miron, 2000). In many such schools, Oakes et al., (2002) point out the low enrollment of students with special needs, a practice that runs counter to the democratic and social justice themes of middle level education.
* Private funding has financed a choice of whole-school reform designs in hundreds of schools across the country in the past ten years. The nonprofit New American Schools, for instance, contributed over $8 million during the past decade to raise achievement. They report few gains in student performance, a result attributed to the lack of district support and schools’ capacities to change (Berends, Bodilly, & Kirby, 2002).

The complexity of factors illuminated by these alternative approaches is, likewise, reflected in the research base. Quality research attempts to address the multiple variables associated with reforming schools, but currently lacks models to serve as frameworks (Chatterji, 2002). Specifically related to middle level research, the challenges of investigating variables associated with student achievement are described as follows:

* Multiple factors such as policy, curriculum, and organization (for a framework, see Hough & Irvin, 1995).
* Types of performance indicators such as academic efficacy, number of books read, parental involvement (see Mulhall, Flowers, & Mertens, 2002).
* Ethnographic studies and quantitative studies using self-report data (see Hough & Irvin, 1997, for studies that focused on achievement).
* Model building such as links between school inputs, curricular leadership dimensions, and school outcomes (see Brown, Claudet, & Olivarez, 2002, for a model of variables and links to school effectiveness).

The research base currently provides no clear-cut and definitive answer to the challenge of raising achievement. Nevertheless, patterns are emerging that provide direction for practitioners. This research summary presents the findings, grouped as classroom practices or school practices related to student achievement.
Research on Classroom Practices and Achievement

Substantial changes in teacher practices, classroom instruction, and assessment produce improved student performance (Jackson & Davis, 2000; Wenglinsky, 2000). Whether these improvements surpass the effects of student socio-economic status (SES) levels and other external factors is debatable. Wenglinsky (2002) examined the NAEP data on over 7,000 eighth graders in 1996 and found the effects of classroom practices and other teacher characteristics on student achievement to surpass the effect of SES on students. On the other hand, Mertens and Flowers (in press), in a study of 102 Mid South Middle Start schools, concluded that SES was more highly associated with student achievement than effective classroom practices and teaming practices. Bruce and Singh (1996), using sample data from the U. S. Department of Education National Education Longitudinal Study, 1988 (NELS: 88), found the strongest influences on achievement were previous grades in school, then family background, then ethnicity.

Empirically documented classroom practices include the following:
Clear Academic Focus

* Learning goals that are performance based contribute to increased student learning by focusing instruction on targeted outcomes (Brophy & Good, 1986; Cotton, 2000).
* Learning goals based on standards and measured periodically are most effective for student learning (Black & William, 1998, a synthesis of 250 articles). The Southern Regional Educational Board (SREB) report, Raising the Bar (SREB, 2001a), reports effective teacher practices as presenting challenging requirements, giving students actual examples of high quality student work, and sharing the assessment criteria by indicating the amount and quality of work needed to earn an A or B.
* Task orientation where class is businesslike with emphasis on completing work has been associated with higher achievement (Lee & Smith, 1993). A comparison of students in the 90% level of achievement with students in the 10% level of achievement found their perceptions of the learning environment differed on task orientation, student involvement, and rule clarity (Waxman & Huange, 1996).

Teaching Strategies

Quality of instruction has a strong impact on student learning (Brophy & Good, 1986; Stigler & Heister, 1999). This is a widely accepted generalization. The following strategies have been identified as effective by major research studies (Darling-Hammond, 1996; Hattie, Biggs, & Purdie, 1996):

* Establishing high standards and expressing a willingness to help students achieve them are teacher practices associated with higher achievement (Phillips, 1997; Southern Regional Education Board SREB, 2001a, 2001b).
* Implementing assessments to measure progress toward goals and to inform practice (Black & Wiliam, 1998; Darling-Hammond, Ancess, & Falk, 1995; Falk & Ort, 1998; Pressley, Yokoi, Rankin, Wharton-McDonald, & Mistretta, 1997; Stiggins, 2002) provides descriptive feedback of teaching effectiveness, while building students’ confidence and resulting in adjustment to instruction (Cotton, 2000; SREB, 2001a; Stiggins, 2002). Likewise, teachers learn more if they assess their teaching and the students’ learning and use it to change instruction. Constantly assessing student progress is one of the practices of teachers “who succeed at developing real understanding of challenging subjects” (Darling-Hammond, 1996, p. 11).
* Applying a deep understanding of subject matter and of the characteristics of young adolescents is a requirement of teaching certification. Mertens and Flowers (2003, February) reported that certified middle level teachers in teams with common planning times had the highest levels of effective team and classroom practices. Team practices were defined as curriculum coordination and integration practices, coordination of student assignments, parent contact and involvement, and contact with other building resource staff. Classroom practices were defined as small group, active instruction, integrated and interdisciplinary practices, authentic instruction and assessment, critical thinking practices, reading and writing skill practices, and math skill practices. Numerous other studies present evidence of the association between certified teachers and student learning (Chatterji, 2002; SREB, 2001b; Wenglinsky, 2000). A deep understanding of subject matter provides teachers with the background for higher order thinking and questioning skills (SREB, 2001c).
* Promoting critical thinking and higher order thinking (McLaughlin & Talbert, 1993). Teachers who are able to discern students’ level of thinking and use it to construct knowledge help them to develop a better understanding of content (Darling-Hammond, 1996).
* Learning in context with integrated and interdisciplinary practices, using meaningful tasks (Flowers, et al., 2000) and an integrated curriculum (Jackson & Davis, 2000; also, see Hartzler’s meta-analysis of studies on integrated curriculum and achievement, 2000).
* Implementing mastery learning and the scaffolding of successful steps to develop confidence and skills (Hattie, et al., 1996).
* Promoting student engagement with a task orientation (Waxman & Huang, 1996). Hattie et al. (1996) found promoting higher student engagement was a common feature of 51 studies of learning skills. Student involvement is increased by using effective questioning techniques (Cotton, 2000). Motivation and homework were found to have a significant effect on the achievement of eighth graders (Bruce & Singh, 1996).
* Using small groups and individual attention (Flowers, et al., 2000a) with cooperative learning properly implemented (Johnson & Johnson, 1999).

Support for Teaching and Learning

* Extra help and resources for students. Supporting students through enhancement and extra practice has been related to greater achievement (McLaughlin & Talbert, 1993). Large-scale reform programs repeatedly report achievement gains associated with enhanced practice in reading, math, and writing (Middle Start, 2002; Mid South Middle Start, 2002). In a study of 26 middle schools in Massachusetts implementing Turning Points recommendations (Carnegie Council on Adolescent Development, 1989), math manipulatives and extended writing were associated with achievement gains (DePascale, 1997). Extra resources provided to reforming schools are also an essential component of conditions to enable achievement gains, reported by Balfanz and Mac Iver (2000). Likewise, extra resources for schools, particularly middle schools with high populations of at-risk students, were a condition of 31 Illinois middle schools that implemented Turning Points (Carnegie Council on Adolescent Development, 1989) recommendations and reported achievement gains (see www.turningpts.org).
* Collaborative structures for teachers. Instructional improvements in the classroom occur best in schools that have established organizational conditions to support teacher collaboration, such as, common planning time, teams that engage in positive adult-child relationships, heterogeneous groupings of students, and flexible scheduling (Flowers, et al., 2000a, 2000b; Miles & Darling-Hammond, 1998). For more details on teaming and common planning time, see the section below entitled Research on Schools and Achievement.
* Time for teachers to work together provides support for learning together, solving problems, and discussing values (McLaughlin & Talbert, 1993; Miles & Darling-Hammond, 1998). Common planning time enables teachers to work together on interdisciplinary teams, resulting in student achievement scores improving (Mertens & Flowers, in press).
* Professional development is needed for teachers to gain the skills necessary for school improvement processes, for teaming, for teaching young adolescents, and for implementing alternative assessments and data-based decision making (Flowers, Mertens, & Mulhall, 2002; Wenglinsky, 2000). Among the leadership skills necessary to improve school effectiveness are collecting and evaluating school and classroom data (King, 1999; Stiggins, 2002). Other skills needed to positively influence student achievement are working with special populations, high-order skills for math, and laboratory skills for science with frequent hands-on activities and testing of classroom learning tied to standards (Wenglinsky, 2000). Spigler and Hiebert (2000) concluded, after analyzing teaching videos of eighth grade math classrooms in Japan, Germany, and the United States, that teachers would benefit from examining lessons for more content and more problem solving situations that are aligned to standards. They recommended that teachers learn in groups how to incorporate more content and problem solving.

Research on Schools and Achievement

A prodigious amount of literature exists on middle level schools and practices, however, far less research exits that documents improved student achievement and school variables. No national study has been conducted of the relationships between student achievement data and middle school factors. Regional studies, however, of whole-school reform initiatives that looked at multiple school sites and achievement, first appeared in 1997 (see Backes, Ralston, & Ingwalson, 1999; Center for Prevention Research and Development, 1998; DePascale, 1997; Felner, Jackson, Kasak, Mulhall, Brand, Flowers, 1997; Mac Iver, Young, Balfanz, Shaw, Garriott, Cohen, 2001; Mac Iver, Mac Iver, Balfanz, Blank, & Ruby, 2000; Mertens, Flowers, & Mulhall, 1998; National Forum To Accelerate Middle Grades Reform, 2002). Other types of studies included case studies of high-performing middle schools (Miles & Darling-Hammond, 1998; Trimble, 2002), and school-wide factors, such as climate, as they related to math achievement and attendance (Phillips, 1997).

The research indicates that the following practices are associated with student achievement:

Holistic Approach

The interdependent nature of interventions supports the holistic approach to school reform that is associated with achievement gains. Dickinson and Butler (2001) called this interdependency the ecological nature of the middle school concept that integrates multiple conditions, beliefs, processes, and goals. Schools provide the context that enables and supports effective classroom practices (Wenglinsky, 2002). Aspects of the holistic approach are as follows:

* An alignment of all parts of the system focused on the intellectual development of young adolescents (Jackson & Davis, 2000; Davis, 2001). More than 50% of teachers in high-performing schools in a sample of 60 middle schools in 14 states reported perceptions of clear goals/priorities and of teachers and school administrators working together to improve student achievement (SREB, 2001b).
* High implementation of the school design. Large-scale studies of schools engaged in research-based design models conclude that higher implementation of the design model resulted in higher student achievement (Berends, 2002; DePascale, 1997; Felner, et al., 1997; Mac Iver, Young, Balfanz, Shaw, Garriott, & Cohen, 2001; Mertens, Flowers, & Mulhall, 1998).
* A combination of academic focus and supportive relationships fosters higher achievement, concluded Lee (1993) in a study of 377 middle schools. Darling-Hammond (1996) reached similar conclusions from case studies of New York City schools structured for serious learning and for caring relationships. Schools where teachers know their students well show better student results (Darling-Hammond, 1997; Dickinson & Erb, 1997; Lee, Bryk, & Smith, 1993). Likewise, teachers who help students develop a real understanding of challenging subjects do so by developing student confidence, motivation, and effort (Darling-Hammond, 1996).
* A focus on academic achievement. Phillips (1997) in her study of 5,600 students in 23 middle schools concluded that “academic press,” consisting of a demanding curricula, more homework, and high expectations for students, correlated with mathematics achievement and attendance; she found little support for the communal model of school effectiveness that places importance on positive adult social relations, positive teacher-pupil relations, and democratic governance.

A School’s Capacity for Change

The school’s capacity for change stems from a combination of factors that enable teachers and administrators to problem-solve and implement decisions that help students learn. Various research studies have investigated these factors.

* The principal viewed as a strong leader is associated with articulation of the school’s mission, a safe learning environment, and instructional improvements (Edmonds, 1979; Berends, 2002; Clark & Clark, 2001).
* High levels of trust emerged as an important element of change processes from eight years of research on Chicago Schools’ reform initiative from 1990-2000 (Sebring & Bryk, 2000a; 2000b).
* Teaming and the capacity for change. Teaming is associated with increases in a school’s capacity for change. Nineteen Middle Start schools in Michigan that implemented Turning Points recommendations for two or more years reported increases in student achievement and a greater capacity for change in comparison with 45 other schools with no Middle Start help (Center for Prevention Research, 1998).
* Teams and common planning time provide structures for collaboration that Mertens & Flowers (in press) found to be associated with increased implementation levels of effective classroom practices and teaming activities. The benefits of teaming have been well documented. Structures that support teacher collaboration are a part of successful schools (Darling-Hammond, 1996). Teaming together with common planning time enables teachers to learn and problem solve together (Erb, 2001; Gallagher-Polite, 2001). Teachers working together show a positive association with curriculum coordination, classroom integration, and interdisciplinary practices (Flowers, et al., 2000a).
* Teaming and length of time together. The most effective teams have teachers who have worked together for four or more years and who have smaller student team sizes and common planning time (Flowers, et al., 2000b). Three or more years of teaming together was associated with more team activities and more implementation of effective classroom practices and teaming activities in a sample of 102 Mid South Middle Start schools (Mertens & Flowers, in press).
* Teaming and better teaching. Teachers become better teachers when they work on curriculum and assessments collaboratively (National Commission on Teaching and America’s Future, 1997; Stigler & Stevenson, 1991). Mertens and Flowers (in press), in a study of 120 Mid South Middle Start schools found teaming and common planning times were associated with higher levels of teaming practices, such as integrated and interdisciplinary practices, authentic instruction and assessment, reading skill practices, and math skill practices.
* High levels of professional development result in higher achievement, according to the report, Closing the Gap in Middle Grades (SREB, 2001b). This report recommends professional development to be greater than 16 hours annually.

Focus on Learning

* An academic core. The centrality of an academic core with higher expectations for middle grades is part of a comprehensive framework recommended by Turning Points 2000 and proposed by the Southern Regional Educational Board (SREB) to raise the achievement of all middle grades students (SREB, 2001b; 2001c).
* School goals and measurement. A sense of direction provided by school goals helps align resources, staff development choices, and curriculum decisions. Schools that focus on learning with articulated goals for student achievement and their assessment show greater achievement.
* Data-informed decision making. Turning Points 2002 (Jackson & Davis, 2000) recommends the use of standards and information gleaned from ongoing assessments in a “backward design” of instruction. Analysis of student achievement data provides the information needed to diagnose learning needs and degrees of mastery that influence who to focus on and what to emphasize in instruction. The Council of Great City Schools looked at student achievement patterns in three large urban school districts and concluded that an important strategy for success was the commitment to data-driven decision making (Snipes, Doolittle, & Herlihy, 2002). Likewise, the Southern Regional Educational Board includes the use of data to review and revise school and classroom practices as needed for improved student learning (SREB, 2001b).
* Equitable opportunities to study algebra and other challenging content have been confirmed by several large-scale studies to relate to higher achievement (Christie, 2002; SREB, 2001a; Suter, 2000).

Conclusion

The research base for young adolescents repeatedly tells us that achievement gains are associated with greater implementation of both academic and caring processes focused on student learning. The conditions and structures aligned with these processes consist of quality instruction in the classroom in schools with holistic approaches focused on learning and supported by the school’s capacities for change. Additional factors that promote achievement include school leadership, parental involvement, and community support.

No Child Left Behind (NCLB) provides challenges to schools but also opportunities through increased funding. In a document entitled “Opportunities and Accountability to Leave No Child Behind in the Middle Grades,” Brown (2000) describes 18 funding areas in NCLB to improve achievement. State initiatives that enable schools and districts to access these funds may well enable schools and districts to implement the classroom and school practices that raise achievement.
Acknowledgments

The author wishes to thank the reviewers for their helpful comments on earlier versions of this article: David Hough, John Lounsbury, Steve Mertens, Nancy Mizell, and George White.

An asterisk (*) in the lists below indicates non-empirical publications, commentaries, and practitioner-focused articles as a way for the reader to distinguish such studies from research-based and/or empirical studies.