School of golden touch? A study of school effectiveness in improving student academic performance

Abstract

This paper explores whether different schools exhibit varying levels of effectiveness in improving student academic performance and, if so, which factors might affect school effectiveness. Using data from the China Education Panel Survey (CEPS), this research examines the influence of various school features on dynamic changes in middle school student academic performance. It finds that schools of higher socioeconomic status, with more academically proficient students and with more complete school facilities, are more effective in improving students’ academic performance. Meanwhile, the work shows that local community school rankings, the average quality of school teachers, and academically related practices do not have a significant impact on a school’s effectiveness. In summary, this research extends previous studies on student education attainment from a focus on student performance to a focus on changes in performance. It illustrates the essential roles played by the capacities of enrolled students and of individual/aggregated family socioeconomic status in improving student academic performance.

Introduction

Education is an important tool from which individuals can achieve upward social mobility and avoid downward social mobility. High-quality schools are generally regarded as important means to reach high education attainment (Ye 2015; Tang 2015); therefore, parents often spare no effort to allow their children to enter better primary and secondary schools. In the process of preparing school enrollment, various terms related to entrance exams—including occupying pits (zhankeng), excellent student cultivation (peiyou), pinching the top (qiajian), school choice (zexiao), co-construction (gongjian), and school district housing (xuequfang)—have been created and reflect fierce competition for high-quality education. The competition triggers various discussions on educational equity and school effectiveness. According to some studies, high-quality schools usually have more abundant material resources, more skilled faculty, or better students, and thus their outstanding performance is considered to be based on an unequal distribution of educational resources or enrollment policies (Liu et al. 2009) rather than on school effectiveness itself.¹

In line with the concerns of society, whether there are significant differences in education attainment among schools and which factors may affect school effectiveness have been core issues of the sociology of education. However, thus far, researchers have not reached a consensus, while comparatively consistent findings show that a school’s socioeconomic status/composition² is closely related to student academic performance and enrollment rates (Crosnoe 2009; Palardy 2013). This association embodies not only the family and peer effects of students (Coleman et al. 1966; Dreeben and Barr 1988) but also the effects of school educational resources, teacher quality, and academic practices (Engberg and Wolniak 2014). In China, studies on school effectiveness have focused on the impact of key schools on student academic performance and find that the students at these schools tend to outperform their counterparts at other schools (Ye 2015). Possible explanations for this phenomenon mainly focus on teacher quality and other factors that are difficult to observe and measure directly, such as school management systems and school atmospheres (Wang et al. 2017).

However, it should be noted that whether comparing key schools with ordinary schools or comparing schools of varied socioeconomic statuses, one fact has been easily overlooked, that is, schools and students are not randomly matched, and students enrolled at different schools often have different learning backgrounds and abilities (e.g., key high schools select the best students according to the results of high school entrance examinations). This also creates sample selection problems. Therefore, if we fail to control factors such as foundations of enrollment or differences in family background, we will overestimate the net effect of some schools (Wang et al. 2017). We may create an explanatory bias when discussing how schools/schooling affects student academic performance and future social status (Raudenbush and Eschmann 2015; Downey and Condron 2016).

Given this, some researchers have put forward the concept of “value-added” in focusing on changes in student academic performance over a certain period. Investigating the impact of schools on the value-added of student performance (rather than just the level of education achieved at a certain point of time) is equivalent to controlling the initial level of student education. Thus, it can be used to more accurately identify school effectiveness in promoting students’ academic performance (Thomas 2005).

Therefore, this study is intended to more accurately reveal the role of schools in students’ education attainment based on the “value-added” concept.

Specifically, this study uses data from the first and second waves of the China Education Panel Survey (CEPS) to build a comprehensive model that covers factors of three levels: school, family, and individual. By tracing changes in middle school student academic performance through compulsory education, this article compares the effects of different schools and features of the same school on changes in student academic performance (academic value-added).

In other words, if we relate the schooling process to the processing of raw materials in a factory and student education attainment to products from assembly lines, we explore possible sources of different processing abilities among factories. The present study can help us clarify the relationship between school stratification and students’ education attainment, or, more broadly, the role of schools with different characteristics in the reproduction of social inequality. In addition, the results of this study will contribute to future school education reform and promote the equitable development of education.

The role of schools in individual education attainment

At present, the research on education attainment and education inequality has thoroughly analyzed the family’s role and mechanisms affecting individual education attainment and has reached a relatively consistent conclusion (Li 2006; Liu 2008). However, in terms of school effectiveness, there is still much controversy regarding whether a school can have a measurable impact on individual academic performance and regarding which specific features of a school can affect student academic performance.

In 1966, the publication of the Coleman Report opened the discussion of school effectiveness in the field of the sociology of education. The report found that relative to student family backgrounds, neighborhoods, and peers, school investments in material resources, teachers, and curricula had little impact on students’ standardized test scores (Coleman et al. 1966). Schools had failed to address differences in student academic performance of different socioeconomic and ethnic backgrounds (Downey and Condron 2016). Later, researchers replicated Coleman’s study by adopting more precise research methods but could not agree on whether schooling in general expanded, maintained, or narrowed the academic gap (and macro-scale social inequality) among students of different family backgrounds (Raudenbush and Eschmann 2015; Jennings et al. 2015; Downey and Condron 2016; Domina et al. 2017).

At present, a relatively consistent finding in academia is that some general characteristics of schools, such as their socioeconomic status and quality, are closely related to student academic performance. High socioeconomic status schools can significantly improve student academic performance, prolong their years of education, and raise their educational expectations (Palardy 2013; Wu and Huang 2016). What is traditionally called “good” schools, such as magnet schools in the USA and key middle schools in China, can also significantly improve individual academic performance. Chinese scholars show that compared to ordinary middle schools, students at key middle schools have significant advantages in terms of academic performance and are more likely to enter high school and elite universities (Ye 2015; Tang 2015).

It should be highlighted that the relationship between the socioeconomic status of a school and the individual socioeconomic status of student families are not always consistent. The former is a product of the latter. Major factors shaping differences in education outcomes caused by differences in the socioeconomic status may lie in student backgrounds (i.e., family, peers, and neighborhoods) and differences in schools’ resource allocation and academic practices and management (Palardy 2013). However, similar to research on the overall effects of schools, there are considerable variations in research on the impacts of various resources (such as school material and human resources) and academic practices on student academic performance.

Among disputes on the input effectiveness of school material and human resources, the most widely known are the multi-round debates between Hanushek and Hedges. Hanushek holds that after controlling for student family background, there is no systematic correlation between the allocation of school resources (including the allocation of faculty) and student academic performance. Such school resources include the average expenditures of students, the ratio of students to teachers, teachers’ salaries, class size, school facilities, teachers’ overall education levels, and experience (Hanushek 1989, 1997). However, Hedge argues that there is a positive correlation between school input and student performance and that student per capita expenditures have a particularly significant impact; this finding has maintained sufficient robustness across different studies (Hedges et al. 1994; Greenwald et al. 1996).

Similar controversies exist in research on schools in developing countries and regions. Some studies have found that in underdeveloped areas, school resource input (e.g., per capita funding, teaching aids, library construction, and teacher training) is closely related to student academic performance (Heyneman and Loxley 1983). Others have suggested that similar to what is observed in developed countries, resource investment in developing countries, including education funds and human resources, may not be able to provide the corresponding output (Hanushek 1995). However, both sides agree that minimal levels of school resources such as textbooks and necessary facilities have a substantial impact on student academic performance (Hanushek 1995; Kremer 1995).

In terms of debates on material resource investment, some researchers believe that the effectiveness of resource investment is closely related to the development of local education. In countries or regions with insufficient investment in basic educational resources (mostly in developing countries), an increase in basic investment can significantly improve student academic performance. However, in areas where educational investment is more abundant (mostly in developed countries), additional resources beyond basic investments will not bring about more benefits (Buchman and Hannum 2001).

For China, which is a developing country, studies have confirmed the positive impact of school human resource investment (including the student-teacher ratio and full-time teacher qualification ratio) on student academic performance. Still, disputes remain regarding the role of school material resource investment (e.g., the average area of school buildings per student and the number of books per student) and financial resource investment (educational/shared funds per student, etc.) (Hu and Du 2009; Hou and Shen 2014). However, most of these studies are based on data from western regions. Considering vast disparities in educational investment and resource allocation across different regions, as well as considering the relationship between the effectiveness of resource allocation and overall education investment (Buchman and Hannum 2001), we do not know whether the conclusions drawn from regional samples can be generalized to the country.

In addition to investigating the effect of school resources, scholars have identified the importance of schooling and proposed that the specific process of education may be the most important factor that leads to differences in student academic performance (Raudenbush and Eschmann 2015). Specifically, some studies have explored the relationship between schools’ internal educational processes and student academic performance in reference to time schedules, specific teaching methods, relations between teachers, teacher-student interactions, etc. (Gamoran et al. 2004). In terms of time schedules, some studies have found a positive relationship between the arrangement of the school year and school days and academic performance, but the strength of this relationship remains controversial (Hallinan 1988). However, some researchers still suggest that the impact of families’ differing socioeconomic status on students can be reduced by prolonging school time (e.g., by popularizing pre-school education, extending semesters) (Raudenbush and Eschmann 2015). Many countries and regions regard extending school time and increasing daily teaching time as an important means to improve learning investment (Bellei 2009). Teaching interactions in the classroom, including class size and the quality of teacher-student interactions, can significantly affect student academic performance (Gamoran et al. 1995; Crosnoe et al. 2004).

Other studies have incorporated teaching management into the scope of educational processes and teaching practices and found that a school’s emphasis on learning (i.e., academic pressure) also affects student academic performance (Berends 2015; Palardy 2013).

The abovementioned works explore the impact of the overall school characteristics (including the socioeconomic status and quality of a school) or argue about the specific school features affecting student academic performance, but most of their discussions focus on student education attainment at a particular point of time. However, school admissions are often heavily selective. Therefore, for discussing the effects and internal mechanisms of good schools, if we do not consider student academic level at enrollment but focus on only student performance at a certain point of time, this is likely to overestimate the influence of school features on individual students (Wang et al. 2017).

Because of this, educational circles have put forward the concept of “value-added” or used longitudinal data for analyses. Both emphasize that for analyzing school effectiveness, we must not only pay attention to student performance at a certain point of time but also control student academic levels at the time of enrollment. Thus, we discuss the impact of schools on the value-added of student academic performance in various respects over a certain period (Thomas 2005). This concept and method have been widely adopted in the UK, the USA, and other countries. Existing studies have found considerable differences between school rankings based on students’ raw scores and rankings based on value-added scores. “Good” schools in everyday parlance are not always effective at improving student academic performance, and variables that affect students’ final scores will not always affect students’ progress scores and vice versa (Strand 1997).

Perhaps due to the availability of information, little attention has been given to the value-added concept in China. Some educational administrations and schools have taken similar indicators into account in comparing the enrollment results of different schools (Chen 2013), and a few researchers have conducted much introductory and exploratory work on value-added concepts and methods (Thomas 2005; Bian and Lin 2007), but few empirical studies on school effectiveness have adopted this concept. Several studies have mainly analyzed the influence of individual, family, school, and other factors on students’ college or high school entrance examination results after controlling for students’ middle school entrance examination results (schooling starting point). Such research finds that differences in student academic performance (50% to 60% variation) can be explained by students’ learning foundations before entering schools and academic levels at enrollment. Factors such as school material resources and teacher quality play a very limited role in promoting student academic progress (Ma et al. 2006; Du & Yang 2011; Wang et al. 2017). To some extent, this discovery supports the currently popular “student source theory,” which indicates that student academic performance is more a result of the accumulation of previous factors and that the role of schools is minimal.

Overall, these studies have focused on the development, changes, and gains of academic performance in addition to static indicators such as academic performance and education attainment and have begun to explore the impact of various factors on student academic performance after controlling for their initial levels. However, such works still present shortcomings. First, due to data limitations (generally based on local data), their findings are not based on samples that are representative enough to systematically estimate whether schools have had different effects on promoting student academic performance and the sources of these effects. Second, the limited domestic research has mainly focused on high school education outside of the compulsory education stage, where differences (in the name of the model, key, and ordinary high schools) between schools and differential admission policies are allowed. There has been no research on the school effectiveness of compulsory education (which should be responsible for promoting educational equality) in improving student academic abilities in terms of educational value-added.

Therefore, this study uses CEPS data and middle school students as subjects to analyze the impact of school characteristics (e.g., school quality, resource allocation, schooling processes, school socioeconomic status, and student academic levels at enrollment) on the value-added of student academic performance.

School quality hypothesis

Many studies have found that the key school system plays an important role in students’ education attainment (Tang 2015), but at the compulsory education stage, the government emphasizes educational equity and does not allow for the distinction of key and non-key schools.³ However, while the government does not publish the classification and ranking of schools, many local communities have classified and ranked schools according to unified entrance examinations scores over the years. Therefore, in this study, we regard the ranking of schools in local communities (school reputation) as a measure of school quality and consider this to play an important role in promoting students’ academic progress.

Hypothesis 1: School ranking (school reputation) has a significant impact on the value-added of student academic performance. The higher a school’s ranking, the higher the value-added of student academic performance.

Resource allocation hypothesis

Research on school effectiveness has always been concerned with the impact of investment in school material and teacher resources on student academic performance but has not reached a consensus (Hanushek 1989, 1997; Hedges et al. 1994). While Chinese research has verified the importance of teacher quality in promoting students’ progress (Hu and Du 2009), there remains a lack of systematic research based on representative nationwide samples. In view of this, we assume that the input of school materials and teachers is conducive to promoting student academic performance.

Hypothesis 2A: Investment in material resources at schools has a significant impact on the value-added of student academic performance. The more material resources a school invests in, the greater the improvement in student academic performance.

Hypothesis 2B: Overall school teacher quality has a significant impact on the value-added of student academic performance. The higher the overall level of teacher quality, the greater the improvement in student academic performance.

Schooling hypothesis

Relative to a school’s quality, reputation, and resource allocation, schooling is a more dynamic concept and involves the process through which schools, teachers, and students participate in the construction and realization of established educational goals. It covers not only teaching management implemented by schools to achieve educational goals but also the teaching environments of schools, such as interactions between teachers and students during academic activities.

The importance of strong teacher-student interactions (Gamoran et al. 1995; Zhang and Xie 2017) has been verified by previous studies from the perspective of teacher-student interactions in the classroom. However, the present study is intended to determine whether the overall nature of teacher-student relationships in schools has a significant impact on student academic performance.

In terms of teaching management, the pressure to graduate more students to high schools and extending teaching time are two important school management measures. They can help teachers and students increase teaching or academic input, improve academic performance, and ultimately increase high school promotion rates. We will examine the efficiency of prolonging teaching time and whether this can genuinely improve student performance.

Besides, Chinese research on students’ college entrance examination results suggests that student academic performance may benefit more from a school’s atmosphere and school management system (Wang et al. 2017), but empirical data have not yet supported this. Therefore, this study measures school atmospheres and management systems based on indicators of teaching environments, teaching time arrangements, and the pressure of students’ high school admission rate faced by teachers and discusses their role in promoting student academic performance.

Hypothesis 3A: The teaching environment at a school has a significant impact on the value-added of student academic performance. The more positive the overall teaching environment of a school is, the greater the value-added of student academic performance.

Hypothesis 3B: School teaching management has a significant impact on the value-added of students’ academic performance. Teaching management includes two aspects: high school promotion pressure on teachers exerted by schools and school teaching time schedules.

Hypothesis on student academic level at enrollment

A school’s socioeconomic status can be regarded as an important dimension for measuring student academic levels at enrollment. It reflects the agglomeration effect of the family background of enrolled students and has always been used as a core influencing factor in studies on school effectiveness. Exiting research has found that the academic performance of students enrolled at schools of high socioeconomic status is significantly higher than that of students enrolled at schools of low socioeconomic status (Palardy 2013). However, such an academic advantage may be attributed to the advantage present upon entering school, and thus a school may play a role only in maintaining this advantage (Raudenbush and Eschmann 2015). Therefore, we examine whether a school’s socioeconomic status can also improve student academic performance.

Hypothesis 4A: A school’s socioeconomic status has a significant impact on the value-added of students’ academic performance. The higher a school’s socioeconomic status, the greater the value-added of students’ academic performance.

Another important indicator of student academic performance is students’ learning foundations (academic level) at the time of school enrollment. The research based on regional data has found learning foundations upon enrollment to be an important factor in predicting students’ current academic performance and future academic progress (Wang et al. 2017). Learning foundations can be divided into collective and individual foundations. When students’ overall academic levels are high, it is possible to create a mutually motivating or competitive learning environment, which will encourage students to study harder (Ryan 2001). Therefore, this study further examines relevant findings from two respects: the overall academic level of students and the academic levels of individual students at the time of enrollment.

Hypothesis 4B: The overall academic level of students and individual academic foundations have a significant impact on the value-added of student academic performance. The higher the student academic levels are, and the better the learning foundations of students before entering school, and the higher the value-added of student academic performance.

Data, variables, and methods

Data

Data used in this study were taken from China Education Panel Survey (CEPS) designed and implemented by the National Survey Research Center of Renmin University of China.⁴ The survey is China’s first large-scale panel survey for young students starting from middle school.

CEPS uses the probability proportionate to size sampling (PPS) method and uses 28 county-level units as primary sampling units (PSUs) for 31 provinces, municipalities, and autonomous regions of China. For each PSU, four schools are sampled, and two classes of the seventh and ninth grades are then selected from each school for a cluster survey. A total of 19,487 valid student questionnaires were obtained from 112 schools and 438 classes (several schools have only a single class for one grade) (see Wang 2016 for further details), and 19,007 parent questionnaires, 1412 teacher questionnaires, and 112 school leader questionnaires were collected through the baseline surveys. A total of 9920 former seventh-grade students and their families, teachers, and principals were interviewed in the second wave.

This study focuses on student academic performance and influencing factors in effect during the two waves of surveys. Therefore, it was necessary to select individuals who participated in the two waves of the survey. After eliminating individuals presenting missing variables (including students who transferred schools), a sample of 9266 individuals was finally obtained.

Variables

Dependent variable

The dependent variable of this study is the change in student academic performance. Because students’ scores collected by the survey are the results of mid-term examinations organized by each school, it was difficult to compare across schools. Therefore, we used the standard score achieved on student cognitive ability tests as an approximate measure of academic performance. The cognitive ability test was designed and administered by the CEPS project team. After obtaining the original score, the CEPS project team converted it to the standard score of the 3PL model based on item response theory (IRT) to compare across different schools, regions, and waves of the survey.⁵

Independent variable

Apart from the individual variables adopted, the independent variables used in this study are school-level variables. Considering that the variation in independent variables between the two waves is minor and the CEPS corrected errors and omissions of the baseline survey during the follow-up process, the data used in this study are mainly taken from the second-wave survey. For school quality (school reputation), we used schools’ local reputations (rankings) as an index to measure school quality. According to each school’s ranking in its county (divided into five levels) estimated by respondents of the principal’s survey, we classified schools into three categories: lower-middle, upper-middle, and best.

Resource allocation covers two aspects: material resources and teacher quality. Material resources cover 12 items (e.g., laboratories, computer classrooms, libraries, music rooms), which are treated as 12 dummy variables in this study. When a school does not have one of the above facilities, the variable is assigned as “0” and is otherwise assigned as “1” (regardless of the condition of equipment). We added the items up and obtained a variable with numeric value varies between 2 and 12. The higher the score, the more complete a school’s material resources. In addition, according to information on school teachers provided by the survey of principals, we calculated the proportion of teachers with an undergraduate degree or above as a measurement of teacher quality.

The teaching environment is measured based on interactions between teachers and students. CEPS asked students about their interactions with teachers of each major subject (e.g., “Mathematics/Chinese/English teachers often ask questions/encourage me”). According to answers provided by the seventh and ninth grade students in the first-wave survey, we calculated the average for all students at each school as a measurement of school teaching environments. The higher the value, the more frequent interactions occur, and the more positive and active interactions between teachers and students are, and the better the teaching environment.

School teaching management covers two aspects: teaching time schedules and school’s pressure to graduate more students to high schools. (1) CEPS asked students to specify “the number of days in a week students must attend school” and “students’ daily school schedules” through the survey of principals. Based on this, we calculated the amount of time students spent at school each day and obtained values between 8 and 14 h to measure students’ daily school hours. (2) To measure a school’s pressure to graduate more students to high schools (high school promotion pressure) as perceived by all teachers interviewed,⁶ we calculated an average value where the higher the score, the greater the level of high school promotion pressure faced by teachers.

The socioeconomic status/social composition of schools is based on parents’ educational level and occupational information collected by CEPS through its first-wave survey. We calculated the proportion of parents with a college degree or who are professional technicians or staff of government departments or state-owned enterprises. The higher the proportion, the higher the socioeconomic status of the school as a whole.

For measuring the overall student academic level at enrollment, CEPS asked for each student’s class ranking and the number of students included in each sixth-grade class. Based on this, we calculated the percentiles of class rankings of each student at the time of graduation from primary school. We then calculated the median for each school to measure the overall academic level of each school.⁷ The higher the value of this variable, the higher the overall student academic level at enrollment of a school.

The variable of learning foundation measures student academic level at the individual level. CEPS asked students how difficulty they felt about the three major subjects of the sixth grade in primary school. According to the students’ assessments (from “very little” to “very difficult”), we calculated the average scores for students as an approximate measure of their learning foundations upon entering middle schools. The higher the score, the more learning difficulty for them at the time, and thus the worse their learning foundation.

Control variable

We measure four levels of control variables: individual, family, class, and school. Individual-level variables include gender (e.g., female student), household registration type (e.g., rural household), student learning habits (learning efforts), and student participation in additional tutoring.

In terms of student learning habits/efforts, the survey asked students to review their academic behavior over the past year⁸ and then calculated the average score for each student as a measure of his or her learning habits/efforts. In addition, according to student participation in tutoring outside of class during the second-wave survey, we establish a dummy variable for participation in additional tutoring. For those participating in tutoring on any major subject, this variable was valued at “1” and was valued at “0” otherwise.

Family-level control variables include the following: having an only child or not, family economic status (parents’ self-assessment), parents’ highest education level (based on parents’ highest degree), and parents’ occupation (we take the value for a parent who is a technical professional or staff in a government department or state-owned enterprise; otherwise, we take the value of the father’s occupation or the mother’s occupation when the father’s occupation information is not available).

Class-level control variables include headteacher information such as gender, education (full-time bachelor degree or above), and years of teaching (setting a year as a unit and including the square of years).

In addition, due to a large number of schools participating in the survey, the same wave of CEPS survey in many instances spanned two semesters, which may have led to different lengths of study time for students at different schools between the two waves of the survey. To avoid the resulting errors, we calculated the time interval between the two waves of surveys with the unit “month.”⁹

Descriptive statistical results for all variables of this study are shown in Table 1.

Table 1 Descriptive statistics of the variables

Model setting

We focus on the impact of various school-level characteristics on student academic performance (measured by cognitive ability). The data cover two levels: the school and individual levels. The individual level of students is nested within the school level. To accurately estimate the impact of school-level characteristics on students’ individual academic performance, we use a multilevel linear model with a random intercept for our statistical analysis.

Level 1: Individual level

$$ {Y}_{ij=}{\beta}_{0j}+\sum \limits_{k=1}^n{\beta}_{0k}{X}_{kij}+{\beta}_z{Z}_{ij}+{\varepsilon}_{ij} $$

Among the above, Y_ij represents the change in the academic level of the jth school and the ith student over the two waves of the survey (i.e., the seventh and eighth grades); X_kij represents the value of the jth school, the ith student, and the kth independent variable; Z_ij represents all control variables at the individual level; ε_ij represents the random error at the individual level; and β_0j is the random intercept, indicating the value of changes in student academic performance for the jth school when all variables are zero.

Level 2: School level

$$ {\beta}_{0j}={\gamma}_{00}+\sum \limits_{v=1}^m{\gamma}_{0v}{W}_{vj}+{\gamma}_{0s}{S}_{\mathrm{j}}+{\mu}_{0j} $$

Among the above, W_vj represents the value of independent variables at the jth and vth school levels, S_j represents control variables at the school level, μ_0j is the random error at the school level, and γ₀₀ represents the value of changes in student academic performance when variables at all school levels are zero.

School characteristics and school processing ability

To clarify the effects of school features on promoting the value-added of student academic performance, we establish two sets of models. Models 1 to 4 take the static academic performance (the standard score of cognitive ability test for the eighth grade is set as an approximate measurement value) as a dependent variable. Models 5 to 7 take the value-added of student academic performance (the difference between the standard score on a cognitive ability test for the seventh and eighth grades) as a dependent variable. A comparison of the results of the two models can help us clarify some original cognitive misunderstandings and more accurately demonstrate the role of school features on student academic performance. Table 2 provides the statistical estimate results for each model.

Table 2 Factor analysis of school processing ability: based on a hierarchical linear model

Before estimating each model, we first fit the zero model without independent variables (not shown here) to examine the extent to which school-level factors explain the dependent variables. The results show that when the static academic performance is a dependent variable, roughly 31% of the variance can be explained by the school-level factors. However, when the value-added of the academic performance is taken as a dependent variable, the school-level factors can explain roughly 11% of the variance of the dependent variable, that is, the explanatory ability of the school-level factors is considerably lower.

Models 1 to 4 first analyze the influence of factors of different levels on student academic performance according to the second-wave survey. This static approach to studying student scores at a certain point of time has been the most common way to study student academic performance. The estimated results of the models confirm the findings from previous similar studies, that is, after controlling for individual and family factors, there is a significant correlation between school features and student academic performance.

Model 1 shows that before adding other school-level independent variables, school reputation (ranking in the county/local community), school material resources, and overall teacher quality are significantly related to students’ academic performance. The higher the ranking of a school within its county, the more abundant its material resources, the higher the proportion of teachers with a bachelor degree or above, and the better student academic performance. However, after introducing the teaching environment variable (model 2) (i.e., the frequency of encouragement and praise from teachers as perceived by students), the variable of teacher quality is no longer significant. Instead, teacher-student interactions and teachers’ encouragement of their students, in particular, are related to teacher quality, but the former is more important than the latter in influencing student academic performance. After further controlling for the overall student academic level at enrollment (model 4), variables of the school’s teaching environment are no longer significant either.

In addition, the effect of school rankings (school reputation) is not constant. After considering a school’s socioeconomic status (model 3), the effect of school rankings declines, and the difference between schools ranked in the upper-middle and the lower-middle is significant only at the level of 0.1. After further introducing the overall student academic level at enrollment and learning foundation upon enrollment (model 4), there are no significant differences between the school rankings. However, students from the top schools present significant advantages in terms of academic performance.

Also, two variables measuring school teaching management (models 3 and 4) (i.e., students’ school time and teachers’ pressure to increase high school promotion rates) have no significant effects on student academic performance.

Generally speaking (model 4), student academic performance is closely related not only to school quality (school reputation) but also to the completeness of school material resources and a school’s socioeconomic status. Students at the best local schools have always had significant advantages in terms of academic performance. Improvement in school material resources is also conducive to improving student academic performance. Consistent with previous research findings, a school’s socioeconomic status profoundly affects student academic performance. For every 10% increase in the proportion of white-collar workers or college graduates among students’ parents, the student academic performance can be improved by roughly 0.05 standard points (= 0.515/10).

At the same time, a school’s overall academic level at enrollment and the individual learning foundation at enrollment also play an important role in student academic performance. Statistical estimates show that when the overall academic level at enrollment, which is measured by the median class ranking of enrolled students in sixth grade, increases by 1%, student scores can be increased by approximately 0.01 standard points. At the individual level, consistent with this, the poorer the learning foundation of students in sixth grade, the lower the student scores in eighth grade. These two findings clearly show that their initial levels largely influence students’ current academic performance and demonstrate a need to control for their initial level when studying students’ current academic performance (as shown in models 5 and 7).

However, some school-level factors related to teachers’ characteristics, including teacher quality, school teaching environments reflected by teacher-student interactions, and teachers’ pressure to graduate more students to high schools, have no significant correlations with student academic performance. The effect of teacher quality is not significant, which may be related to the generally good positioning of teachers in various schools.¹⁰ CEPS data show that the average proportion of teachers with a bachelor degree and above is roughly 83% (see Table 1), and that variability among schools is limited. So it is difficult to find a measurable impact. Before considering students’ learning foundations, school teaching environments have a significant impact on students’ current academic performance, but after applying learning foundations, the effect of schools’ teaching environment measured based on teacher-student interactions is no longer significant. This may be due to the weak correlation between teacher-student interactions and students’ learning foundations. Regarding teachers’ pressure to increase students’ high school promotion rates, as this study examines eighth grade students who still have some time before taking high school entrance exams, pressure may not translate into effective behavior. For another dimension of school teaching management, students’ hours spent at school daily, we also failed to find a significant relationship between this variable and student academic performance.

In a word, when we take student academic performance from the second-wave survey as a dependent variable, we find that the higher the ranking (school reputation), the richer the material resources, the better the overall academic level and individual learning foundations of students enrolled, the higher the socioeconomic status of parents, and the better student academic performance. This not only coincides with previous research findings but also confirms empirical observations to a certain extent. Such findings based on an observation that parents’ school choice behaviors and the rise of “school district housing” have been triggered. However, when we consider the change (value-added) in student academic performance from another angle, are these school characteristics still effective? To address this question, we introduce our second series of models, the value-added series models (models 5–7).¹¹

Models 5 to 7 analyze the influence of various factors on student academic performance from a dynamic point of view to minimize the impact of initial individual differences and more clearly identify the “net” role of factors at the school level. Unlike the first series of models (models 1–4) and people’s levels of experience and understanding, the second series of models’ estimates show that factors closely related to students’ current academic performance do not always promote student academic improvements equally.

Once the dependent variables are changed to the value-added of student academic performance, the difference between upper-middle and lower-middle schools is no longer significant (models 5 and 6). After controlling for the socioeconomic status of schools, top-ranked schools also lose significant advantages (model 5 compared to model 6). In terms of resources, school material resources have always had a significant effect, while teacher quality has no significant effect. The academic level of student sources (model 7), including the overall academic level of a school’s students at enrollment and students’ learning foundations upon entering a school, is significant. Generally speaking, a school’s socioeconomic status, material resources, and students’ academic level at enrollment are not only closely related to students’ current academic performance but also have a significant impact on student academic improvements over a certain period.

Specifically (model 7), the higher the socioeconomic status of a school, the greater the improvement in student academic performance. The data show that for every 10% increase in a school’s socioeconomic status, student academic performance can be improved by roughly 0. 04 standard points (= 0.397/10) over a certain period. The higher the overall academic level of the students at enrollment, the more remarkable the progress students can achieve, and for every one score decrease in students’ learning foundations in the sixth grade, student progress declines by roughly 0.206 standard points. In addition, teacher quality and teaching environments reflected by teachers’ encouragement of students and school management systems have no significant impact on student academic progress. Therefore, only hypotheses 2A (material resources), 4A (school socioeconomic status), and 4B (student source level) are verified while our other hypotheses are not.

Generally speaking, this study finds that when the increase/value-added of student academic performance is analyzed as a dependent variable, after controlling for other variables, schools with higher overall socioeconomic status, more complete material facilities, and better academic preparation for students at enrollment can effectively improve student academic performance in the long term; students with a good learning foundation before entering middle school are more equipped to improve their academic level. By contrast, with all other variables being equal, schools’ high ranking in their local regions, better teacher quality, more extended teaching periods. Or with teachers’ stronger pressure to increase high school promotion rates are not always effective at improving students’ academic performance even though their students’ current academic performance may be better.

Considering the two sets of models together, we are not difficult to find that a school’s socioeconomic status, material resources, and students’ academic level at enrollment have significant effects, regardless of which dependent variable we use. The high socioeconomic status of a school not only represents a high-quality peer environment/effect (Palardy 2013) but also strong family resources and support for a given school. Therefore, a strong academic atmosphere and resource advantages created by a school can not only affect student academic performance but also promote their academic progress. Students who study at schools with a high overall student academic level at enrollment also have more room to progress, which may also be reflected in peer effects. Related studies have found that peer groups heavily affect child and adolescent development. Peers also serve as role models and promote the development of norms (peer pressure) and encourage students in the same settings to develop similar values and learning attitudes and to promote each other’s academic progress (Ryan 2001). In addition, as far as so-called “good” schools are concerned, while their students do perform better, these “good” schools do not always have a significant effect on improving student academic performance, that is, after controlling for overall family background and students’ academic levels at enrollment, “good” schools recognized by local communities do not always have stronger processing ability. The same is true of the quality of teachers.

Further research (through the establishment of an HLM random coefficient model, not shown) also demonstrates that a school’s advantages in socioeconomic status, material resources, and overall student academic level at enrollment are consistent for students with different learning foundations. Even when students with weak learning foundations enter “good” schools, they do not experience a superior academic improvement.

Conclusion and discussion

Family and school are the two most important places for education. The role of the family in student academic performance has been widely acknowledged, but researchers have not reached a consensus on the impacts of school-related factors on student academic performance or on the specific ways in which such impacts occur.

The present comparative study found that compared to those in developed countries, schools in developing countries have stronger impacts on student academic performance (Heyneman 1976). For China, the study found a correlation between schools (especially school characteristics and resources) and student performance. The academic performance of key school students is significantly better than that of students at non-key schools, and in general, the teachers and material resources of key schools are superior to those of non-key schools (Wang et al. 2017). Such research results not only conform to parents’ observations of reality and lay the foundation for their choices and actions but also constitute a basis for government policy-making. Parents strive to send their children to the best local schools, causing the price of “school district housing” to soar and fueling public doubt in the equality of education. The government expects to balance educational resources by encouraging prominent schools to merge with weak schools, establishing education groups, and implementing other ways to bridge the gap between schools in terms of prestige, teacher quality, and material resources (Wang 2014; Yang 2015).

However, can schools truly convert bad qualities into good ones and greatly promote student academic progress? Can sending children to a good school recognized by the local community improve their academic performance and help them achieve a reversal? Is there a considerable difference between different schools in terms of training students? Most previous studies have not been able to answer these questions as they failed to control differences in students’ learning foundations and the schools’ overall academic levels of students at the time of enrollment.

Based on previous studies, the present study extends the dependent variable to the change/value-added of student academic performance over a certain period with the intention to more clearly determine the quality of schools and the role of specific school features in improving student performance through an analysis of dynamic changes in student scores. More specifically, through this study, we hope to determine whether schools with higher rankings and more resources can better improve student academic performance. The results of this study can help parents make future education decisions and inform the government to formulate education policies. Our findings help us better understand the role of schools in improving student academic performance and more accurately ascertain the impacts, mechanisms, and direction (expansion/maintenance/shrinkage) of school effectiveness amidst social inequities brought about by student family backgrounds.

Contrary to common perceptions, this study finds that while student academic performance enrolled at “good” schools is indeed better, “good” schools are not always able to improve student performance more effectively from a dynamic point of view. In other words, student academic performance does not make a qualitative leap upon entering prominent schools. The same is true for overall teacher quality and the effectiveness of school management systems.

However, a school’s overall socioeconomic status, student academic level at enrollment, and the completeness of a school’s material resources and facilities can improve student performance to a certain extent and more effectively. The overall socioeconomic status of a school and student academic level at enrollment reflects the importance of learning with others (peer effects). On the one hand, the socioeconomic status of a school represents the overall family backgrounds of its students, and student family resources can be used by a school as cultural and social capital to encourage its students’ academic achievement. On the other hand, this also reflects a kind of peer atmosphere and academic environment formed by the gathering of students with similar family backgrounds, affecting student learning attitudes and performance (Coleman et al. 1966; Dreeben and Barr 1988). The overall academic level of students at enrollment reflects the starting point of student learning, and the academic environment developed as a result of competent students gathering in one school. The completeness of school material resources also reflects the capacity of a school to gather, mobilize, and attract resources to a certain extent. A school can, in turn, use these resources to promote student academic progress.

However, further results show that while the above school characteristics can indeed promote student academic progress to a certain extent, they are unlikely to help students with a poor learning foundation at the time of enrollment achieve a “reversal.” Instead, regardless of whether a student’s learning foundation is strong or weak, school effectiveness is the same for all students. Students with a poor learning foundation will not make better progress than those with a better foundation upon entering higher socioeconomic status schools. However, they may make slightly better progress than those of a comparable level studying at lower socioeconomic status schools.

In contrast, the individual-level factors, including learning foundation and learning effort (learning habits), play a more important role in promoting academic performance. When students confronted learning difficulty in the sixth grade, this will greatly limit their future progress, and cultivating good learning habits will effectively promote future academic performance.

Generally speaking, this study verifies longstanding wisdom that an internal cause forms the basis for change and development in terms of dynamic academic development. Rather, if we want students to achieve better results and progress faster, it is not enough to send students to good schools. More importantly, students should lay a solid foundation for learning and cultivate good learning habits from an early age. However, schools also play a role to a certain extent. In improving student academic performance, a school’s peer environment and academic atmosphere play a more important role than its material resources and teacher quality.

To some extent, this discovery overturns the conventional view that schools can “turn stone into gold.” In contrast, this study confirms that “gold will shine everywhere.” The learning process is a process of accumulation. Through early accumulation and comparing notes with competent peers, individuals are more likely to accumulate more and develop more potential later on.

In addition, the findings of this study reflect the importance of family background in school education. Student family backgrounds affect student academic performance through their efforts. More importantly, the collective atmosphere created by the gathering of students with similar family backgrounds also plays an important role in student academic performance. As the social and economic status of families is stable, the socioeconomic status of schools has become the focus of educational reform in some countries (Crosnoe 2009; Palardy 2013). To reduce education inequality, governments and societies should encourage students with low socioeconomic status to enter schools of high socioeconomic status, as well as allow students from disadvantaged families to engage in school environments that are conducive to improving their academic performance.

As far as this study is concerned, differences in socioeconomic status between schools of compulsory education may more reflect the residential segregation reflected in the school district system. Preventing the effects of further social inequity and residential segregation on student education and occupational attainment via schools’ social compositions and teaching environments must be prioritized in the planning of urban residential patterns and the formulation of school enrollment policies.

Of course, this study still presents many limitations. First, the study uses student scores on cognitive ability as a proxy variable for academic performance. While there is a strong correlation between the two, cognitive ability emphasizes students’ basic logical thinking and problem-solving skills. It does not always directly involve the specific knowledge taught in school curricula and is thus not fully equivalent to academic performance. Therefore, deviations may exist in the findings of this study. Second, a period of 1 year is used as the time interval to measure the value-added of student academic performance. In examining academic development, an observation period of 1 year is relatively short. As some students may not achieve their potential over this timeframe, it is difficult to demonstrate the final results of academic development. If the changes in student academic performance were monitored over a more extended observation period, our research findings might have been more meaningful. Third, this study is an early attempt to offer a tentative account of school effectiveness, and we hope to use more waves of CEPS data to carry out a more in-depth and detailed exploration in future work.

In addition, there are limitations in the data collection methods that compromise our analysis. Some individual factors, such as family background and learning habits, will affect students’ first and second grades at the same time. Whether the impact on academic value-added originates from one’s family or oneself as an individual or school-related factors is not determined in this paper. If the latter is the case, these effects can be partially attributed to school effects. More detailed studies may also take into account the potential impacts of summer effects on student academic progress (academic behavior during summer vacation more closely reflects the role of the family) and the ways to mitigate them to obtain the net effect of school and so on. We can only hope that the present work can play a role in generating valuable information and helping the sociology of education uncover the operating mechanisms of school effectiveness.

At last, it should be emphasized that this study discusses the relationship between various characteristics of schools and changes in student academic performance rather than whether school education affects student academic performance. Therefore, the results of this study do not suggest that school education in China does not promote student academic performance. Furthermore, the impact of schools on students should encourage the development of moral, intellectual, physical, and esthetic education, as the development of learning/cognition constitutes only one aspect of the role of schooling. Thus, the present study is only a partial investigation of school effectiveness, and in the future, we hope that more information can help us conduct comprehensive research on school effectiveness. Finally, this study explores the impact of schools on compulsory education on student academic performance. As high schools show greater variability, the findings of this study cannot be extended to a discussion of high schools.