Abstract
The use of learning objectives and success criteria has been reported to have a positive impact on student achievement and attitudes to learning (Clarke, 2008,2013; Lenoret al., 2014; Mansell et al., 2009). However, the consistent use of specific learning objectives and success criteria is not established in schools in Shanghai and there exist few studies that investigate their influence on learning.
This study explores the impact of learning objectives and success criteria on learning and pupil attitudesin primary school mathematics lessons. The findings indicate that the use of learning objectives and success criteria is associated with an increase in student achievement in a paper and pencil test (effect size 0.6). An exploration of student attitudes revealed that both motivation, self-efficacy and self-regulation was reportedly increased (P<0.001) when both learning objectives and success criteria are shared in mathematics lessons. It was observed that success criteria were more strongly correlated with higher motivation, self-efficacy and self-regulation than learning objectives. This indicates that the use of success criteria; andlesser extent learning objectives, may result in higher levels of pupil achievement as a consequence of higher levels of student motivation, self-efficacy and self-regulation.
Furthermore, this study provides evidence of the successful integration of formative assessment practices into primary schools in Shanghai through collaboration between state education and a private education group.
Key words:learning objectives; success criteria; motivation, self-efficacy
Introduction
The importance of assessment for learning (AfL) has long been associated as having a positive impact on pupil learning (Florez and Sammons,2013;Hodgson and Pyle,2010;Tierney and Charland,2007). Since the 1990s, over 250 articles about AfLhave been published.The seminal work of Wiliamand Black (1998; 2009) found that formative assessment strategies can significantly improve student academic achievement.Since, John Hattie (2008), whose meta-analysis of more than 800 publications found that providing formative assessment in the classroom had a significant impact on student academic achievement; ranking fifth among all factors influencing academic achievements.
There are four main practices commonly associated with AfL (Black and Wiliam, 1998):
- establishing clear learning objectives and success criteria
- eliciting and interpreting learning evidence for example through questioning
- providing effective feedback to close the gap in learning
- encouraging peer and self-assessment
AfLis also reported tohave an impact on learners’ learning motivation, self-efficacy and metacognition all of which may be factors in supporting higher levels of reported student academic achievement (Stiggins,2006;MacPhail and Halbert 2010;OECD,2008). Moreover, it has been shown that the use of AfL can also promote the development of studentself-regulation often identified as attitudes and aptitudes thatunderpin academic achievement (Lenoret al.,2014; Assessment Reform Group, 1999; Mansell et al. 2009; Clark, 2015). A number of research articles claim that self-regulation raises levels of focus, flexibility, confidence and achievement and includes core elements such as setting goals, planning, monitoring progress and adjusting learning methods(Bandura 1986, 1997; Zimmerman 2002; Pintrich 1999, 2004; Black and Wiliam, 2009). Reported evidence indicates that AfL can have a positive impact on student academic achievement that may be a product of improved motivation, self-efficacy and self-regulation.
Whilst AfL is an established feature of education in many parts of the world (OECD, 2008), it is not a feature of education in Shanghai, China.However, curriculum and education reform for basic education across China places emphasis on the application of formative assessment practices to reduce the pressures associated with summative methods that are a feature of current systems; especially in primary schools.Shanghai MunicipalityEducation Commission implemented “standards-based teaching and assessment” for elementary school from 2013,requiring elementary school teachers to base assessment on curriculum standards and using multiple assessment approaches.However,there has been limited impact on teacherpractice because the policy lacks specific instructional and operational guidance onthe implementation of formative and performance assessment. Lack of effective teachers training is also another reason for limited application of formative assessment.(Zhao Shiguo,2019)The predominant curriculum and pedagogical modelused by elementary teachers in Shanghai is the “Mastery Model for Teaching and Learning”that places emphasis pay on teaching than assessment,which is typically paper-and-pencil test-based summative in approach. Consequently, there are few studies on the use of formative assessment in China.
Education reform in China (2019) is directed by the Ministry of Education re-emphasised the importance of introducing formative assessment. Curriculum, assessment and reform are executed at province and municipality level by an education commission. At district level, national and local educational initiatives are led by a research and training instituteand its team of trainers, researchers and inspectors. District level research and traininginstitutehold responsibility for developing teacher practice and curriculum. Working with researchers and education leaders from Wellington College China, Pudong District research and training institute, the largest district in Shanghai, planned and implementedtraining onAfL practices for a sample of selected schools. It was decided to focus on the four aspects of AfL with teachers initial trained in the implementation of learning objectives and success criteria. This decision was made because effective application of the other three aspects of AfL are dependent on effective learning objectives and success criteria (Hazel Crichton ﹠Ann McDaid,2016; Hanover Research,2014.
This study seeks to explore the impact of learning objective and success criteria on learning in primary schools in Shanghai. Primary school mathematics teachers in one district of Shanghai, PudongDistrict, were trained in the use of specific learning objectives and success criteria and the impact on student achievement and student motivation, self-efficacy and self-regulation analysed.
Research Design and methodology
This study involves the research and training centre in Pudong, Shanghai, training a sample of teachers to implement specific learning objectives and success criteria in primary school mathematics lessons over a semester. The subsequent impact on student achievement in a paper and pencil test along with student reported motivation, self-efficacy and self-regulation.
Definition of key terms
Below are the definitions used during the work undertaken across primary schools in Shanghai, China.
Learning objective: The skills and knowledge that a student should possess on successful completion of a lesson or phase of study
Success criteria: The measures used to determine whether, and how well, learners have met the learning objectives.
Motivation: The attribute that moves us to do or not to do something (Gredler, Broussard and Garrison, 2004)
Self-efficacy: The belief we have in our own abilities, specifically our ability to meet challenge ahead and complete them successfully (Akhtar, 2008).
Self-regulation:Self-regulation learning is an active constructive process whereby learners set goals for theirlearning and monitor, regulate, and control their cognition, motivation, and behaviour,guided and constrained by their goals and the contextual features of the environment. (Pintrich and Zusho2002,p. 64)
Training teachers
Five teachers from three schools were selected to participate in the quasi-experimental study on a voluntary basis. All teachers were responsible for teaching Grade 4 students (9 to 10 years old) mathematics. The teachers were trained in the development and use of learning object and success criteria as part of a partnership between the Pudong Research and Training Institute and Wellington College China in which through a series of workshops seminars and demonstration classes teachers were introduced to and applied AFL strategies. Prior to the formal intervention, an initiation meeting was also held, during which the participating teachers were briefed on the design of the research project. Teachers worked collaborativelyto determine learning objectives and success criteria forlessons. During the study,as an inspector, one of the authors visited the schools twice a week, during which the teachers would be observed teaching, gather feedback progress in the implementation of learning objectives and success criteria and to share ideas on developing practice.
Sample group
The sample involved 187 students in Grade 4(9-10 years old) in primary schools across Pudong, a district in Shanghai. Five mathematics classes in Grade 4 were selected randomly in three schools. An independent sample t-test was carried out for student knowledge of learning objectives, success criteria and attitudes to mathematics of students in the five classes showed that there was no significant difference between classes and schoolsfor each dimension. The treatment group comprised five mathematics classes of Grade 4 pupils in three schools that experienced teaching using consistent use of learning objectives and success criteria. The control group was made up of all other Grade 4 pupils in the same three schools.
Research approach
This study adopts a quasi-experimental design, in which mathematics teachers for five Grade 4 classesin three schools employ specific learning objectives and success criteria as a teaching intervention. The control sample comprises other grade 4 mathematics classes in the same three schools.
The study conducted a pre-intervention and post-intervention tests with 194 and 187 pupils respectively which comprised paper and pencil mathematics test, questionnaire and interviews with a sample of pupils. For the purposes of control, the number and gender of the five classes experiencing the intervention were relatively similar. During the study, all students in the sample were taught the unit of “understanding of decimals” with teachers leading lessons in the intervention classes employed the consistent use of learning objectives and success criteria. All classes experienced four hours of mathematics every week during the study. Learning objectives and success criteria for the unit were jointly determined through the form of collective lesson preparation by teachers leading the intervention classes. The study lasted eight weeks.
Pre and post measures comprised capturing student achievement in pre and post mathematics tests and gathered student perspectives through a questionnaire that contained items focusing on learning objectives, success criteria, motivation, self-efficacy and self-regulation; see below, followed by structured interviews with a sub-sample of students who were asked similar questions to those on the questionnaire.
Paper and pencil test
The test undertaken was consistent for all students in both the control and treatment cohorts. The test items were developed by inspectors at the research and training centre and were consistent with tests typically used across the district for primary school mathematics. The content focused on the topic “understanding decimals” and followed guidance from the curriculum content and textbook used extensively across the municipality of Shanghai. The test was conducted in consistent controlled conditions.
Questionnaire
The questionnaire was organised into the following three sections: 1. motivation; 4 items, 2. self-efficacy, 1 item and 3. self-regulation; 3 items. Student responsesto the questionnaire were recorded using a likert-scale; “strongly disagree”, “disagree”, “uncertain”, “agree” and “totally agree” with the following values applied: 1, 2, 3, 4 and 5 respectively. Factor analysis carried out on responses to the questionnaire indicated that the KMO value is 0.862, and the coefficient of internal consistencywas 0.812, indicating that the questionnaire displayed adequate reliability and validity.
Structured interview
Structured interviews were conducted with the students in the study through a stratified random sampling approach. The questions posed by the researchers were structured around those in the questionnaire.
Data analysis
SPSS 20.0 statistical software was used in this study for statistical analysis of the obtained data. Descriptive analysis, correlation analysis and regression analysis were used. For comparison of test outcomes between the control and intervention group, effect size was calculated between the two samples.
Results
Analysis of test outcomes
Student performance in a paper and pencil mathematics test undertaken post intervention was compared between the control group and intervention group. It was demonstrated that the intervention was associated with higherachievement in the test with an effect size 0.6, a relatively high effect size, relative to the control group. Thus, the evidence indicates that students in the intervention group
performed significantly better in the mathematics assessment.
Table 1: Post-test paper and pencil test data (n=70)
Questionnaire
Data from the questionnaire misreported in Table 2. Analysis indicates that student motivation increased by 10.3% post intervention. This was accompanied by an increase by 9% of students reporting that they respond to teacher questions and an 8.2%in reported student participation in class discussion. However, a small increase of 2.6%was recorded in students committed to completing homework post intervention.Studentreported self-efficacyincreased by 8.2% post intervention. Student self-regulation was enhanced post intervention as determinedby 7.4% increase in students frequently self-reflecting, an increase of 9.6% of pupils previewing or reviewing work but only a minor increase (1.2%) post intervention in the proportion of students who reported changing their approach to learning schedule to support learning in mathematics.Overall, reported data suggests that levels of motivation, self-efficacy and self-regulation increased post intervention.
Table 2: Student responses to the questionnaire (n=194(pre), n=187(post))
Correlation analysis between learning objectives, success criteria and student motivation, self-efficacy and self-regulation
a.Learning objectives and student approaches to learning
Correlation analysis for learning objectives is presented in tables 3 to 5. Analysis reveals that:
- Learning objectives displayed a correlation with motivation (p<0.01) with a slight correlation coefficient increase of 0.02 post intervention
- Learning objectives displayed a greater positive correlation with self-efficacy (<0.05 pre and <0.01 post intervention respectively) with a larger correlation coefficient increase of 0.112 post intervention
- Learning objectives are positively correlated with self-regulation (P<0.01) with an increase in correlation coefficient of 0.036 post intervention
Table 3 Correlation analysis of learning objectives and student motivation
Table 4 Correlation analysis of learning objectives and self-efficacy
Table 5Correlation analysis of learning objectives and self- regulation
b.Correlation analysis of success criteria and student approaches to learning
Correlation analysis for success criteria is presented in tables 6 to 8. Analysis reveals that:
- Success criteria displayed a positive correlation with motivation (P<0.01) with an increase in correlation coefficient of 0.62 post intervention
- Success criteria displayed a positive correlation with self-efficacy (<0.05 pre and <0.01 post intervention respectively) with a larger correlation coefficient increase of 0.211 post intervention
- Success criteria displayed a positive correlation with self- regulation (P<0.01) with an increase in correlation coefficient of 0.17 post intervention
Table 6 Correlation analysis of success criteria and motivation
Table 7 Correlation analysis of success criteria and self-efficacy
Table 8 Correlation analysis of success criteria and self-regulation
The analysis suggests that success criteria displays a stronger correlation with motivation, self-efficacy and self-regulation than learning objectives. This is most markedly apparent in the increases post intervention for student motivation and self-efficacy.
Regression analysis learning objectives and success criteria on self-regulation
a.Regression analysis learning objectives on student approaches to learning
Regression analysis for learning objectives is presented in tables 9. Analysis reveals that:
- In general, there is a positive regression between learning objectives and motivation, self-efficacy and self-regulation both pre and post intervention
- Learning objectives displayed a positive regression with student motivation; 17.9% (adjusted R2=0.179) pre intervention and 19.7% (adjusted R2=0.197) post intervention
- Learning objectives displayed a lower regression with self-efficacy; 2.4% (adjusted R2=0.024) pre intervention and 7.6% (adjusted R2=0.076) post intervention
- Learning objectives displayed a positive regression with student self-adjustment: 14.1% (adjusted R2=0.141) pre intervention and 17% (adjusted 0.17) post intervention
Table 9 Regression analysis of learning objectives on self-regulation
- Regression analysis of success criteria and student approaches to learning
Regression analysis for success criteria and self-regulation is presented in tables 11. Analysis reveals that:
- In general, there is a positive regression between succession criteria and self-regulation both pre and post intervention
- Success criteria displayed a positive regression with student motivation; 15.4% (adjusted R2=0.154) pre intervention and 20.8% (adjusted R2=0.208) post intervention
- Success criteria displayed a lower regression with self-efficacy; 2.1% (adjusted R2=0.021) pre intervention and 13.6% (adjusted R2=0.136) post intervention
- Success criteria displayed a positive regression with student self-adjustment: 16.2% (adjusted R2=0.162) pre intervention and 17.6% (adjusted 0.176) post intervention
Table 0Regression analysis of success criteria on self-regulation
It is evident that learning objectives and success criteria have a varied but positive correlation with three elements of self-regulation; motivation, self-efficacy and self-adjustment, but that there is a stronger correlation for success criteria with student motivation, self-efficacy and self-regulation.
Discussion
This study has demonstrated that AfL practices synonymous with pedagogy in schools in the UK, and beyond, can be developed an applied in primary schools in Shanghai, were such practices are less well developed (Cui Yunhuo,2010;Zhong Qiquan,2012;Yang Xiangdong,2009). The actions of the research and training institute in Pudong, Shanghai, resulted in primary school maths teachers consistently and effectively using learning objectives and success criteria.
Teachers who took part in the quasi-experiment, said they had been accustomed to focusing on their teaching plan before taking part in it, focusing more on how to teach than on how to assessment, and often the classroom assessment is simply seen as oral praise, criticism ,excises or quiz . After participating in this quasi-experiment, they realized deeply that classroom assessment is a very professional thing, and it is a process of reasoning based on learning evidence. If they can make good use of classroom assessment, especially the two key strategies of AfLlearning objectives and success criteria ,- they can not only improve students’ learning enthusiasm to a great extent, a sense of wellbeing and involvement, but also to promote their more autonomous learning.
The consistent and effective use of learning objectives and success criteria was associated with higher levels of student performance in a paper and pencil test relative to students in the control group. Thus, these findings are congruent with other work in this field (Reed,2012; Lenoret al., 2014; Mansell et al., 2009), though they contribute a the relative minor body of work in this field for education in China. Possible reasons for higher student performance in the test will be explored below.
Students from the intervention group reported that learning objectives and success criteria are important in assisting learning in mathematics; most markedly for success criteria. Moreover, the use of both learning objectives and success criteria are correlated with higher levels of student motivation, self-efficacyand self-regulation. This is consistent with findingsof Locke and Latham (1990) who postulated that the use of learning objectives and success criteria, particularly, make explicit what successful learning entails which in turn permits students to envisage achieving the desired outcomes. Locke and Latham(1990) showed that goal setting has positive impact on student performance, hence the use of learning objectives and success criteria in the present study may have made explicit exactly ‘what’ was to be learned and ‘what’ success would entail. Martin(2006) found that ‘personal bests’ had a great positive impact on educational aspirations, love for school, participation in class, and insistence on tasks. This studyindicates that after intervention, student understanding of learning objectives is positively correlated with motivation and self-regulation to a moderate degree, although the correlation between self-efficacy is very weak. This may be due to giving students a clear direction and expectations and giving students a sense of control over their learning, thus improving their motivation to learn. This is supported in through student interviews:
“I think sharing learning objectives has a great impact on me. In class, I know more about what to learn in this class, and can better understand this class, so that I can concentrate more in class” (ht4501)
“If you don’t know learning objectives, you don’t know what to learn in this class” (zp4617)
“…because learning objectives helped me know the direction of learning” (ht4508)
“..the teacher told me to learn the learning objective, which helped me prefer to raise my hand to answer the questions” (ht4702)
“Tell us the learning objectives, and we can more clearly understand what we need to learn and where we need to learn” (ht4731)
John Hattie (2009) found that success criteria had a positive impact on student learning. This indicates that, alongside learning objectives, making explicitwhat success entails provides students clarity on what successful performanceinvolves is important in supporting learners.This study demonstrates that success criteria is correlated, to a greater degree withmotivation, self-efficacy and self-regulation than learning objectives. This may be because success criteriaprovide students with a clear understanding of what is required and what successful performance entails and this clarity, in turn,raises levels of motivation to learn. Moreover, success criteriaprovide a reference for students to evaluate their own performance and as such adjust their behaviours and thinking in order to improve performance or overcome misconceptions and challenges.This is supported by student interviews:
“I think by telling me the success criteria, I can know exactly where I went wrong.” (zp4612)
“.. if we complete the success criteria; we will prove that we have learned the content of this lesson.” (gq4301)
“…because the success criteria can let us know what we know and what we do not know” (zp4616)
“Because knowing the success criteria can help us evaluate ourselves.” (ht4735)
“I think it is very important for teachers to tell us the criteria for success in the classroom. Because knowing the criterion of success, we can better measure ourselves, … we can more effectively do more problems and improve our maths.” (ht4531)
“I think the criteria for success have had a huge impact on my mathematics. For example, it lets me know whether I had mastered all the knowledge for this class. If I don’t master all or have some problems, I can practice more and improve on my weak points” (ht4536)
“The success criteria are more important than the learning objective, I can improve my work in this class which makes me learn more.” (ht4301).
Therefore, the higher levels of pupil achievement in thisstudy may be a consequence of the use of learning objectives and perhaps more specifically success criteria raising student levels of motivation, self-efficacy and self-regulation.
Conclusion
In conclusion, the consistent use of learning objectives and success criteria is associated with increased student performance in a paper and pencil maths test for primary school students studying mathematics in Pudong, a district in Shanghai. Success criteria, and lesser extent learning objectives, are positively correlated with increasedmotivation, self-efficacy and self-regulation for students.This may be becausesuccess criteria outline the key steps that students need to complete in order to achieve learning objectives (Shirley Clarke, 2004). Thisstudysuggests that success criteria have a higher correlation with learning motivation, self-efficacy and self-regulation than learning objectives.Importantly, this study has demonstrated the successful integration into primary school in Shanghai, China of formative assessment practices synonymous with practices in the UK, and beyond.
Therefore, it is proposed that both wider development of learning objectives and success criteria along with further application of the four aspects of AfL will assist in realising educational reform in Shanghai and beyond. Furthermore, it demonstrates the effective collaboration between the established education system in Shanghai with Wellington College China. Such partnerships may serve as an effective means of delivering the desired education reform across China.
References
Black, P. and Wiliam, D. (2009) Developing the theory of formative assessment.Educational Assessment Evaluation and Accountability21(1):5-31
Hanover Research,Agust,(2014).The Impact of formative Assessment and Learning Intentions on Student Achievement, www.hanoverresearch.com.
Hazel Crichton ﹠Ann McDaid,(2016). Learning Intention and Success Criteria: Learners’ and Teachers’ views. The Curriculum Journal ,27:2,190-203. Stiggins R.,(2006).Assessment for learning: A key to Motivation and Achievement .Phi Delta Kappaninternational, 2(2):1-19.
Nicol,D.Macfarlane-Dick (2006).formative assessment and self-regulated learning：A Model and Seven Principles of Good Feedback Practice.”Studies in higher Education,p.7.
Lenor Santos Jorge Pinto (2013).the development of self-regulation through assessment criteria [J].Procedia-Social and Behavioral Sciences 112(2014):907-915.
Wiliam, D.; Lee, C.; Harrison, C. & Black, P. J. (2004). Teachers developing assessment
for learning: impact on student achievement. Assessment in Education: Principles Policy
and Practice, 11(1), 49-65.
Pintrich, P. R. &Zusho, A. (2002). Student motivation and self-regulated learning in the collegeclassroom, in: J. C. Smart & W.G. Tierney (Eds) Higher Education: handbook of theory andresearch (vol. XVII) (New York, Agathon Press).
Deborah K. Reed (2012).Clearly Communicating the Learning Objective
Matters!, Middle School Journal, 43:5, 16-24.
Florez, M. T., & Sammons, P. (2013). Assessment for learning: Effects and impact. Reading: CfBT Education Trust.
Department for Education and Skills(2007). Assessment for Learning 8 schools project report.
Retrieved July 9, 2019, from http://www.standards.dfes.gov.uk
Clarke, S. (2008). Active learning through formative assessment, London: Hodder Education. p.81.
Clarke, S. (2013). Outstanding formative assessment. Oxon: Hodder Education.
Cui Yunhuo (2010).Assessment for learning: the new A New Paradigm of students academic achievement assessment. Educational ScienceResearch,(3):15.
Yang Xiangdong(2009). The role and Reconstruction of Classroom assessment，The Global Education,Vol.38(9):42-46.
Zhong Qiquan(2012).The Challenge of Classroom assessment, The Global Education, Vol.41(1):10-16.