Holistic Data Analysis for Effective School Evaluation by Uzma Shakeel, Vice Principal at Generation’s School in Karachi
We live in an age where data is highly valued. Yet, collecting and gathering data in itself does not necessarily lead to effective evaluations. Once the purpose of collecting the data is clear and determined, the key is to examine and interpret the data holistically. In doing so, we need to be mindful of the following questions:
- How accurate and precise is the data?
- What does it tell us?
- What does it not tell us?
- Which data might have been overlooked?
- What questions evolve out of this data?
The third and fourth questions mentioned above are derived from one of the most prevalent cognitive biases i.e., survivorship bias. Survivorship bias occurs when researchers only consider existing or surviving observations and fail to consider observations that have ceased to exist. (Elston, 2021)
When too many planes weren’t returning from missions during World War 2, the Statistical Research Group (SRG) was asked to look into the problem. The SRG’s task was to figure out how to best protect the planes. While the majority believed that much more armour must be put on the parts with most bullets, Abraham Wald – a Hungarian mathematician – offered a different perspective when studying the problem. He stated that the majority’s decision to reinforce the planes in the most commonly hit areas emerged as a result of survivorship bias. This was because crucial data from the planes that didn’t survive was ignored and not being taken into consideration. Hence, the extra armour did not belong to parts of the plane that could survive a lot of bullets, but to the parts that couldn’t.
Being an educationist, I have come across the concept of survivorship bias while analysing data for school evaluation. Hence, I would like to share some of my insights on how school leaders can see beyond the obvious for in-depth analysis of data.
Schools have a lot of data including, student and staff attendance, student academic achievements, enrolment, withdrawals, teacher retention, parental information, fee structure, etc. All this data becomes valuable when schools use it for their benefit. Keeping the concept of survivorship bias in mind, I have picked the following three areas based on my personal experience to facilitate schools to analyse data holistically for effective school evaluation.
1. Teacher retention
To fully understand and evaluate teacher retention, we should definitely look into the reasons why teachers would dedicate their lives to institutions, but at the same time, we must consider the reasons for staff turnover. Analysing this data would be beneficial in overcoming the problems that are off the radar and hence improve teacher retention. Proactive schools keep a record and identify the causes of the turnover. Although some turnover is considered necessary and healthy for schools, too much instability can be harmful and may create serious organisational challenges (Allensworth et al., 2009)
While annual appraisals are an essential tool for identifying teacher growth and improvement, keeping track of teachers’ performance over a period of years would provide better information about their trajectory. Moreover, lesson observations and learning walks provide deeper insight into problems that teachers might be facing.
2. Student retention
Many institutions believe that increasing student enrolment is important for their success and growth. However, it is critical to observe the increasing or decreasing trend of admission withdrawals over a reasonable number of years in order to apply student retention strategies sensibly. Just focusing on student enrolment could result in survivorship bias. Hence, student retention is another equally important factor for an institution’s advancement and success.
School leaders, faculty, students, and parental engagement play an imperative role in ensuring student retention. School leaders provide the students with opportunities such as scholarships and academic/physical facilitation, i.e. enhancing student’s academic performance and supporting their physical wellbeing at school. The faculty provides a safe environment, academic counselling, and advice that helps students to engage in collaborative activities. Ultimately, students themselves also contribute towards retention. For example, their self-motivation and the ability to choose a role model significantly contributes to their learning success. (Lau, 2003)
Parents’ feedback gathered through annual or periodical surveys should be analysed critically for better evaluation of student retention too. Sometimes parents would not want to disclose the reasons for leaving school. Hence, it is essential to gauge their verbal feedback and engage them in meaningful school activities.
3. Students’ academic performance
Analysis of students’ academic performance is the core activity of any learning organisation. Sometimes we tend to take students’ learning behavior objectively and make decisions based on a particular set of results during the year. However, we then ignore their performance over a period of years and fail to observe their learning pattern due to non-availability of complete data or poor documentation. In addition, an important aspect of the students’ learning that is often overlooked is their participation and achievements in cross-curricular activities. Usually, parents feel that engaging in cross-curricular activities at school has an adverse effect on students’ academic success. However, my long experience suggests that students who engage themselves in cross-curricular activities and keep a balance seem to have higher academic achievements and success. Recent research also shows that cross-curricular involvement has a positive influence on students’ academic performance, class attendance, and self-image. (Anjum, 2021)
Conclusion
Keeping survivorship bias in mind, a systemic approach towards holistic data analysis will yield better and deeper insights and minimise missed opportunities for identifying problems. This process will help towards effective and comprehensive school evaluation.
Learn more about the benefits of School Self-Evaluation
Read our Cambridge standards for school self-evaluation
Bibliography:
Allensworth, E., Ponisciak, S., & Mazzeo, C. (2009). The Schools Teachers Leave: Teacher Mobility in Chicago Public Schools. Chicago, IL: University of Chicago Consortium on Chicago School Research. https://www.researchgate.net/publication/234655051_The_Schools_Teachers_Leave_Teacher_Mobility_in_Chicago_Public_Schools
Anjum, S. (2021). Impact of Extracurricular Activities on Academic Performance of Students at Secondary Level. International Journal of Applied Guidance and Counseling, 2(2), 7-14. DOI:10.26486/ijagc.v2i2.1869
Lau, L. K. (2003). Institutional factors affecting student retention. Education, 124(1), 126-137.
Elston, D. M. (2021). Survivorship Bias. Journal of the American Academy of Dermatology. DOI: https://doi.org/10.1016/j.jaad.2021.06.845
