Are Virtual Preservice Teacher Observations as Effective?

I am researching remote classroom observations of preservice teachers in order to understand their level of effectiveness in comparison to face

-to-face observations. This research is important to future recommendations for the observation of preservice teachers in the Indiana University East School of Education. Research Question

Is there a difference in the effectiveness of preservice teacher evaluation using virtual observation technology rather than face-to-face university supervision?

Introduction

The field of teacher preparation is an ever-evolving landscape. Where corporate American can manage human capital when dealing with inanimate objects, the field of education deals with biotic factors moving both ways. This means that the physical analysis of soft skills, dispositions, grit, behavioral management skills, time management skills and a plethora of other “unseen” teacher traits must be a staple of teacher education programs. In order to assess these traits, states require varying numbers of field placement hours or preservice teachers.

These field placements come with a flow of university supervision and observations. Since the creation of teacher education training, these observations have been conducted between a hired professional (internal or external to the university). However, the COVID-19 pandemic of 2020-2021 created barriers that made face-to-face observations difficult or impossible to conduct.

Luckily, the preceding decade brought about a competitive evolution of Web 2.0 video tools. As a result, some higher education institutions turned to Google suite for file storage and asynchronous evaluation of preservice teacher performance, while others used tools such as GoReact (the product focus in this study)..

My personal passion for this research study is to compare and contrast the benefits and shortcomings of remote (virtual) preservice teacher evaluations. The goal of this research is to provide empirical evidence of the effectiveness of cloud-based remote observation tools on the development and future success of preservice teachers.

Review of the related literature

Heafner, Petty, & Hartshorne (2011) lay a foundation for this study when they state, “Impending changes within teacher education, whether motivated by national or state educational mandates, teacher shortages, shifting teacher education candidate needs, growth of second-career professionals seeking employment, or tighter operating budgets, have paved new and innovative avenues for teacher preparation programs.” All of these factors play crucial roles in the shift of preservice teachers throughout the country. For the purpose of this study, we will look away from the motivation behind remote observations and focus solely on the literature which evaluates its effectiveness. Nevertheless, it is important to note that the COVID-19 pandemic of 2020-2021 has created more research opportunities due to “shifting teacher educational candidate needs”.

What is the purpose of a preservice teacher observation?

Windsor, Kriewaldt, Nash, Lilja, & Thornton (2019) suggest that teacher evaluations create professional conversations using evidence, feedback and dialogue. Windsor et al. (2019) state that observations, “...provide preservice teachers with structured opportunities to reflect upon and possibly rethink their beliefs.” Windsor et al. (2019) go on to argue that observations help ensure professional conversations between mentor teachers, university educators and preservice teachers; thus, helping ensure professional growth and preparation.

In addition to mentorship and professional growth, preservice observations act as methods of accreditation for licensing institutions. According to the Council for the Accreditation of Educator Preparation (2015) evidence of educator preparation programs should be purposeful, show great depths of reflection, show a holistic approach to training and provide both quantitative and qualitative data points. One of these points of data comes from preservice teacher observation notes. Therefore, preservice teacher observations provide experience, professional development, and timely feedback for the growth of future educators, all the while, providing accreditation data necessary to perpetuate schools of education with national and state licensing capabilities.

What is an effective preservice teacher observation?

In a 2010 guide titled The Classroom Observation Process, Jenny Hadingham, Assistant Director of the Centre for University Learning at Rochester University suggested the following steps to effective classroom observation. Step one is the pre-observation briefing of the preservice teacher. Hadingham (2010) suggests that this is the most overlooked step of the teacher evaluation process. Hadingham (2010) goes on to express that the observer needs to present the observation as a “developmental process in which the reviewer’s role is entirely one of constrictive observation, followed up by feedback, designed to support the reviewee.”

Following the pre-observation briefing, it is time for the classroom observation. In step two, the observer’s role is to record what the lecturer did well and expand on what areas could be done differently.

The third section of an effective preservice teacher observation is the post-conference. Hadingham (2010) explains that the “de-brief is a fundamental part of the classroom observation process,” and should “take place within a few days of the classroom observation (so that the information is still fresh in mind).”

Finally, comes the observation report. There are numerous ways universities collect data around the country. Some students use these observations to create relationships that develop into future jobs. However, these mentor relationships are more than just data points.

The set up of effective preservice teacher observations have held ties to the physical classroom protocols associated with the steps listed above. However, a decade after Hadingham’s 2010 study was published, the technology available and extenuating circumstances forced many schools of education to consider video evaluations as an alternative to university supervision in physical spaces.

Why consider virtual observations as a means of evaluating preservice teachers?

Chisenhall (2015) stated that Santagata, Zannoni, and Stigler (2007) as well as Star and Strickland (2008) demonstrated that when preservice teachers are able to observe classroom teaching through video observations they are better able to understand teacher practices, learn to observe the way students think, and observe quality instructional techniques aligned with current research and best practices that may not be observed in a face-to-face setting (Angelici & Santagata, 2010).

The aforementioned research suggests that self-evaluation is more effective using video tools because it allows teachers to “replay” their successes and growth areas. These tools also allow university supervisors to create time stamps that help students align standards, objectives and goals with the rubric associated with observation assessments. In the long-run this creates documentation of preservice teacher growth from field placement to field placement.

What have previous studies found out about virtual observations?

Wash, Bradley & Beck (2014) concluded that remote observations could be used to reduce preservice teacher anxiety while offering comparable detailed feedback. Additionally, Wash et al. (2014) discovered a secondary positive impact on schools of education due to scheduling relief. This created more instructional opportunities for faculty members stretched too thin between faculty duties and observations.

MacMahon, Gradaigh & Ni Ghuidhir (2019) found that virtual observation of preservice teachers had the potential to empower all stakeholders through opportunities offered in the development of stronger links between schools and teacher education institutions. This study also discovered that virtual observations created authentic experiences which enabled professional dialogue, social learning and credible feedback for immediate review (McMahon et al., 2019).

Along with these positive attributes of implementing virtual observation practices, researchers have discovered many hurdles. When interviewing university supervisors regarding the pros and cons of virtual observation, O’Neil, Krause & Douglas (2017) reported three main concerns for virtual observations. These included reduced interaction between the preservice teacher and cooperating teacher, lack of audio quality to hear preservice teacher/student interactions and connectivity/quality issues with video quality.

Additionally, concerns within the O’Neil et al. (2017) study of university supervisors

suggested that there was a disconnect between relationship building. Meaning some preservice teachers reported lacking mentorship connectivity between their US and CTs.

Even though there were reports of benefits and drawback to the process of virtual observations, the majority of available research suggests little to no difference in the quality of evaluations in physical versus virtual spaces. Foundational research by Heafner et al. (2011) states, “By comparison, we purport that observational experiences for all participants were similar and interchangeable without compromising the quality or value of the internship in promoting professional growth.

Method

The above research supports the conversation for schools of education to consider integrating virtual observations into their curriculum track. However, it is important to note that further research needs to be conducted to isolate various areas of effectiveness.

Research Questions

1. When looking at teachers who are considered “completors” by CAEP, are their differences between school corporation evaluations of those observed using physical university supervision and those using virtual observation tools?

2. When completing a self-evaluation survey what are the differences in the perceptions of quality between the preservice teacher, the cooperating teacher and the university supervisor?

3. When looking at past and present EdTPA scores, are there quantitative differences between cohort averages?

Research Design

The first process in the design of this research study is to create a 10-15 question survey regarding the perceived quality of the GoReact platform. Questions in the survey will range from demographics of participants to self-regulated perceptions of virtual observation quality. Additional questions will be open-ended for qualitative feedback of the GoReact experience.

Alongside the distribution and collection of survey data, interviews will be conducted to create qualitative data points. Interviewees will include preservice teachers, cooperating teachers, professors and/or university supervisors.

Comparative data will be analyzed from completor evaluations. Analysis of completor data will focus on the level of administrative evaluations of practicing teachers who have graduated from our program prior to the shift to GoReact virtual observations and those after. These data will be used to validate the difference of quality between teacher performance using physical and virtual observation methods.

Finally, the study will utilize EdTPA scores of student teachers. These data will be evaluated prior to and following the shift to the GoReact virtual observation platform. Analysis will focus on similarities and differences between student productivity in video submissions and narratives.

Context

The past decade has provided many research studies in the comparative analysis of physical and virtual observation effectiveness. As seen in the literature review in this proposal, the majority of research provides lenses into specific academic context (i.e. secondary physical education or university supervisor perceptions). This study will look at elementary teachers practicing in grades K-6.

The overall themes of the available literature suggests a negligible difference in the quality of face-to-face and virtual observations; focusing on preference and choice. This study will use teacher evaluations in other forms (EdTPA, Indiana RISE, and Ohio OTES) to compare the variable of observation type changes.

Data Sources and Collection

Evaluation sources will come from survey data collected through the Qualtrics platform. This will generate analytics and graphics to implement in the final report. Qualitative data will be collected through interviews using Zoom. Zoom transcripts will be analyzed and recorded for more accurate quotations in the research narrative. Analysis of this data will be recorded using Google Documents and Sheets.

EdTPA and teacher evaluation (RISE and OTES) scores will be collected in Google Sheets and analyzed in a side-by-side comparison chart. These charts will be turned into a matrix for visual representation of data, which will be implemented into the final research report.

Data Analysis

Analysis will include potential successes and potential limitations to the study. Data will be shared and discussed with other faculty members in order to add additional rationale. Data will also be cross-reference with those set forth in prior research studies. These will all be used to make recommendations for the future implications of virtual observation tools in the Indiana University East School of Education.

Possible implications of research

At Indiana University, we currently use the GoReact virtual observation platform for elementary preservice observations in field placements leading up to student teaching. We still use face-to-face university supervision in student teaching placements. This research could be used to save students time and the university both money and time through the integration of virtual tools.

Depending on the results of the study, this could solidify a future of virtual observations or could help focus budget allotments to strengthen university supervision. Either way, this helps quantify best practices in the evaluation and support of our graduates; thus, strengthening future student learning.

References

Angelici, G. & Santagata, R. (2010). Studying the impact of lesson analysis framework on preservice teachers’ abilities to reflect on videos of classroom teaching. Journal of Teacher Education, 61(4), 1-17.

Chisenhall DE.( 2016) Preservice teachers' sense of efficacy: Video vs. face-to-face observations. [Order No. 10027111]. https://www-proquest-com.proxyiub.uits.iu.edu/docview/1771637085

Council for the Accreditation of Educator Preparation (2015). CAEP Evidence Guide, Version 2.0. http://caepnet.org/~/media/Files/caep/knowledge-center/caep-evidence-guide.pdf?la=en

Karen D. Hager (2020) Integrating Technology to Improve Teacher Preparation, College Teaching, 68:2, 71-78, DOI: 10.1080/87567555.2020.1723475

Hadingham, J. (2010, September 20). The Classroom Observation Process. Retrieved March 7, 2021, from https://www.rochester.edu/college/cetl/assets/pdf/classroomobservationprocess.pdf

Heafner, T.L., Petty, T.M, & Hartshorne, R. (2011). Evaluating Modes of Teacher Preparation: A Comparison of Face-to-Face and Remote Observations of Graduate Interns. Journal of Digital Learning in Teacher Education, 27(4), 154-164.

MacMahon, B., O Gradaigh, S., & Ni Ghuidhir, S. (2019). Super Vision: The Role of Remote Observation in the Professional Learning of Student Teachers and Novice Placement Tutors. Tech Trends: Linking Research & Practice to Improve Learning, 63(6), 703-710. https://link-springer-com.proxyiub.uits.iu.edu/article/10.1007/s11528-019-00432-z

O’Neil, K., Krause, J.M., & Douglas, S. (2017). University Supervisor Perceptions of Live Remote Supervision in Physical Education Teacher Education. International Journal of Kinesiology in higher Education, 1:4, 113-125, DOI: 10.1080/24711616.2017.1328190

Pickering, L.E., & Walsh, J.E. (2011). Using videoconferencing technology to enhance classroom observation methodology for the instruction of preservice early childhood professionals. Journal of Digital Learning in Teacher Education, 27 (3), 99-108. doi: 10.1080/21532974.2011.10784664

Schmidt, D.A., & Lindstrom, D. (2011). Editor’s remarks. Journal of Digital Learning in Teacher Education, 27(3), 91-91. Doi: 10.1080/21532974.2011.10784662

Wash, P.D., Bradley, G., & Judy, B. (2014). Remote Classroom Observations with Preservice Teachers. Southeastern Regional Association of Teacher Educators, 24 (Fall-Winter), 58-65.

Windsor, S., Kriewaldt, J., Nash, M., Lilja, A., & Thornton, J. (2020). Developing teachers: Adopting observation tools that suspend judgement to stimulate evidence-informed dialogue during the teaching practicum to enrich teacher professional development. Professional Development in Education, 1-15. doi:10.1080/19415257.2020.1712452

Zaier, A., Arslan-Ari, I., & Maina, F. (2020). The use of video annotation tools and informal online discussions to Explore PRESERVICE TEACHERS’ self- and Peer-evaluation of Academic Feedback. Journal of Education, 201(1), 19-27. doi:10.1177/0022057420903269