A Tool for Faculty Peer Review of Online Teaching

Author Information
Author(s): 
Ann H. Taylor
Institution(s) or Organization(s) Where EP Occurred: 
Penn State University
Effective Practice Abstract/Summary
Abstract/Summary of Effective Practice: 

To address the need for online course peer review in the College of Earth and Mineral Sciences at Penn State, the Dutton e-Education Institute has designed, implemented, and assessed a peer review process for online teaching. The Peer Review Guide for Online Teaching at Penn State is composed of two parts: 1) an Instructor Input Form to be completed for the reviewer by the reviewee in advance of the peer review and 2) the actual Peer Review Guide for Online Teaching at Penn State, which is to be completed by the reviewer during the peer review. Following the peer review, the reviewer summarizes her observations in a document that is to be included in the reviewee's dossier—identical to the procedure followed in resident instruction. Reviewers are encouraged to share the completed Guide with the reviewee, as well.

Description of the Effective Practice
Description of the Effective Practice: 

The peer review of teaching–like the peer review of research–is a widely accepted mechanism for promoting and assuring quality academic work, and is required the purpose of promotion and tenure at Penn State. The peer review process in resident instruction typically involves a faculty reviewer observing a peer’s classroom. The reviewer then summarizes her observations in a document that is to be included in the reviewee’s dossier.

To address the need for online course peer review in the College of Earth and Mineral Sciences at Penn State, the Dutton Institute has designed, implemented, and assessed a peer review process for online teaching. The Peer Review Guide for Online Teaching at Penn State that we have developed is based on the “Seven Principles for Good Practice in Undergraduate Education,” a summary of 50 years of higher education research that addressed good teaching and learning practices. While instruments such as end-of-course surveys provide a measure of student satisfaction with a course, the Seven Principles provide a useful framework to evaluate the effectiveness of online teaching. Each adapted principle is described in detail in the Guide, including examples of evidence of how a principle may be met in an online course. Resources for additional information are also included.

The Peer Review Guide for Online Teaching at Penn State is composed of two parts: 1) an Instructor Input Form to be completed for the reviewer by the reviewee in advance of the peer review and 2) the actual Peer Review Guide for Online Teaching at Penn State, which is to be completed by the reviewer during the peer review. Following the peer review, the reviewer summarizes her observations in a document that is to be included in the reviewee's dossier—identical to the procedure followed in resident instruction. Reviewers are encouraged to share the completed Guide with the reviewee, as well.

Use of this peer review process has already been fully embraced by one academic department within the College of Earth and Mineral Sciences and is now being expanded to other units. While initial pilot tests utilized online teaching faculty as the peer reviewers, we will begin utilizing faculty who are only familiar with traditional face-to-face instruction as peer reviewers, as well. It is hoped that this "cross-fertilization" will break down many of the myths that still surround online teaching at our institution, as face-to-face faculty will be able to examine online teaching and learning effectiveness first hand.

Supporting Information for this Effective Practice
Evidence of Effectiveness: 

The Peer Review Guide for Online Teaching at Penn State is composed of two parts:

  1. An Instructor Input Form to be completed for the reviewer by the reviewee in advance of the peer review, and
  2. The actual Peer Review Guide for Online Teaching at Penn State, which is to be completed by the reviewer during the peer review.
How does this practice relate to pillars?: 

The peer review process we have developed helps to ensure adequate support for faculty in online course delivery. To begin with, faculty are given a copy of the Guide in advance of teaching their online course so they can review the criteria upon which their teaching will be evaluated. Through that use, the Guide serves as a valuable faculty development tool, especially when there is an opportunity to review the document with a learning designer or other faculty development expert. The peer review process itself provides reviewees with meaningful and focused feedback on their online teaching that includes information about resources they can use to improve their teaching practices. When necessary, departments can use the information obtained through this process to develop a customized faculty development plan for the reviewee that will help that individual improve future performance and teaching satisfaction.

Equipment necessary to implement Effective Practice: 

Faculty reviewers simply need access to the reviewee's online course. The process encourages reviewer and reviewee to communicate throughout the process, especially when evidence of effective teaching practices outlined in the Guide are not clearly available. For example, the faculty reviewer may need to request from the reviewee copies of e-mail interactions between instructor and students when seeking evidence of faculty-student interactions.

Estimate the probable costs associated with this practice: 

The only cost involved with this practice is time. The peer review process is designed to take approximately 2 hours for a reviewer to complete. Some faculty involved in our pilot test, however, reported voluntarily spending longer amounts of time reviewing their peer's online course out of professional curiosity!

References, supporting documents: 

Chickering, A. & Gamson, Z. (1987). Seven principles for good
practice in undergraduate education. AAHE Bulletin (39 )7.

Other Comments: 

We presented our work at the 16th Annual Sloan-C Conference in November 2010.

Contact(s) for this Effective Practice
Effective Practice Contact: 
Ann H. Taylor
Email this contact: 
atb3@psu.edu