Online Learning Efficacy Research Database

Author Information
Author(s): 
Katie Linder
Author(s): 
Mary Ellen Dello Stritto
Institution(s) or Organization(s) Where EP Occurred: 
Oregon State University
Effective Practice Abstract/Summary
Abstract/Summary of Effective Practice: 

As the field of online education continues to grow and mature, and as skepticism for online teaching and learning practices remains a prevalent challenge to the field, a searchable database of online efficacy research is needed. The Online Learning Efficacy Research Database (https://ecampus.oregonstate.edu/research/projects/online-learning-effica...) offers users a comprehensive list of studies that detail the outcomes of student learning compared across online, blended/hybrid and face-to-face modalities. Users can search the database by keyword, discipline, modality, sample size, education level, date range or academic journal.

Description of the Effective Practice
Description of the Effective Practice: 

There is research that demonstrates the equivalency of learning outcomes for online and blended environments versus face-to-face modalities. However, a comprehensive collection of this research was not available in an easy-to-find and up-to-date database for faculty and other higher education stakeholders to review. The goal of The Online Learning Efficacy Research Database (https://ecampus.oregonstate.edu/research/projects/online-learning-effica...) was to create a searchable resource of academic studies on the learning outcomes of online and/or hybrid education in comparison to face-to-face environments. Database users can find a comprehensive list of studies that detail the outcomes of student learning compared across online, blended/hybrid and face-to-face modalities. Users can search the database by keyword, discipline, modality, sample size, education level, date range or academic journal.
The studies included in the database are chosen based on two main criteria:
• The study includes a comparison of instructional modalities including face-to-face, fully online, hybrid/blended or web-facilitated. (In some studies, online modalities, such as hybrid/blended and fully online, are compared with one another.) Studies that do not include at least one modality comparison are excluded.

• The study includes measurement of at least one student performance outcome, such as exam scores, assignment scores, course grades, GPA or another outcome. Studies that only measure student perceptions or satisfaction are excluded from the database.
Currently, the database has 219 citations across 74 discrete disciplines from 161 academic journals. The Online Learning Efficacy Research Database is updated each month with additions of new studies that fit the database parameters.

Supporting Information for this Effective Practice
Evidence of Effectiveness: 

Since the Online Learning Efficacy Research Database launched in November 2017, there have been more than 10,300 pageviews from 41 unique countries.
While those numbers speak highly of the need for this type of database, most of the measurable outcomes we can report thus far are anecdotal. Many individuals across the country have contacted us to praise the database, with one calling it “useful work that fills a void” and another referring to the database as “genius.” Students have contacted us regarding how to cite the database in their research papers. At conferences and via email individuals have told us that they are sharing the database with their colleagues. Several people from across the country have also contacted to us to suggest additions to the database for us to review.
We have also been contacted by researchers asking us about a range of areas outside of the database parameters including topics such as online teaching and learning research by student affairs professionals, comparisons of synchronous and asynchronous classrooms, and impacts of early access to online courses for registered students, among many other topics. The number of people who have contacted us with such a diverse range of research inquiries implies that the database has also situated the ECRU as a leader in online teaching and learning research resources.
We anticipate that the database will be of service to dissertations and meta-studies, but it is too early to measure the impact in that way.
Attached to this nomination are two examples of comments the database has received on Twitter.

How does this practice relate to pillars?: 

Access:
Faculty are being asked to take their courses online, but they are interested in learning about the effectiveness of online education before they spend time designing an online class. This database addresses that need and provides an opportunity for faculty and other distance education stakeholders from across the world to gain access to the most up-to-date efficacy studies that can help inform their decisions to move courses and programs online. Since many decisions regarding which courses and programs to move online are made in disciplinary contexts, this database offers an important resource since it is searchable by field of study.
There are also multiple ways to search the database, providing better user functionality and ease. The database allows users to find discipline-specific research that compares two or more modalities (e.g. online versus hybrid). Users can search the database by keyword, discipline, modality, sample size, education level, date range and journal name. The database also includes the ability to filter results by discipline, modality, sample size and peer review status.

Learning Effectiveness:
This database is not meant to convince faculty that online teaching and learning is always effective, but rather provide a resource for them to locate references and seek out those studies themselves to make their own assessments of the outcomes of those studies. It is important for faculty to do this comparative assessment prior to bringing their course online in order to effectively create a learning environment where online student learning outcomes are equivalent to a face-to-face environment.
To provide context for this resource, since it is accessed by researchers from a range of institution types and disciplines, a frequently asked questions page (https://ecampus.oregonstate.edu/research/projects/online-learning-effica...) has been included with the database detailing the criteria used for inclusion and how the search criteria for each database citation was determined.
Faculty Satisfaction:

Faculty locally and nationally have contacted us to praise the database and inform us that they are sharing the database with colleagues. We have been contacted by a range of researchers with a diverse range of research inquiries, further supporting the concept that this database is meeting a need in the field.

Student Satisfaction:
Students have contacted us regarding how to cite the database in their research papers. We anticipate that the database will be of service to students writing dissertations and meta-studies as well.

Scale (institutional commitment to achieve capacity enrollment via cost effectiveness):

It was important to us that this database be free and accessible to scholars and distance education stakeholders around the world. Oregon State University Ecampus has invested resources not only in creating this database but also in sharing it widely as an open educational resource. In addition to promoting the database at conferences through presentations and poster sessions, we have also invested in print and digital marketing efforts so that online teaching and learning stakeholders in a range of institutions and positions can learn about this resource.

Equipment necessary to implement Effective Practice: 

To build the database, we used PHP, MySQL, CSS, JavaScript/jQuery, and Font Awesome.

The research database is a PHP/MySQL application. The database was designed to meet the filtering, searching, and sorting requirements specified by the research team. The Ecampus web team designed the user interface with accessible, responsive web design techniques. They used Font Awesome iconography to represent categories and key characteristics of each citation.

Ecampus staff with expertise in web applications, database design, and research design and methodology were integral to the project

Estimate the probable costs associated with this practice: 

The cost associated with the database was primarily the staff time required to search and gather research articles, build a spreadsheet, design the database and web interface, and maintain the database. This project is an illustrative example of the ongoing collaboration between the Ecampus Research Unit and our IT/web development and marketing teams. Staff from each group came together to envision the database goals, frame user needs and user experience, and discuss the possibilities for database search criteria. This partnership continued as the ECRU staff began searching for and cleaning the database entries, being attentive to formatting the data in a way that would be useful for the IT/web development team. As the IT/web development team built the database and accompanying website, several meetings occurred to test and provide feedback from a user perspective.
The marketing team also provided input on user experience from a non-researcher perspective and helped the ECRU to create a dissemination plan to share out about the database through conference presentations, industry media advertisements, social media posts, and emails to colleagues and distance education stakeholders.
The Online Learning Efficacy Research Database could not have been built without the contributions and creativity from both the Ecampus Research Unit, the IT/web development team, and our marketing team.
The development of the database took approximately 12 weeks to complete, with team members from the teams described above offering their time as needed and as available. Updating and maintaining the database requires approximately 16 hours of ECRU staff member time per month.

References, supporting documents: