The Open SUNY Center for Online Teaching Excellence (COTE) has developed an online course design rubric and process that addresses both the instructional design and accessibility of an online course that is openly licensed for anyone to use and adapt. The aim of the Open SUNY COTE Quality Review (OSCQR) Rubric and Process is to support continuous improvements to the quality and accessibility of online courses, while also providing a system-wide approach to collect data across campuses, institutions, departments, and programs that can be used to inform faculty development, and support large scale online course design review and refresh efforts systematically and consistently.
The Open SUNY Center for Online Teaching Excellence (COTE) has developed an online course design rubric and process that addresses both the instructional design and accessibility of an online course that is openly licensed for anyone to use and adapt. The aim of the Open SUNY COTE Quality Review (OSCQR) Rubric and Process is to support continuous improvements to the quality and accessibility of online courses, while also providing a system-wide approach to collect data across campuses, institutions, departments, and programs that can be used to inform faculty development, and support large scale online course design review and refresh efforts systematically and consistently. The OSCQR rubric and process are currently being used by 23 SUNY institutions.
Working with multi institutional teams of SUNY online instructional designers, librarians, distance learning directors, and technologists, Open SUNY COTE staff started with the Chico rubric, 20 years of SLN research-informed best online practices, the SUNY office of general counsel’s memorandum on accessibility considerations, and conducted a gap analysis with Quality Matters, iNACOL, and Bb exemplary courses. The resulting rubric was also informed by the Community of Inquiry model (Garrison, Anderson, and Archer, 2000), The 7 Principles for Good practice in Undergraduate Education (Chickering & Gamson, 1987), The Adult Learner(Malcom Knowles, 1973), Bloom's Taxonomy (Bloom et al., 1956) and How People Learn (Bransford et al., 1999), and mapped to the Open SUNY COTE fundamental competencies for online teaching. We are continuing the process of building out the citations and annotations associated with each standard in the rubric here: OSCQR Rubric Annotations and Refresh resources – each standard is explained and supported by citations from the literature. A formal literature review is planned and in progress.
There are two components to OSCQR:
A course review with the OSCQR Rubric produces an Action Plan that is framed from the perspective of the Community of Inquiry (CoI) model, to help reviewers assess and target opportunities to improve the course’s social presence, cognitive presence, and teaching presence, in addition to the overall online course educational experience. It substantively addresses accessibility. The OSCQR Accessibility Rubric is based on the recommendations of SUNY’s Office of General Counsel in their 2013 memo, “Accessibility Considerations in the wake of SUNY’s Online Initiatives.” The rubric has been reviewed by members of the FACT2 Accessibility Task Force, and address the legal considerations required to be compliant with Section 508 of the Rehabilitation Act of 1973, New York State Enterprise IT Policy NYS-P08-005, Section 504 of the Rehabilitation Act, and the Americans with Disabilities Act.
The OSCQR Rubric is unique and differs from other online course quality rubrics in several ways. It is not restricted to mature online courses. The Rubric can be used formatively with new online faculty to help guide, inform and influence the design of their new online courses. It is non-evaluative. Conceptually the rubric and process approach course review and refresh as a professional development exercise, to guide faculty in their understanding of improving course design from an effective practices perspective, rather than as a course evaluation, or quality assurance procedure. It prioritizes changes. The Action Plan, that is automatically generated by the course review process, presents recommendations for course design improvements based on the review, and assists in prioritization of course revisions based on the estimated time to make those improvements. The rubric also provides suggestions for course design improvements for each standard that can be selected from a menu of options by each reviewer to supplement reviewer feedback. The OSCQR Rubric can be customized, i.e., standards can be added, edited, and /or eliminated.
The use of the Rubric informs the quality of the instructional design of online courses. It is used formatively to guide online course design, as well as summatively to continuously improve the quality and accessibility of online courses. High levels of adoption (23 campuses so far) and consistently high levels of satisfaction, confidence, and reported learning from both faculty and students are evidence of our effectiveness in the systematic use of online course quality standards in professional development activities, as well as course quality review processes.
Open SUNY COTE Spring 2015 online student survey
2469 student respondents from 42 SUNY campuses
Open SUNY Spring 2015 online faculty survey
402 respondents to the Spring 2015 Faculty Survey.
This practice relates to pillars in the following ways.
Access: Open SUNY is a system-wide initiative of the State University of New York (SUNY) designed to expand access to online learning and meet the increasing demands of the workforce throughout New York State and the world. Open SUNY launched in January 2014 with an initial wave of 6 SUNY campuses and 8 Open SUNY+ online degree programs. The second wave of Open SUNY+ programs launched in January 2015 with 17 campuses and 56 Open SUNY+ online degree programs. Central to Open SUNY are a set of signature elements based in quality assurance for online learning and a series of system-level supports for faculty teaching online, student access to NYS high needs online degree programs and student success, campus and system-wide infrastructure, and innovative instructional models.
Learning Effectiveness: Open SUNY Course Supports, a pillar of the Open SUNY Center for Online Teaching Excellence, has been designed to support online learning effectiveness in online courses and degree programs with a flexible and comprehensive online course quality review and refresh process that is required of of all online courses in Open SUNY+ online degree programs. The OSCQR rubric and process were developed to establish online course quality standards and a process to scale continuous online course quality improvement. Both students and faculty report high levels of engagement and learning in our annual surveys.
Faculty Satisfaction: When online faculty are well prepared to teach online in well designed online learning environments, and have opportunities to continuously improve their online teaching practices, as well as their online course designs, they have positive satisfying experiences teaching online. Open SUNY COTE (formerly the SUNY Learning Network) is well known for success in large-scale online faculty development, and high levels of online faculty satisfaction (Fredericksen, et al, 2000). Today we are using the OSCQR Rubric in new online faculty development. We are also training online instructional designers and online faculty peer course reviewers to use it as a tool to review existing online courses to identify areas that need improvement, and we are issuing badges to online practitioners that receive training, that conduct reviews as a self-assessment or as a peer review, and that have online courses that have been reviewed. Our annual faculty surveys continue to show high levels of faculty satisfaction.
Student Satisfaction: Open SUNY COTE (formerly the SUNY Learning Network) is well known for high levels of online student satisfaction and reported learning (Fredericksen, et al, 2000). Today, by formalizing online course quality standards in the form of the OSCQR Rubric, and systematically applying instructional design and accessibility standards to all Open SUNY+ online courses and degree programs, we have continued to see high levels of student satisfaction and reported learning in our annual surveys.
Scale (institutional commitment to achieve capacity enrollment via cost effectiveness): Now fully operationalized, Open SUNY has scaled and institutionalized the Open SUNY+ signature elements and supports and designed an online course review/refresh process that is open to all SUNY campuses, online programs, courses, and faculty to ensure that all SUNY online students have access to high quality accessible online courses.
No special equipment is necessary to implement OSCQR. What you will need are:
The probable costs associated with the use of the OSCQR Rubric and process are minimal on a small scale, except any costs associated with training faculty, or incentivizing course reviews. The process of conducting course reviews and making online course improvements to refresh course designs takes time, which is often not insignificant depending on many factors. We estimate a course review to take an individual 6-10 hours. The review and refresh of one online course is estimated to be 20% of an FTE for one academic term (16 weeks). Conducting team course reviews with collaborations on course refreshes that include librarians, technologists, and instructional designers, as well as the subject matter expert (SME) is resource intensive. OSCQR has been designed to flexibly support various implementation models, including use as a self-assessment in online faculty professional development, online instructional designer course review, peer faculty/SME course review, and the collaborative multi disciplinary team approach to online course review and refresh. The learning review part of the process to review and plan the next set of course design improvements involves the online instructor and instructional designer.
There is no license fee for use of the rubric. It is shared with a creative Creative Commons license: CC BY-NC-SA 3.0 US. Because the OSCQR Rubric is licensed under Creative Commons, and the Dashboard is licensed under LGPL, the entire process can be shared, used by anyone with no cost, and can be customized to address individual campus environments.
Technical Costs to self-host the OSCQR Rubric and Dashboard:
With a regular Google account there is a 1 hour computing time limitation and there is a limitation of only being able to create 250 spreadsheets a day. We suggest having two Google Apps for Business accounts – one for running the Dashboard, and one for the Self Serve Rubrics. Running out of time or hitting the maximum number of spreadsheets created per day with the limits of two different Google business accounts is unlikely.
Link for Google Apps for Business
Quotas for Google Apps for Business
We have one person responsible for the technical application design of the rubric and dashboard. To change or improve the application in terms of its design, analytics, reports, requires someone with Google spreadsheet programming skills. HelpDesk support and trouble shooting support are also necessary to support those accessing the interactive online rubric and dashboard.
References and supporting documents are found below posted as links.
We welcome inquiries.
Alexandra M. Pickett (conceptual designer)
Dave Ghidiu (application developer)
Robert Piorkowski (operational architect)
Dan Feinberg (project manager and accessibility rubric designer)
Erin Maney (course reviewer)