The Online Student Support Services Scorecard Implementation

Author Information
Victoria Brown, Ed. D.
Dana R. Willett
Kamarie B. Carter
Joshua Z. Book
Institution(s) or Organization(s) Where EP Occurred: 
Florida State University System
Institution(s) or Organization(s) Where EP Occurred: 
Texas State University
Effective Practice Abstract/Summary
Abstract/Summary of Effective Practice: 

The Florida State University System developed and implemented the Online Student Support Services Scorecard during Fall semester 2017 to assess the services provided to online students throughout the system’s universities. At the SACSCOC Conference in December 2017, Florida Atlantic University presented the finalized scorecard, where a distance learning administrator from Texas State University requested permission to adopt and adjust the scorecard to assess the services offered at Texas State University. Texas State University implemented an adaptation of the scorecard in summer 2018. Implementation of the scorecard aligns with the OLC pillar of student access.

Description of the Effective Practice
Description of the Effective Practice: 

NOTE: All scorecard activity described in this narrative occurred prior to OLC’s adoption of the scorecard as an addition to the OLC Scorecard Suite.

The Online Student Services Scorecard is a rubric with 44 quality indicators in 11 categories of admissions, financial aid, pre-enrollment advising, veteran services, career counseling, post-enrollment, orientation, library, disability, technology support and graduate student support. Each of the categories has a range of standards from 2 to 9. The rubric has three levels with scoring for the item from 0 to 2 points. To earn 2 points the institution meets the exemplary service definition, 1 point signifies the service is available at the institution, and 0 points indicate limited or no service.

This scorecard was created through a multi-stage process involving multiple stakeholders in the Florida State University System. The finalized scorecard was presented to the Florida State University System Steering Committee for the Online Education 2025 Strategic Plan and was approved for dissemination in March of 2017. In 2017, both the Florida State University System and Florida College System adopted the Online Student Support Services Scorecard. In 2018, Texas State University adopted and implemented the scorecard. The development of the scorecard and implementation by the Florida State University System, Florida College System, and Texas State University are discussed below.

Florida Atlantic University:

The scorecard was intended to be used as a tool for ongoing improvement assessment. As such, the implementation of the tool should be a process rather than a one-time evaluation of the services resulting in on going improvement. The steps for that process are outlined in Image 1. As was the intended first step, the distance learning leader at Florida Atlantic University began the introspection (Step 1) and initiated conversations (Step 2) with other units on campus to collaboratively evaluate the level of services offered. These conversations built the awareness of the need of student support services for online students and fostered discussion of how to provide those services (Step 3).

In the Fall of 2018, the Center for eLearning held a series of meetings to develop a strategic plan. The directors of the various units across campus were invited to provide input into the expansion of distance learning support services. At that meeting, the scorecard was shared with the directors. The scorecard enabled the directors to see the importance of providing the services for the students. A second meeting was held to establish workflow processes and allocation of resources to support the expansion. In that meeting the directors decided that all students were OWLS and should be integrated into the workflow of their offices rather than the establishment of a parallel student services system (Step 4). The group recommend the following:

• Development of a liaison model- the liaison would represent distance education within each of the service units on campus to support cultural change.
• The liaison would be responsible for
o Building awareness
o Coordinating training
o And ensuring equivalent access to services
• To form a workgroup for online student success to develop a seamless process for handing the students off to the appropriate person with different departments.
• To provide eCounsel/eAdvising with a provision of “live” communication between the student and the advisor.
• Expand the online student service team to include a person for development of improved communication through social media.

Texas State University:

At the SACSCOC December 2017 conference, Florida Atlantic University presented the scorecard to the conference participants. After the presentation, Texas State University immediately requested permission to adopt and modify the scorecard accordingly for its online programs and services. By January 2018, Florida Atlantic University granted this permission to the Office of Distance and Extended Learning at Texas State University.

Leadership in the Office of Distance and Extended Learning at Texas State University presented the Online Student Support Services Scorecard to the Texas State University Distance and Extended Learning Steering Committee in January 2018 for discussion and consideration of adoption. The committee subsequently approved the use of the scorecard as an evaluation instrument (Step 1). Using the scorecard, in May and June 2018, an online services committee assessed services offered to online students (Step 2). After the committee gathered results, the distance learning office sent the scorecard to respective units and departments on campus, asking them to conduct a follow-up self-evaluation of the services currently offered by their units (Step 3).

The Office of Distance and Extended Learning reviewed the results of the administrative review of student services using the scorecard, then reviewed the results of the self-review that the units completed using the components of the scorecard they filled out.

The office will conduct the following three gap analyses:

• Administrative perceptions vs. aspirations of the scorecard rubric
• Self-review perceptions vs. aspirations of the scorecard rubric
• Administrative perceptions vs. self-review perceptions

Reminder: All scorecard activity described in this narrative occurred prior to OLC’s adoption of the scorecard as an addition to the OLC Scorecard Suite.

Supporting Information for this Effective Practice
Evidence of Effectiveness: 

Results of the Recommendations to Date for Florida Atlantic University (Step 5):

• The first student support training session was offered at Florida Atlantic University eLearning Appreciative Advising Workshop
• Laptops and headsets with microphones were distributed to offices needing the additional technology
• Expansion of services provided

The Office of Distance and Extended Learning at Texas State University is currently waiting on the first Steering Committee meeting of fall 2018 to evaluate the scorecard results and discuss potential ways to create an aspirational model, using the scorecard, for the Texas State Distance and Extended Learning Committee and Texas State University departments (Step 4).

Although we are in the midst of obtaining results from the scorecard, we have seen a great deal of responses from departments. Twelve out of the 15 units surveyed on campus have provided results from the quality scorecard, and there have already been many discussions on potential new services for online students. Attached are the returned quality scorecards from the different units on campus that were surveyed.

Other Universities Who Used the Scorecard to Improve Services

o University of North Florida
o University of Florida
o University of West Florida
o University of Central Florida
o University of South Florida

These universities are in various stages in the use of the scorecard to improve online student services.

How does this practice relate to pillars?: 

The implementation of this scorecard relates to the pillar of access for effective practices. This scorecard assesses what services are provided within a multitude of higher educational departments including academic, administrative, and technical units. The scorecard assesses services such as tutoring, advising, counseling, pre-enrollment services, post-enrollment services, accommodations for disabilities, library services, student involvement, technical service, and many more. The Florida State University System and Texas State University’s utilization of this scorecard meets the pillar of access by determining if online students have access to vital resources needed to succeed in college. Analysis of the scorecard assists in creating a strategic plan to implement or improve upon certain services.

Equipment necessary to implement Effective Practice: 

A computer with access to Excel is needed to implement this scorecard. The scorecard is interactive through Excel and allows the user to choose a score from a drop-down menu while allowing a place for notes and recommendations. The scorecard can be shared via email, so access to the internet and an email account is required. Phone and internet services are also required to contact units of an institution or research services offered on their web-pages.

Estimate the probable costs associated with this practice: 

Staff time – effort and time of staff are required to send out, fill out, and analyze the scorecard.

References, supporting documents: 

New Strategic Plan at the Center for eLearning:
eLearning Appreciative Advising Workshop:
Online Student Support One-Stop Web Page:
Improving Online Student Support Services Steps Diagram (attached)
Online Student Support Services Scorecard – Original (attached)
Adapted Online Student Support Services Scorecard – Texas State University (attached)
Email sent to student service departments at Texas State University (attached)
Returned Online Student Support Services Scorecard – Texas State University Departments (multiple department scores - attached)
Preliminary Gap Analysis – Texas State University (attached)

Contact(s) for this Effective Practice
Effective Practice Contact: 
Kamarie B. Carter
Email this contact:
Effective Practice Contact 2: 
Dana R. Willett
Email contact 2:
Effective Practice Contact 3: 
Victoria Brown, Ed. D
Email contact 3: