Dynamic Rubrics

Author Information
Author(s): 
Conklin, Heather C., Tuten, J. Terrell, and VanderMeulen, Matt
Institution(s) or Organization(s) Where EP Occurred: 
Ashford University
Effective Practice Abstract/Summary
Abstract/Summary of Effective Practice: 
Ashford University’s use of cutting-edge, web-based, dynamic rubrics in eight online writing-intensive courses has improved the efficiency and effectiveness of grading, feedback provision, and assessment. The rubricsallow for assessment of assignments, competencies, program outcomes, and institutional outcomes simultaneously using real-timedata, assessment of student performance, and tailored feedback on writing assignments. To date, the dynamic rubrics have generated over 300,000 data points.  The rubrics align with the pillars of learning effectiveness, access, and scale, which may produce greater faculty and student satisfaction.
Description of the Effective Practice
Description of the Effective Practice: 
According to The New York Times, “The Association of American Colleges and Universities recently asked employers who hire at least 25 percent of their workforce from two- or four-year colleges what they want institutions to teach… 89 percent said they wanted more emphasis on ‘the ability to effectively communicate orally and in writing’, 81 percent asked for better ‘critical thinking and analytical reasoning skills’ and 70 percent were looking for ‘the ability to innovate and be creative.’” Critical thinking and writing skills are essential for academic and professional success; however, a significant amount of high quality feedback is needed to help students develop these skills. Providing such feedback on each assignment is difficult, time-consuming, and labor-intensive. Instructors must manually incorporate feedback into every assignment, which often leads to inconsistent feedback that may not directly address each student’s individual learning needs.
 
To improve feedback quantity and quality, Ashford University implemented cutting-edge, web-based, dynamic rubrics into eight online, high-volume, entry-level, writing-intensive courses during the summer of 2008. Ashford University is the first university to use dynamic rubrics for large-scale assessment of student performance on writing assignments. Starting summer 2010, new rubrics will be launched in additional Ashford University undergraduate courses. 
 
The design and implementation of Ashford University’s dynamic rubrics were driven by several key needs:
·         Consistent and rich feedback tailored to students’ individual learning needs;
·         Efficient grading and feedback provision on writing assignments in an online environment with approximately 50,000 students; and
·         In-depth, real-time data to enhance student learning, program/institutional assessment, curricular improvement, and accountability.
 
Ashford University’s dynamic rubrics allow for the simultaneous assessment of writing assignments, competencies, program outcomes, and institutional outcomes. Rubric criteria are built from assignment descriptions. These criteria are tagged with competencies, program outcomes, and institutional outcomes. The integrative properties of the rubrics occur when instructors grade student writing assignments using the software; assessment data is collected passively.  
 
In addition to collecting assessment data, rubric criteria can contain pre-written feedback with embedded active hyperlinks where appropriate, as well as provide each student feedback tailored to his/her specific learning needs on every writing assignment. The integration of grading, providing feedback, and assessing using the rubrics provides scalability, consistency, and efficiency, which directly benefits students, instructors, assessment personnel, and administrators.
 
Collecting performance data for all students allows for individual- and aggregate-level analyses. Analyses can be cross-sectional or longitudinal in design. To date, the dynamic rubrics have generated over 300,000 data points. Rich, real-time assessment reports can be created from the high-quality, rubric-generated data. The assessment reports allow for targeted, data-driven improvements to courses, programs, and the institution to enhance curricula, accountability, and student learning.
 
Ashford University is designing real-time individual student performance reports, known as Rich Report Cards. They include customized feedback with embedded active hyperlinks, allowing students to get guidance they need when they need it. The provision of real-time, consistent, rich, and tailored feedback through writing assignments and Rich Report Cards target students’ specific learning needs and provide guidance to improve learning. 
Supporting Information for this Effective Practice
Evidence of Effectiveness: 
 
Ashford University examined the amount of feedback on 100 randomly selected written assignments in an entry-level General Education writing course pre- and post- rubric implementation. Students received over five times the amount of feedback when writing assignments were graded using dynamic rubrics. In another study, student scores increased from the first to the last writing assignment. 
How does this practice relate to pillars?: 
 
Ashford University examined the amount of feedback on 100 randomly selected written assignments in an entry-level General Education writing course pre- and post- rubric implementation. Students received over five times the amount of feedback when writing assignments were graded using dynamic rubrics. In another study, student scores increased from the first to the last writing assignment. 
Equipment necessary to implement Effective Practice: 
 
Ashford University’s dynamic rubrics are built using the Waypoint Outcomes software. The software is commercially available. Institutions can customize rubrics to meet their own needs or use Ashford University’s rubric model.
 
In addition to the software, an integrative assessment-centered framework is necessary for building and implementing rubrics because the alignment of competencies, assignment criteria, and outcomes is essential for obtaining useful assessment data. This alignment has spawned several considerations for building dynamic rubrics: 1) use of the rubrics for grading, assessment, and/or other purposes; 2) use of odd v. even numbered item response values; 3) number of response items necessary to show growth/change in student performance; 4) weighted versus non-weighted criteria; 5) normalized scores; and 6) equivalence in item response values across assignments, courses, and programs. 
Estimate the probable costs associated with this practice: 
 
Ashford University’s dynamic rubrics afford savings on time, labor, and resources for instructors and assessment personnel because grading, feedback, and assessment processes are conducted simultaneously. Furthermore, detailed assessment data can be analyzed and reported in real-time, thus greatly reducing the potentially high costs associated with curricular and institutional changes (or lack thereof) resulting from delayed, insufficient, and/or poor quality assessment data. Finally, the software for building the dynamic rubrics is not priced on a per student basis, and students do not have to pay any fees to receive feedback reports. 
 
References, supporting documents: 

 

Contact(s) for this Effective Practice
Effective Practice Contact: 
Terrell Tuten