Sponsor Videos

Canvas
Pearson
Software Secure

Keypath Education

Conference News


Twitter
LinkedIn FaceBook YouTube GooglePlus www.instagram.com/onlinelearningconsortium

 

Download the Mobile App
IOS  |  Android
OLC Mobile App


Make your travel arrangements


Yoga with Jan

Add to my registration

 

American Higher Education in Crises book cover

Join keynote speaker Goldie Blumenstyk for a book signing.

Books are available for pre-purchase for $16.95 (+tax). 
Read more


Conference Program now posted! This year's line-up includes:

 

OLC Excellence and Effective Practice Award Recipients Announced

 

Add/remove sessions from the Program Listing on the website or in the mobile app to create a list of sessions you want to attend!

My Schedule



Join Keynoters Goldie Blumenstyck (Chronicle of Higher Education) and Phil Hill and Michael Feldstein (MindWires Consulting)


BYOD to learn, explore, and share knowledge within this lab environment

Technology
Test Kitchen

Save the Dates

22st Annual OLC International Conference
November 16-18, 2016 | Orlando, Florida | Walt Disney World Swan/Dolphin Resort

OLC Innovate 2016 - Innovations in Blended and Online Learning
April 20-22, 2016 | New Orleans, LA | Sheraton New Orleans Hotel

Assessing Student Learning in Online Courses Using Learning Analytics

#Twitter: 
#olc42943
Presenter(s)
Florence Martin (University of North Carolina at Charlotte, USA)
Additional Authors
Abdou Ndoye (Qatar University, USA)
Session Information
October 15, 2015 - 11:15am
Track: 
Learning Effectiveness
Major Emphasis of Presentation: 
Practical Application
Institutional Level: 
Multiple Levels
Audience Level: 
All
Session Type: 
Discovery Session
Location: 
Atlantic Hall
Section: 
A
Position: 
2
Session Duration: 
45 Minutes
Session: 
Discovery Session 2
Abstract

Learning analytics techniques and data measures for different assessment types (Comprehensive-type, discussion board, reflective focused, and project-based assessments) in online courses are explored.

Extended Abstract

The landscape of technology in education has changed over the last two decades. Online learning has become prevalent. In this study, we examine online learner-centered assessment and how it helps with online teaching and learning to measure the students' progress through the lens of learning analytics. Siemens and Long (2011) proposed that "learning analytics should ultimately be focused on disruption and transformation in education, changing the very nature of teaching, learning, and assessment." Learning analytics is defined as "the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs" (Ferguson, 2012). Learner-centered assessments shift the move from grades, marks and credits to learning, outcomes, and graduating with the skills needed as a professional.

Assessments in Online Learning
The development of online learning in higher education requires schools and teachers to shift their thinking and practices in terms of learning effectiveness. The asynchronously online learning courses provide opportunities for a student centered approach to learning and assessment. In fact, the online learning environment provides a platform for more performance based assessment through immediate feedback, opportunities for individual practice and guidance (Reeves, 2000). These opportunities provided by the online learning environment place educators in the ideal and most desired position of not only monitoring learning events as they happen but also the possibility to take corrective measures and adjust teaching to improve student learning (Romero-Zaldivar, Pardo, Burgos, & Delgado Kloos, 2012). Other benefits that the online learning platform provides with respect to assessment include better monitoring opportunities for student learning and more immediate feedback (Buckingham & Fergusoon , 2012 ; Benson, 2003), more individual practice opportunities (Shuey 2002 ; Orme, 2004). Scholars (Blummer, 2007; Orme, 2004)) have studied assessment practices and techniques that could foster learning effectiveness and continuous improvement in an online learning environment.

Four types of online learning assessment techniques are commonly used. They include: Comprehensive-type, discussion board, reflective focused, and project-based assessments.
Comprehension-type assessment: These are selected response type of assessment. In this type of assessment, students are usually given choices to select a type of answers among many others. Comprehension type assessments aim for students to understand remember and/or memorize concepts and ideas (Van den Broek et al., 2005). They are mostly meant for formative type assessment as they are quicker tools to gather information that can be used to monitor student learning and make necessary adjustments (Cornelius, 2013; Wormeli, 2007 ; Tomlinson, 1999). These include assessment such as multiple-choice, true false, matching ranging etc.

Discussion board: Discussion boards are well-suited for promoting collaboration and interaction among online learners. According to Shuey, (2002), these can be used to assess skills such as reasoning, collaboration, negotiation, argumentation, and debating (clark, Sampson, Weinberger, & Erkens, 2007) and teamwork etcÉ This method of assessment promotes active learning and also allows student to support each other in the form of a learning community and therefore the development of multiple perspectives (Gikandi et al., 2011; Mackey & Evans, 2014)

Reflection-focused assessments: These assessments focus not only on the correct answers of a given problem, but they emphasize the thought processes that lead to that answer (Frederick, 2002). These are assessments for which students are expected to articulate more elaborate responses to questions related to skills and knowledge learned in the course. These assessments allow students to formulate their responses using theoretical and practical knowledge. Examples of these assessments include short answers, essay, minute papers, research papers, reflection papers etc.

Project-based assessment: These assessments include presentations and projects. These types of projects require manual grading based on rubrics. Examples of such skills include the ones related to oral communication (i.e interacting with audience, keeping eye contact, tone of voice etcÉ) which cannot be assessed from written work since they require live or recorded demonstration. Projects based assessment provides an opportunity for students to work in groups or individually, and the interaction among group members can be analyzed as part of this assessment.

Purpose of this Study
The purpose of this study was to identify learning analytics techniques and data measures for different assessment types in online courses. Using a set of structured and non-structured data the study suggests different techniques and analyzes that can be used to enhance online learning.

Method
Data Measures
Some of the data Measures were collected and tabulated in Excel worksheet for the different assessment types in a preservice instructional technology course taught at a southeastern university in the United States. The course was taught in a 15 week time period and had 7 modules and each module included a variety of instructional components including an elearning module, a quiz, and hands on projects. There were 18 students in this online course.

Tableau for Quantitative Data Analysis and Visualization
The data was then imported into a data visualization tool, Tableau. Different visualization techniques were applied. Below is a screenshot of the tableau software. The analysis shown in this paper was not to find statistically meaning relationship but to identify information that can support an instructor to formatively evaluate teaching and learning.

ManyEyes for Qualitative Data Analysis Visualization
Qualitative data were imported into IBM's many eyes to run different visualization patterns.
Results, Discussion and Implication

Results and Discussion
Visualization from the data and discussion of the findings will be presented at the presentation.

Implications
The overarching goal of this research is to provide evidence to support the improvement online teaching and learning. This project will advance that goal of using learning Analytics data for assessment in online and blended learning. Instructors may use this knowledge in online teaching. Instructional designers may use this knowledge to recommend best practices on online course design. Administrators may use these results to design successful online programs. Educational researchers may use this framework to analyze data from the various online assessments within the learning analytics framework. Bringing together these points of view will help improve online teaching and learning.

References will be shared at the session (not included here due to the 1000 word limit)

Lead Presenter

Florence Martin is an Associate Professor in the Instructional Systems Technology program at the University of North Carolina Charlotte. She received her Doctorate and Master's in Educational Technology from Arizona State University. She has a bachelor's degree in Electronics and Communication Engineering from Bharathiyar University, India. Previous to her current position, she taught at University of North Carolina Wilmington for seven years. She also worked on instructional design projects for Shoolini University, Viridis Learning, Maricopa Community College, University of Phoenix, Intel, Cisco Learning Institute, and Arizona State University. She worked as a co-principal investigator on the Digital Visual Literacy NSF grant working with Maricopa Community College District in Arizona. She researches on designing and integrating online learning environments.