CONTEXT:
Most instructors would agree that it is challenging to deliver asynchronously a course like statistics, which involves the application of computational and analytical skills. Most students would also opt to take such a course face-to-face if they are not constrained by the venue or schedule. Having observed a high rate of D/F/W (grade "D" or "F" or withdrawal) during the first few weeks when she taught two sections of this 16-week asynchronous, online course for upper-division undergraduate students, one of several instructors for the course conducted a pilot study to assess the effect, if any, on student learning of adding a synchronous component to the course.
RESEARCH QUESTION:
Does adding a synchronous component to an asynchronous online statistics course improve student learning?
METHODS:
The sample consisted of 31 undergraduate students each from the Fall 2014 and Spring 2015 semesters. For each semester, the students were selected from two sections of the same 16-week online course taught by the same instructor. Students that were still registered in the course at the end of the semester were included; therefore, the sample excluded students that had withdrawn during the semester and included students that stopped participating in course activities but did not withdraw. During data analysis, however, the latter group, consisting of a total of five students, was excluded as being outliers.
Between the 6th week and 13th week of the fall semester, the instructor offered students four web meetings that were an additional, optional course activity, since the course was originally developed by other faculty without such meetings. During these meetings, she reviewed course materials and assignments and students had the opportunity to ask questions. In the following semester, similar web meetings were incorporated into the course calendar from the beginning. In both cases, students received extra credit for participation in these optional meetings.
Student learning was measured by the final grade before addition of any extra credit points (total points earned as a percentage of maximum possible points). This metric excluded extra credit resulting from participation in the web meetings or other reasons. Outliers were defined as the percentages that were (a) below the value calculated from the first quartile of the data set minus 1.5 times the interquartile range or (b) above the value calculated from the third quartile of the data set plus 1.5 times the interquartile range. Linear regression analyses were performed, using Microsoft Excel¨ 2010, to test the null hypothesis that the number of web meetings attended by a student and the student's final grade were independent. The analyses were run with the data of the fall semester and spring semester separately and then combined. The level of significance (alpha) was set at 0.05.
RESULTS:
After the exclusion of outliers from the regression analyses, none of the regression coefficients is statistically significant at alpha = 0.05. The p-values are in the 0.94 to 0.99 range.
CONCLUSIONS:
Based on the results of this study, the null hypothesis is not rejected. No statistical significance is found to support the research hypothesis that the number of web meetings attended predicts improved student learning outcome.
DISCUSSION:
Teaching abstract concepts and applications in the online, asynchronous environment is challenging, partly because students must be motivated to study course materials on their own and ask questions to seek clarifications and thorough understanding. Experience shows that many students are reluctant to do this either because they are shy or because they are indifferent. It was anticipated that the synchronous component added would at least help the shy students, since the indifferent would be less likely to participate in the optional activity. The results of this pilot study show that the modified delivery format does not have an effect on student learning as measured by the final grade before extra credits. Interestingly, anecdotal evidence gathered from student comments appear to show that the students find the web meetings helpful.
This study has several limitations. Since it is a pilot study, the sample size is small. It is not a randomized controlled study and there is no comparison group. A large portion of the course materials and assignments were revised in the second semester; therefore the grades of the two semesters are not directly comparable although in both semesters the course would meet the overall course and program objectives. Also, after each web meeting, an electronic handout of the presentation slides was posted in the course, so students who did not attend a web meeting could access the handout. They could have benefited from being presented the materials in a way different from the original lecture and following the worked examples on the slides. This would imply that the study would have underestimated the positive impact of the web meetings. The instructor provided extensive one-on-one tutoring to the students that came forward to ask for assistance, which might have helped students achieve higher grades even when they did not attend the web meetings. Some students might have a family member, friend, or tutor that could help them. Finally, generalization to other subjects besides statistics or other academic levels besides upper-division undergraduate students may be inappropriate since the students might possess different skill sets.
The study does have the advantage that the student population taking this course is relatively homogeneous and consistent, with a high proportion being working adults and all being students in academic programs in health professions.
A follow-up study on a larger scale should examine some of the potential confounding factors. A separate intervention is planned that would involve online course-specific tutoring that is not currently available to distance learners of this institution as a student support service.
SESSION OUTCOMES / IMPACT ON INDUSTRY:
Participants of this session will be informed of the results of recent research that might influence their practice. They may also be motivated to conduct similar research and add to the knowledge base that will benefit all researchers and practitioners.
