Using Media Analytics to Enhance Course Design

Session Evaluations Contest
Questions?

Comments?
Average Per Semester Video Viewing
First Video/DB Alignment Attempt
Video Duration Recommendations
Average Student Engagement
Course Issues
Video number and duration varied week to week
Number ranged from 2 to 6 per week
Duration ranged from 6 minutes to 55 minutes
Alignment between videos, DB assignments, and quizzes/exams unclear
Use of embedded videos in discussion board prevented optimal use of LMS features


Introduction to World Religions
Initially developed in 2012
One of the early courses to integrate video
Large enrollment (100+) survey course
Each religious tradition explained by different faculty specialists
Videos were made by department with Camtasia
Discussion Boards (DB) significant component

FSU ODL Media Usage

2011
2012
2013
2014
2015 Projected
# of Contributions
1,526
4,984
9,332
8,500
9,593
# of Contributors
9
428
979
1,302
1,670
# of Media Plays
5,324
20,226
126,486
286,564
353,293
Office of Distance Learning
Online programs offered since 2001
18 graduate programs
5 undergraduate programs
584 distinct courses
339 graduate courses
245 undergraduate courses
Using Media Analytics to Enhance Course Design
John L. Crow & Dr. Annette Jones
Office of Distance Learning
Florida State University
Improvements to Address Issues
Standardized each week to have 3-4 lectures, each about 10 minutes (8-12 minutes)
Included captions & transcripts of lectures
Redesigned DB prompts to better align with videos and engage more critical thinking
Designed lectures to complement LMS DB functionality
General Recommendations to Instructors
Keep videos less than 10 minutes in length – 7 minutes may be target
Provide transcripts and captioning
Align text and video content with assessments and activities
Reconsider the purpose of the lecture video
Clarify difficult concepts
Synthesize themes from different instructional materials
Provide examples
Do demonstrations/scenarios
Work problems
Provide animation and images that help students visualize content

Session Evaluations Contest
Session Evaluations Contest
Open OLC Conferences Mobile App
Navigate to session to evaluate
Click on "Rate this Session"
Complete Session Evaluation*

(As part of our "green" initiatives, OLC is no longer using paper forms for session evaluations.)
*Contact information required for contest entry but will not be shared with the presenters.
Winners will be contacted post-conference.
Each session evaluation completed (limited to one per session) = one contest entry
Five (5) $25 gift cards will be awarded to five (5) individuals
Must submit evals using the OLC Conferences mobile app
Next Steps for Religion Course Revision
Explore ways to create more interaction within videos, for example, embedded quizzes – increase completion rates
Repackage the use of transcripts and close-captioning as a learning tool, not just as an accessibility tool
Record new, tightly focused lectures (multiple faculty or just one)
Refine and improve DB questions
Student Feedback (Con)
I believe after 7 minutes, it gets boring and hard to follow.
The course lecture videos can be dragged on for too long at some points, but for the most part I enjoy them.
Length is the first. It can be a little daunting and intimidating when presented in video form. Ones attention span is already so narrow when on the computer already that it requires incredible focus to stay engaged. […] This means that I end up using the transcripts as an alternative very frequently. 
I would like to see the lecture in one video rather than multiple videos.
Sometimes the course lecture videos are long and boring and not straight to the point.

Example of Lecture Location Changes (Old)
Student Feedback (Pro)
[The videos] are really not that long and easy to follow. The transcripts are perfect addition to the lecture!
I really like how each lecture video is numbered and relates to the corresponding numbered discussion board. Very helpful if needing to go back and review a certain section of the lecture video.
I think the course lectures are helpful. They aren't too long which is good because it keeps me interested.
I like the course discussion boards because they make me stay on track with the course material each week, without them I would probably never read the chapters or primary texts or listen to the lectures.
There isn't much to dislike about the course lecture videos. They are informative and a good length.

Student Feedback from Revised Course
Find the lecture videos helpful in understanding course material
Utilize both the video captions as well as transcripts in addition to the video
Typically watch a video in its entirety and don't consider video length when choosing to watch
Overwhelmingly view videos on laptops rather than mobile devices

Revised Course Student Engagement
Example of Lecture Location Changes (New)
Engagement Across Course Comparison

21
Click to edit Master title style
10/12/2015

#
10/12/2015

#
Click to edit Master title style
Click to edit Master text styles
Second level
Third level
Fourth level
Fifth level
Click to edit Master text styles
10/12/2015

#
Click to edit Master title style

Click to edit Master text styles
10/12/2015

#
Click to edit Master title style
Click to edit Master text styles
Click to edit Master text styles
Second level
Third level
Fourth level
Fifth level
Click to edit Master text styles
Click to edit Master text styles
Second level
Third level
Fourth level
Fifth level
10/12/2015

#
Click to edit Master title style
Click to edit Master text styles
Second level
Third level
Fourth level
Fifth level
Click to edit Master text styles
Second level
Third level
Fourth level
Fifth level
10/12/2015

#
Click to edit Master title style
Click to edit Master text styles
10/12/2015

#
Header: Adobe Garamond Pro, 40 pt
Click to edit Master text styles
Second level
Third level
Fourth level
Fifth level
Title: Adobe Garamond Pro,
Centered, No All Caps, 54 pt
Presenter Information,
Adobe Garamond Pro, 24 pt
Click to edit Master title style
Click to edit Master text styles
Second level
Third level
Fourth level
Fifth level
10/12/2015

#
Click to edit Master title style
Click to edit Master text styles
Second level
Third level
Fourth level
Fifth level
10/12/2015

#
Click to edit Master title style
Click to edit Master text styles
Second level
Third level
Fourth level
Fifth level
10/12/2015

#

6
Video Viewing Time 2013 vs 2014

Semester Days (Three Semesters)


Avg Play Time Per Day




Average Completion Rate (Spring 2013-Spring 2015)

[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]





Total Lecture Video Plays (May 2013-April 2015)





10/12/2015

Click to edit Master text styles
Second level
Third level
Fourth level
Fifth level

#
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]

Course Titles
* = Graduate Courses


Time (Minutes)




Fall 2015 Video Completion Rates (First 7 Weeks)

[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]
[CELLREF]




Video Duration