Lecture Capture Reporting

1

Lecture Capture -Reporting

A detailed guide for staff on the Reporting

available in Lecture Capture

February 2019

Lecture Capture Reporting

2

Contents 1.0 Understanding Course and Student Analytics ............................................................................................. 3

1.1 Viewing Course and Section Analytics ..................................................................................................... 3

1.2 Viewing Student Analytic Data ................................................................................................................ 5

2.0 Engagement and Analytics Metrics Defined (What do the analytics mean?) ............................................. 5

2.1 Attendance (This is not related to Man Met’s SEM\Attendance Monitoring System) ........................... 6

2.2 Video Views ............................................................................................................................................. 6

2.3 Video views for all classes in the section ................................................................................................. 7

2.4 Video views by students .......................................................................................................................... 8

2.5 Video views by students for a class ......................................................................................................... 8

2.6 Notes........................................................................................................................................................ 9

3.0 Configuring YOUR Engagement Score Preferences ..................................................................................... 9

3.1 To define engagement scores ................................................................................................................. 9

3.2 Understanding Engagement Pre-sets .................................................................................................... 10

4.0 Using Engagement Sliders to Create Targeted Engagement Views and Export ........................................ 10

5.0 Comparing Class Grades to Engagement Metrics ..................................................................................... 12

6.0 Downloading Student Analytics to CSV ..................................................................................................... 15

Lecture Capture Reporting

3

1.0 Understanding Course and Student Analytics The ANALYTICS tab for a course/section provides information and feedback regarding student

engagement (this has no link to Man Mets Student Engagement\Attendance Monitoring system)

and classroom participation. Hover over a metric label for a brief description of the value being

shown.

Instructors can view analytics for all classes in a course, for individual classes, and student

engagement information about students for the whole course, or students for a single class. These

analytics are filtered by engagement, attendance, content views, questions asked, notes taken,

and overall participation.

Instructors customise what engagement means to them. They can select pre-set engagement

profiles or customise individual metrics to achieve meaningful engagement scores.

See Engagement and Analytics Metrics Defined (section 2) for a detailed description of each of the

tracked metrics, and how to view and interpret them for both classes and students.

To access course and student analytics:

From the DASHBOARD, click ALL CLASSES for the course you want to see, then click

the ANALYTICS tab.

The selections on the left side of the page allow you to select to view analytics for Classes or

for Students.

You can also select to export analytic metrics to a spreadsheet, and/or export select analytics for

students to your LMS Gradebook.

Clicking Engagement lets you customize the "importance" of certain aspects of participation,

depending on what is most important to you as the instructor. This can help you identify which

students are not active in the way you want them to be.

1.1 Viewing Course and Section Analytics The Analytics tab for a section provides a variety of engagement and analytic data for students and

the classes in your section.

Lecture Capture Reporting

4

See What do the analytics mean? for detailed information on what each of the metrics actually

tracks and displays across all classes and for individual classes.

To view section analytics:

From the Class List page for a section, click the ANALYTICS tab.

The Classes page appears by default, providing overview information for All Classes.

The line-graph at the top shows the information by class date.

Selecting different metrics below the graph highlights or "bolds" that line on the graph. It also

changes the unit of measure (X-axis) on the graph appropriate to the metric selected.

For example, clicking Video Views changes the measurement to a number, indicating how many

times the video for the class was viewed.

Click Hide unselected below the metrics to remove the unselected lines from the graph.

Hover your mouse over a date - the metric buttons change to show the count/percentage for

each item for that date. This is shown in the below figure.

Use the classes drop-down list above the graph to view engagement information for a specific

class.

The Least engaged students list identifies which students do not appear to be participating in

classes or class discussions. Get more information on student participation from the Student

Analytics page.

The Confusing content list provides links to the class content locations that students have flagged

as confusing.

TIP: Review the Confusing content to determine what parts of your presentations students found

confusing.

Lecture Capture Reporting

5

1.2 Viewing Student Analytic Data The Students page of the Analytics tab provides detailed engagement information about each

student in the class.

See Engagement and Analytic Metrics Defined for detailed information on what each of the

metrics actually tracks for individual students both across classes and for particular classes.

To view student analytics:

From the DASHBOARD, click on the Students to Watch statistic shown for a course.

From the Class List page for a course, click the ANALYTICS tab, then click Students from the left

side of the page.

You can select to view the student engagement information for all classes, or use the Classes

drop-down list at the top to view student information for a particular class.

If you select a particular class, you can then use the drop-down on the right side of the page above

the table to view the student details for either the Video or the Presentation published to the

class (if it has both). This is identified in the figure below.

You can also use the Search box to find a particular student if needed.

You also have the option Exporting student information to a Downloadable CSV file, or to your LMS Gradebook if you access Echo360 through an LMS.

2.0 Engagement and Analytics Metrics Defined (What do the analytics mean?) Engagement and analytic information uses six different metrics that combine to determine the

overall "Engagement" score given for students and classes. The Engagement Settings page lets you

assign relative weight to each metric, depending on what is important to you, or what aspects you

want to keep an eye on.

Lecture Capture Reporting

6

However, in order to properly interpret the information being presented, you need to know how

we define each of them, and what information is actually being tracked/logged.

Attendance

Video Views

Video views for all classes in the section

Video views for a specific class

Video views by students

Video views by student for a class

Presentation Views

Questions/Q&A

Notes

Activities

2.1 Attendance (This is not related to Man Met’s SEM\Attendance Monitoring System) Attendance (please note this is not linked to our Attendance Monitoring System)

Definition - If the student clicks on your lecture capture then this is recorded as an attendance in

the reporting.

2.2 Video Views Definition: The number of times students have clicked play in the classroom to view a video. You

can see the total number of views for each class, or the number of views for each student in a

given class. When viewing total number of views for a class, keep in mind that every student view

is counted, whether by different students or multiple times by a single student. Repeated views by

a single student are counted if there is a 15 minute gap between views.

Depending on where you are on the Analytics page, the video views information can be more or less granular.

Lecture Capture Reporting

7

2.3 Video views for all classes in the section In the Classes section of the Analytics tab, if you are viewing All Classes and click Video views, the number indicated provides the total number of views of the video, by class date. However, if 32 views are shown for the date, it could be one student viewing it 32 times, or 32 students viewing it once. Each view is counted as soon as a student selects to play the video in the classroom. Repeated views by a student are counted if there is a 15 minute gap between views.

Video views for a specific class In the Classes section of the Analytics tab, select a specific class

from the drop-down list. The bar graph changes to show information for only that class. In

addition, a second drop-down list appears, allowing you to see Video view or Presentation view

information. Video is shown by default.

NOTICE that the X axis (horizontal) measurement now reflects the timestamp/location in the video

for the view, and the Y axis (vertical) indicates how many views that location in the video has

received. Once again, the count is cumulative; 32 views by one student of a location is counted the

same as 1 view of the location by 32 students.

Hover your mouse over a location in the graph to see the timestamp for that particular location.

Lecture Capture Reporting

8

2.4 Video views by students In the Students section of the Analytics tab, select All Classes from the class drop-down list. All

video views are shown for each student. Again this figure is cumulative; a student with 20 video

views could mean 20 views of a single video, or one view of 20 different videos. A view is counted

as long as the student played the video in the classroom and a gap of at least 15 minutes occurred

between repeated views of a video by a single student.

2.5 Video views by students for a class In the Students section of the Analytic tab, select a particular class from the class drop-down list,

then select Video.

The Video views for that class are shown for each student. This lets you know which students have

viewed that video at all, and which have viewed it multiple times. It also shows what percentage

of the video the student viewed. Remember that a video view count is logged if a student played

the video in the classroom. This page lets you know how much of the video each student actually

watched.

Lecture Capture Reporting

9

2.6 Notes Definition: The number of words entered by students into the Notes panel and/or Study Guide for

the section or class.

This is another reasonably straightforward metric, simply letting you know which students are

taking notes in the Echo360 interface, and how extensive their note-taking is. You cannot see

student notes. All you can do is identify who is taking notes.

The word count is determined by counting the number of distinct groups of letters students are

entering. A "word" is qualified as a set of letters between spaces. (This is as opposed to typing test

word-per-minute counts, where a "word" is qualified as a group of 5 keystrokes.) All this really

means is that a student who uses very big words in their notes will have a smaller "notes word

count" than a student who uses lots of small words.

3.0 Configuring YOUR Engagement Score Preferences An engagement score is a combination of metrics that you customise to make them meaningful to

you. You specify how important each metric is relevant to the other metrics. This creates YOUR

definition of "engagement".

The metrics work together to form the engagement score provided for each student. The

Engagement score is determined based on the student's activity (or inactivity) surrounding your

classes and the media you are presenting, as weighted by the "metric importance" setting you give

each of those metrics. Essentially the student engagement score scales student performance

based on the weight you establish for each of the analytic metrics.

See Engagement and Analytics Metrics Defined for a detailed description of each of the metrics,

what it tracks, and how you can view each for both classes and students.

You can change your definition of engagement at any time. The Engagement settings page also

provides pre-defined presets that you can use or base your settings on. See Understanding

engagement presets below.

NOTE: Engagement Settings MUST be set for EACH SECTION. Engagement settings do not carry

over from one section to another. All metrics are evenly weighted by default.

3.1 To define engagement scores Navigate to the Class List page for the course (click ALL CLASSES from the DASHBOARD or select

the course from the COURSES drop-down list on the main menu).

Click the ANALYTICS tab for the course.

Select Engagement.

Lecture Capture Reporting

10

By default, all of the metrics are evenly weighted. You have the following customization options:

Change the default settings by sliding the end of each metric's bar to the right or left to increase or

decrease the importance of that metric.

Click one of the Presets provided. See Understanding engagement presets below.

Do both! Select a Preset, then customize it by moving the slider to the right or left as appropriate.

All metrics add up to 100%: The "weight" of each metric is a portion of 100% - meaning if you

increase one metric, the rest of the metrics must decrease by a corresponding amount. This is

reflected in the percentages shown for each metric to the right of the slider.

Click APPLY to apply these settings to this course.

Click RESET to return the metrics to their original (or last saved) settings.

You are strongly encouraged to read Engagement and Analytics Metrics Defined to become

comfortable with the details each metric tracks. That will help you weigh them more accurately

based on the things you think are important for students to do to succeed in your classes.

3.2 Understanding Engagement Pre-sets The Engagement Pre-sets are designed to weight the metrics in a way that reflects the importance

of that pre-set. The three pre-sets provided reflect the most common engagement types used by

instructors to determine whether a student is properly participating in the class or not.

These can be used as they are defined in the pre-set, or they can be used as a base from which you

may customise your definition of "what's important".

Click each pre-set to see what settings are used there, and feel free to adjust them for your own

needs.

4.0 Using Engagement Sliders to Create Targeted Engagement Views and

Export

Lecture Capture Reporting

11

Because Echo360 allows you to customise what engagement means to you, you can use the

engagement settings to change (as often as you like) how Engagement is calculated using the data

that already exists for the section/students.

For example, you may decide that you want to see how much of the video media each student is

viewing. You CAN navigate to the Analytics > Students tab and select a class, then select Video as

the media type, and see each student's viewing percentage from there. Then select a different

class, then selecting video...select another class, then select Video... There must be an easier way.

And there is!

You can change your Engagement weighting so that Video Views are the only thing that count

toward engagement scoring. In this case, each student's engagement score for each class will

equal the % viewed of that class' video. When rolled up to the Section level, the total Engagement

score for each student will be the aggregate view % across the classes that have already occurred.

To do this, move the metric's slider you want (in this case Video Views) all the way to the

right AND slide the remaining metrics' sliders all the way to the left. The metric you want should

read 100%.

The image below shows how this setting would look in the Engagement page of the Analytics tab.

BE SURE to click Apply once you have made changes.

Now when you return to the Students page of the Analytics tab, the Engagement Scores for each

student will reflect the percentage of the class videos each has watched. Rolled up across All

Classes, the Engagement percentage shown is an aggregate of the percentage of videos viewed for

classes that have already occurred. This is shown in the below figure.

Lecture Capture Reporting

12

This process can be applied to any of the Engagement metrics available, and can be changed as

often as you like. Changing the engagement sliders simply changes the Engagement calculation

using existing data. Once you click Apply, the calculations are re-performed and the Engagement

percentage shown reflects the new value.

If you want, you can use this method to identify two different metrics as the sole inputs to

engagement. Perhaps you weigh Video Views and Presentation Views equally and want to see

which students are viewing ALL the class materials. Move BOTH sliders (Video Views and

Presentation Views) all the way to the right, and move the remaining sliders all the way to the left.

The percentage shown for the two active metrics should read 50%.

BETTER YET, since Engagement is an exportable value, to a CSV file, having it be recalculated

based on a single metric allows you to change and then export that value as needed. For example,

if you perform the above process to change Engagement Percentage to equal Video Views, then

export grades for all classes that have already occurred, the percentage written to the gradebook

will reflect the video viewed percentage for each student across the selected classes. The same

would be true for an Engagement export to CSV.

Once again, be sure to click Apply once you make changes to the Engagement sliders.

5.0 Comparing Class Grades to Engagement Metrics As an instructor, the Engagement metrics for student interactions are useful for knowing how

involved each student is. However, it may be more useful if you can compare those metrics to the

grades actually being received by each student in your class. This comparison can help you

determine exactly what types of interaction translate to student success.

Echo360 allows you to import grade figures into the Analytics page, then provides a scatterplot

graph that shows the correlation (intersection) of a selected metric with the grade given for each

student.

Lecture Capture Reporting

13

In the above figure, student grades have been uploaded, and the graph shows the correlation

between the Engagement score and the course grade provided by the uploaded CSV file. Selecting

a point on the graph shows the details for that student to the right of the graph, also shown in the

above figure.

See What do the analytics mean? for a detailed description of each of the metrics and what it

tracks. See also Configuring YOUR Engagement Score Preferences for more information on setting

your engagement preferences, to more accurately track what kind of student interaction is

important to you (as total defined engagement).

To populate the correlation graph

Navigate to the Class List page for the course.

Click the ANALYTICS tab, then select Engagement as indicated by the arrows in the above figure.

From the Metric Correlation Graph section at the bottom of the page, click UPLOAD GRADES.

A dialog box appears with a template download link.

Lecture Capture Reporting

14

Click the Download gradebook template link.

The downloaded CSV file contains names and email addresses for each student.

Locate and open the downloaded CSV gradebook template file. It will have the same name as the

term-course-section you are viewing.

Enter a grade value for each student into the "final_grade" column of the spreadsheet. Do not

change any of the other data.

While the column reads "final_grade" you can use any grade you want (mid-term, exam grade,

etc) to compare against Echo360 student metrics.

Use percentage for grade values: The gradient of the "Course Grade" or Y axis of the correlation

graph ranges from 0 to 100, with the assumption that the grade being provided is a percentage

score.

Save the updated CSV file (in CSV format).

If you closed the Upload grades dialog box, click UPLOAD GRADES again, to open it.

Click UPLOAD, then navigate to and select the saved CSV file containing grade values.

Once the CSV file is processed, the plot points on the graph will change, to correlate with the

grades provided for each student. You can upload and re-upload updated grades as often as you

like, but the graph only shows the most recently uploaded data.

Once grades are uploaded, you can:

Use the X-axis drop-down list located below the graph to change which metric you want to

compare against the grades for each student.

Hover over a graph point to see which student the point corresponds to - the information appears

to the right of the graph while you are hovered over the point.

Click on a graph point to retain the student information on the right.

Use the Search box to search for students whose information you want to view.

Lecture Capture Reporting

15

You are also encouraged to read What do the analytics mean? to become comfortable with the

details each metric tracks. That will help you use each more accurately for comparing against the

overall course grade given to each student.

6.0 Downloading Student Analytics to CSV While the information on the Classes and Students pages on the Analytics tab is helpful,

sometimes you may want to download the data to a file, to review it offline, to compare it to other

programs' grading exports, or to manipulate the data outside of Echo360.

To download analytic data to a CSV

Navigate to the Class List page for the course (click ALL CLASSES from the DASHBOARD or select

the course from the COURSES drop-down list on the main menu).

Click the ANALYTICS tab, then click Export & Gradebook from the options on the left.

Use the drop-down list to select which metric you want to download to a CSV file. Your options

include:

Activity Participation (%)

Activity Participation (Raw #)

Activity Score (%)

Activity Score (Raw #)

Attendance

Engagement

Notes word count

Presentation Views

Questions

Video Views

Click START EXPORT JOB.

The new export job appears in a list below the Export selection options. When it is complete, it is

listed with a Download link as shown in the below figure. You may need to refresh the page to see

the Download link.

Most exports will finish almost immediately. Large exports (i.e., many classes and/or many

students) may take a little more time to complete.

Lecture Capture Reporting

16

Click the Download link next to each export type to download the file.

If opened in a spreadsheet program such as Microsoft Excel, each downloaded CSV export

contains columns for each class, and rows for each student, with the selected metric value shown

in the corresponding cell for each student for each class. Students are listed alphabetically by last

name.

Each metric or Export Type has one (and only one) active, downloadable file. Each time you

initiate a new export of that metric, the old file is overwritten by the new one. The date/time of

the last initiated export is shown for each so you know how current the data in the downloadable

file is.

For reference over the term, you may decide to download (and save somewhere) the old file

before generating the new one, in order to compare the numbers from one point in the term to

the next.

See What do the analytics mean? for definitions of each metric and general information on how

they are gathered and calculated.