University System of Georgia Learning Management System Transition Task Force

Final Report

30 April 2011

Submitted by Donna C. Llewellyn, Ph.D.

Task Force Chair

Executive Summary The University System of Georgia Learning Management System Transition Task Force was formed in August 2010 and met from the beginning of September 2010 through to the end of April 2011. The task force was tasked with recommending the next learning management system (LMS) that will meet the 21st century needs of the University System’s students and faculty. After collecting input from the stakeholders and doing independent research into a number of potential platforms, the task force hereby makes its final recommendation for the University System of Georgia to move to Desire2Learn at the end of the current contractual period with Blackboard.

Background Charge: On August 18th, 2010, Dr. Susan Herbst, Executive Vice Chancellor and Chief Academic Officer for the University System of Georgia charged Dr. Donna Llewellyn to chair a task force to select a new learning management system for the System. While the memorandum with the entire charge is given in Appendix 1, it is worth stating the guiding principles here. Guiding Principles: • Recommend a product that meets 21st century needs of students and faculty

supporting the improvement of retention and graduation rates • Recommend a product that will be used for multiple purposes (e.g. academic

instruction/research/training/continuing education/economic development) • Recommend a student focused minimum LMS suite to maintain affordability and

increase efficiency • The task force will partner with IT to recommend an enterprise solution with an

architecture that provides optimal performance/stability and supports increased enrollments of 100,000 additional students by 2020

• The work of the task force will be an open and transparent process to include all stakeholders

Task Force Membership: Donna Llewellyn worked in collaboration with Dr. Linda Noble, Associate Vice Chancellor for Faculty Affairs, to select members of the task force with the joint goals of achieving broad representation across institutional sectors and mission, while simultaneously maintaining a small enough group that would enable efficient and effective work. A complete list of the task force members is given in Appendix 2. It should be noted that representatives from the Georgia Department of Education and the Technical College System of Georgia were invited to serve on the task force. Students in Georgia move from our K-12 schools to both our Technical Colleges and the University System institutions. Therefore, it was seen as vital that we invite these constituencies to participate in our discussions and deliberations with the intent of improving articulation and future collaborations between the different educational stakeholders of the state. These “outside” partners participated, but they were not voting members with respect to the choice of finalists or the final recommendations. Further, there was a sizeable representation from the central office and from ITS (Information Technology Services) on this task force. It was seen as imperative that the task force move forward with complete information that sometimes is only available at the system level, and that any final recommendations be implementable by ITS. Again, these members fully participated but did not vote on the recommendations.

Process Meetings: The task force had a kick-off meeting on September 2, 2010 at Macon State College. The group then met approximately every two weeks (except during holiday periods) via Wimba, ending with another face-to-face meeting on April 28, 2011 again at Macon State College. The agendas and minutes of each of the meetings were posted on the task force web site (www.usg.edu/lms) and the minutes are included in Appendix 3. In addition to the regularly scheduled meetings, the task force also met electronically for webinars hosted by each of the five finalist platforms and then for a debrief session after each of these. (See more details about these webinars in the “Deliberations” section below.) A schedule of these webinars and a summary of the discussions from the debriefing sessions are in Appendix 4. Feedback and Input: It was vital for the task force to communicate with and solicit input from the stakeholders throughout the university system. This was accomplished through a variety of means including: • A web site was set up to enable widespread distribution of relevant documents along with periodic “blog” postings to encourage participation from the stakeholders. A summary of the feedback received from these blog postings is given in Appendix 5. • The task force facilitated five campus-based regional town hall meetings (each was simultaneously available and archived via Wimba to the entire USG).

o November 11, 2010 at Darton College o November 15, 2010 at Kennesaw State University o November 16, 2010 at Georgia Southern University o November 19, 2010 at Georgia State University o December 9, 2010 at Macon State University

A summary document of the feedback received from these meetings is in Appendix 6. • The task force presented and received feedback at various USG conferences and meetings including:

o Georgia Summit o Rock Eagle o Student Advisory Council o USG Faculty Council

• Online surveys were administered to the faculty and students of the system. Reports of the results of these surveys were posted on the web site and are included in Appendix 7. In addition, the vista administrators of the system were surveyed to learn of any special issues from these support personnel. • Sandbox environments were set up for each of the five finalist platforms. Accounts were administered at each campus and users were encouraged to complete an evaluation survey to give their feedback on each of the five platforms. Reports of those survey results are given in Appendix 8. • The task force members and a few other invited participants (those most vocal during the rest of the process) were encouraged to complete a thorough rubric evaluating the sandbox environments. A report summarizing those results are given in Appendix 9.

Decision Deliberations Criteria: At the initial task force meeting, the group started the discussion about the “non-negotiables” – those things that the new generation LMS must satisfy. This list was further refined during the following months. This resulted in a “Guiding Criteria” document that was shared via the task force web site and was used to narrow our consideration to a set of eight platforms. This document was posted on the task force web site and is included in Appendix 10. Reference Checks: The chair of the task force then did a reference check of each of the eight platforms to check two conditions for further consideration: the responsiveness of the vendor/community and the historical track record of providing LMS functionality successfully in a large scale implementation. Following this step, the task force then further limited their consideration to five platforms: Blackboard Learn 9, Desire2Learn, Moodle, Pearson Learning Studio, and Sakai. One of the technical requirements laid out by ITS was that the recommended platform must be able to be hosted internally by the USG. It was determined during further investigations that there had been a miscommunication about one platform, Pearson Learning Studio, and that it appears that this ability is not present from this provider. Since this was learned late in the process (after it had been announced that they were a finalist), they were kept in the pool of candidates under consideration. Demonstrations and Q&A Webinars: The task force invited each finalist platform to respond to questions related to their functionality and future road map. For the open source contenders, this was directed both at appropriately selected community members. Each platform was also invited to host a webinar in order for the task force members to see a demonstration of the platform, to go over the answers to the posed questions, and to ask follow-up questions. For the open source platforms, the task force enlisted the support of commercial affiliates to host the webinars and to carry out the demonstrations, and then included community members in the Q&A portion of the webinars. The list of requested demo actions as well as preliminary questions is included in Appendix 11. Note that in parallel to this effort, ITS supplied each platform with their own list of questions related to technical requirements. Evaluation Rubric: For the final decision process, the task force agreed on an overall rubric for evaluation. This rubric is included in Appendix 12. The task force paid special attention to the issues of conflict of interest throughout their deliberations. One member of the ITS staff who had been a support member of the task force left the system to join Wimba. This person was immediately removed from all task force communications, he turned over his physical files to another ITS staff member on the task force, and he confirmed that he had deleted all of our communication and documents from his computer. It was further decided that no members of the ITS staff or the central office staff should have a vote in

the final recommendation. Moreover, since all vendor communication was filtered through the chair, she agreed not to have a vote in this decision. However, all task force members were encouraged to participate throughout and this included giving their input all the way through to the final deliberations. It is important to note that all perspectives were needed in this final decision, but that the USG central office staff, ITS staff, and the chair did not cast votes. Final Review: At the final meeting of the task force, all of the feedback and input data were reviewed and discussed. Each task force member present then completed the final rubric to the best of their ability individually. The group then had a lengthy discussion, reaching a consensus on the rating for each item. The rating results in each category along with a ranked order of the five platforms in each area are given in Appendix 12. One voting member of the task force had a conflict at her institution and so could not be present at the final meeting to participate in the deliberations or the vote. The chair checked in with her after the meeting and she concurred with the final recommendations.

Recommendations Primary Recommendation: To meet the needs of the students and faculty of the University System of Georgia in the 21st century, the USG LMS transition task force strongly recommends adoption of the Desire2Learn platform as the next generation USG Learning Management System. This platform outranked all of the competitors in all areas of consideration other than the ease of transition. The lead in that category was Blackboard Learn 9, which is not a surprise given that the system will be moving from a Blackboard product currently in use. However, Desire2Learn was a fairly close second place contender in this area. This recommendation is a result of the task force members taking all input data into account, considering the multiple and varied contexts and needs across the system’s 35 institutions, and the need to provide a fiscally responsible solution. Note that due to the necessity for ITS to negotiate the most beneficial economic terms for the implementation of any recommendation, the task force did not have final cost figures for the different platforms. However, the ITS representatives did provide their current best known figures and these were taken into consideration. Again, due to the sensitive nature of this endeavor, these specific cost estimates are not provided in this report. Instead a table outlining the relative costs in various categories across the finalist platforms is provided in Appendix 13. Especially vital is the need for ITS to be able to support the selected solution centrally while simultaneously providing the flexibility and customization that the individual institutions require, given the fiscal realities of the USG budget at this time. Moreover, this must be balanced with the need to provide our faculty and students with a usable, stable, and technologically advanced LMS platform. The task force was united in its determination that Desire2Learn was the only contender that satisfied all of these constraints. The task force also understood that due to the dire nature of the current budget situation that there is a possibility that there are zero additional dollars to invest in a new LMS solution. One potential solution to this dilemma is to wait one year for the necessary investment in infrastructure. This would enable ITS and the campuses to carry out low-cost planning during FY12 and then be ready to move forward provided that the economy has improved for FY13. This does present a potential risk, however ITS has stated that this is a feasible plan. While the task force believes that it is vital to invest in the teaching and learning environment, if it is indeed true that the System has zero new dollars available for spending on license fees during the next two years, then the task force agreed that it is acceptable to remain with Blackboard with their scaled back contract offer (current price guaranteed for up to five years) until such time as the fiscal situation improves enough to implement our primary recommendation of Desire2Learn. It should be noted that even this option does carry with it an additional cost of training and migration. Further, without a heavy investment in hardware and related infrastructure, this solution will not

meet the highly desired need for customization at the campus level. Please see more information about this option and all of the cost issues in Appendix 13. It should be noted that, as mentioned earlier, the Pearson Learning Studio platform can not be hosted internally by the USG. Therefore, since this was a criteria set forth by ITS, the task force does not recommend this platform under any circumstances. Related Policy Recommendations: The task force also discussed several related policy issues for the USG. They unanimously make the following recommendations: • The chosen LMS provider should state their commitment to accessibility issues and if possible, this should be written into the contract. The students in the USG clearly stated this as a priority in our stakeholder survey, and it is the right and legal thing to do. • There should continue to be a policy and procedure allowing institutions to opt out of the centrally provided and supported LMS. This policy should include the caveat that such a decision on the part of any institution should not result in higher costs for the other institutions in the USG or ITS. In choosing Desire2Learn, the task force gave a large weight to the flexibility that it will permit between campuses as well as on individual campuses. However, the task force understands that no one LMS solution is right for all contexts, and that therefore, if the case can be made that there exists a better LMS for a particular institutional situation then that should be respected. However, this should in no way have a negative fiscal impact on the System or the other institutions. • All institutions should have one LMS for all of their faculty and students, regardless of academic discipline. The students of the USG were very clear that they strongly prefer a unified LMS platform on their respective campus. The downsides from allowing multiple LMS platforms on a single campus strongly outweigh the potential benefits of this kind of flexibility at an institution.

Appendices 1: Charge memorandum 2: List of task force members 3: Task force meeting minutes 4: Webinar schedule and summary of debriefing meetings 5: Summary of blog postings 6: Summary of task force town hall input 7: Faculty and student stakeholder survey reports 8: Sandbox evaluation surveys 9: Task force sandbox evaluation rubric and summary of results 10: Guiding criteria document 11: Finalist questions and demo requests 12: Final rubric form and summary of results 13. Relative costing information and considerations

Appendix 1 Charge Memorandum

DR. SUSAN HERBST CHIEF ACADEMIC OFFICER & EXECUTIVE VICE CHANCELLOR PHONE (404) 657-1332 270 WASHINGTON STREET, S.W. FAX (404) 463-6682 ATLANTA, GEORGIA 30334 SUSAN.HERBST@USG.EDU

"Creating a More Educated Georgia" www.usg.edu

Aug. 19, 2010

TO: Donna Llewellyn, Director, Center for the Enhancement of Teaching and Learning, Georgia Institute of Technology FROM: Susan Herbst, Executive Vice Chancellor and Chief Academic Officer cc: Institutional Presidents Vice Presidents for Academic Affairs Vice Presidents for Student Affairs Chief Business/Fiscal Officers ACIT Representatives GeorgiaVIEW Institutional Administrators RE: Learning Management System Task Force Overview The University System of Georgia currently hosts a centralized Learning Management

System (Blackboard Learning System - Vista 8) for 32 institutions as part of the GeorgiaVIEW Service. Vista 8 operational support from Blackboard ends in January 2013 and there are currently serious issues of parity between Blackboard’s Vista 8 and their newest product - Learn 9.1 (for example, lack of multi-institutional functionality, integration with the Student Information System, learning context hierarchy, lack of a clear migration path). For these reasons, we must initiate a project to bring together USG stakeholders to evaluate and select a Learning Management System that will support the strategic goals of the University System of Georgia and the needs of our institutions.

Action Although there is a major IT component to this decision, the ultimate decision regarding next

steps resides with Academic Affairs. Therefore, the University System of Georgia has identified you to chair a task force whose primary goal is to recommend a next generation Learning Management System. The task force should consist of representatives from the faculty, students, information technology, and other appropriate end-users and agencies.

Guiding Principles for the Task Force

• Recommend a product that meets 21st century needs of students and faculty supporting the improvement of retention and graduation rates

• Recommend a product that will be used for multiple purposes (e.g. academic instruction/research/training/continuing education/economic development)

• Recommend a student focused minimum LMS suite to maintain affordability and increase efficiency

"Creating a More Educated Georgia" www.usg.edu

• The task force will partner with IT to recommend an enterprise solution with an architecture that provides optimal performance/stability and supports increased enrollments of 100,000 additional students by 2020

• The work of the task force will be an open and transparent process to include all stakeholders Timeframe To ensure time for a successful implementation of the new LMS and provide at least two

implementation groups of campuses, the selection process must not only be thorough and inclusive of all stakeholders, but timely as well. The expectation is for the task force to submit a vendor recommendation, concluding the RFP process, by March 2011.

Next Steps Below are the next steps we foresee being completed as soon as possible. Doug Hyche, the

director of GeorgiaVIEW (program within USG that manages the day-to-day operations of Vista 8), will organize a meeting with you to provide background and data to assist with these initial endeavors.

• Finalize the membership of the task force • Develop a stakeholder analysis of those affected by LMS selection/implementation • Develop a communication plan and strategy • Establish time commitments and expectations for members • Identify the selection/review process

I wish to thank you and Georgia Tech for their willingness to share their expertise in guiding the selection of our system’s next learning management system. The inclusion and cooperation amongst our institutional stakeholders during the selection process is critical as we prepare to select a student-centered solution that effectively supports the core mission of the University System of Georgia: teaching and learning.

Appendix 2 List of task force members

USG Learning Management System Transition Task ForceMembership and Contact Information

Member Institution Role Email PhoneDonna Llewellyn Georgia Institute of Technology task force chair donna.llewellyn@cetl.gatech.edu 404-894-2340Tracy Adkins Georgia Perimeter College vista administrator tadkins@gpc.edu 678 891 3456 Donald (Will) AllisonGeorgia Southern University student representative da00928@georgiasouthern.edu 678-839-5300Jill Brady North Georgia College and State Univ registrar jebrady@northgeorgia.edu 706-864-1882Christina Clayton Department of Education virtual learning, DOE cclayton@doe.k12.ga.us 404-617-2048Sherry Clouser University of Georgia distance education sac@uga.edu 706-542-0525Gabby Grant University of West Georgia student representative ggrant1@my.westga.edu 718-749-2277Charles Hodges Georgia Southern University faculty representative chodges@georgiasouthern.edu 912/478-0497Robert Keown Georgia Virtual Technical College virtual learning, technic rkeown@tcsg.edu 404.679.1667Betty LaDuke Georgia Perimeter College faculty representative Bettie.laduke@gpc.edu 404-654-0968Barry Monk Macon State College faculty representative barry.monk@maconstate.edu 478-757-2656Darryn Ostrander Darton distance education darryn.ostrander@darton.edu 229 -317-6838Chris Randall Kennesaw State University faculty representative crandal2@kennesaw.edu 770.423.6410Kimberly Tanner Valdosta State University disability services katanner@valdosta.edu 229-245-2498Curt Carver USG CIO curt.carver@usg.edu 404-657-1588Virginia Michelich USG student achievement virginia.michelich@usg.edu 404-463-0502Linda Noble USG faculty affairs linda.noble@usg.edu 404-656-0764Barry Robinson USG project management barry.robinson@usg.edu 706-583-2168Mike Rogers USG faculty affairs mike.rogers@usg.edu 706-583-2207Doug Hyche USG GAView doug.hyche@usg.edu 706-583-2164left task force 2/9/11

Appendix 3 Task force meeting minutes

2 September 2010 Kick-off Meeting of the USG LMS Transition Task Force Macon State College 10:00 am – 4:00 pm Attendees: all members of the task force except Kimberly Tanner Decisions 1. Barry Robinson (ITS) will take minutes for the first meeting. The job will rotate at each meeting 2. Additional stakeholder groups identified

a. Academic Advisory Committees b. GOML

3. Draft Timeline – I did not record any changes to the timeline 4. Task Force will gather initial requirements from USG stakeholders to formulate criteria to narrow down to a handful of LMS choices 5. Direct all vendor communications to Donna 6. Vet all proposed surveys, official communication from task force with Donna 7. Team will meet every two weeks via Wimba Action Items 1. Task force team members will provide Donna with:

a. Any contact info changes to contact list (by COB 9/3/10) 2. Mike Rogers will post SREB LMS slides to T-square (by COB 9/3/10) 3. Add to communication plan the following(Barry/Doug by COB 9/3/10)

a. Stakeholders i. Academic Advisory Committees (see Linda Noble for breakdown)

4. Initial Requirements compiled by task force members a. Post your previously compiled “initial requirements” list to t-square, Resources, Initial Requirements, or email to Donna – (Task Force Members by 9/7/2010) b. Post GeorgiaVIEW previously compiled list of sample desires/requirements to t-square, resources, initial requirements– (Doug Hyche by 9/3/2010)

5. Create t-square folder for peer institution resource material – (Donna by 9/7/2010) a. Post any known peer institution/system information/resources that relate to LMS, LMS selection, or email to Donna for posting – (Task Force Members by 9/10/2010

6. Post Parity lists between current implementation of Blackboard Vista 8 and possible alternatives – Doug Hyche by 9/7/2010) 7. Send out followup memo to USG community (similar/same listservs as charge memo) announcing the start of the project (Donna/Linda by 9/9/2010) 8. Double check ability of Task Force members to add files to t-square – Donna by 9/3/2010 9. Post Ga Summit and Rockeagle info the t-square site – Doug Hyche by 9/3/2010

10. Review communications plan and send comments/suggestions to Donna (Task force members by 9/10/2010) 11. Create list of questions for students to be used in a survey – (Will / Gabby, Virginia by 9/10/2010) 12. Brainstorm what social media can be used to attract student response – Gabby/Will by 9/10/2010 13. Email your “must have requirements to Donna – Task force members by 9/10/2010 14. Review polling tools – Donna 15. Develop questions for faculty surveys – Bettie LaDuke, Linda Noble, Doug Hyche by 9/10/2010 16. Create online surveys for different sectors on external website – Donna by 9/17/2010 17. Send out list of 9/2/10 meeting action items – Donna by 9/3/2010 18. Send Donna lists of LMS vendors you are aware of – Task force members by 9/7/20 19. Create survey for Vista Administrators – Tracy Adkins by 9/10/2010 Informational 1. Curt Carver offered ITS resources to create sandbox environments for narrowed down LMS choices 2. Robert Keown (TCSG) has offered task force members the ability to review the Angel LMS integrations 3. Need volunteer to represent LMS Task Force at Rock Eagle conference (Sherry Clouser is possibility) 4. Let Donna know if you will be attending the Ga Summit conference – (task force members by 9/10/2010) Communication Ideas/Concerns Parking Lot 1. How do we deal with vendor requests for information (for right now, forward all to Donna) 2. Specific Stakeholder interaction versus overall communication plan 3. Develop Questions / Surveys, reuse existing resources

16 September 2010

Meeting of the USG LMS Transition Task Force

Via Wimba

2:30 pm – 4:00 pm

Attendees: all members of the task force except Christina Clayton, Robert Keown, Virginia Michelich, Linda Noble

Secretary: Darryn Ostrander

Agenda o Check in and Welcome o Call for a volunteer secretary to take meeting minutes

o Volunteer: Darryn Ostrander o Review of Minutes from September 2 meeting

o No comments on September 2 minutes o Update on action items

o Updated Phone number o New files posted to T-Square o Email to USG community from Donna went out on Wednesday 9/15 – copy will

be sent to task force members o Darryn Ostrander agreed to help at Summit o Rock Eagle

Info on T-Square October 20-22 Still need volunteers – confirm via email

o List of LMS options Went through list that has been gleaned from email messages and an

online list Members mentioned others and other lists

o Upcoming presentations o Georgia Summit – Donna and Darryn o Rock Eagle – Mike Rogers, Sherry Clouser, Darryn, others? o RACRA (September 30th) – Jill Brady o Student Advisory Council (November 6th) – asked for volunteers, Gabby Grant

volunteered, others requested via email o Priority lists

o Non-negotiables – went over list that had been contributed. Discussed that need to clarify 508 Compliance so that this is clear to all audiences. Kimberly offered to do that.

o Extremely important – went over the list that had been contributed. Clarified several of the items including access to source code and

o Nice to Have - went over list that had been contributed. Discussion that many of these items are really features that are present in some form in Vista but that need improvement to be really useful. Bettie LaDuke offered to go through and clean up this list to make that clear.

o Surveys – discussed that there will be several surveys aimed at different audiences. Chuck Hodges wanted to be sure that the data would be captured by campus. Doug Hyche assured us that this will be possible. Time frame is to have the surveys administered in October. Idea is to close them in early November.

Students • Will Allison has started on several surveys on survey monkey and

will post them Faculty

• Betty Laduke and Doug Hyche have gathered surveys from several other schools. Looking to edit rather recreate.

o Vista Administrators Tracy has posted a survey in T-Square

o Donna: Feedback has come from the website o Next Steps/Action Items/Next Meeting

o Finalize Surveys – aim for a close to final draft of the main three surveys by our next meeting.

Student Survey • Gabby Grant • Will Allison • Virginia Michelich

Faculty Survey • Bettie LaDuke • Doug Hyche

Vista Administrators • Tracy Adkins

o Start eliminating contenders from the list of candidate LMS products – go through the list looking at if they would be able to satisfy our non-negotiables – really a quick first pass – by our next meeting.

Kimberly Tanner Tracy Adkins Barry Monk Chris Randall

o Start scheduling town hall meetings – Donna will work with Barry Robinson, Doug Hyche, Mike Rogers, and Linda Noble to start planning and scheduling these. Let Donna know via email if you wish to help with this.

o Next meeting Wednesday September 29, 10:30 – noon via Wimba (might need to move this afternoon)

USG Learning Management System Transition Task Force Meeting 29 September 2010 Via Wimba In attendance: All members except Will Allison (in class), Curt Carver, Christina Clayton, Kimberly Tanner. Guest: Hilliard Gastfriend. o Check in o Call for a volunteer secretary to take meeting minutes

o Darryn volunteers to be secretary o Review of Minutes from September 16 meeting

o Minutes are accepted o Feedback from the community and from Georgia Summit

o Issues related to non-negotiables: o We asked, “What is our current LMS missing that you want in the new one?” o We asked, “What do you want to go away in our next system?” o We asked “What tools do other LMS’ offer that you wish you had access to?”

o Upcoming Presentations/Meetings o September 30 – Jill Brady at RACRA o October 5 – Tracy Adkins and Bettie LaDuke will be doing presentation at GPC

Faculty Development Day o October 20-22 – Mike Rogers, Sherry Clouser, Darryn Ostrander at Rock Eagle o November 6 – Gabby Grant and Bettie LaDuke at Student Advisory Council

(GCSU) o Updated Priority List

o List is updated (version 2) o This will continue to be updated as we do town halls and surveys

o Surveys Vista Admin Faculty Student survey

o Surveys Continued Timing Process Data Analysis

o Update on action items o LMS Contenders – we are making progress o Town Hall Meetings

o Review of timeline and where we are o Other Business o Next Steps/Action Items/Next Meeting

o October 12 meeting at 2:30 via Wimba o Action Items

Send feedback to the thee presenters of the surveys within a week Linda and Donna will work on the communication piece for the Surveys

USG Learning Management System Transition Task Force Meeting 12 October 2010 Via Wimba Minutes o Attendees: Barry Robinson, Bettie LaDuke, Charles Hodges, Chris Randall, Curt

Carver, Darryn Ostrander, Donna Llewellyn, Doug Hyche, Gabby Grant, Hilliard Gastfriend, Jill Brady, Kimberly Tanner, Mike Rogers, Sherry Clouser, Tracy Adkins, Virginia Michelich, Will Allison

o Secretary for today’s meeting: Jill Brady o Review of Minutes from September 29 meeting

o Minutes were approved as presented o This meeting is being archived o Presentations/Meetings

o GaSummit presentation is archived in the location below. http://deimos.apple.com/WebObjects/Core.woa/Browse/gcsu.edu.4742336568.04742336573

o RACRA – September 30 – Jill reported that the group was receptive to the news of the LMS search, but no new feedback was gathered.

o Rock Eagle - October 20-22 – Mike, Sherry and Darryn will present next week. The session will not be recorded.

o Student Advisory Council – November 5-6 – Gabby and Bettie will present. o Faculty Development Day (GPC) – Bettie and Tracy presented to 100 faculty

(October 7) regarding the LMS search. Many faculty voiced concerns regarding the migration, but the mood was not negative. They will have additional sessions with faculty on each campus the week of November 1 (including Wimba sessions).

o Update on action items o Surveys

Vista Admin (Tracy) • Discussion of various questions – Tracy will revise this week.

Students (Gabby/Will) • Discussion of various questions –Gabby and Will will revise this

week. Faculty (Bettie)

• Discussion of various questions – Bettie will revise this week. Final edits on the surveys will be completed this week. Surveys will be made available widely via communication with VPAAs

and CIOs from Linda Noble and other student, staff and faculty lists. Surveys will be open through the second week of November. Hilliard Gastfriend will analyze the data and have initial results available

in late November or early December. o LMS Contenders

508 Compliance (Kimberly)

• Contact has been made with vendor regarding their compliance. These vendors may be divided between compliant, working towards compliance and will likely not be compliant during the timeline of the taskforce.

o Town Hall Meetings Four regional meetings will be held in November—Northwest (KSU),

Southeast (GaSouthern), Central (GaState), and Southwest (Darton). One catch all (MaconSt )meeting will be held in early December. Simultaneous Wimba sessions will be held where possible. Suggested sites were reviewed.

o Other Business o Sandboxes perhaps available during January/February (after 3(?) contenders are

selected in mid-December). This will depend on ability for ITS to get these up and running in a reasonable amount of time.

o Next Steps/Action Items/Next Meeting o Sessions with Vendors

January/February, in parallel, with sandbox trials. Face to face if possible. By invitation - The taskforce would provide a list of questions in advance

for the vendors. We would meet in Macon (perhaps) with the full task force. Issues would then be reviewed.

A suggestion was made that a webinar with each vendor be held prior to the face to face meetings to review the features of the software.

o Next meeting – Thursday, October 28 at 11 am.

USG Learning Management System Transition Task Force Meeting 28 October 2010 Via Wimba

Minutes

Attendees: Donna Llewellyn, Tracy Adkins, Will Allison, Jill Brady, Curt Carver, Sherry Clouser, Doug Hyche, Robert Keown, Bettie LaDuke, Virginia Michelich, Barry Monk, Linda Noble, Chris Randall, Barry Robinson, Mike Rogers, Kimberly Tanner o Check in o Call for a volunteer secretary to take meeting minutes o Barry Robinson offered to do this o Review of Minutes from October 12 meeting Approved o Presentations/Meetings

o Rock Eagle The idea of pilots was a theme

o Clarification of Terms: o sandbox short term – no teaching, just playing around without real student

data, evaluate prior to recommendation o sandbox phase 2 –teaching, with real student data o pilot – after recommendation is made, production level support for limited

teaching o early adopters – first phase implementation, production level support

o Discussion of issues related to pilots and sandboxes – see summary under sandboxes later in the minutes.

o Bettie and Tracy will be doing a series of sessions for all of their campuses next week

o Update on action items o Surveys

o Hilliard – good response thus far, should pick up as domain announcements go out. Reports will be provided each Friday

o LMS Contenders o 508 Compliance o Technical Requirements o Discussion about migration issues

o Town Hall Meetings o update

o Plans for Sandboxes o Type 1 Sandbox (Playing) – issues related to how many users, how inclusive they

can be, etc. o Type 2 Sandbox (Teaching) – there are additional issues related to student data,

archiving, level of service guarantee, and effect on timeline o Subgroup will study these issues and will report back to task force before next

meeting o Future Meetings

o Will continue biweekly wimba meetings

Next meeting is November 11 at 2:30pm USG Learning Management System Transition Task Force Meeting 11 November 2010 Via Wimba Minutes Attendees: Donna Llewellyn, Tracy Adkins, Sherry Clouser, Bettie LaDuke, Hilliard Gastfriend, Charles Hodges, Virginia Michelich, Linda Noble, Barry Robinson, Kimberly Tanner

o Check In  o Call for a volunteer to take minutes 

o Linda Noble offered to do this  

o Review of minutes from October 28 meeting o Approved 

 o Presentations/Meetings 

o Student Advisory Council at Georgia College & State University  These students validated much of what is on our requirements list  Bettie LaDuke will send list of specific requests (includes better interface 

with Outlook/Exchange calendar, better downloading functionality, interface with mobile devices, flexibility in choice of software to complete assignments, more gradebook functionality and would like faculty to use this function more often) 

o Various meetings on Georgia Perimeter College campuses  Much of these requests also validated our requirements list  Tracy Adkins will send list of specific requests 

o Meetings reported other than the scheduled Town Hall Meetings where members are talking about the LMS Transition 

Donna  Llewellyn is meeting with the Directors of Teaching and Learning Centers on November 12 

Linda Noble is meeting with the University System of Georgia Faculty Council on November 19  

o Update on Action Items o Surveys 

Deadline to participate extended to November 19  Various methods of communicating the extension were determined  Hilliard reported briefly on number of respondents 

• Response rate is good from the 4‐year institutions but under‐represented at research universities and two‐year colleges 

o LMS Contenders o Updates were given about some of the lms’s who had not responded to 

Kimberly’s request for info about compliance  

o Town Hall Meetings o Donna is attending the meeting at Kennesaw State (Nov 15) o Bettie will attend the meeting at Georgia State (Nov. 19) o Mike and Charles will attend the meeting at Georgia Southern (Nov 16) 

 o Meetings with Other Stakeholders 

o Donna will find a way to meet with these various groups either face‐to‐face or via Wimba 

o USG Librarians were added to the list of stakeholders to meet with  

o Sandboxes o Donna reported the results of the CIO Survey on the possibility of doing the 

sandboxes in an actual teaching environment  Results were mixed with no clear trend 

o It was determined that the sandboxes would not be conducted with live teaching 

o Donna asked for volunteers from the task force to create teaching “scenarios” for testing in the sandboxes 

o Various ways to give institutional access to the sandboxes were discussed  Institutional liaisons will be needed  Notice to campus CIOs will be needed because their staff/faculty will be 

involved o A separate testing group consisting of ITS support staff and Task Force members 

will also participate in the sandboxes  

o Discussion – process for choosing finalist list o All information from all sources (508 compliance, scalability, survey results, 

meetings, etc.) will be considered to narrow down the list  

o Plans for vendor visits o Need for strong academic affairs presences at these meetings was expressed o BlackBoard will be at ITS in Athens on December 8 

Task Force members welcome to attend  Specific questions will be developed and distributed to BlackBoard by 

December 1st to be addressed at Dec 8th meeting o Meetings with other vendors will be scheduled soon – these will include 

opportunity to meet both with ITS staff as well as task force members.  All will be given questions ahead of time.  

o Future Meetings o Next Task Force meeting is November 22 at 2:30 o Meeting Conflict on December 9th with Town Hall at Macon State was noted 

 o Action Items will be circulated via email to all task force members 

USG Learning Management System Transition Task Force Meeting 22 November 2010 Via Wimba Minutes Attendees: Adkins, Allison, Brady, Carver, Clouser, Hodges, Hyche, LaDuke, Llewellyn, Michelich, Monk, Noble, Randall, Robinson, Rogers, Tanner o Check in o Call for a volunteer secretary to take meeting minutes Barry Robinson will take minutes o Review of Minutes from November 11 meeting Approved o Presentations/Meetings Team reported the results of the LMS Town Halls and other meetings. The archives

of the town halls are posted on the website. GSU wanted more process transparency. We will try to set up a blog on the website.

Will Allison requests that we have a student focused virtual town hall meeting. o Surveys 94 admins submissions 3650 faculty submissions 12500 student submissions Institutional reports will be created and distributed (process to be determined – most

likely sent to CIOs and VPAAs on each campus) o LMS Contenders Update on compliance issues. We now have nine lms contenders that we will concentrate

on. All of the ones that have been ruled out show no signs of compliance or have been completely unresponsive to our questions about this issue. The nine that are still being considered are all aware of compliance issues and have some process in place to respond to these issues. AMAC will assist with external review as needed for finalists.

Now, we will check references of each of these nine contenders to arrive at a list of 5 (or so) semi-finalists. At that point, our list will be public.

Reference questions 1. size of installation – how many students/faculty/courses/campuses, size of

infrastructure/hosted or self-hosted? 2. how long a client/user of the platform, where did they migrate from? 3. how did the migration go when they moved to this platform? 2. level of

responsiveness (support and feature requests) 4. frequency of upgrades – who dictates schedule, how determined? 5. would you recommend lms (why/why not?) 6. how active is the user community 7. how hard to support faculty and student use? Does it support learning?

Others? – input by 11.23 to donna. o Discussion – process for choosing finalist list o Plans for vendor visits

will not meet f2f with Blackboard, will schedule virtual vendor demos for mid December - early January

o Sandboxes Start with existing testing scenarios, recruit folks to do the testing (sub-committee of task force plus other stakeholders) Form a sub-committee, Vista admins as campus liaisons

o Other Business o Future Meetings

Learning Management Selection Task Force Meeting Minutes 12/8/10

Chuck Hodges called the meeting to order at 2:00 pm as the presiding officer in lieu of Donna Lewellyn absence due to an institutional commitment. (Action items are indicated in bold.) Members present for the meeting were: Barry Monk, Chuck Hodges, Linda Noble, Mike Rogers, Barry Robinson, Bettie Laduke, Chris Randall, Darryn Ostrander, Gabby Grant, Kimberly Tanner, Tracy Adkins, Will Allison, Sherry Clouser, Curt Carver. Also present was Hilliard Gastfriend. Minutes from the previous meeting were approved as is. A revised time line was presented and discussed. Chuck indicated that completing the reference check in the prescribed time was proving to be a very difficult task due to the slowness with which vendors were submitting references. This delay impacts the due dates of all future steps. A vote was taken to approve the amended time line and it passed. Action Item: Donna will now submit to Dr. Susan Herbst the revised timeline asking for approval to proceed with the revised dates as indicated. Once it has been approved, a notice will be posted on the web site with this information. The next town hall is to take place at Macon State on 12/9/10. Members from the task force attending are Donna Lewellyn, Mike Rogers and Doug Hyche. Donna will be meeting the CIO’s of the R1 institutions via conference call on 12/10/10 regarding the task force. Discussion was held regarding a student town virtual hall meeting. The decision was made to not hold such a meeting but instead create blog where students would have additional opportunities for input. Help was asked for in getting the word out to such groups as the Student Advisory Council and Student Affairs group. Action item: Linda Noble volunteered to reach out to these groups with this information. There was a discussion about several individual campus activities regarding campus based LMS selection groups. Discussion regarding the results of the surveys ensued. A time table was presented by Hilliard regarding availability of the survey results. A report of the system-wide quantitative (multiple choice questions) results will be posted on the website by the end of the month. At that time, each campus CIO will also be able to receive a data-dump of all of the campus’ responses. The full qualitative report will be available to the task force in early Spring. This data will be used primarily in the implementation process.

It was reported that Donna is checking references of the recommended vendors/communities who have submitted references. Donna is meeting with the USG legal representative concerning the RFP/sole source process and will bring information regarding legal aspects back to the LMS group when appropriate. No other business was introduced. Future meeting schedule will be posted through an email from Donna. Submitted by Mike Rogers

USG Learning Management System Transition Task Force Meeting 18 January 2011 (postponed from January 10 due to ice storm) Via Wimba Attending: Llewellyn, Adkins, Brady, Clouser, Hodges, Hyche, LaDuke, Monk, Ostrander, Randall, Robinson, Rogers Minutes o Check in and get volunteer secretary Barry Robinson agreed to be the secretary o Review of Minutes from December 8 meeting Approved – will be posted on website this afternoon o Update about survey reports Campus-filtered raw data sent to CIOs on 1/7/2011 Quantitative (close-ended results) reports will be ready for posting within the next two

weeks Qualitative (open-ended results) will be ready later in the spring o Review results of reference checks Reviewed all reference checks. Two out of the eight platforms did not have references

respond so they were ruled out at this point. Extensive discussion about what was learned through the reference process. Taskforce discussed how many contenders to keep in the process. Only one was eliminated due to lack of track record with large scale clients at this time. Donna will notify them ahead of this information being made public.

o Next Steps: Arrange vendor/community online meetings –

Demos Questions (send questions one week ahead of meeting)

• Subgroup formed to create first list of questions for these meetings Start working on sandboxes

Implementation issues Scenarios Evaluation

Overall Evaluation Rubric o Future Meetings

January 28 at 2:00 February 10 at 11:00 March 1 at 2:00 Will also need participation in the vendor/community online meetings

o Other Business o Action Items

USG Learning Management System Transition Task Force Meeting 28 January 2011 Via Wimba Attending: Llewellyn, Adkins, Allison, Carver, Clouser, Hodges, Hyche, LaDuke, Michelich, Monk, Noble, Ostrander, Robinson, Rogers, Tanner Minutes Check in and get volunteer secretary

• Darryn Ostrander agreed to be the secretary Review of Minutes from January 18 meeting

• Approved Discussion with attorney about open records laws and how they impact us

• Daryl Grinswold – spoke about opens records issues Update from Hilliard about survey reports

• Quantitative (close-ended results) reports are complete and in the process of being formatted

• Qualitative (open-ended results) is scheduled for spring delivery Update on communication with vendors/communities Next Steps:

Arrange vendor/community online meetings – Report out of subgroup Will run parallel process of technical questions and functional questions –

different expertise and audience. Each will have a rubric that will feed into the overall evaluation process

Demos Questions (send questions one week ahead of meeting) – discussed draft

list – make sure to add roadmap questions to functional list Start working on sandboxes – Report out of subgroup

Implementation issues Scenarios – will use eCore courses migrated to each platform (if possible) Evaluation – will have a rubric for anyone who uses the sandbox. These

will fold into the overall evaluation procedss Cost Considerations – ITS will start to build cost models for each of the

remaining platforms with information from vendors and community members. Overall Evaluation Rubric – will consider all four components – initial technical

questions, initial functional/roadmap questions, sandbox experiments, and cost considerations.

Future Meetings

• February 10 at 11:00 • March 1 at 2:00

Other Business

USG Learning Management System Transition Task Force Meeting 10 February 2011 Via Wimba Attending: Adkins, Carver, Clouser, Gastfriend, Grant, Hodges, LaDuke, Llewellyn, Monk, Noble, Ostrander, Randall, Robinson, Rogers Minutes Check in

• Chuck Hodges agreed to be the secretary Review of Minutes from January 28 meeting

• Approved Donna Llewellyn provided several reminders:

• Task Force members must notify her regarding prior or existing associations with any of our current LMS contenders.   

• Doug Hyche has resigned from the Task Force due to leaving USG for a job with Wimba, 

which is a Blackboard company. 

• Members of the Task Force representing USG will NOT vote on our final recommendation for an LMS. 

• Donna will not vote. 

Donna informed the Task Force that information has been added to the Task Force website informing site visitors of the Task Force process from now to the end of the recommendation. Hilliard Gastfriend provided an update on our survey results. The analysis of the quantitative data collected with the student survey is nearly complete. The analysis of the qualitative data has begun, but there is a long way to go. Hilliard has the assistance of a graduate student from UGA for the qualitative analysis. The analysis of the quantitative data from the faculty survey is complete and will be posted on the website soon. Donna informed us that all finalists at this stage have been sent a memo regarding technical requirements, webinar and demo needs, and sandbox logistics Curt Carver informed us that all CIOs at USG institutions were asked to refuse requests from our finalists for special product promotions such as webinars.

Curt provided an update on contract discussions and cost calculations. The five finalists have been (or will be in the very near future) contacted regarding contract discussions/cost calculation information. This is to allow the task force to have costing information as part of the input to their final recommendation. Six points in the future negotiation of the contracts were explained. Hilliard asked if a specific length of the eventual contract has been discussed. Curt said that a few different time frames have been discussed, but no time frame has yet been discussed. Curt informed the Task Force members of a new LMS vendor has been contacting USG institution Presidents directly. He explained what the USG central office is doing to make sure that this does not interfere with our Task Force process. Linda Noble commented that our Task Force members need to do all that we can to encourage participation, as appropriate, on our campuses with the remaining parts of the selection process. Barry Robinson reported on a recent visit of ITS to the University of West Georgia to discuss eCore needs with a System-wide LMS. He urged the Task Force to provide them a way to communicate their needs to the Task Force. Linda suggested that Barry facilitate a meeting of the West Georgia personnel with Donna in the near future. Donna and Barry agreed. Donna discussed next steps regarding setting up demos/webinars from the LMS finalists and setting up sandboxes. Barry provided information regarding logistics of account management and support of the sandboxes. Bettie LaDuke showed a revised rubric to be used while the Task Force members examine the sandbox environments of the sandboxes. Donna reminded us of future meetings.

USG Learning Management System Transition Task Force Meeting 1 March 2011 Via Wimba Minutes Attending: Llewellyn, Brady, Hodges, LaDuke, Ostrander, Randall, Robinson, Rogers, Tanner Volunteer secretary: Barry Robinson Review of Minutes from February 10 meeting – minutes approved Update on vendor communications

Instructure asked to be reconsidered – After a review and a discussion with several task force members, it wawould not be appropriate. Campus Cruiser – some of the reference check emails ended up in their junk mail, they were allowed to reopereferences responded so they are not being considered further. Let Donna know if you get vendor pressure 

• Update on web meetings: o Schedule Each vendor/community partner will have two hours for demo’ing their platform, answering our pre-posted then for an open Q&A. We will follow that with a meeting in our wimba site for recap and feedback. Your comments should also be posted in the appropriate resource folder in t-square. Details are posted in 3/1/11 meeting archive o Logistics The webinar portion will be hosted by the vendor/community partner – you will receive log on info ahead o Details are posted in 3/1/11 meeting archive • Update on sandboxes (Barry) o Schedule – has been pushed back a bit but we should still have 3 weeks for evaluations. Some institutionsbreak but all will have time to complete. o Accounts – there will be faculty and student role accounts available at each institution. It will be up to thethey hand these out (and if multiple users can use one account at the same time). I will send some guidelinespriorities (action item). o Support - Tracy Adkins is taking the lead on this (thank you Tracy). There is a listserv set up for all the vcommunicate and get assistance. o Scripts – These will be available through the vista admins. The feedback will be programmed in vovici tovery quick turn around on the results so that we can use it in our deliberations. o Evaluation Rubric – Task force members will use a more detailed evaluation rubric o Data analysis – As mentioned, the stakeholders will input their feedback into vovici for ease of analysis, tainto excel. Overall evaluation rubric – still in process Final meeting schedule – We will meet in Macon for our final deliberations. Date to be arranged.

USG Learning Management System Transition Task Force Final Decision Meeting 28 April 2011 Macon State College Attendees Donna Llewellyn, Tracy Adkins, Curt Carver, Robert Keown, Bettie LaDuke, Chris Randall, Will Allison, Linda Noble, Darryn Ostrander, Barry Robinson, Mike Rogers, Kimberly Tanner, Chuck Hodges, Sherry Clouser, Gabby Grant, Barry Monk Agenda/Notes 10:00 Call to Order Introductions (matching the faces with the voices): Around the table – name, institution, role Reminder of the voting process and next steps Who gets to vote – Members of the task force from the system institutions excluding Central Office/ITS staff and excluding Chair Form of our recommendation to the Chief Academic Officer What will happen next – Donna will submit a report to the Chief Academic Officer with the recommendation and a background of process and justification. At that time, “gag order” on the task force will be lifted (we advise all members to refer questions from outside your institution to Millsaps at central office). It is then up to the Chief Academic Officer of how to proceed with our recommendation. 10:20 Call for a volunteer secretary to take meeting minutes – Sherry Clouser volunteered. Minutes from March 1 meeting were approved 10:25 Review of Charge to task force 10:35 Overview of the inputs to this decision process (our criteria for the decision)

Review of the Rubric Teaching and Learning Environment Getting there Support and Community Moving Forward Cost of Ownership Is anything left off of the rubric? No, consider each component broadly.

Overview of the data we have collected Feedback from our stakeholders: Website blog entries Town hall meetings

Additional Feedback from campus town hall meetings that were not included in central task force town hall meetings– importance of the master course functionality, announcements, notifications, information feed, need for training

Surveys Sandbox surveys How will we weight all this data? That will be up to each task force member. Reference Checks

No written records for these. The references were more for preliminary data to clear that each considered platform had large scale implementations already successfully in place and to check on the responsiveness of the vendor and the community members.

Webinars for the taskforce Taskforce evaluation of the sandbox environments Noon Lunch (in the cafeteria) To Kimberly - Any comments from AMAC or students?

o A student who is blind was able to get through all of the systems. o No system was perfect. Any contract should include statements about

addressing accessibility issues. Even features that were accessible were sometimes cumbersome. Question about whether or not the alt tags migrated – if they did not, this was actually a simple fix.

To the students – Any further comments?

o Gabby preferred D2L and Blackboard over Moodle and Sakai in the sandbox environments.

o Will worked with a diverse group of students and found varied opinions regarding which was the best. Everyone liked something different. Students care that we have a good LMS, but it won’t be a big deal to them which one it is. They will learn it pretty easily.

To anyone - What about interaction with the vendor and/or user community?

o Comments about each ITS evaluation of the technical issues related to each environment

o The Pearson model is changing – not enough details to completely analyze the impact of this on our decision process.

o D2L offers a centrally hosted system that can be controlled by each institution. The institutions would have to work together to work out upgrade schedules.

o Blackboard says they will have the ability to offer central hosting with institutional customization in January; as of right now, this is not possible without major hardware and infrastructure investments.

o With Sakai and Moodle, there is concern about customization for 35 different campuses.

ITS investigation into the cost of ownership – this is still ongoing. Some ballpark numbers were shared. Independent work on the rubric. Discussion on each line of the rubric to reach consensus. What types of recommendations can we make?

• Sole recommendation • Ranked order of two or three • Primary/Secondary Recommendation • Other…

4:00 Preliminary Vote

1. Sole choice is… BB9 – 2 D2L – 7

2. If cost is chief concern, go with: BB9 (3.75M) – 4 BB9 (barebones) – 1

3. Allow to opt to: D2L Sakai – 5

Discussion about the opt-out option If the central solution is not best for a campus, should we recommend an opt-out solution? Not a specific solution, but that there be a process for a campus to opt out. If a campus opts out, it should select a single solution. Concern about saying that there is a cost issue. Would the GA citizenry blow up about that. Reference back to the chart (rubric results) – D2L wins most categories. Discussion about cost issue. According to current figures, there is about a $1M difference over 5 years between D2L and Bb9 all in solutions. Therefore, given rubric and ensuing discussion, the cost consideration vote was retaken and the new result is that if the system office determines that there are ZERO new dollars to spend on an lms that

they should continue with the “barebones” BlackBoard solution (the promise of no increase from current price of ~$500,000/year) for up to five years until there is money to implement our first choice solution of the next generation LMS: Desire2Learn. 5:00 Next Steps Donna will be writing up the report. Until it has been delivered to the central office, we are to respectfully decline to answer. Then the report will also be posted to the lms task force web site. Refer questions to John Millsaps, if desired.

Appendix 4 Webinar schedule and summary of debriefing meetings

Webinar Schedule Date Time Platform March 7, 2011 9am: Sakai March 7, 2011 4pm: Desire2Learn March 14, 2011 4pm: Blackboard Learn 9 March 15, 2011 10am: Moodle March 15, 2011 4pm: Pearson Learning

Studio

1 

 

Summary of Debriefing Sessions after Vendor Demo Webinars Prepared for April 28, 2011 Meeting of the USG LMS Transition Task Force 

 • Blackboard: 

 Positive Features  Negative Features 

Faculty Already Familiar  Master Course Concept Not at Level Hoping For 

Mobiles  Lack of Learning Content Structure 

Rubrics  Way TAs are handled in Large Sections 

  

• Desire2Learn:  

Positive Features  Negative Features 

Like functionality  May  be a steep learning curve 

UI a little better than Sakai  Why go from one company‐based product to another? 

Has a WebCT feel to it   

Liked the features   

Much more controllable than Sakai   

Accessibility is not an afterthought   

Assess for everyone (disabilities & learning styles)   

All Features Requested were in the product   

Seem to have a lot of experience migrating content   

  

• Moodle:  

Positive Features  Negative Features 

Simple  Migration without Moodle Rooms may be impossible 

Master Course function addressed what faculty wanted 

No rubric feature 

Students can flag questions in the quiz function 

No real “wow” factor 

Able to link external blog or use the internal one 

Nothing value‐added above other LMS’s 

  Does not appear to work with mobile devices unless we go through a secondary vendor 

  Edit view has very small icons and could be difficult for some to work with 

  Did not address access for students with disabilities 

2 

 

   

  

• Pearson eCollege:  

Positive Features  Negative Features 

They handle migration of courses  Instruction videos not captioned  

Definite improvement over older versions of eCollege  

No equation editor option for students  

Simple, but not simplistic  

Cloud  computing  will  impose  some limitations 

 They have a good philosophy regarding accessibility 

 

They archive everything, forever  

API concepts are nice  

No MapleTA  

USG central office staff have familiarity with Equella 

How non‐course organizations are formed  

Good attention to social media (although this could also be done with the other systems) 

 

Learning  Edge  owns  Equella  and  that company  is  prone  to  over‐commit  and under‐deliver  

Integration of multi‐media  

 

Student outcomes dashboard  

 

Exam time multiplier (good for faculty, but could be time intensive for administrators) 

 

 

Can alter test time for individual students  

 

Reporting features  

 

Multiple roles available and customizable  

 

Pleasing interface  

 

Good content organization as the default (although this could also be done with the other systems) 

 

 

 

3 

 

  

• Sakai:  

Positive Features  Negative Features 

Did everything expected and wanted  Implementation Works well at Tech for communities  Interface is very different – culture shock?   Will need to cluster institutions for economy of scale   Will need structure for institutions to request 

upgrades   May not be conducive to cross‐institutional 

collaboration   May not be customizable enough   Students may have a problem with tool bases   Some good indicators regarding accessibility   Identifies for non‐students in collaborative sites   System‐level resources to support institutional 

flexibility    

 

Appendix 5 Summary of blog postings

Summary of LMS Task Force Blog Postings Prepared for April 28, 2011 Meeting of the USG LMS Transition Task Force 

 Question #1: Thoughts on Centralized and Locally Controlled Solutions 

• 22 comments posted with multiple posts by same individuals • Three responses requested central control • Three responses requested central control with local flexibility & customizability • One response requested local control only • Not all comments responded directly to the question 

o Next LMS must accommodate the majority of the faculty but be flexible for innovators  

Question #2: Student Input • Call for questions to student representatives on the Task Force resulted in zero postings 

 Questions #3: Comments on list of 8 Finalists: 

• BlackBoard had two posts in support and four against adoption o Negative posts included: 

some problems with coding errors, application deficiencies, and system design flaws 

too similar to WebCT  Several would only use BlackBoard if required to 

• Epsilen and Pearson each had one post against adoption • Four posts recommended adoption of an open source platform and two of those recommended 

Moodle over Sakai o A total of 9 posts were very positive towards Moodle with some concerns, however, 

about its grade book function and online help support • A few posts addressed desired requirements independent of an evaluation of any of the 8 

finalists. These requirements included: o No Java and Flash/increased compatibility with multiple browsers o Ease of faculty interaction o More flexible content o Eliminate redundant asks when designing courses o Must work with Wimba o Better options to imbed other technologies (e.g., Turnitin) 

 Question #4: What are the most important issues the TF should consider when considering future plans of the five finalists: 

• Although there are several lengthy posts under this question, most are not direct responses to the specific question. The posts express concerns about the process used to recommend the next LMS, the makeup of the task force, the time line, and the evaluation of sandboxes that are not in a real teaching environment. 

• One post addressed the question and indicated an important issue is total cost of ownership. 

Appendix 6 Summary of task force town hall input

University System of Georgia Learning Management System 

 Town Hall Meeting Comments 

The purpose of this document is to briefly describe the comments made by the participants in the University System of Georgia Learning Management System Town Hall Meetings.  

 The meeting archives were reviewed and the comments of the participants were logged.  A careful listening to the proceedings resulted in the development of a coding scheme that included the following categories:  

1. Tools and Features 2. Integration 3. Platform 4. Migration 5. Training and Support 

 

The meetings were intended to elicit input from faculty, staff and students about the impending change of the University System of Georgia Learning Management System (LMS). As such, the proceedings were open ended and participants were encouraged to comment on any aspect of both the LMS and the selection process .  A total of 145 comments were captured over the course of the five meetings with 75 relating to tools and features.  There were 19 comments about integration and 12 that concerned specific LMS platforms. In addition there were nine comments about migration, and four concerning training and support.  

Tools and Features The most frequent topic of the Town Hall Meeting was Tools and Features.  “Tracking” was the most frequently cited feature with eight comments. The grade book and internal features such as “Wimba like”, “wiki’s” and “media player” were mentioned seven times each.  Four mentions of email were made and there were three each of assessments, editing and “flexibility”. Other topics that were brought up were portfolios, reporting, user control and whiteboards.   

Integration The integration of the prospective LMS with other technologies was a popular topic at the Town Hall meetings. There was a significant interest in proprietary technologies such as Live Text, Microsoft, and various social media sites. In addition the integration of the LMS with mobile phones and libraries was mentioned.   

Platform  Comments about the platform ranged from questions about institutional branding, sandbox testing,  and the LMS’s that are under consideration.  Several participants asked about the consideration of an open source LMS while others asked what specific LMS’s were under consideration. 

Migration The migration of content from the current LMS to the new system was an important topic. The participants stressed the need for an effective technology that would retain as much of their existing content as possible.  

Training and Support There were several comments about the importance of training and support. Respondents indicated that training well prior to the initial use of the LMS was vital and that ongoing training and support was equally important. 

 

 

 

 

. 

 

 

 

 

Appendix 7 Faculty and student stakeholder survey reports

BOARD OF REGENTS OF THE UNIVERSITY SYSTEM OF GEORGIA

 

USG Learning Management System Taskforce 

  Faculty Survey Results 

 January 2011 

 

   

LMS Taskforce Faculty Survey January 2011    Page 2 of 21 pages  

Table of Contents Survey Background ......................................................................................................................... 3 

Comments on Survey Analysis .............................................................................................. 3 

Question 0: Please identify your primary institutional affiliation. ........................................... 4 

Question 1:  Have you used an LMS at your primary institution within the past two years? ... 5 

Question 2:  Please select the reasons you DO NOT use an LMS. ........................................... 7 

Question 3:  Please select the LMS you use most at your primary institution. ........................ 8 

Question 4:  How do you use the LMS you use most? ............................................................ 9 

Question 5:  If you use the LMS you use most with your face‐to‐face classes, what kinds of 

actions do you take with it?   .............................................................................................. 11 

Question 6: Whether you are currently using an LMS or not, how would you describe your 

level of expertise with this type of system? ......................................................................... 12 

Question 7:  How important are the following administrative features of an LMS to you? ... 13 

Question 8:  How important are the following instructional features of an LMS to you? ...... 15 

Question 9:  How important are the following technical features of an LMS to you? ............ 17 

Question 10: What is your primary discipline of instruction? ............................................... 19 

Question 11: What degree levels do you normally teach? ................................................... 21 

Question 12: Please provide any additional comments below ................................................ 21 

   

LMS Taskforce Faculty Survey January 2011    Page 3 of 21 pages  

Survey Background 

 The University System of Georgia (USG) Learning Management Transition (LMS) taskforce 

was charged in August 2010 with recommending an LMS to replace the current system (Vista 8) 

which will no longer be supported by Blackboard effective January 2013. The taskforce has a 

broad range of representation including students and faculty as well as other USG stakeholders.  

For more information about the taskforce, please visit the website: http://www.usg.edu/lms. 

In an effort to gather input from the main users of the USG LMS users, the taskforce 

distributed several surveys.  This document catalogues the results of the survey that was 

distributed to the faculty at all of the USG campuses.  

  Comments on Survey Analysis  

The survey contained several categories of questions: Choose one, choose many (‘select all 

that apply’), choose one with option to specify ‘other,’ choose many with the option to specify 

‘other,’ yes/no, and free or open response. Because of the large number of respondents, some 

of the questions that allowed respondents to specify their own category of response created 

difficulties in the analysis. For example, question 10 was stated as follows:  

Q10: What is your primary discipline of instruction? 

This question listed 23 choices for possible answers and an ‘Other, please specify’ option. 

We were surprised that of the approximately 2,900 responses, 238 (approximately eight 

percent of the sample) chose to select the Other category. We had the option of simply listing 

LMS Taskforce Faculty Survey January 2011    Page 4 of 21 pages  

all 238 responses but believed a qualitative analysis of the responses would yield more accurate 

and more instructive information. In the aforementioned question, a new table of results was 

created which now contained 35 items, under the subheader “Revised Table.” This analysis was 

quite complex and time consuming and also complicated any reports we were to write, but we 

believed it was the best and most accurate approach.  

 

In this report, if a question offered the opportunity for respondents to specify an ‘Other’ 

response, the revised table has been presented. The original table and the codified ‘Other’ 

responses appear in the Appendix. The numbers labeling each response refers to a master list 

of all responses used by the coders in the analysis. 

                        

LMS Taskforce Faculty Survey January 2011    Page 5 of 21 pages  

Question 0: Please identify your primary institutional affiliation.  Institution  Count % Total

Abraham Baldwin Agricultural College  27  0.78% 

Albany State University  23  0.67% 

Armstrong Atlantic State University  79  2.29% 

Atlanta Metropolitan College  14  0.41% 

Augusta State University  78  2.26% 

Bainbridge College  27  0.78% 

Clayton State University  73  2.11% 

College of Coastal Georgia  55  1.59% 

Columbus State University  70  2.03% 

Dalton State College  68  1.97% 

Darton College  102  2.95% 

East Georgia College  29  0.84% 

Fort Valley State University  24  0.69% 

Gainesville State College  241  6.97% 

Georgia College & State University  115  3.33% 

Georgia Gwinnett College  113  3.27% 

Georgia Highlands College  94  2.72% 

Georgia Institute of Technology  62  1.79% 

Georgia Perimeter College  287  8.30% 

Georgia Public Library Service  1  0.03% 

Georgia Southern University  225  6.51% 

Georgia Southwestern State University  56  1.62% 

Georgia State University  98  2.84% 

Gordon College  62  1.79% 

Kennesaw State University  350  10.13%

Macon State College  88  2.55% 

Medical College of Georgia  162  4.69% 

Middle Georgia College  47  1.36% 

North Georgia College & State University 142  4.11% 

Savannah State University  21  0.61% 

South Georgia College  30  0.87% 

Southern Polytechnic State University  64  1.85% 

University of Georgia  242  7.00% 

LMS Taskforce Faculty Survey January 2011    Page 6 of 21 pages  

Institution  Count % Total

University of West Georgia  139  4.02% 

USG System Office  1  0.03% 

Valdosta State University  127  3.67% 

Waycross College  20  0.58% 

Total  3456   

    Question 1:  Have you used an LMS at your primary institution within the past two years? (Respondents could only choose a single response.)   

Response  Chart  Frequency  Count 

Yes      81.7%  2790 

No      18.3%  626 

Not Answered      40 

  Valid Responses  3416 

  Total Responses  3456 

                  

LMS Taskforce Faculty Survey January 2011    Page 7 of 21 pages  

Question 2:  Please select the reasons you DO NOT use an LMS. (Respondents were allowed to choose multiple responses; responses sorted by count, not as they appeared in survey; ”Other” responses placed at bottom of item list.)  Revised Table 

Response  Percentage  Count

I have not received adequate training.* 18.75%  138

It does not fit with my teaching style or pedagogy.* 17.66%  130

Other tools (e.g., such as my own web site, Web 2.0 tools, etc.) are more useful to me in delivering and managing my course.* 

15.22%  112

It is not used at my institution or department.* 11.96%  88

There is no need for me to provide my course materials digitally/electronically. 

7.07%  52

Don’t know what an “LMS” is**  5.16%  38

Staff, administrator, or not currently teaching** 4.89%  36

It is not reliable.  4.89%  36

It does not provide adequate features.* 5.03%  37

It is too hard to learn.* 4.35%  32

New teacher or new to USG**  2.45%  18

Unclassifiable**  0.95%  7

Student preference**  0.27%  2

Time constraints**  1.09%  8

Fear of constantly changing technologies** 0.27%  2

Total    736

* Response count augmented after analyzing “other” responses. ** New category added after analyzing “other” responses.  

LMS Taskforce Faculty Survey January 2011    Page 8 of 21 pages  

Question 3:  Please select the LMS you use most at your primary institution. (Respondents could only choose a single response.) 

 

Response  Chart  Frequency  Count 

Blackboard Vista      11.1%  302 

CourseDen      4.0%  108 

Distance Learning VSU Vista (BlazeVIEW) 

    2.4%  65 

elearning@gsc      3.6%  98 

eLearning Commons (eLC)      6.6%  178 

GaVIEW Vista      5.6%  152 

GeorgiaONmyLINE (GeorgiaVIEW) 

    0.8%  23 

GeorgiaVIEW      9.7%  264 

GeorgiaVIEW (Blackboard Vista) 

    8.0%  218 

GeorgiaVIEW (WebCT Vista) 

    10.2%  276 

GeorgiaVIEW Vista      7.8%  212 

GeorgiaVIEW Vista @ Waycross College 

    0.4%  10 

GeorgiaVIEW Vista 8      10.7%  291 

GeorgiaVIEW@ASU      1.1%  31 

GeorgiaVIEW@BC    0.2%  6 

GGC Bb Vista      1.8%  49 

iCollege      6.7%  182 

MGC Online (GeorgiaVIEW) 

    0.8%  23 

Moodle      1.0%  27 

MSC Vista (GeorgiaVIEW)      1.3%  36 

Online Learning @ GCSU    0.1%  2 

Pirates' VIEW      0.8%  23 

Sakai    0.2%  6 

LMS Taskforce Faculty Survey January 2011    Page 9 of 21 pages  

Response  Chart  Frequency  Count 

SSU eLearning    0.3%  7 

T‐Square      1.8%  50 

uLearn      2.7%  72 

Not Answered      36 

  Valid Responses  2711 

  Total Responses  2747 

 Question 4:  How do you use the LMS you use most? (Respondents were allowed to choose multiple responses; responses sorted by count, not as they appeared in survey; ”other” responses placed at bottom of item list.)  Revised Table 

Response  Percentage Count 

To supplement my face‐to‐face classes* 43.21% 2188 

To teach online classes (classes with 95% or greater activity online) 

20.20% 1023 

To teach hybrid classes (classes with 51% ‐ 94% activity online) * 

14.85% 752 

Committee work  7.56% 383 

Student advising  3.48% 176 

Special student interest groups or clubs 2.37% 120 

Specific LMS tool**  2.41% 122 

Research*  2.33% 118 

Faculty training or resource**  0.91% 46 

Faculty communication/common area** 0.59% 30 

Unique/Unclassifiable**  0.57% 29 

LMS Taskforce Faculty Survey January 2011    Page 10 of 21 pages  

Response  Percentage Count 

Administration training or resource** 0.41% 21 

Student special projects/independent study** 0.24% 12 

Student communication/common area** 0.22% 11 

Unspecified communication/common area** 0.20% 10 

Access to other tools**  0.12% 6 

As a student**  0.10% 5 

Continuing education/external courses** 0.10% 5 

To view or assist other teacher's classes** 0.08% 4 

Managing GTAs/student workers**  0.06% 3 

Total  5064 

* Response count augmented after analyzing “other” responses. ** New category added after analyzing “other” responses. 

 

                

LMS Taskforce Faculty Survey January 2011    Page 11 of 21 pages  

Question 5:  If you use the LMS you use most with your face‐to‐face classes, what kinds of actions do you take with it?  (Respondents were allowed to choose multiple responses; responses sorted by count, not as they appeared in survey; ”other” responses placed at bottom of item list.)  

Revised Table 

Response  Frequency  Count

I post a copy of my syllabus.  13.79%  2254

I post documents, files, and articles.* 13.65%  2231

I post assignments so students can access them electronically.* 

11.80%  1929

I use the grade book feature so students can access their grades.* 

11.33%  1852

I post my PowerPoint presentations. 10.83%  1770

Students use it to submit assignments to me. 8.91% 1457

I set up discussions so students can collaborate virtually.* 7.39% 1209

I give quizzes and tests online.*  6.97% 1139

I post videos/audios/podcasts***  5.97% 976

I set up small groups for assignments, discussions, and/or other reasons.* 

4.31% 704

I set up chat rooms.  2.91% 476

Access outside tools/materials**  0.59% 96 

I use the SafeAssign feature to check for plagiarized work. 0.48% 79 

Communication/email**  0.47% 77 

Unique/unclassifiable**  0.20% 32 

Calendar**  0.14% 23 

General announcements**  0.14% 23 

LMS Taskforce Faculty Survey January 2011    Page 12 of 21 pages  

Response  Frequency  Count

Surveys/feedback**  0.04% 7 

Roster tool**  0.04% 7 

Learning modules**  0.02% 4 

Sign‐up tool**  0.02% 3 

Track student access**  0.01% 2 

Total  16,350

* Response count augmented after analyzing “other” responses. ** New category added after analyzing “other” responses. *** Original category amended 

 Question 6: Whether you are currently using an LMS or not, how would you describe your level of expertise with this type of system? (Respondents could only choose a single response.)  

Response  Chart  Frequency  Count 

Never used      8.9%  276 

Beginner      16.1%  500 

Intermediate      50.4%  1566 

Advanced      24.6%  764 

Not Answered      33 

  Valid Responses  3106 

  Total Responses  3139 

 

 

   

LMS Taskforce Faculty Survey January 2011    Page 13 of 21 pages  

Question 7:  How important are the following administrative features of an LMS to you? Please indicate the importance of each feature/function by ranking them from Not Important to Very Important.  

Ability to automatically contact students or release information based on student performance data 

Ability to selectively release content to students based on assessment results or achievement towards course outcomes 

Ability to share documents/information   (such as office hours, test questions, documents) across courses  Ability to view reports on student performance and activity  Ability to set permissions to prevent certain content from being downloaded  Ability to administer mid‐term and end‐of‐course evaluations  Ability to easily create formulas for calculating grades  Ability to receive notification when content is updated  Ability to access technical support 24/7 by telephone, e‐mail, or live chat 

               

LMS Taskforce Faculty Survey January 2011    Page 14 of 21 pages  

 

LMS Taskforce Faculty Survey January 2011    Page 15 of 21 pages  

Question 8:  How important are the following instructional features of an LMS to you? Please indicate the importance of each feature/function by ranking them from Not Important to Very Important.  

Ability to create interactive learning objects, modules, or units for users with different learning styles 

Ability to manage and create groups for online student collaboration  Ability to provide private space in the online course for students to reflect  Ability to provide a space where students can develop their own ePortfolios  Ability to access library materials and research assistance  Ability to align course content with common course outline objectives or general 

education competencies (Example: Provides the ability to map course content in learning modules to the same learning objectives or competencies) 

Ability to conduct and archive live class sessions online  Ability to provide secure online testing  Ability to detect plagiarism  Ability to easily create and edit mathematical and scientific equations  Ability to maintain a pool of test questions that are accessible for use among multiple 

course sections that I teach  Ability to maintain a pool of test questions that can be used by other faculty members 

in my department    

LMS Taskforce Faculty Survey January 2011    Page 16 of 21 pages  

LMS Taskforce Faculty Survey January 2011    Page 17 of 21 pages  

Question 9:  How important are the following technical features of an LMS to you? Please indicate the importance of each feature/function by ranking them from Not Important to Very Important.  

Ability to have online discussions among students in multiple sections of the same course 

Ability to include audio or video content that can be downloaded to mobile devices  Ability to access course from hand‐held devices (Examples: iPhones, Blackberry, smart 

phones, mobile phones)  Ability to include Web 2.0 tools (blogs, wikis, RSS feeds, other new technologies) in 

the LMS itself  Ability to integrate blogs, wikis, RSS feeds, other new technologies from outside 

sources with the LMS  Ability to provide visual notification of the submission of new information (Examples: 

discussion board postings, assignment submissions, quiz and exam submissions)   

LMS Taskforce Faculty Survey January 2011    Page 18 of 21 pages  

LMS Taskforce Faculty Survey January 2011    Page 19 of 21 pages 

Question 10: What is your primary discipline of instruction? (Responses sorted by count, not as they appeared in survey. ”Other” responses placed at bottom of item list.)  

Revised Table 

Response  Percentage  Count 

Education*  12.45%  366 

Health Professions and Related Clinical Sciences* 11.90%  350 

English Language and Literature/Letters* 10.14%  298 

Business, Management, Marketing, and Accounting* 8.61%  253 

Mathematics and Statistics  7.18%  211 

Social Sciences*  6.33%  186 

Computer and Information Sciences* 5.75%  169 

Biological and Biomedical Sciences* 5.44%  160 

Physical Sciences*  4.59%  135 

Psychology  3.47%  102 

History  3.44%  101 

Visual and Performing Arts*  3.40%  100 

Communication, Journalism, and Related Programs* 3.03%  89 

Foreign languages, literatures, and Linguistics 2.65%  78 

Liberal Arts and Sciences, General Studies and Humanities* 2.04%  60 

Engineering  1.29%  38 

Library Science  1.26%  37 

Not faculty/Do not teach**  0.92%  27 

Physical Education, Kinesiology and Wellness** 0.82%  24 

LMS Taskforce Faculty Survey January 2011    Page 20 of 21 pages 

Response  Percentage  Count 

First Year Experience**  0.82%  24 

Philosophy and Religious Studies  0.65%  19 

Agriculture, Agriculture Operations, and Related Science* 0.58%  17 

Legal Professions and Studies  0.44%  13 

Unique/unclassifiable**  0.37%  11 

Counseling and Human Development** 0.34%  10 

Geography**  0.31%  9 

Economics and Finance**  0.31%  9 

Architecture*  0.37%  11 

Earth Sciences**  0.24%  7 

Natural Resources and Conservation* 0.27%  8 

English for Speakers of Other Languages** 0.14%  4 

Ethnic, Cultural, and Gender Studies* 0.20%  6 

Continuing Education**  0.10%  3 

Family and Consumer Sciences**  0.10%  3 

Aviation**  0.07%  2 

Total  2940 

         

LMS Taskforce Faculty Survey January 2011    Page 21 of 21 pages 

Question 11: What degree levels do you normally teach? (Respondents were allowed to choose multiple responses.)   

Response  Chart  Frequency  Count 

Undergraduate      86.7%  2574 

Graduate      31.0%  920 

Professional      7.2%  213 

Continuing Education      3.4%  100 

  Valid Responses  2968 

  Total Responses  2968 

 Question 12: Please provide any additional comments below. Feel free to comment on an individual question or an area that was not covered in the survey but you believe should have been.  

(Comments and analysis appears in qualitative report.)  

 

 

 

 

 

 

 

 

  

 This report was prepared under the direction of Dr. Hilliard Gastfriend. If you have any questions, please contact him at: hilliard.gastfriend@usg.edu for clarification or corrections. 

BOARD OF REGENTS OF THE UNIVERSITY SYSTEM OF GEORGIA

USG Learning Management System (LMS) Taskforce 

 Student Survey Results 

 January 2011 

 

  

LMS Taskforce Student Survey January 2011   Page 2 of 33 pages 

Table of Contents  Survey Background ......................................................................................................................... 4 

Comments on Survey Analysis ........................................................................................................ 4 

Q1........................................................................................................................................ 6 

Q2........................................................................................................................................ 7 

Q3........................................................................................................................................ 8 

Q4........................................................................................................................................ 9 

Q5...................................................................................................................................... 10 

Q6...................................................................................................................................... 11 

Q7...................................................................................................................................... 12 

Q8...................................................................................................................................... 14 

Q9...................................................................................................................................... 15 

Q10. ................................................................................................................................... 16 

Q11. ................................................................................................................................... 17 

Q12. ................................................................................................................................... 18 

Q13. ................................................................................................................................... 19 

Q14. ................................................................................................................................... 21 

Q15. ................................................................................................................................... 22 

Q16. ................................................................................................................................... 23 

Q17. ................................................................................................................................... 25 

Q19. ................................................................................................................................... 27 

LMS Taskforce Student Survey January 2011   Page 3 of 33 pages 

Q20. ................................................................................................................................... 29 

Q21. ................................................................................................................................... 29 

Q22. ................................................................................................................................... 30 

Q23. ................................................................................................................................... 30 

Q24. ................................................................................................................................... 31 

Q25. ................................................................................................................................... 31 

Q26 .................................................................................................................................... 31 

Q27. ................................................................................................................................... 32 

Q28. ................................................................................................................................... 32 

Q29. ................................................................................................................................... 32 

Q29. ................................................................................................................................... 33 

Q30. ................................................................................................................................... 33 

   

LMS Taskforce Student Survey January 2011   Page 4 of 33 pages 

Survey Background 

 The University System of Georgia (USG) Learning Management Transition (LMS) taskforce 

was charged in August 2010 with recommending an LMS to replace the current system (Vista 8) 

which will no longer be supported by Blackboard effective January 2013. The taskforce has a 

broad range of representation, including students and faculty as well as other USG 

stakeholders.  For more information about the taskforce, please visit the website: 

http://www.usg.edu/lms. 

In an effort to gather input from the main users of the USG LMS users, the taskforce 

distributed several surveys.  This document catalogues the results of the survey that was 

distributed to the students at all of the USG campuses.  

 

Comments on Survey Analysis  The survey contained several categories of questions: Choose one, choose many (‘select all that 

apply’), choose one with option to specify ‘other,’ choose many with the option to specify 

‘other,’ yes/no, and free or open response. Because of the large number of respondents, some 

of the questions that allowed respondents to specify their own category of response created 

difficulties in the analysis. For example, question 14 was stated as follows:  

 

Q14: From which of the following mobile platforms would you be likely to access an LMS? 

 

LMS Taskforce Student Survey January 2011   Page 5 of 33 pages 

This question listed 6 choices for possible answers and an ‘Other, please specify’ option. We 

were surprised that of the 9,410 responses, 1,148 (approximately 12 percent of the sample) 

chose to select the Other category. We had the option of simply listing all 1,148 responses but 

believed a qualitative analysis of the responses would yield more accurate and more instructive 

information. In the aforementioned question, a new table of results was created which now 

contained 12 items, under the subheader “Revised Table.” This analysis was quite complex and 

time consuming and also complicated any reports we were to write, but we believed it was the 

best and most accurate approach.  

 

In this report, if a question offered the opportunity for respondents to specify an ‘Other’ 

response, the revised table has been presented. Since some questions offered in the survey had 

conditional responses and redirection to different questions depending upon a respondent’s 

answers, the question numbers have been renumbered in numerical order for clarity; that is, 

the numbers appearing in these results do not match identically with the original question 

numbers appearing in the survey. 

LMS Taskforce Student Survey January 2011   Page 6 of 33 pages 

Q1. In which institution in the University System of Georgia are you currently enrolled? (If you are a transient student please select the university at which you have transient status.) Note: Private colleges and out‐of‐state colleges are not considered USG Institutions and should be filled‐in with the out‐of‐state option below.  Institution  Count  % Total Abraham Baldwin Agricultural College  111  0.89% Albany State University  35  0.28% Armstrong Atlantic State University  241  1.92% Atlanta Metropolitan College  67  0.53% Augusta State University  202  1.61% Bainbridge College  141  1.13% Clayton State University  531  4.24% College of Coastal Georgia  270  2.16% Columbus State University  348  2.78% Dalton State College  409  3.27% Darton College  694  5.54% East Georgia College  117  0.93% Fort Valley State University  111  0.89% Gainesville State College  537  4.29% Georgia College & State University  165  1.32% Georgia Gwinnett College  276  2.20% Georgia Highlands College  241  1.92% Georgia Institute of Technology  68  0.54% Georgia Perimeter College  1221  9.75% Georgia Public Library Service  2  0.02% Georgia Southern University  526  4.20% Georgia Southwestern State University  372  2.97% Georgia State University  415  3.31% Gordon College  746  5.96% Kennesaw State University  979  7.82% Macon State College  591  4.72% Medical College of Georgia  171  1.37% Middle Georgia College  248  1.98% North Georgia College & State University  539  4.30% Savannah State University  9  0.07% South Georgia College  39  0.31% Southern Polytechnic State University  346  2.76% University of Georgia  1149  9.17% University of West Georgia  263  2.10% University System Office  6  0.05% Valdosta State University  319  2.55% Waycross College  21  0.17% Total  12526   

LMS Taskforce Student Survey January 2011   Page 7 of 33 pages 

Q2. If you have transient status at more than one USG institution, please select the institutions below:    Institution  Count  % Total Abraham Baldwin Agricultural College  15  2.20% Albany State University  17  2.49% Armstrong Atlantic State University  13  1.90% Atlanta Metropolitan College  9  1.32% Augusta State University  16  2.34% Bainbridge College  8  1.17% Clayton State University  21  3.07% College of Coastal Georgia  6  0.88% Columbus State University  11  1.61% Dalton State College  13  1.90% Darton College  38  5.56% East Georgia College  5  0.73% Fort Valley State University  10  1.46% Gainesville State College  22  3.22% Georgia College & State University  16  2.34% Georgia Gwinnett College  15  2.20% Georgia Highlands College  18  2.64% Georgia Institute of Technology  16  2.34% Georgia Perimeter College  63  9.22% Georgia Public Library Service  2  0.29% Georgia Research Alliance  1  0.15% Georgia Southern University  41  6.00% Georgia Southwestern State University  10  1.46% Georgia State University  58  8.49% Gordon College  29  4.25% Kennesaw State University  28  4.10% Macon State College  21  3.07% Medical College of Georgia  4  0.59% Middle Georgia College  17  2.49% North Georgia College & State University  16  2.34% Savannah State University  8  1.17% South Georgia College  11  1.61% Southern Polytechnic State University  16  2.34% University of Georgia  38  5.56% University of West Georgia  11  1.61% USG System Office  2  0.29% Valdosta State University  28  4.10% Waycross College  10  1.46% Total  683   

LMS Taskforce Student Survey January 2011   Page 8 of 33 pages 

Q3. If you have transient status at more than one USG institution, please select the institutions below:   Institution  Count % TotalAbraham Baldwin Agricultural College  4  1.58% Albany State University  6  2.37% Armstrong Atlantic State University  5  1.98% Atlanta Metropolitan College  3  1.19% Augusta State University  4  1.58% Bainbridge College  4  1.58% Clayton State University  9  3.56% College of Coastal Georgia  2  0.79% Columbus State University  7  2.77% Dalton State College  6  2.37% Darton College  10  3.95% East Georgia College  2  0.79% Fort Valley State University  8  3.16% Gainesville State College  7  2.77% Georgia College & State University  6  2.37% Georgia Gwinnett College  7  2.77% Georgia Highlands College  5  1.98% Georgia Institute of Technology  7  2.77% Georgia Perimeter College  14  5.53% Georgia Southern University  11  4.35% Georgia Southwestern State University  9  3.56% Georgia State University  22  8.70% Gordon College  11  4.35% Kennesaw State University  18  7.11% Macon State College  10  3.95% Medical College of Georgia  6  2.37% Middle Georgia College  6  2.37% North Georgia College & State University 4  1.58% Savannah State University  3  1.19% Skidaway Institute of Oceanography  1  0.40% South Georgia College  3  1.19% Southern Polytechnic State University  2  0.79% University of Georgia  16  6.32% University of West Georgia  4  1.58% Valdosta State University  11  4.35% Total  253       

LMS Taskforce Student Survey January 2011   Page 9 of 33 pages 

Q4. If your home institution is out of state, please type it in below: (Sorted alphabetically)  Institution  Count Indeterminate/ Not applicable  39Not specified  18Alabama  10Florida  10International  7Tennessee  7South Carolina  6Georgia  5Online  5New York  4Kansas  3North Carolina  3Ohio  3California  2Illinois  2Iowa  2Maryland  2Pennsylvania  2Virginia  2Arizona  1Arkansas  1Louisiana  1Massachusetts  1Mississippi  1Oklahoma  1Texas  1Virgin Islands  1              

LMS Taskforce Student Survey January 2011   Page 10 of 33 pages 

Q5. Please select your student classification. (Respondents could only choose a single response)  

Revised Table  Response  Percentage Count

Freshman ‐ Student who has earned 0 ‐ 29 college semester credit hours.*  29.46% 3551

Sophomore ‐ Student who has earned 30 ‐ 59 college semester credit hours.* 

23.49% 2831

Junior ‐ Student who has earned 60 ‐ 89 college semester credit hours. 16.48% 1986

Senior ‐ Student who has earned 90 or more college semester credit hours.* 

15.43% 1860

Graduate ‐ Student pursuing a Master’s, Education Specialist, or doctoral degree (Ph.D or Ed.D.).* 

10.50% 1265

First Professional ‐ Student enrolled in Veterinary Medicine, Law, Medicine, Dentistry, or Doctor of Pharmacy. 

0.97% 117

Post‐Baccalaureate ‐ Student who has a Baccalaureate degree who is taking first professional or graduate courses but who is not working toward a degree. 

0.86% 104

Unclassified Undergraduate (Other) ‐ Student who has a bachelor's degree who is enrolled for credit at the undergraduate level but not pursuing another degree. 

0.61% 73

Transient ‐ Student taking one or more courses which he/she has permission to transfer to his/her home institution. 

0.49% 59

Medical/Dental/Veterinary Medicine Interns and Residents. 0.19% 23

Unspecified/indeterminate**  0.62% 75

2nd Bachelors**  0.29% 35

Dual/Joint Enrollment**  0.26% 31

Certificate/Add‐on/Endorsement**  0.25% 30

LMS Taskforce Student Survey January 2011   Page 11 of 33 pages 

Faculty/staff**  0.08% 10

Post baccalaureate,  Seeking Associate** 0.02% 3

Total   

* Response count augmented after analyzing “other” responses. ** New category added after analyzing “other” responses. 

  Q6. How would you describe your race/ethnicity? (Respondents could only choose a single response)  

Response  Chart  Frequency  Count 

White or Caucasian (Non‐Hispanic Origin) 

    60.6%  7303 

Black or African‐American (Non‐Hispanic Origin) 

    22.6%  2727 

Prefer not to answer      5.7%  683 

Asian or Pacific Islander      3.7%  440 

Hispanic      3.6%  436 

Multiracial      3.4%  409 

American Indian or Alaskan Native 

    0.4%  44 

Not Answered      118 

  Valid Responses  12042 

  Total Responses  12160 

            

LMS Taskforce Student Survey January 2011   Page 12 of 33 pages 

  Q7. Please enter your current major of study.  (Sorted by number) 

Response  Percentage  Count Nursing  16.05%  2000** Early Childhood Education    6.22  775** Biology    5.18  646** Psychology    5.06  631** Management    4.98  620** Accounting    3.84  478** Computer Science    3.83  477** Health*    3.49  435 Business*    3.47  432 Education*    3.31  413 Political Science    2.69  335** Criminology    2.66  331** Mass Communications    2.44  304** English    2.29  286** Marketing    1.99  248** General/Undecided*    1.95  243 Information Technology*    1.76  219 Engineering*    1.72  214 History    1.62  202** Middle Grades Education    1.38  172 Mathematics    1.34  167** Sociology    1.33  166** Art    1.26  157** Chemistry    1.26  157** Finance    1.18  147** Pharmacy*    1.16  144 Secondary Teacher Education    1.09  136** Social Work*    0.99    123 Management Information Systems    0.97  121** Athletics/Exercise Science/Kinesiology*   0.79  99 Medicine*    0.72  90 Agriculture*    0.61  76 Consumer Sciences*    0.58  72 Economics    0.51  64** Physical Therapy*    0.51  64 Law/Legal*    0.51  63 

LMS Taskforce Student Survey January 2011   Page 13 of 33 pages 

Response  Percentage  Count Library Sciences*    0.47  58 Business Teacher Education    0.46  57 Technology Support Services    0.45  56** Other*    0.44  55 Physical Education    0.43  54 Spanish Language & Literature    0.40  50** Physics    0.37  46** Theatre    0.35  44** Anthropology    0.33  41** Counseling*    0.32  40 Environmental Studies    0.32  40** Biology/Secondary Education    0.30  38 Performance    0.30  37** Veterinary*    0.26  33 Dentistry*    0.25  31 Science*    0.24  30 Aviation*    0.23  29 Language & Literature*    0.23  29 Construction Management*    0.22  28 Architecture*    0.20  25 Speech‐Language Pathology    0.19  24** Integrated/Interdisciplinary*    0.18  23 Biochemistry*    0.18  22 Geography    0.18  22** Music Teacher Education    0.17  21 Foreign Language*    0.16  20 Occupational Therapy*    0.15  19 Social Studies*    0.14  18 French Language & Literature    0.13  16** Teaching Field – Mental Retardation    0.11  14 Geology    0.10  13** Global Studies    0.10  13** International Economics    0.10  13** Philosophy    0.10  13** Earth Science/Secondary Education    0.10  12 Economics, general    0.08  10 Biotechnology*    0.07  9 Environmental Sciences    0.07  9 Real Estate    0.07  9 Recreation    0.07  9** 

LMS Taskforce Student Survey January 2011   Page 14 of 33 pages 

Response  Percentage  Count Chemistry/Secondary Education    0.06    7 Economics/Secondary Education    0.05  6 German    0.05  6** Physics/Secondary Education    0.02  3 Theory & Composition    0.02  3 TOTAL    12,462 *New category added after analysis of survey responses to option “other” 

**After analysis of survey responses to the option “other,” responses were coded and added to the counts of appropriate survey options 

     Q8.  The majority of the courses I take at my institution are: (Respondents could only choose a single response)  

Response  Chart  Frequency  Count 

Traditional Face‐to‐Face Instruction ‐ Little or no online component 

    59.1%  6562 

Traditional Face‐to‐Face Instruction ‐ Heavy online component 

    18.0%  1997 

Online Only/Distance Learning 

    11.6%  1284 

Hybrid (50% online, 50% face‐to‐face) 

    11.4%  1264 

Not Answered      480 

  Valid Responses  11107 

  Total Responses  11587 

         

LMS Taskforce Student Survey January 2011   Page 15 of 33 pages 

Q9. Please select all LMS you have experience with. (Respondents were allowed to choose multiple responses)  Revised Table 

Response  Frequency Count 

GeorgiaView VISTA*  29.76% 7150

Blackboard  25.36% 6093

WebCT  25.13% 6037

Blackboard Learn  11.53% 2771

I do not know any Learning Management Systems. 2.48% 596

ANGEL  2.34% 563

Moodle*  1.29% 310

FirstClass  0.52% 125

T‐Square  0.36% 87

Misclassified/Unclassifiable**  0.30% 72

Publisher/Textbook Website**  0.27% 64

Desire2Learn  0.23% 55

Sakai  0.23% 55

Portfollio/Assessment**  0.10% 23

Other LMS**  0.09% 21

Anlon  0.02% 4

  24,026 

*New category added after analysis of survey responses to option “other” 

**After analysis of survey responses to the option “other,” responses were coded and added to the counts of appropriate survey options 

 

  

LMS Taskforce Student Survey January 2011   Page 16 of 33 pages 

Q10. Please select the primary LMS you currently use. (Respondents could only choose a single response)  

Response  Chart  Frequency  Count 

Blackboard Vista      12.4%  1435 

CourseDen      1.6%  185 

Distance Learning VSU Vista (BlazeVIEW) 

    1.1%  128 

elearning@gsc      3.8%  441 

eLearning Commons (eLC)      8.4%  977 

GaVIEW Vista      5.7%  664 

GeorgiaONmyLINE (GeorgiaVIEW) 

    1.3%  155 

GeorgiaVIEW      9.0%  1043 

GeorgiaVIEW (Blackboard Vista) 

    14.7%  1705 

GeorgiaVIEW (WebCT Vista) 

    12.1%  1401 

GeorgiaVIEW Vista      8.0%  929 

GeorgiaVIEW Vista @ Waycross College 

  0.1%  17 

GeorgiaVIEW Vista 8      4.7%  544 

GeorgiaVIEW@ASU    0.3%  39 

GeorgiaVIEW@BC    0.3%  30 

GGC Bb Vista      0.9%  104 

iCollege      6.8%  793 

MGC Online (GeorgiaVIEW) 

    1.0%  119 

Moodle    0.2%  26 

MSC Vista (GeorgiaVIEW)      2.5%  289 

Online Learning @ GCSU    0.2%  19 

Pirates' VIEW      1.0%  113 

Sakai    0.0%  5 

SSU eLearning    0.1%  7 

LMS Taskforce Student Survey January 2011   Page 17 of 33 pages 

Response  Chart  Frequency  Count 

T‐Square      0.4%  46 

uLearn      3.1%  364 

Not Answered      9 

  Valid Responses  11578 

  Total Responses  11587 

   Q11.  Approximately how many times do you log‐in to the primary LMS you currently use? (Respondents could only choose a single response)  

Response  Chart  Frequency  Count 

Never      2.0%  233 

Monthly      2.9%  332 

Weekly      6.8%  783 

1‐2 times per week      8.5%  975 

3‐5 times per week      23.0%  2642 

6‐10 times per week      22.9%  2630 

11‐20 times per week      17.9%  2063 

>20 times per week      16.0%  1837 

Not Answered      29 

  Valid Responses  11495 

  Total Responses  11524  

          

LMS Taskforce Student Survey January 2011   Page 18 of 33 pages 

 Q12. Please check all features you actively use in the primary LMS you currently use. (Respondents were allowed to choose multiple responses)  

Response  Chart  Frequency  Count 

Grades/MyProgress      89.9%  8460 

Course Email      72.6%  6831 

Online Quizzes and Exams      66.3%  6238 

Online Assignment Turn‐in      66.1%  6221 

Discussion Boards      65.2%  6140 

Calendar      41.3%  3889 

Online Lecture (Recorded)      16.2%  1526 

Task Management      15.0%  1413 

Live Chat      13.4%  1257 

Cloud‐based File Storage      7.4%  700 

Online Lecture (Live)      6.4%  599 

  Valid Responses  9410 

  Total Responses  9410 

                   

LMS Taskforce Student Survey January 2011   Page 19 of 33 pages 

Q13. Please rate the importance of each feature in an LMS.  

Calendar 

Cloud‐based File Storage 

Course Email 

Discussion Boards 

Grades/MyProgress 

Live Chat 

Online Assignment Turn‐in 

Online Lecture (Live) 

Online Lecture (Recorded) 

Online Quizzes and Exams 

Task Management 

LMS Taskforce Student Survey January 2011   Page 20 of 33 pages 

LMS Taskforce Student Survey January 2011   Page 21 of 33 pages 

Q14. From which of the following mobile platforms would you be likely to access an LMS? (Respondents were allowed to choose multiple responses)  Revised Table  Response  Percentage  Count 

Apple iOS (e.g., iPhone, iPad)  30.74%  3137 

Google Android (e.g., Droid)  25.35%  2587 

RIM Blackberry OS*  17.75%  1811 

Windows Phone  9.14%  933 

Do not have access/do not want**  5.06%  516 

Palm webOS*  3.75%  383 

Symbian OS (e.g., Nokia phones)  2.73%  279 

Specific cell/all cell model phones**  2.12%  216 

Laptop/Netbook computers**  1.63%  166 

Unclassifiable/Misclassified**  1.55%  158 

Specific mobile/all mobile devices**  0.14%  14 

e‐Readers**  0.05%  5 

Total    10,205 

*New category added after analysis of survey responses to option “other” 

**After analysis of survey responses to the option “other,” responses were coded and added to the counts of appropriate survey options 

        

LMS Taskforce Student Survey January 2011   Page 22 of 33 pages 

Q15.  Which features of the primary LMS you currently use would you most likely access via a mobile device? (Check all that apply.)  

Revised Table 

Response  Frequency Count

Grades/MyProgress  20.43% 6099

Course Email*  18.60% 5552

Course Announcements*  13.83% 4129

Calendar*  10.73% 3204

Discussion Boards  10.31% 3077

Online Quizzes and Exams  5.85% 1746

Online Assignment Turn‐in*  5.41% 1614

Task Management  4.11% 1226

Online Lecture (Recorded)  3.11% 928

Live Chat  2.32% 692

Online Lecture (Live)  2.11% 629

Cloud‐based File Storage  1.71% 509

Will not access/cannot access**  1.33% 396

Class content**  0.06% 17

Unclassifiable/multiple responses** 0.04% 12

Everything**  0.02% 7 

Syllabus**  0.02% 5 

Unique responses**  0.01% 3 

Access to other web tools**  0.01% 2 

LMS Taskforce Student Survey January 2011   Page 23 of 33 pages 

Response  Frequency Count

Learning Modules**  0.01% 2 

Total  29,849

 *New category added after analysis of survey responses to option “other” 

**After analysis of survey responses to the option “other,” responses were coded and added to the counts of appropriate survey options 

    Q16.  Which features do you consider the most useful in the primary LMS you currently use? Please rank your top three choices below. Ignore the "Not Ranked" column.  

Calendar 

Cloud‐based File Storage 

Course Email 

Discussion Boards 

Grades/MyProgress 

Live Chat 

Online Assignment Turn‐in 

Online Lecture (Live) 

Online Lecture (Recorded) 

Online Quizzes and Exams 

Task Management 

 

LMS Taskforce Student Survey January 2011   Page 24 of 33 pages 

LMS Taskforce Student Survey January 2011   Page 25 of 33 pages 

Q17. For which of the following do you use the primary LMS you currently use? (Respondents were allowed to choose multiple responses)  Revised Table  Response  Percentage  Count Classroom  53.97%  8090** Studying  28.77  4313** Research Work   8.03  1204** Personal Work   3.95   592** Student Organizations   3.72   557 Social Organizations   1.33   200 Other*   0.14    21 None*   0.09    13 TOTALS    14,990 *New category added after analysis of survey responses to option “other” 

**After analysis of survey responses to the option “other,” responses were coded and added to the counts of appropriate survey options 

  Q18. How do you rate your expertise as a user of technology items in the following areas?   

Educational work (classroom, studying) 

Research work 

Personal work (outside of school) 

Entertainment 

LMS Taskforce Student Survey January 2011   Page 26 of 33 pages 

LMS Taskforce Student Survey January 2011   Page 27 of 33 pages 

Q19.  How strongly would you agree or disagree with the following statements regarding potential barriers to your using technology in learning?    

Accessibility is an important issue in technology. 

I didn't have adequate technical support. 

I don't have the skills to use technology. 

Instructors didn't know how to implement it. 

Software accessibility was not an issue. 

Technology added extra work with little connection to the course. 

Technology is too complicated. 

Technology is too expensive. 

Technology takes too much time. 

The technology didn't work on my computer. 

  

LMS Taskforce Student Survey January 2011   Page 28 of 33 pages 

LMS Taskforce Student Survey January 2011   Page 29 of 33 pages 

 Q20.  What method(s) do you use to learn new technologies? (Respondents were allowed to choose multiple responses)  Response  Percentage  Count Self Taught  38.13%  7560** Help from family, friends, colleagues 

25.60  4981** 

Ask IT support staff  11.21  2180** Learn at your workplace  10.94  2128** Taking online computer classes   7.04  1369 Attend face‐to‐face seminar   5.88  1144** Course (not online)*   0.23      44 Instructor/Classmates*   0.17      33 Other*   0.07      14 None*   0.01        2 TOTALS    19,455 *New category added after analysis of survey responses to option “other” 

**After analysis of survey responses to the option “other,” responses were coded and added to the counts of appropriate survey options 

 Q21.  If you have a problem with technology for a class, where are you MOST likely to seek assistance first? (Respondents could only choose a single response)  

Response  Chart  Frequency  Count 

Friends or colleagues      28.4%  2308 

Course instructor      27.4%  2229 

Self‐help website      18.7%  1523 

Talk to technical help support on phone 

    9.9%  807 

E‐mail technical support      7.3%  597 

Use technical help support Chat 

    4.8%  390 

Talk to technical help support in person 

    3.5%  284 

Not Answered      127 

  Valid Responses  8138 

  Total Responses  8265 

LMS Taskforce Student Survey January 2011   Page 30 of 33 pages 

Q22. Please rate your overall satisfaction with the primary LMS you currently use. (Respondents could only choose a single response)  

Response  Chart  Frequency  Count 

Completely dissatisfied      3.4%  274 

Could be better      33.1%  2691 

Indifferent      10.5%  858 

Satisfied      40.2%  3271 

Completely satisfied      12.8%  1042 

Not Answered      69 

  Mean  3.260 

  Standard Deviation  1.146 

  Valid Responses  8136 

  Total Responses  8205 

   Q23. How difficult is it for you to navigate the primary LMS you currently use? (Respondents could only choose a single response)  

Response  Chart  Frequency  Count 

Very easy/No trouble      30.8%  2507 

Easy      46.7%  3802 

Indifferent      16.3%  1322 

Difficult      5.5%  447 

Very difficult      0.7%  56 

Not Answered      71 

  Valid Responses  8134 

  Total Responses  8205 

     

LMS Taskforce Student Survey January 2011   Page 31 of 33 pages 

Q24. Please rate the usefulness of the primary LMS you currently use in improving your success in a course. (Respondents could only choose a single response)  

Response  Chart  Frequency  Count 

Not useful at all      4.3%  352 

Slightly useful      19.6%  1593 

Indifferent      17.0%  1382 

Mostly useful      37.0%  3007 

Very useful      22.0%  1783 

Not Answered      88 

  Valid Responses  8117 

  Total Responses  8205 

   Q25. What do you like most about the primary LMS you currently use?   (Deleted, available in qualitative analysis report)   Q26. What do you dislike most about the primary LMS you currently use?   (Deleted, available in qualitative analysis report)              

LMS Taskforce Student Survey January 2011   Page 32 of 33 pages 

Q27.  In your opinion, how important is it to have ONE standard learning management system across the University System of Georgia institutions? (Respondents could only choose a single response)  

Response  Chart  Frequency  Count 

Extremely important      36.9%  2990 

Important      34.3%  2784 

It does not matter      22.9%  1859 

Not important      3.1%  250 

Not important at all      2.8%  230 

Not Answered      92 

  Valid Responses  8113 

  Total Responses  8205 

   Q28.  Do you think you would find an interactive tutorial helpful if you were a first time user? (Respondents could only choose a single response)  

Response  Chart  Frequency  Count 

Yes      67.6%  5439 

No      13.5%  1087 

Uncertain      18.9%  1525 

Not Answered      102 

  Valid Responses  8051 

  Total Responses  8153 

   Q29.  If the primary LMS you currently use had more appealing features do you think you might log on more? Please explain.  (Deleted, available in qualitative analysis report)    

LMS Taskforce Student Survey January 2011   Page 33 of 33 pages 

Q29.  If you had comments or clarifications on an earlier question, please type them below.  (Deleted, available in qualitative analysis report)    Q30. If you have comments about other issues relating to LMS not covered in this survey, please type them below.   (Deleted, available in qualitative analysis report)                       This report was prepared under the direction of Dr. Hilliard Gastfriend. If you have any questions, please contact him at: hilliard.gastfriend@usg.edu for clarification or corrections. 

Appendix 8 Summary of stakeholder sandbox evaluation surveys

Created using Vovici

USG LMS Sandbox Evaluation Survey Type: LMS Sandbox Standard Report Date: 4/18/2011 Time Zone in which Dates/Times Appear: (GMT-05:00) Eastern Time (US & Canada) Total number of responses collected: 661

1: What Institution do you represent? (Institution:)(Respondents were limited to brief text responses)

Response Chart Frequency Count

Georgia State University 5.3% 35

Abraham Baldwin AgriculturalCollege 1.8% 12

Georgia Perimeter College 4.5% 30

RESA 0.3% 2

Southern Polytechnic StateUniversity 1.8% 12

Georgia Highlands College 2.4% 16

Gainesville State College 4.1% 27

Bainbridge College 2.6% 17

Waycross College 3.9% 26

University of Georgia 2.1% 14

Macon State College 4.1% 27

Georgia Health SciencesUniversity 2.3% 15

Georgia College & StateUniversity 1.5% 10

WebMBA 1.4% 9

Created using Vovici

Georgia Southwestern StateUniversity 4.5% 30

East Georgia College 2.7% 18

Gordon College 1.1% 7

University of West Georgia 2.9% 19

Darton College 3.5% 23

Augusta State University 0.9% 6

Valdosta State University 7.9% 52

Kennesaw State University 5.4% 36

USG Outreach (Best Practices) 0.5% 3

Columbus State University 1.7% 11

Middle Georgia College 2.6% 17

SREB 2.0% 13

Savannah State University 0.5% 3

Albany State University 2.4% 16

Atlanta Metropolitan College 3.5% 23

USG Training Initiatives 1.1% 7

College of Coastal Georgia 0.5% 3

Clayton State University 0.3% 2

Alternative Media AccessCenter (AMAC) 0.8% 5

North Georgia College & StateUniversity 1.7% 11

Armstrong Atlantic StateUniversity 3.2% 21

Created using Vovici

GeorgiaONmyLINE 2.4% 16

Fort Valley State University 2.0% 13

Georgia Southern University 5.7% 38

Georgia Gwinnett College 1.7% 11

South Georgia College 0.5% 3

Other Responses 0.3% 2

Valid Responses 661

Total Responses 661

Created using Vovici

2: Are you an Student or Instructor?(Respondents could only choose a single response)

Created using Vovici

3: Did you evaluate Blackboard Learn 9?(Respondents could only choose a single response)

Created using Vovici

4: Blackboard Learn 9 Instructor Evaluation (Test Results)

Actions:

4: Blackboard Learn 9 Instructor Evaluation (Satisfaction Rating)

Created using Vovici

Created using Vovici

5: Overall, do you recommend this platform (Blackboard Learn 9)for adoption?(Respondents could only choose a single response)

Created using Vovici

6: Did you evaluate Desire2Learn?(Respondents could only choose a single response)

Created using Vovici

7: Desire2Learn Instructor Evaluation (Test Results)

Actions:

7: Desire2Learn Instructor Evaluation (Satisfaction Rating)

Created using Vovici

Created using Vovici

8: Overall, do you recommend this platform (Desire2Learn) for adoption?(Respondents could only choose a single response)

Created using Vovici

9: Did you evaluate Moodle?(Respondents could only choose a single response)

Created using Vovici

10: Moodle Instructor Evaluation (Test Results)

Actions:

10: Moodle Instructor Evaluation (Satfisfaction Rating)

Created using Vovici

Created using Vovici

11: Overall, do you recommend this platform (Moodle) for adoption?(Respondents could only choose a single response)

Created using Vovici

12: Did you evaluate Pearson LearningStudio?(Respondents could only choose a single response)

Created using Vovici

13: Pearson LearningStudio Instructor Evaluation (Test Results)

Actions:

13: Pearson LearningStudio Instructor Evaluation (Satfisfaction Rating)

Created using Vovici

Created using Vovici

14: Overall, do you recommend this platform (Pearson LearningStudio) for adoption?(Respondents could only choose a single response)

Created using Vovici

15: Did you evaluate Sakai?(Respondents could only choose a single response)

Created using Vovici

16: Sakai Instructor Evaluation (Test Results)

Actions:

16: Sakai Instructor Evaluation (Satfisfaction Rating)

Created using Vovici

Created using Vovici

17: Overall, do you recommend this platform (Sakai) for adoption?(Respondents could only choose a single response)

Created using Vovici

18: Did you evaluate Blackboard Learn 9?(Respondents could only choose a single response)

Created using Vovici

19: Blackboard Learn 9 Student Evaluation (Test Results)

Actions:

19: Blackboard Learn 9 Student Evaluation (Satisfaction Rating)

Created using Vovici

Created using Vovici

20: Overall, do you recommend this platform (Blackboard Learn 9)for student adoption?(Respondents could only choose a single response)

Created using Vovici

21: Did you evaluate Desire2Learn?(Respondents could only choose a single response)

Created using Vovici

22: Desire2Learn Student Evaluation (Test Results)

Actions:

22: Desire2Learn Student Evaluation (Satisfaction Rating)

Created using Vovici

Created using Vovici

23: Overall, do you recommend this platform (Desire2Learn)for student adoption?(Respondents could only choose a single response)

Created using Vovici

24: Did you evaluate Moodle?(Respondents could only choose a single response)

Created using Vovici

25: Moodle Student Evaluation (Test Results)

Actions:

25: Moodle Student Evaluation (Satisfaction Rating)

Created using Vovici

Created using Vovici

26: Overall, do you recommend this platform (Moodle)for student adoption?(Respondents could only choose a single response)

Created using Vovici

27: Did you evaluate Pearson LearningStudio?(Respondents could only choose a single response)

Created using Vovici

28: Pearson LearningStudio Student Evaluation (Test Results)

Actions:

28: Pearson LearningStudio Student Evaluation (Satisfaction Rating)

Created using Vovici

Created using Vovici

29: Overall, do you recommend this platform (Pearson LearningStudio) for student adoption?(Respondents could only choose a single response)

Created using Vovici

30: Did you evaluate Sakai?(Respondents could only choose a single response)

Created using Vovici

31: Sakai Student Evaluation (Test Results)

Actions:

31: Sakai Student Evaluation (Satisfaction Rating)

Created using Vovici

Created using Vovici

32: Overall, do you recommend this platform (Sakai) for student adoption?(Respondents could only choose a single response)

Created using Vovici

USG LMS Sandbox Evaluation Survey Type: USG LMS Task Institutional Report Date: 4/18/2011 Time Zone in which Dates/Times Appear: (GMT-05:00) Eastern Time (US & Canada) Total number of responses collected: 661

5: Overall, do you recommend this platform (Blackboard Learn 9)for adoption?(Key: Top, Neutral, Bottom, N/A)

2 Chart Top Neutral Bottom Total Mean

Student 0 0 0 0 0.000

0.0% 0.0% 0.0% 100.0%

Instructor 75.3% 192 28 35 255 2.027

75.3% 11.0% 13.7% 100.0%

20: Overall, do you recommend this platform (Blackboard Learn 9)for student adoption?(Key: Top, Neutral, Bottom, N/A)

2 Chart Top Neutral Bottom Total Mean

Student 64.5% 21.5% 78 17 26 121 2.405

64.5% 14.0% 21.5% 100.0%

Instructor 0 0 0 0 0.000

0.0% 0.0% 0.0% 100.0%

8: Overall, do you recommend this platform (Desire2Learn) for adoption?(Key: Top, Neutral, Bottom, N/A)

2 Chart Top Neutral Bottom Total Mean

Student 0 0 0 0 0.000

Created using Vovici

0.0% 0.0% 0.0% 100.0%

Instructor 67.1% 149 34 39 222 2.230

67.1% 15.3% 17.6% 100.0%

23: Overall, do you recommend this platform (Desire2Learn)for student adoption?(Key: Top, Neutral, Bottom, N/A)

2 Chart Top Neutral Bottom Total Mean

Student 70.8% 80 19 14 113 2.088

70.8% 16.8% 12.4% 100.0%

Instructor 0 0 0 0 0.000

0.0% 0.0% 0.0% 100.0%

11: Overall, do you recommend this platform (Moodle) for adoption?(Key: Top, Neutral, Bottom, N/A)

2 Chart Top Neutral Bottom Total Mean

Student 0 0 0 0 0.000

0.0% 0.0% 0.0% 100.0%

Instructor 28.7% 58.3% 62 28 126 216 3.532

28.7% 13.0% 58.3% 100.0%

26: Overall, do you recommend this platform (Moodle)for student adoption?(Key: Top, Neutral, Bottom, N/A)

2 Chart Top Neutral Bottom Total Mean

Student 24.8% 57.5% 28 20 65 113 3.504

24.8% 17.7% 57.5% 100.0%

Created using Vovici

Instructor 0 0 0 0 0.000

0.0% 0.0% 0.0% 100.0%

14: Overall, do you recommend this platform (Pearson LearningStudio) for adoption?(Key: Top, Neutral, Bottom, N/A)

2 Chart Top Neutral Bottom Total Mean

Student 0 0 0 0 0.000

0.0% 0.0% 0.0% 100.0%

Instructor 37.7% 19.3% 43.0% 78 40 89 207 3.251

37.7% 19.3% 43.0% 100.0%

29: Overall, do you recommend this platform (Pearson LearningStudio) for student adoption?(Key: Top, Neutral, Bottom, N/A)

2 Chart Top Neutral Bottom Total Mean

Student 51.4% 19.3% 29.4% 56 21 32 109 2.743

51.4% 19.3% 29.4% 100.0%

Instructor 0 0 0 0 0.000

0.0% 0.0% 0.0% 100.0%

17: Overall, do you recommend this platform (Sakai) for adoption?(Key: Top, Neutral, Bottom, N/A)

2 Chart Top Neutral Bottom Total Mean

Student 0 0 0 0 0.000

0.0% 0.0% 0.0% 100.0%

Instructor 34.7% 19.1 46.2% 69 38 92 199 3.281

Created using Vovici

%

34.7% 19.1% 46.2% 100.0%

32: Overall, do you recommend this platform (Sakai) for student adoption?(Key: Top, Neutral, Bottom, N/A)

2 Chart Top Neutral Bottom Total Mean

Student 52.8% 20.8% 26.4% 56 22 28 106 2.566

52.8% 20.8% 26.4% 100.0%

Instructor 0 0 0 0 0.000

0.0% 0.0% 0.0% 100.0%

Appendix 9 Task force sandbox evaluation rubric form and summary of results

Areas of Consideration

Doe

sn't

Mee

t

Mee

ts

Exc

eeds

Sup

erio

r

aver

age

Doe

sn't

Mee

t

Mee

ts

Exc

eeds

Sup

erio

r

aver

age

Doe

sn't

Mee

t

Mee

ts

Exc

eeds

Sup

erio

r

aver

age

Teaching and LearningCommunication/Collaboration Features 0 1 2 3 1.55 0 1 2 3 1.47 0 1 2 3 0.58 - Calendar 1 6 2 4 1.69 2 6 4 2.17 5 5 2 0.75 - Announcements 6 5 1 1.58 5 6 1 1.67 4 8 0.67 - Course email 1 6 3 1 1.36 4 7 1 1.75 6 6 0.50 - Discussions 3 5 4 2.08 2 7 3 2.08 2 9 1 0.92 - Chat 3 6 1 1.50 2 4 5 1 1.42 5 6 0.55 - Whiteboard 4 6 1 0.73 4 4 2 1 1.00 8 2 0.20 - Groups 3 5 3 2.00 6 5 1 1.58 3 8 1 0.83 - Wikis 4 4 1 1.67 3 6 0.67 6 5 1 0.67 - Blogs 6 4 1.40 2 3 3 1 1.33 6 5 0.45 - Mobile messaging 6 2 1 1.44 2 4 2 1.00 7 2 0.22Ease of Use Comments:

Content Features 1.41 1.66 0.67 - File types supported 6 5 1 1.58 5 5 2 1.75 2 8 1 1.00 - Ease of integrating media (a/v) 1 3 8 1.58 4 6 1 1.73 4 6 0.60 - Ease of linking to a Web site 1 4 7 1.50 4 6 1 1.73 2 7 1 0.90 - Creating a page 1 4 6 1 1.58 5 6 1 1.67 3 5 3 1.00 - File upload (one file/zipped) 6 4 2 1.67 7 4 1 1.50 2 9 0.82 - File Manager/folder structure 8 3 1 1.42 6 5 1 1.58 2 9 1 0.92 - File sharing between class sections 2 3 4 1.22 3 3 2 1 1.11 4 4 1 0.67 - Learning Modules design/flexibility 1 4 6 1 1.58 1 6 5 2.33 5 6 1 0.67 - Content Management 6 4 1 1.55 2 6 2 2.00 5 6 0.55 --Create from Master Course? 4 5 1.56 5 3 1 1.56 4 4 0.50 - Portfolios 1 8 2 1.09 6 2 2 1.60 9 2 0.18 - Student workspace 3 4 2 0.89 1 3 5 2 1.73 7 2 1 0.40 - Glossary 1 8 3 1.17 9 2 1 1.33 7 3 1 0.45Ease of Use Comments:

Blackboard 9 Desire2Learn Moodle

Areas of Consideration

Doe

sn't

Mee

t

Mee

ts

Exc

eeds

Sup

erio

r

aver

age

Doe

sn't

Mee

t

Mee

ts

Exc

eeds

Sup

erio

r

aver

age

Doe

sn't

Mee

t

Mee

ts

Exc

eeds

Sup

erio

r

aver

age

Blackboard 9 Desire2Learn Moodle

Assessment Features 1.42 1.57 0.59 - Quizzes, Surveys 2 9 1 1.92 3 8 1 1.83 3 7 1 0.82 --Modify by student? 4 3 1.43 5 2 1.57 5 3 0.38 - Database of questions 4 7 1 1.75 4 7 1 1.75 4 5 2 0.82 - Selective Release/time control 1 5 6 1.42 4 3 5 2.08 8 4 0.33 - Import/export 1 9 1 1 1.17 1 5 5 1 1.50 2 9 0.82 - Discussions for grades 5 5 2 1.75 5 4 2 1.73 5 5 1 0.64 - Group assessments 1 6 3 1 1.36 1 7 2 1 1.27 8 2 1 0.36 - Presentations 1 5 3 1.22 1 5 2 1 1.33 3 6 0.67 - Plagiarism detection tools 3 4 1 0.75 3 4 1 0.88 6 2 0.25 - Assignment Dropbox 8 3 1 1.42 4 7 1 1.75 2 10 0.83Ease of Use Comments:

Class Management Features 1.46 1.63 0.79 - Roster 1 7 3 1.18 1 4 5 1.40 2 6 2 1 1.18 - Tracking reports 1 5 5 1.36 5 6 1.55 2 6 3 1.09 - Group management 5 4 2 1.73 7 4 1.36 2 8 0.80 - Selective release 1 5 5 1.36 3 5 3 2.00 4 7 0.64 - Learning module/structure/flexibility 3 6 1 1.80 4 5 2 1.82 6 3 1 0.60 - Import/export/migration 1 6 3 1 1.36 5 5 1 1.64 5 6 0.55 - Design/structure flexibility 6 4 1.40 1 3 6 1 1.64 4 5 1 0.70Ease of Use Comments:

Gradebook Features 1.44 1.55 0.88 - Import/export to Excel 9 3 1.25 6 4 1 1.55 2 6 3 1.09 - Flexibility of views 6 4 2 1.67 6 4 1 1.55 3 7 1 0.82 - Workflow 1 9 1 1 1.17 6 5 1.45 3 7 0.70 - Customizeability 5 6 1 1.67 5 5 1 1.64 2 8 1 0.91Ease of Use Comments:

Areas of Consideration

Teaching and LearningCommunication/Collaboration Features - Calendar - Announcements - Course email - Discussions - Chat - Whiteboard - Groups - Wikis - Blogs - Mobile messagingEase of Use Comments:

Content Features - File types supported - Ease of integrating media (a/v) - Ease of linking to a Web site - Creating a page - File upload (one file/zipped) - File Manager/folder structure - File sharing between class sections - Learning Modules design/flexibility - Content Management --Create from Master Course? - Portfolios - Student workspace - GlossaryEase of Use Comments:

Doe

sn't

Mee

t

Mee

ts

Exc

eeds

Sup

erio

r

aver

age

Doe

sn't

Mee

t

Mee

ts

Exc

eeds

Sup

erio

r

aver

age

0 1 2 3 0.75 0 1 2 3 1.165 6 1 0.67 5 5 1 1.641 9 1 1.00 3 8 1.733 6 2 1 1.08 2 6 2 1.004 7 1 0.75 7 4 1.36

10 1 1.09 6 5 1.456 4 1 0.55 6 3 1 0.502 8 1 0.91 1 3 3 1.297 4 0.36 1 6 3 1.204 7 1 0.75 4 5 1 0.706 3 0.33 3 5 1 0.78

0.89 1.441 8 1 1 1.18 7 2 1 1.401 8 1 1.00 7 3 1.30

10 1 1.09 5 6 1.551 10 1 1.00 1 6 4 1.27

11 1 1.08 7 4 1.3610 1 1.09 8 2 1 1.36

2 4 1 0.86 4 3 1 1.634 6 2 0.83 1 8 2 1.093 6 2 0.91 7 2 1.224 3 2 0.78 1 5 2 2.135 4 1 0.60 6 1 2 1.565 4 1 0.60 1 4 2 2 1.566 3 1 0.50 6 3 1.33

SakaiPearson

Areas of Consideration

Assessment Features - Quizzes, Surveys --Modify by student? - Database of questions - Selective Release/time control - Import/export - Discussions for grades - Group assessments - Presentations - Plagiarism detection tools - Assignment DropboxEase of Use Comments:

Class Management Features - Roster - Tracking reports - Group management - Selective release - Learning module/structure/flexibility - Import/export/migration - Design/structure flexibilityEase of Use Comments:

Gradebook Features - Import/export to Excel - Flexibility of views - Workflow - CustomizeabilityEase of Use Comments:

Doe

sn't

Mee

t

Mee

ts

Exc

eeds

Sup

erio

r

aver

age

Doe

sn't

Mee

t

Mee

ts

Exc

eeds

Sup

erio

r

aver

age

SakaiPearson

0.89 1.111 7 4 1.25 1 3 7 1.552 5 1 0.88 3 4 1 0.752 7 3 1.08 4 6 1 1.733 4 5 1.17 2 4 4 1.204 6 1 0.73 7 3 1 1.458 2 1 0.36 3 5 2 0.903 5 2 0.90 5 3 1 1 0.802 6 1 0.89 3 4 2 0.894 1 1 0.50 5 1 1 0.431 8 3 1.17 7 2 1 1.40

0.73 1.283 6 2 0.91 8 2 1.206 3 2 0.64 2 4 2 1 1.222 7 1 0.90 5 4 1.444 5 2 0.82 2 3 4 1.225 5 1 0.64 1 5 4 1.306 4 1 0.55 7 3 1.305 5 1 0.64 7 3 1.30

1.14 1.461 7 2 1 1.27 1 6 2 1 1.301 8 2 1.09 5 4 1 1.602 5 3 1.10 5 3 1 1.563 4 4 1.09 1 5 3 1 1.40

0.00

0.20

0.40

0.60

0.80

1.00

1.20

1.40

1.60

1.80

Commun

icatio

n/Coll

abora

tion F

eatur

es

Conten

t Fea

tures

Asses

smen

t Fea

tures

Class M

anag

emen

t Fea

tures

Gradeb

ook F

eatur

esBlackboardDesire2LearnMoodlePearsonSakai

Appendix 10 Guiding criteria document

Guiding Criteria A. These criteria have guided our work up to this point: The platform must: Be well-established and stable operationally. Be able to be integrated with a student information system Be committed to being fully compliant with accessibility laws and recommendations Meet basic functionality requirements of our current system.

B. These criteria will guide our work as we move into the next stage of our work: The platform should: Satisfy the technical requirements set forth by the university system ITS department Allow the university system to provide necessary services while being fiscally

responsible Offer advanced functionality and usability to our stakeholders Allow for integration with various third party solutions that our stakeholders expect to

use in conjunction with the platform Have a future roadmap of product development that ensures that while the platform

stays relevant and effective, it does not inflict painful migrations along the way Have a responsive and active support network, whether this is from a vendor

company or from a community of users Offer access to the source code or to APIs, etc. to allow for internal USG technical

experts to provide appropriate service to the system stakeholders

Appendix 11 Finalist questions and demo requests

 

Functional Questions for Vendors  

Demo Portion: 

1. How can classes/sections be created from a master course?  Please demonstrate. 2. How is content created?  Please demonstrate.   

i.e.  webpage/text, assignments, quizzes 3. Please demonstrate your built‐in content creation tools such as the WYSWIG HTML editor 4. Please demonstrate how a quiz can be created via the following methods:   

a. From within the system b. Through applications such as Respondus c. Via publisher testbanks 

5. How can quiz settings be modified for a single student?  Please demonstrate. 6. How are quizzes completed by students and how are they graded? 

a. How are written‐response (essay) questions graded?  Please demonstrate. 7. How are assignments completed by students and how are they graded?  Please demonstrate 8. Please show how mathematical expressions can be input by a student. 9. How are grades calculated and displayed in the grade book?  Please demonstrate. 10. How are rubrics created and used?  Please demonstrate. 11. How are plagiarism‐detection tools accessed and used?  Please demonstrate. 12. How can groups be created and used?  Please demonstrate. 13. What communication tools are available?  Please demonstrate any of the ones listed below and 

any others you wish to show us:        ‐‐Are chat rooms with webcam support included?        ‐‐Is instant messaging included?        ‐‐Are wikis included?        ‐‐Are blogs included? 

14. How is student activity tracked?  Please demonstrate. 15. How are publisher course cartridges loaded?  Please demonstrate or  illustrate. 16. Please demonstrate this platform’s mobile capabilities/features. 

 

 

Q&A Portion: 

1. Explain how master/child classes/sections works.  For example, suppose a master course is set up with a number of children sections.  Can information (assignments/announcements/quizzes, etc.) be sent down from master to some sections but not all, and without erasing anything that has been done already in the child sections.  Similarly, can information be sent from child to child section? 

2. What types of content migration tools are available? 3. How do you handle audio/video content? 4. What tools are included for students? 

       ‐‐Portfolios?        ‐‐Workspace?  

5. What does file storage/management look like from both the instructor and student perspective? 6. Are there student features for taking notes? 7. How do you integrate with the following third party solutions?   

a. Wimba Classroom/Voice/ Pronto 

b. Turnitin  

c. Turning Technologies 

d. Pearson’s MyMathLab  

e. MapleTA  

f. Respondus LockDown Browser 

g. Respondus  

h. Smarthinking  

i. Others? 

8. Please discuss browser/java requirements? What browsers/OS are supported?  What is your process for certifying new versions? 

9. Do you allow access to an API? 10. Do you have a “become user” or true superuser type account available for administrators? 11. Would you describe how your LMS can/does integrate with a student information system (ie 

systems like Banner)? 12. Can course/student information be imported into your system? Is this a batch or realtime load?  

If an import is a batch process, how are students and/or other instructors added after the batch is complete? 

13. Can grade/student information be exported from your system and loaded back into the student information system? 

 

14. Can guest instructors or visitors/mentors be added to the course (independent of the load from the student information system)?  If so, how is their access to student's personal information limited (contact information, grade information, etc.)? 

15. What other roles does the platform allow (instructional designer, TA, Lead TA, etc.)? 16. Can a student (or instructor) limit what information can be seen about other students in a 

course (student privacy concerns)?  For example, if a mass email is being sent to the students on the class roster, will the email addresses be hidden from others participating in the course? 

17. Does the platform include a way to include pictures in the class roster? 18. Can students access a tutorial/introduction to the LMS upon initial login? 19. Is this platform compliant with Section 508c of the Rehabilitation Act?  What are the 

accessibility features? 20. Does this platform support real‐time captioning? 21. Does this platform support live video feeds (remote sign language interpreting)? 22. How can users with disabilities access this platform using text to speech software (JAWS, 

VoiceOver), speech to text software (Dragon Naturally Speaking), and other accessibility software (refreshable Braillers, electronic notetakers, etc.)? 

23. Does this platform offer text enlargement features for users with low vision? 24. Does this platform use images for technical support?  Are alt tags used? 25. Does this platform’s tutorial videos captioned?  Do they have video descriptors? 26. Please discuss your mobile features and strategy. 27. Please discuss the philosophy of upgrades and future versions – how complex is it to migrate 

from one version to the next and how is the timetable of upgrades determined?  Is there support for past versions if an institution decides not to upgrade on your timetable? 

28. Please discuss your product roadmap. 

Appendix 12 Final overall rubric form and summary of results

Blackboard 9 Desire2Learn MoodleTeaching and Learning•Functionality•Ease of use•Accessibility•Flexibility in roles•Flexibility in use cases•Ability to use third party plug-ins•Integration with banner and other systems•Mobility•Browser/Java issuesGetting There•Migrating course content and tool functionality from Vista•Migrating content from other situations•Training needs•Implementation set up issuesSupport and Community•If commercial, responsiveness of vendor•If commercial, availability of support/timeliness of support from vendor•If commercial, other add-ons, and other product tie-ins•If open-source, size and relevance of community•If open source, responsiveness of community•If open source, availability of commercial affiliate support

Blackboard 9 Desire2Learn Moodle•All – ability to support in-house at ITS and on campuses•All – infrastructure needsMoving Forward•Future roadmap in terms of functionality•Frequency and severity of upgrades – needs for frequent downtime/migration•Ease of migration to future versionsEase of migration to other productsCost of Ownership•Hardware costs•Human costs•Implementation costs•Support costs

Teaching and Learning•Functionality•Ease of use•Accessibility•Flexibility in roles•Flexibility in use cases•Ability to use third party plug-ins•Integration with banner and other systems•Mobility•Browser/Java issuesGetting There•Migrating course content and tool functionality from Vista•Migrating content from other situations•Training needs•Implementation set up issuesSupport and Community•If commercial, responsiveness of vendor•If commercial, availability of support/timeliness of support from vendor•If commercial, other add-ons, and other product tie-ins•If open-source, size and relevance of community•If open source, responsiveness of community•If open source, availability of commercial affiliate support

Pearson Sakai

•All – ability to support in-house at ITS and on campuses•All – infrastructure needsMoving Forward•Future roadmap in terms of functionality•Frequency and severity of upgrades – needs for frequent downtime/migration•Ease of migration to future versionsEase of migration to other productsCost of Ownership•Hardware costs•Human costs•Implementation costs•Support costs

Pearson Sakai

Blackboard 9 Desire2Learn Moodle Pearson SakaiTeaching and Learning 1.00 1.72 0.81 0.47 1.50

Getting There 2.00 1.67 0.33 0.00 0.33

Support and Community 0.90 1.70 0.60 0.50 1.00

Moving Forward 1.00 1.00 0.50 0.50 1.00

Cost of Ownership 1.50 1.75 1.00 1.67 1.00

First Choice Second Choice Third Choice Fourth Choice Fifth ChoiceTeaching and Learning Desire2Learn Sakai BlackBoard 9 Moodle Pearson

Getting There BlackBoard 9 Desire2Learn Moodle/Sakai Pearson

Support and Community Desire2Learn Sakai BlackBoard 9 Moodle Pearson

Moving Forward Bb9/D2L/Sakai Moodle/Pearson

Cost of Ownership Desire2Learn Pearson Blackboard 9 Moodle/Sakai

Appendix 13 Relative costing information and considerations

There are many components in the total cost of ownership of a learning management system. The primary ones used by ITS to provide cost estimates are: • License fees – these include the base fee along with some bundled services • Hardware costs • Personnel costs – for hardware support, software support, system administration,

etc. • Training costs – both for ITS staff and for training of the trainers at the

institutions • Course and content migration support • Integration costs – both with third party services and for customization at the

different institutions • Reporting costs • End-user costs

For the purposes of task force deliberations, four primary figures were used: • License fees • Infrastructure costs • Support personnel costs • Total cost of ownership (as calculated by ITS)

Due to the ongoing nature of the negotiations and the need to keep some level of security around those negotiations until the task force results are made public, the actual dollar figures were estimated and then ranked from lowest to highest (in lieu of firm figures). The task force feels that the other (non-cost related) support in favor of the task force recommendations is strong enough that unless the actual rankings of the cost figures changes dramatically that this estimation process is sufficient. For the open source platforms, there are no license fees. However, it was recognized that there are commercial affiliates for each of these platforms and that some of the costs incurred for transition and implementation could be in the form of outsourcing rather than in-house expenses. The task force did not differentiate between these two types of expenses. For two of the commercial platforms, there were two different options presented. For the primary deliberations, in each case, the option of the most comprehensive offering from the platform was considered. However, when discussing secondary recommendations in the case of severe fiscal constraints, the more scaled back offers were considered. For Pearson Learning Studio, this involved a new offer that we have been asked not to reveal in a public document at this time. However, this is a new offering from Pearson, so it was not checked during the reference checks and there is no assurance of reliability or quality. Furthermore, it does not allow for ITS internal hosting and so it was not ruled acceptable by the task force. For Blackboard, there is an offer to keep the same price as is now in place ($590K) for an additional five years. This is a scaled back offer since the USG would still incur the extra license fees for the add-on services that are used system-wide (including for example Wimba/Elluminate, Connect, and Mobile). As noted in the report, this is only an acceptable alternative if the over-riding constraint is money for

licenses. Note that if this solution is chosen then there must still be an investment in the transition (migration and training), and either a heavy investment must be made in hardware, or there will not be customization available for the individual campuses, which is one of their highest priorities for the new LMS. Here are the cost comparisons for the full offerings of each platform (0 - $0 cost, 1= lowest, 5 = highest)

LMS Offering Infrastructure Costs 

Support Staff Costs  License Costs 

Total Cost of Ownership (5 years) 

Blackboard Learn 9  3 3 1  2

Desire2Learn  2 2 2  3

Moodle 1.9.11  3 4 0  4

Pearson Learning Studio  0 1 3  1

Sakai 2.7.1  3 4 0  4

In the total cost of ownership over five years, the gap between Blackboard 9 and Desire2Learn is currently estimated at about $1.3M. The same time period gap between Desire2Learn and the open source platforms is estimated at $3.5M.