LMS Evaluation Information
About the LMS Evaluation
The Learning Technologies Resource Centre has embarked on a multi-stage evaluation of next-generation Learning Management Systems to find a replacement for WebCT Campus Edition 4.1 as it reaches end-of-life status when the current licence agreement expires in 2008.
(more ...)
WebCT Campus Edition 4.1 will no longer be supported or upgraded by the vendor at the close
of the current contract. Considering the recent merger of WebCT and Blackboard under the Blackboard
name and the explosion of new Learning Management Systems on the market, we felt this was the
opportune time to assess our online teaching and learning needs at McMaster and select a product
that will meet those needs at the end of the contract and beyond. (Enterprise WebCT products will
still be available and will be included in the evaluation.) Given the fact that it will take approximately 1 year to migrate to a new system it is crucial
that we start the process now. The various and diverse prespectives at this institution (academic,
administrative, technical) necessitate that this process be
as inclusive as possible. This website will be used to update the community on our
activities, events and progress. Please feel free to drop us a comment by clicking on the "Comments" link
on the left side of this page.
Process Updates
April 29, 2008 - Evaluation complete
Next generation Learning Management System has been selected
The Learning Technologies Resource Centre (LTRC) has completed an extensive multi-year project to find the best possible replacement for WebCT Campus Edition 4.1 (currently in use). The LTRC along with the LMS Selection Committee and Senior Administration chose the Blackboard Learning System after extensive analysis of twelve potential products and the teaching and learning needs of the community.
For further information please visit our new Implementation information site.
October 22, 2007 - RFPs Issued
LMS Selection/Evaluation Process Update: Executive Summary
Download the full report.
As WebCT 4.1 approaches its end-of-life date, the LTRC has embarked on a Learning Management System (LMS) selection and evaluation process to procure a system that best meets the needs of McMaster University . In December of 2006, the LTRC issued a Request for Information (RFI) as an international public open call to Learning Management System (LMS) vendors and channel partners (private and open-source). Twelve submissions were received. The first objective was to reduce the number of submissions to a manageable number in order to proceed with the Request for Proposal (RFP) and hands-on trials/assessments. In order to narrow down the product list to continue to the next phase of evaluation, i) faculty, staff, and students were sent surveys on feature preferences; ii) current WebCT usage was analyzed; iii) discussions with faculty and staff were conducted, and iv) continued investigations into current literature and processes at other educational institutions were conducted.
The surveys and usage analysis reveal that it is difficult to determine what is important because what is important is context-dependent . The survey data, the WebCT usage data, and the experience of the LMS staff show that there is a high degree of diversity in the teaching, learning, administrative, and management needs . Consideration of the technology adoption model in the context of current WebCT use illustrates that the best way to facilitate current integration is to ensure a similar level of functionality to WebCT 4.1 . Exploring the diversity of needs shows that the best way to provide flexibility is to look for the largest possible range of options . Last, there are certain technical and administrative constraints that need to be met in order to facilitate teaching & learning innovations and to function within the context of the University's infrastructure, policies, and direction .
Looking at the available data in multiple contexts can provide an insight into which products are better suited to our institution. Each of the twelve RFI returns was scored based in three major ways and compared. First, each product was analyzed with respect to how closely it matched current WebCT functionality. Second, each product was analyzed with respect to the degree to which it provided options for course and program management as well as flexibility. Third, the products were ranked against key technical, administrative, and management criteria.
The initial report presented to the LMS Selection/Evaluation Committee for discussion on Wednesday, September 26 th , 2007 recommended seven RFP submissions. The committee however, agreed that sending RFPs to seven vendors was too many to effectively and efficiently evaluate. Instead, a maximum of five products were picked for RFP based on the following criteria: i) pick the top three products based on the analysis in Section 6 of the report; ii) include Moodle as the open-source option if Moodle is not one of the top three products (as it consistently scored the highest among the open-source returns); iii) include the top FirstClass product as the FirstClass option, if either FirstClass product is not part of the top three.
The LTRC issued 5 RFPs on October 15 th , 2007 for the following five products: Learning Environment (Desire2Learn), eLearning Server (Intrafinity), Vista (Blackboard), Moodle (Open Knowledge Technologies) , and FirstClass (Palantir Information Systems) . Download the full report.
May 18 , 2007 - Faculty/Staff and Student Surveys closed
Thank you to all who participated
The online surveys have now closed. We received 206 responses to the faculty/staff survey and 1899 responses to the student survey. The winners of our draw for Titles Gift Certificates were Lonnie Magee, Economics Dept. from the faculty/staff/technical surveys and Angela Graham, graduate student in History from the student survey.
April 3, 2007 - New Information Sessions Scheduled
New Information Sessions Scheduled
A series of information sessions for the Learning Management Systems (LMS) selection process will be held for interested faculty and staff. The information sessions are scheduled as a casual “drop-in”. LTRC staff will be present to discuss the LMS process, the current surveys that have been distributed, and to discuss any concerns/needs about a new LMS for the institution.
Dates:
Thursday April 12th, 9:30 am – 3:30 pm
Thursday April 19th, 9:30 am – 3:30 pm
Wednesday April 25th, 9:30 am – 3:30 pm
Location: LTRC Classroom, Basement of Thode Library: B117/A
Please send any queries to: lmseval@ltrc.mcmaster.ca
March 27, 2007 - Distribution of Faculty/Teaching Support Survey Begins
LMS Information Session Kicks off Faculty/Teaching Support Survey
On Tuesday, March 27 at 9 a.m. the LTRC held an information session for faculty and teaching support staff.
The distribution of the faculty/teaching support survey has now commenced. This survey is an opportunity for the community
to let the LTRC and Selection Committee better understand what features in a LMS are important in the context of
teaching and learning. If you do not receive a survey invitation by email by the end of Wednesday March 28th, please
fill out the Request for Survey form. An email with a survey link will then be sent to you.
March 22, 2007 - Information Session
Learning Management System Information Session
The Learning Technologies Resource Centre will hold an information session on Tuesday, March 27 from 9 to 10 a.m. in TSH B106 to discuss the current Learning Management Systems (LMS) selection process. All interested faculty and support staff are invited to attend.Agenda:
- Brief LMS history at McMaster
- Selection process overview
- Faculty/support staff needs survey
- Q & A
March 21, 2007 - First Committee Meeting
Learning Management System Evaluation Committee Holds First Meeting
The LMS evaluation/selection committee held its first meeting on Friday, March 16 to review the LMS selection process, the data collected to date and the next steps in the process. The Request for Information to vendors contained 450 technical, administrative and academic checkpoints. Ten vendors responded to the RFI. The LTRC has developed online rubrics so that our user community can help us weight the importance of each of the checkpoints. The weighting will be applied to the vendor data and a ranking of the submissions will help determine which vendors receive a Request for Proposal in the next phase of the process. The final evaluation phase will involve hands-on testing of selected products.
Due to the overwhelming response of people interested in helping with the evaluation we have chosen a collaborative process that allows any and all members of the academic community to contribute. A survey of the relative importance of the academic checkpoints in the RFI will soon be online. Faculty, students and academic support staff are encouraged to complete the survey and help the committee determine which products have the potential to meet the teaching and learning needs at McMaster. The technical rubric is near completion and was contributed to by many technical staff on campus familiar with Learning Management Systems. A general information session is also scheduled for Tuesday, March 27 at 9 a.m. in TSH-B106.
Our LMS Evaluation website will help to keep you informed about issues related to the process.
You can also direct correspondence to lmseval@ltrc.mcmaster.ca or drop a note in the comments section of the lmseval webpage noted above.
| Adrienne Visocchi |
Kinesiology |
| Andy Hrymak |
Chemical Engineering |
| Anna Moro |
Linguistics and Languages |
| Brian Gould |
Mohawk/McMaster Consortium |
| Dale Roy/Paola Borin |
CLL |
| Dick Day |
Psychology, Science |
| Jeffrey Trzeciak |
Library |
| John Alley |
UTS |
| John Bell |
Humanities Media/Computing |
| Nicole Wagner |
PhD Student, Business |
| Olive Wahoush |
School of Nursing |
| Rocco Piro |
CSU, FHS, Comm. Studies |
| Sophia Holness |
Registrar’s Office |
| |
|
| Bart Strong |
LTRC |
| Colin Czerneda |
LTRC |
| Herman Poon |
LTRC |
| Katrina Espanol-Miller |
LTRC |
| Laila Bastedo |
LTRC |
| Tracey Butler |
LTRC |
| Wayne Cai |
LTRC |
A summary of the presentation is also available:
November 29, 2006 - Evaluation Process
LTRC Embarks on a Learning Management System Evaluation Process
The Learning Technologies Resource Centre has embarked on a multi-stage evaluation of next-generation Learning Management Systems to find a replacement for WebCT Campus Edition 4.1 as it reaches end-of-life status when the current licence agreement expires in 2008.
WebCT Campus Edition 4.1 will no longer be supported or upgraded by the vendor at the close of the current contract. Considering the recent merger of WebCT and Blackboard under the Blackboard name and the explosion of new Learning Management Systems on the market, we felt this was the opportune time to assess our online teaching and learning needs at McMaster and select a product that will meet those needs at the end of the contract and beyond. (Enterprise WebCT products will still be available and will be included in the evaluation.)
Given the fact that it will take approximately 1 year to migrate to a new system it is crucial that we start the process now. The LTRC will be providing assistance and training during the migration in order to make the transition as smooth as possible. It is anticipated that we will be running the old and new systems simultaneously for at least 1 year and keeping a copy of the old system for 3 years in order to retrieve archived data. The complete process involves eight phases as described below and will be driven by faculty/student needs and participation:
| Phase I |
Request for Information and product qualification using over 450 technical, instructional and administrative checkpoints (The RFI was released November 13th) |
| Phase II |
Summative Evaluation (Stakeholder evaluation committee) January, 2007 |
| Phase III |
Selection of qualifying systems based on above criteria – February, 2007 |
| Phase IV |
Request for Proposal to selected shortlist of qualified vendors and software trial |
| Phase V |
Evaluation of faculty/student feedback regarding systems on trial – March, 2007 |
| Phase VI |
Detailed recommendation by evaluation committee including cost analysis and awarding of contract (estimated completion date – April, 2007) |
| Phase VII |
Pre-deployment of LMS (installation, testing, documentation, web pages) June, 2007 |
| Phase VIII |
Implementation of new LMS, Training and migration (July 2007 through July 2008) |
Many of the next generation Learning Management Systems utilize communication tools that students have become familiar with outside their academic environment. Online collaboration tools, conferences, discussion forums, blogs, wikis, e-portfolios and podcasts have become important elements in the learning process of today’s student. LearnLink, for example, was built on the principles of collaborative learning and it is entirely possible that there is an LMS that will blend the best practices of both LearnLink and WebCT. This is one of the many questions that we would like answered in our evaluation process so that McMaster’s future online teaching and learning needs are appropriately met.
If you have any thoughts or concerns or would like to participate in the process in some way please contact us at lmseval@ltrc.mcmaster.ca
Bart Strong
Executive Director
Learning Technologies Resource Centre
Additional Resources
The Request for Information (RFI)
is now available for download in Word Document format. To download the file to your computer right click on the link below and click on 'Save
Target As' (Internet Explorer) or 'Save Link As' (Firefox, Netscape):
Last updated:
May 18, 2007
|