Detailed Checklist To Compare LMS Vendors


Share this Post


eCoursys Desk

When most of LMS platforms claim to provide best features, how do you compare & choose the best among them?
This article lists the characteristics of high quality LMS platform. You can use these points as checklists to compare competing LMS platforms & choose the one that best suits your organization needs.
  • Uses a straightforward, simple process for learner registration/enrollment
  • Easy and secure self-registration process to gain access to the LMS for external users (who are not automatically registered by virtue of being an employee)
  • Allows learners to auto-enroll for courses
  • Automatically places learners in wait lists if courses are full, with automatic notification to affected stakeholders
  • As a configuration option (where courses are not mandatory), allows learners to select, register and remove courses or curricula from their course listing/learning track on their own with no supervisory or administrator intervention
  • Allows automated identification, disabling, and archiving of learner accounts and course sections that have been inactive for a set amount of time
  • Allows guest (i.e., non-credit) access to course
  • Allows administrators to easily override settings made for groups to account for particular training needs of individuals
  • Allows administrators to easily combine and un-combine course sections
  • Allows a variety of billing options: credit card, corporate purchase orders, departmental account numbers, etc.
  • Notifies users when actions are taken in the system that affect them, both through internal system notification functions and by e-mail.
  • Notification email messages can be customized using a combination of free text and system variables
  • A notification history function is included
  • Notifications and calendar events for learning events and deadlines can be inserted into web-based e-mail calendars, like Gmail and Hotmail.
  • Allows pushing of general notifications out to only specific groups
  • Sends reminders about certifications that are about to expire (so that they can renew)
  • Provides the ability to annotate and communicate actions taken, approvals, errors, etc. in regards to administrative actions, for future reference or for other administrators
  • Has robust internal email editor features resembling standalone email programs (such as Outlook)
  • Offers batch options for tasks involving groups of system objects
  • Allows administrators to batch register groups of learners
  • Allows administrators to create backups in batch
  • Allows administrators to batch set permissions and roles for users
  • Allows administrators to batch configure courses, learning tracks, and curricula
  • Allows time shifting of batch processes of database or processor-intensive tasks to minimize performance disruption during peak usage times
  • Allows administrators to set prerequisites so that learners are evaluated for meeting prerequisites before being able to enroll in a course
  • Can be configured to deliver pre-assessments to allow learners to “test out” if they demonstrate mastery of the material for a course
  • Includes options for configuration of waiving course/curriculum requirements based on demographic attributes other than course completion or pre-assessments
  • Establishes equivalencies so that learners can receive credit for courses and/or waive the requirement to take courses that share the same material as a course already taken
  • Provides an easy, powerful, and intuitive process for importing and configuring content
  • Provides the ability to internally create and/or configure ancillary learning objects like glossaries that can interwork with courses and apply globally to more than one course within the LMS
  • Imports course packages of unlimited size (especially important if your eLearning contains rich media, or courses are very long)
  • Allows elements of a course to be updated without creating a new version of the course (for instance, swapping out the SCORM manifest file without having to upload an entire replacement course package)
  • Allows creation of custom categories and folders of content, for display to users as well as for internal authoring and management
  • Allows assignment of custom thumbnails and icons for content objects displayed in the library or catalog
  • Allows course administrators to duplicate or roll course objects/records forward to new sessions without requiring re-entry of data
  • Has a course organizer feature that allows administrators to easily design and reconfigure a course by dragging and dropping course elements and reordering them
  • Uses robust security architecture to maintain system access
  • Allows learners to self-register for an LMS account using a unique email address and follow-up email requiring a validation response, to avoid duplicate accounts, incomplete user profiles, etc.
  • Incorporates e-signatures
  • Conforms to secure application infrastructure standards such as ISO/IEC 27001
  • Allows encryption of sensitive data
  • Provides a single sign-on, so that users who have logged in to the enterprise intranet (through a portal, etc.) can get into the LMS without additional login
  • Allows login to the LMS to transfer to other enterprise systems (especially HR)
  • Requires user logon only once per LMS session
  • Requires each user to be uniquely identifiable (e.g., user name or user ID)
  • Runs all user requests through a common security checkpoint in the system architecture
  • Defines a wide variety of permission and role levels that are applicable to a range of organizational structures and use case scenarios for the system
  • Restricts course enrollment to pre-authorized learners
  • Incorporates permission levels and supporting features that allow input of SME review comments that are tagged to screens, with search and filtering capability. (this is usually only found on an LCMS
  • Offers “organization aware” features that allow administration based on external data feeds concerning organization roles and permissions
  • Allows content-level permissions that specify conditions to be met for learners to be able to view
  • Features levels of permission corresponding to clearly defined levels of administrative responsibility
  • Performs with minimal latency under a variety of use case scenarios and load conditions
  • Handles large numbers of concurrent users
  • (for behind-the-firewall solutions) Takes minimal time to actually deploy. In 2016, a survey of corporate LMS managers found that nearly one-half (43%) of organizations took more than three months after initial installation to begin enrolling their learners.
  • Handles user load efficiently, provisioning and scaling resources to smoothly accommodate fluctuations (especially spikes) in numbers of concurrent users
  • Works equally well (all functions, including course delivery) on all standard Internet browsers, including a reasonable span of legacy versions of those browsers (backward compatibility with 2 year-old versions is often used as a rule of thumb)
  • Has reasonable system requirements that are attainable within your organization
  • Uses normalized architectures for hardware and software implementations
  • Can be load balanced across multiple servers, with the LMS application distributed across multiple application servers and the database itself located elsewhere.
  • Can be clustered
  • Has robust mechanisms for coping with machine failure
  • Contains a course catalog including many details of courses, especially:
    Objectives, Credits, Course #, Cost, Associated career track(s), Associated competencies, Delivery method, Prerequisites, Functional area, Location (if synchronous), Job skill, Product line, Subject, Associated resources, Seat time, Keywords, etc.
  • Has global search function for learners that searches on all of the above plus text within content and allows grouping of search results.
  • Includes the ability to enter and capture such items as course development costs
  • Provides support for learner surveys and training needs surveys, with options for free text as well as closed ended questions.
  • Includes training budget/cost tracking and projecting features that stores and reports (by learner, course, organization, year, etc.) such items as:
    – Budget authorizations
    – Funds allocated
    – Funds still available
  • Supports the current and all required legacy versions of relevant standards such as SCORM, AICC, IMS Common Cartridge, etc
  • The Standard Data Elements used by the product comply with the DoD Data Architecture (DDA) and Defense Data Dictionary System(DDDS)
  • Supports multi-byte (Unicode) fonts (esp. Asian language characters) and right-to-left languages. This requires that the LMS interface text is stored as data, separate from source code.
  • Offers flavors of the interface in foreign languages for global reach
  • Supports the ADL Training and Learning Architecture (TLA).
  • Includes the ability to enter and capture such items as course development costs
  • Provides support for learner surveys and training needs surveys, with options for free text as well as closed ended questions.
  • Includes training budget/cost tracking and projecting features that stores and reports (by learner, course, organization, year, etc.) such items as:
    – Budget authorizations
    – Funds allocated
    – Funds still available
  • Allows attaching, associating, and consolidating diverse content pieces into a single course
  • Includes ways to link content and assignments in blended learning courses so that it is clear that the components are part of a single course and can be assessed and tracked as such
  • Provides a way to centrally manage links so that changes can easily be made to external URLs associated with learning objects.
  • Is optimized for reusability in general (not just measured by SCORM support). Some LMSs have their own internal content repository that allows internal mixing and matching of objects in designing a course, curriculum, or learning track.
  • Supports a wide variety of delivery architectures. For instance, an eLearning architecture involving a content repository that may be on a different server than the LMS and is supplied by another vendor.
  • Can deliver and/or manage as large a variety of learning objects and activities as possible based on such fundamentally different approaches as:
    – Informal learning
    – Constructivist learning environments (problem-centered, discovery-based, game-based, inductive, etc.)
    – Collaborative/social learning
    – Blended learning
    – Game-based learning
  • Can be configured (via proxy server, etc.) to avoid the cross domain scripting issue so that courses not residing in the LMS domain can be launched. This includes launching courses from content repositories in different locations within the corporate intranet, as well as on the Internet.
  • Costs less for the base application license compared to the cost of other similar systems with similar capabilities and feature sets. This includes all TCO (total cost of ownership) costs.
  • Has a licensing agreement that is flexible and easily scalable to reflect changing numbers of learners and administrators. This is especially important if you project substantial growth in your organization, or have “extended enterprise” users
  • Allows you to meter usage of the system by individual business units, so that you can spread the cost fairly
  • Costs less for recurring and ongoing support compared to the cost of other similar systems
  • Costs less for the database (if included separately) compared to the cost of other similar systems
  • Costs less for add-ons such as APIs to external applications compared to the cost of other similar systems
  • Offers hosted (also termed SaaS or cloud) and/or component-based architecture solutions to take advantage of these potentially cost-saving options
  • Provides an internal function to create and deliver a wide variety of assessment types
  • Can export assessments created within the LMS for use in other content or LMSs.
  • Allows grades to be assignable to a variety of objects, including discussion and blog postings
  • Provides the ability to export and import from Excel or CSV into the grade book
  • Tracks time it takes for learners to answer questions via time stamps, including time stamps on essay submission
  • Includes management features for group assignments, and apply assignments/topics to multiple groups at once
  • Allows embedding of videos, animations, audio, and graphics into assessments
  • Supports infrastructure for managing (assigned on an individual or class basis) of:
    – Mentoring
    – Coaching
    – Groups
    – Projects
    – OJT
    – Shadowing and apprenticing
    – Rotational assignments
    – Career programs
    – Conferences/forums/seminars/workshops
  • Allows learners to take notes as they interact with learning materials. These notes should be persistent between sessions and automatically associated with locations in the content (possibly with a “sticky note” look and feel). If the learner wishes, their notes can be posted, either internally in the LMS, or publicly outside of the LMS, through APIs to applications like Twitter and Facebook.
  • Includes collaboration functions to enable users to communicate with each other, instructors, course administrators, system administrators, etc
  • Provides the ability to send SMS text messages to learners (as either system alerts or actual learning content—note that a considerable amount of higher ed online learning in developing countries is done through SMS)
  • Provides specific functions that enable learners to provide feedback on the content
  • Incorporates a recommender system either based on:
    – User ratings
    – Predefined learning paths
    – System inferences based on info in user profiles, performance data, etc.
  • Supports competency management and Individual Development Plan (IDP) HR enterprise infrastructures
  • Automatically links training interventions and competency objects based on user approval
  • Allows learners to create and manage e-portfolios
  • Maps individuals/groups to a course/curriculum dynamically based on rule sets determined by enterprise requirements
  • Imports/exports competency-related data in common database formats such as XML or MS Access
  • Capable of tracking, reporting and storing a wide range of learner performance data by individual, by group and by cohort groups
  • Allows tracking via a flexible, interoperable standard such as the xAPI, and uses a published, industry-accepted controlled vocabulary for xAPI verbs.
  • Tracks learner data while learners are taking eLearning in a wide variety of file formats and authoring tools
  • For LMSs that incorporate social media and other features to support informal learning, tracking includes “social monitoring” tools
  • Includes the ability to add custom fields to track additional learner information, so that they can be included in analyses and reports
  • Provides the ability to print a variety of tracking-related items, including test scores
  • Allows a learner to view their own online course results on a lesson-by-lesson basis as well as:
    – Time spent
    – Date and time last accessed
    – Number of test tries
    – Course grade
  • Allows administrator design/upload and learner delivery of course completion certificates
  • Includes electronic signature capability on external form(s), for example, the government SF-182. Signature features for government installations should include SSL, PKI, and encryption for all authorizing levels.
  • Allows easy printing of certificates, surveys, and evaluations
  • Provides survey functions as follows:
    – Create and edit
    – Import
    – Copy
    – Define properties
    – Preview
    – Define survey link location (for embedding survey in eLearning, website, sending by email, etc.)
  • Has different options for design of surveys, such as multiple choice, Likert scale
  • Includes data migration tools for moving data permanently from a legacy system to the new one
  • Imports and exports to external systems in real-time and batch mode. This data typically includes not only learner demographics and identification but such things as competencies, certifications, and IDPs (individual development plans).
  • Enables add-ons and integration using an open architecture (see 7.5 Open architectures for more details)
  • Supports use of a third-party database (in compliance with Open Database Compliancy (ODBC) requirements
  • Import and export of learner and course tracking data using standardized data interchange formats (e.g., XML, JSON, CSV) without writing high-LOE integration applications
  • Supports the kind of metadata your organization needs or uses (LOM, Dublin Core, etc.).
  • Includes a convenient mechanism for adding metadata or descriptive labeling to not only courses, but also to other objects (SCOs, files, activities, etc.)
  • Uses metadata to search the course catalog(s)
  • Presents options for display of metadata to learners and administrators at relevant nodes in their workflow
  • Allows configuring how metadata tags will be input by content developers (checkbox, date, popdown, text box, etc.)
  • Has the ability to manage profiles for organizations, not just users
  • Has the ability to matrix learner characteristics demographically, organizationally, etc. (for example, assign learners to more than one job role, in more than one organization)
  • Can be searched on any field
  • Incorporates a recommendation engine that pushes recommendations for learning objects to learners based on attributes of learning the learner has consumed, according to their profile. This could function in much the same way as recommendations.
  • Includes features for:
    – Scheduling of operational and training assets
    – Weapons practice ranges (DoD)
    – Expendables
    – Computer learning facilities
    – Staff assignments
    – Planning training events
  • Displays filtered view to learners of relevant items above
  • Offers a wide variety and number of predefined reports
  • Offers permission levels with different kinds of access to reports.
  • Offers flexible, robust abilities to create custom reports, both internally and by using external tools (including those supplied by other vendors such as Crystal Reports)
  • Prints reports easily, with appropriate options
  • Provides direct access to tables used within the LMS for developing queries and reports. This should be documented in table and data structure specifications provided with the product. This is usually a requirement for government installations.
  • Provides reporting on certification status of groups and individuals, including upcoming renewals, missed renewal deadlines, etc.
  • Provides content-centric reports and/or visualizations
  • Provides user-centric reports and/or visualizations
  • Provides user interface customization , so that both learners and administrators can optimize for their particular needs
  • Can easily create and restore archives of system (e.g., transactions), user (e.g., profiles, enrollments), and content data (e.g., courses and curricula) in a proprietary (such as Blackboard Vista .bak) or open format
  • Is easy to install and reconfigure
  • Manages the administration process efficiently with built-in workflows (for approvals, for instance)
  • Administrative interfaces are clear, simple, and optimized for usability. Administrator interfaces are no less important than learner interfaces. Just because learner interfaces are well-designed does not mean the administrative interfaces will be also(!). This is particularly important where there is a need for non-technical staff to perform administrative functions
  • Includes options for remote administration from outside the enterprise intranet (through the Internet) and possibly via a handheld device
  • Provides features that allow administrators to view role structures in a graphical representation (diagrams, outlines, etc.)
  • Provides instructors with the ability to access grading tools from their mobile device
  • Provides clear, specific error messages that aid in troubleshooting
  • Has a feature to store favorite locations within the system
  • Allows saving of a workspace
  • Displays interfaces that are consistent and standardized throughout all screens
  • Provides a dashboard that indicates to the learner their progress towards meeting their learning goals, as well as common system functions.
  • Uses straightforward, simple, and intuitive paths for performing administrator and learner job task functions. You should test your most common and important use cases on the system to verify this. See 8 Process for choosing an LMS, step 12.
  • Has a feature to store favorite locations within the system
  • (for VLEs only) Allows users to use their telephone rather than VoIP (computer microphone and speakers) to make an audio connection to a session. The telephone numbers provided by the system should be either local or toll free.
  • (for VLEs only) Incorporates an option for the system (upon entering a session) to call the user back to initiate the audio connection, rather than requiring the user to call in from their phone and possibly incur long distance charges.
  • Allows learners and administrators to print transcripts, course completion certificates, and learner records with appropriate options
  • Allows a learner to be able to view a transcript of all training that has been recorded in the LMS for their account along with status and status date
  • Incorporates the ability to import and store information about learner badges and for instructors to create them, using a framework such as Mozilla Open Badges.
  • Has a scalable architecture that allows the system to expand as the number of users increases. The following factors should be taken into account in your planning:
    – Number of concurrent users (current and in the foreseeable future)
    – Database licensing (by seat or site)
    – Database volume restrictions
  • Has a scalable architecture, enabling evolution of the client installation without forcing them to go through frequent major version upgrades
  • Allows configuration of a data distribution network (interconnect distribution peers through a common distribution server)
  • Has a good reputation among acquisition and system owner communities. Ask the vendor who their other clients are, what they use the system for, and see if you can talk to these clients about their experience using the system. Look for negative comments posted on the Internet by members of these communities.
  • Has a clear technology roadmap with a reasonable time frame for new versions and additions of new features
  • Has consulting experience and arrangements, especially with complex issues such as data migration
  • Listens to your concerns during interactions with them, especially during demo sessions of their product. How they are in these situations probably reflects how responsive and attentive they will be to your concerns as a customer.
  • Is of a stable size, as measured by number of employees, annual revenue, capitalization, etc.
  • Has a robust ongoing budget for R&D
  • Has a large number of successful clients. Who the clients are and their industry stature can be important, especially in terms of their similarity to your mission or infrastructure. If you can, find out the number of total users served by the LMS product within this client base.
  • Has worked with many content developers using a variety of different kinds of content. Ask for references at organizations that have deployed content similar to yours.
  • Is familiar with your business model, market, and content typestem]
  • Has robust support for training of all categories of users: learners, instructors, system administrators, content managers, etc.
  • Has robust support documentation in a wide variety of forms including tutorials, help, examples, references, and user manuals
  • Has a variety of Help Desk support options for administrators and learners (telephone, chat, email, etc.). These need to be in synch with the way your organization normally requests help.
  • Is available as close to 24/7 and world-wide as possible
  • Offers extensive training options: eLearning, video tutorials, ILT sessions, webinars, etc.
  • Includes an orientation tutorial for new users
  • Has a feedback function for suggestions on improving the LMS
  • Provides technical consulting services options for customizations, implementation, configuration, architecture design, needs analysis, change management services, etc
  • Provides support for industry-standard streaming protocols for audio and video
  • Minimizes latency in delivering high bandwidth media through robust network throughput and streaming server capacity
  • Provides a library function for upload and tracking of user-generated, internal media (especially videos), or provides direct access to web-based media (such as videos on YouTube)
  • Provides an enterprise Glossary function
  • Provides the ability to push screen shots of the facilitator’s screen to participants
  • Supports interactive video formats
  • Supports a wide variety of media (see below) and media file formats.
  • Has robust support for mobile devices

Have questions?

Feel free to contact us with your feedback & questions.