Strategy for Developing a Taxonomy

By David Shaw, pmp, pcip

Tutorials available on Youtube

The strategy for developing a taxonomy should start with a statement of goals and purpose. Pick the objectives the taxonomy will support and limit the scope:

• Limit domains, users, strategies
• Identify target audiences, contributors, stake-holders, content sources, volume, overall objectives and related strategies
• Do a “needs analysis”

The target audiences, contributors, stake-holders, content sources, and volumes should be identified in an environmental scan and inventory.

A needs analysis focuses on the requirements related to the goals, aspirations and needs of the users and/or the user community and feeds them into the taxonomy analysis process. The main purpose of needs analysis is the user's satisfaction. The output of this should be a statement of scope and objectives to establish a framework for the development of the taxonomy.

Once the business need is understood, a ‘Build’ or ‘Buy’ decision can be made based on criteria such as these:

• Does a standard already exist
• Can we use it as-is
• Can we adapt it
• Should we develop our own

Every case is different but in general a standard should be used as-is. Some users are probably already familiar with the standard and changing it in any way will sow confusion. Maintenance of an adapted version may be difficult if the standard changes. If the adaptation is extensive, the effort involved may be as much as developing a purpose-built (bespoke) taxonomy.

A good taxonomy should have these characteristics with respect to the established goals and purposes:

Usability is a quality attribute assessing the ease of use and learnability of a human-made object. Key quality components in usability are learnability, efficiency, memorability, error propensity and feelings of satisfaction.

Manageability refers to objects that can be managed and are governable; easily handled, worked and shaped; and can be contrived readily to meet needs.

Flexibility is the capability to meet diverse needs, both now and in the future.

Comprehensiveness is the scope of coverage, the degree to which it meets the needs. This may include the expansion of terms using a thesaurus or other mechanisms to address differences in the usage of terminology.

Effectiveness is the capability of producing a desired result. When something is deemed effective, it means it has an intended or expected outcome. 

Efficiency is the extent to which time, effort or cost is well used for the intended task or purpose. The web three-click rule is an example of an objective in efficiency.

Best practices for developing taxonomy are:

• Develop hierarchy
• Normalize
• Review
• Test
• Iterate

 
The process of developing a hierarchy is:

• Identify and involve stakeholders, subject matter experts (SME) and end-users/customers.
• Develop a consensus through iteration, and develop an evaluation plan and criteria.
• Develop a governance framework and decide how to manage the life-cycle, change requests and history.
• Conduct an audit to determine what taxonomies, tags, keywords, and controlled vocabularies are already in use internally; how content is generated, where it is located and used; how is the life-cycle of the content managed; and what are the business processes and workflow. Include any known functional constraints in any planned application for managing content.
• Draft a high-level architecture using knowledge gleaned in the audit and from research in external resources. Develop a broad, shallow taxonomy with no more than three levels and organized around major domains. Establish whether terms are meaningful and reconcile language issues and terminology. Balance the taxonomy and metadata – this is where art comes into play. In the absence of consensus, create a thesaurus – but note that planned applications might not support a thesaurus.
• Normalize the tree by aggregating likes. Remove duplicates and merge terms. Standardise terms and flatten the tree. Ideally it should be no more than 10-12 siblings wide and 3-4 levels deep; although scientific and engineering taxonomies are likely to be much larger. What they gain in coverage they sacrifice in effectiveness and efficiency. (Large models are less effective for many end-users because they must have a deeper knowledge of the terminology in the domain.)
• Review the resulting model with Subject Matter Experts, peer review and voice of the customer reviews. Test the taxonomy by prototyping applications in desktop exercises, and Wash Rinse and Repeat as many times as necessary to get it right. At least four iterations are usually required.

Guidelines for Knowledge Management

By David Shaw

I read a lot about information and knowledge management. Often it seems airy-fairy. By this, I mean, how do you implement it in the real world? Many people think you just go out and buy a knowledge-management system (KMS).

There! Another check box ticked. And another enterprise system destined for an expensive failure.

So this month I have a few slides for you taken from my workshop on knowledge management. Hope they give you some concrete ideas.

eLearning Return on Investment

By David Shaw

Classroom-based training is still effective for many types of skills that do not transfer well to computer-mediated learning. For other types classroom training has a high cost for instructors, facilities, travel and lodging, and lost time on the job.

Because of the high cost, training is often not done. This has a hidden counter cost for the organization in lost gains in productivity from a lack of skills training, and often an increase in risk and liability when the training is required by legislation or regulation.

As an example, one client was given 60 days by a regulatory authority to deliver mandatory training on violence in the workplace to 2903 employees. To take people off the job and cycle them through classrooms had an estimated cost of $379,100 ($131/employee) including an impact on service delivery. The eLearning cost using an approved off-the-shelf course was $151,200 ($52/employee). One savings was in not having to pay 900 people overtime to backstop those away in class while still maintaining service delivery.

Online learning has come a long way from the early days of computer-based training. Today other benefits of eLearning include peer-to-peer support and collaboration, 24/7 asynchronous availability, onlne resources and web links not feasible in the classroom.

Some of the possible benefits include:

• Improved productivity by increasing skills
• Improved productivity from less time away from the job for training
• Increased efficiency through the  convenience of availability 24-hours a day, 7 days a week
• Increased effectiveness through just-in-time training opportunities
• Centralized knowledge management.
• Built-in trainee enrollment and course management.
• Management of compliance and recertification.
• Increased effectiveness through consistent content, context and interpretation.
• Reduced cost though condensed delivery (e.g., eLearning can take ~50% less time).
• Elimination of travel and living expenses.
• Measurement of employee knowledge retention with pre- and post-assessments.
• Courses can be retaken any time to refresh knowledge.
• Reduced liability.

If you do any significant level of training, eLearning should show a good return on investment (ROI). There are numerous examples of ROI calculators on the web. (Actually most of them are break-even calculators. True ROI requires an NPV or IRR calculation.)

Some of the data you will need is given in this table:

Item	Classroom	eLearning
Number of training cycles per year
Number of trainees per cycle
Trainee Average Hourly Wage
Number of days per training cycle
Classroom cost per day		0
Classroom equipment cost per day		0
Travel and living cost per trainee		0
Material cost per trainee		0
Instructors’ annual salaries		0
Instructors’ travel and living cost per training cycle		0
Cost of back-stopping employees
Number of unique courses per year
Average cost of developing each course
Annual LMS cost	0
Other

Learning Management Systems are a Cloudy Commodity

By David Shaw

Industry statistics show that overall IT departments spend 80% of budget on maintaining core data-centre technology and only 20% on the applications (e.g., an LMS) that deliver the real value for an organization. IT is a cost centre, not a strategic asset.

It’s worse with eLearning. Our experience is that successfully implementing and managing an LMS/LCMS is beyond the capability of most IT organizations. Yes, they can install it and get it running but beyond that they simply don’t have the domain experience. Their experience is in structured data (databases) and they regard courseware as unstructured data. Worse, they believe eLearning is essentially PowerPoints running on a web server; they don’t know how to stream video on the intranet; they won’t support half the file formats common in eLearning courseware; and they’re too concerned about Bring Your Own Device to even think about mobile courses.

Cloud Computing can change this.

Like mobile it seemed that Cloud Computing would always be out there somewhere, always several years in the future. But in the same way consumers pushed mobile computing into the enterprise without IT, they are consuming and promoting cloud computing in many guises and bringing it into rogue business units. Here are just a few examples of Cloud Services that you may have used or heard about:

• Automatic upgrades of software on your computer
• Amazon, FaceBook, Google, Yahoo, YouTube and others reside on vast globally distributed Cloud Computing platform
• Stores like Apple iTunes, Google Play, Windows Store
•  File hosting services like Box, Dropbox, Flickr, Google Drive, iCloud, Instagram, Microsoft (Live) SkyDrive, UbuntuOn
• Free eMail services like Hotmail (Live Mail), Gmail, Zoho Mail, AIM Mail, iCloud Mail, Outlook, Yahoo!, and a dozen others
• Online backup services like IDrive, KinectD, MozyHome, Nero, Norton
• Google Docs, Microsoft Office 365, Zoho office tools
• Google Chromebook cloud computers
• Windows 8 pushing users onto the MS Cloud as the default
• Apple pushing users onto iCloud
• Microsoft’s Lift London studio developing new games exclusively for the cloud
• Wix is a really cool cloud service for developing web sites

What these have in common is that they are a form of Cloud Computing called Software as a Service (SaaS). Other cloud types not discussed here (and potentially of greater interest to the IT department) are Platform as a Service (PaaS) and Infrastructure as a Service (IaaS).

What does SaaS mean for eLearning? There are four basic ways for an organization to operate an LMS or LCMS:

•  Internal Hosted
• Hosted Service
• Shared Service
• Software as a Service

Internal Hosted is a non-cloud single-tenant model.  This is the traditional scenario where an LMS/LCMS is implemented and hosted by an IT department or business unit on local servers. The main advantage is direct control over governance and security. The disadvantages are lengthy and difficult procurement processes, and the need for capital investment that may be hard to get. Capacity is always a step function, so you capitalise servers over the forecast demand or experience loads that cause reduced service from too little capacity.

\

Hosted Service is also a non-cloud single-tenant model. This is the same as Internal Hosted except that it has been outsourced to a service provider who supplies the servers and manages the LMS/LCMS software application. This is sometimes called Managed Service. The servers can be located on premises or externally, depending on the arrangement. The advantage is that you don’t need IT approval for a capital expense because this is usually a leased service. The disadvantage is that you do need IT approval. This model also suffers from over- or under-capacity issues.

Shared Service is a non-cloud several-tenant model. This is an Internal Hosted model in which a few separate business units or organizations share a software application run by a centralised service provider to reduce costs. For example, enterprise web applications like Drupal, Joomla, SharePoint, WebSphere, WordPress and many others can be used to set up more than one web-site domain.

Typically partitioning of user data and content is very weak. Typically the oversight model is weak and bureaucratic. Often the users do not have end-to-end ownership of process. Security and governance, including content approval, are often subject to the business processes and delays of the chief business owner. New functionality, performance levels, service-level agreements and changes to the service are difficult to negotiate. This model also suffers from over- or under-capacity issues. It’s a popular government mantra but only a few practitioners have shown significant cost savings.

Software as a Service is a multi-tenant cloud model. This is an external software application from a service provider that runs on top of a stack of PaaS and IaaS cloud services. SaaS uses hypervisor virtualization to separate tenants. The software and database are architected for multiple tenants with strong partitioning of user and content data.

Security is managed by the SaaS provider. For protection, usually data is fragmented across several geophysical locations and often encrypted. The service-level agreement may be standard and difficult to customize. There is no capital cost – a major benefit. This model matches capacity to demand dynamically, and charges only for usage.

However, under the Electronic Communications Privacy Act of 1986, the Patriot Act of 2001 and the Foreign Intelligence Surveillance Act of 1978/2012 the US government can still access any data without a warrant if it is stored in the USA or if the service provider falls under US jurisdiction. Access does not have to be for reasons of national security – the legislation allows warrantless access for political and economic reasons, too.

Also, as an aside, any email stored in the USA over 180 days is not private or privileged.

In 2013 the FBI’s priority was to get new powers under the 1994 surveillance law called the Communications Assistance for Law Enforcement Act. The FBI wants the power to mandate real-time surveillance of email, cloud services, or online chat providers like Skype. This is everything from Dropbox and online games to Live Mail, Outlook Mail, Gmail and Google Voice.

As a simple matter of due diligence and security from economic espionage data should not be stored with US service providers unless extremely strong encryption is used on the client-side (keys should not be stored in the cloud). This is a thorny issue for service-level agreements.

Some of the benefits of SaaS are:

• Lowering the opportunity cost of running technology. Opportunity cost is the cost of not doing something, thus foregoing future benefits.
• Allowing for a shift of costs from capital expenditure to operating expenditure, which makes it much easier to acquire technology.
• Lowering the total cost of ownership (TCO) of technology (resources are pooled).
• Giving organizations the ability to add business value by renewed focus on core activities by spending 20% on core technology and 80% on better business applications.
•  Enabling IT to focus on the strategic aspects of its role.
• Empowering small- and medium-sized organizations through access to global-scale technology at utility prices.

The National Institute of Standards and Technology (NIST) has defined several essential characteristics for a cloud service:

• On-demand self-service so an end user can sign up and receive services without the long delays that have characterized traditional IT.
• Broad network access via standard platforms (desktop, laptop, mobile, etc.).
• ·Resource-pooling across multiple customers who share bandwidth, load balancing, servers and databases.
• ·Rapid elasticity so the service can scale to cope with demand peaks. This means the service level is always matched to the demand so you don’t pay for over-capacity or lose users from under-capacity.
• Measured service with billing metered and delivered as a utility service the same way we consume electricity and water.

Some so-called cloud services charge by the number of users but we would argue that this is just a form of hosted self-service. A true utility service, e.g., electricity, doesn’t care how many people are in the house.

An LMS is a good candidate for SaaS. At this time there are several hosted self-service LMS providers: Articulate Online, BizLibrary CompanyCollege LMS, Docebo Cloud, ePath Learning ASAP, Epignosis LTD, eFront Enterprise LMS, Zenler Online and probably more.

An LMS meets some of the criteria for a cloud service, such as:

•  It is not a differentiating or core business application
•  It is usually web-facing
• Users need access from desktop, laptop, mobile, etc.
• Demand can be spiky when training campaigns are underway
• Global delivery through content-distribution networks
• It can be managed by business departments

An LCMS is not a good candidate for SaaS. It meets some criteria for not using a cloud service, such as:

•  It is not web-facing (it’s a back-end system to a web-facing LMS)
•  It does not have to scale rapidly to meet demand
• It is a value-added service that differentiates the organization (especially with content re-use)
•  It involves uploading many and often large files that consume bandwidth

Start thinking LMS cloud in long-term strategies (with appropriate security). Think about bringing an LCMS in-house.