“Time to Learn”: A new approach to measuring​ the value of corporate learning | blog post

Introducing a new perspective

Countless studies have shown that e-learning/blended learning is almost in every case better than the traditional classroom training in the corporate landscape – from the three points of view of distribution, retention and cost-effectiveness. But how do we differentiate between a good and a not-so-good course and, more importantly, how do we make sure that the courses provide the expected value from the point of view of corporate management?

This article focuses on how you as a senior manager, training professional or person responsible for HR can assess whether the individual courses provided in your course catalogue are actually generating the expected learning value from your corporate perspective.

When we discuss the impact of corporate training it is important to differentiate between the value to the individual and the value to the organization. While a certain course might be of high value to the individuals in attendance, the same course may not be very valuable from a corporate or organizational perspective.

Consider this as an example: Training in a new IT system might be done via a brilliant external course, highly valued by those who attend. However, the course is expensive so only a few people in the organization can be allowed to attend (train the trainer approach). While each of the attendees score the course value as high, the real organizational value might be low because of the poor reach or distribution. This can certainly be partly mitigated afterwards by additional activities done by those who actually did attend the course; however, this adds to the initial high cost.

Since most corporate training is (or should be) focused on the value to the organization rather than the value to the individual, it follows that we need ways to assess the corporate value of a given course separately from the ratings given by individuals in attendance.

In this article we will introduce a formal way of assessing this organizational value as well as suggest a new variable to that which is commonly seen.

Organizational value of training

At an organizational level, consideration is commonly given to rather simple statistics such as number of courses offered/taken/passed, time spent on each course, and individual-based ratings such as pass rates, evaluation scores and the like. In a few rare cases these individual statistics might even be spiced up with something about skills gap to show how well the training is suited to the "theoretically" defined needs of the organization.

We therefore need to look at new ways of measuring the impact or organizational value of learning. Cost is always a factor and a very simple solution might be to simply divide the training cost of a course by the number of participants. This demonstrates one of the most widely used KPI’s. However, this can only give us an idea of the cost-effectiveness and does not in any way provide measurement of the learning value; therefore, it says nothing about whether the course accomplishes organizational goals.

Learning components consists of only two overall generic elements, namely:

  • Quality of the content (QoC), measured by individual evaluation scores, pass rates, measurable effect on skills gap, etc., and
  • Effectiveness of the distribution (EoD), measured by the number of students per course, total number of courses delivered, etc.

Therefore, analysis of the two basic components above allows us to stipulate that the theoretical organizational value (OV) of any given training can be described as OV = QoC * EoD.

Further, from this we may make a more accurate assessment of the training value to the organization by defining a more sophisticated KPI as Cost/Organizational Value, or simply KPI= Cost/OV. This gives us a statistical analysis which is much more relevant when compared to that of the often and erroneously used KPI consisting solely of cost-effectiveness.

Introducing the factor of timing

While this certainly sheds light on the reach and quality of the training, it quickly becomes obvious that we have merely come to a generic theoretical value for the organization that doesn't take into account timing as a crucial factor in corporate or organizational training, due to compliance issues, new IT systems and the like.

This leads us therefore to introduce a third variable: course timing – or Time to Learn(TTL).

Over the years many studies have repeatedly demonstrated a few basic and essential insights into how people learn, such as:

  • People forget– the retained knowledge from training deteriorates over time and as a consequence training must be provided as close as possible to the point in time where it is to be used.
  • Practice is essential- People will improve their skills and therefore add value to the organization only if they continuously practice in order to reinforce what they've been taught.
  • Training before Practiceis better than Practice before Training. This means that people with formal training and good practice skills tend to also correctlypractice doing the right things. Practice before training inevitably results in the formation of bad habits that become very difficult to unlearn.

From the above insights we can deduct that the timing and order of the training/practice cycle is key to the organizational value of the training. It follows that timing is certainly a crucial factorwhen assessing the organizational value of training.

In conclusion, therefore, we can state that in order to improve the organizational value of the training, we need to ensure that it is provided as close to the point in time where the corporation "goes live” with whatever changes it is making, be they new systems, procedures or other adjustments. The parameter Time To Learnfocuses on what may be referred to as "neither too soon nor to late....rather, just right".

How to measure “Time To Learn”?

So how can we measure TTL? – well, it turns out that this is relatively easy to do.

From studies we know the approximate rate of “knowledge deterioration” over time, and while this rate is highly correlated to each individual, most studies seem to agree that even after a few days deterioration becomes significant.

However, we also know from our everyday work life that if we are provided with a new system or set of procedures, we do our very best to get things done, even though we are not sure exactly how. This will begin as a waste of time and will eventually lead to bad habits that are not fully aligned with the original intention, potentially resulting in incorrect usage of systems and processes leading to non-compliant behavior, or, even worse, inducement of financial losses.

This leads us to a simple conclusion: It must be better to have the training close to the point in time where its needed. The closer the better and the further away the worse.

As we already discussed, TLL essentially tells us that the value of any given training is dependent on the right timing. Theoretically the ideal timing would be in the immediately prior to the time when it is needed. From this it follows that the full value potential would be close to 100% if the training is timed ideally, giving us a score of 100% if the training is +1/-1 day away from the time it is needed.

Since most studies indicates a rapid deterioration in the knowledge retained over time, so should the TTL factor show a similar rapid decline. Furthermore, this deterioration is not linear and the same can be said about acquiring new bad habits, so the TTL should be based on a similar geometric scale like the one shown in the figure below.

Training done days before/after “go live”

% of theoretical value

We can now add a TTL variable to the equation. In doing so we find that a more accurate assessment of the Organizational Value is be formulated as: Quality of Content * Effectiveness of Distribution * Time to Learn, or simply OV = QoC * EoD * TTL

The KPI used to measure the true Cost-benefit of a course at an Organizational level is still Cost/OV; however, now OV is defined in depth with a combined value measurement of QualityDistributionand Timing.

So where does all this take us?

By expanding definition of Organizational Value of training, a number of interesting insights emerge.

It is no surprise that with better quality of training, a better overall score will be achieved. However the inverse is even more insightful, that is: Poor quality of content can be partially compensated for by effective distribution and good timing.

A common scenario in corporate training is that many people need to be trained in a short period of time. Since e-learning is in all cases easier to distribute andcorrectly time, it can be argued that evenbelow average quality e-learning will have a higher Organizational Value than high-quality classroom training.

As mentioned earlier most studies show that e-learning is in most cases better than classroom training. However, the same studies also suggest that bringing a class-room component into the e-learning based training (blended, flipped or similar) will generally improve the “Quality of Content (QoC)”. However, if this improvement in quality is done at the expense of timing, the organization ends up with a highly-valued course for the individuals, but with a low Organizational Value.

The common error is that the training will enjoy the reputation of being brilliant due to the individuals giving high evaluation marks while in fact contributing little or nothing to the overall value of the organization and it's workforce members.  Many companies waste large sums of money, sometimes even their entire annual training budgets, on ineffective programs due to poorly-designed measurement variables that ultimately have little or no value to the organization.

To summarize, this article intends to provide insight on how to efficiently and effectively measure the value of corporate training programs, both on the individual and organizational level. It is vital to work with professionals deeply acquainted with course design and statistical analysis of training outcomes. These professionals will successfully guide the business to establish and measure training for the two pillars of corporate learning...... cost-effectiveness and organizational learning value.