​

NEW KNOWLEDGE AND SKILLS have little business value unless they translate into improvements in on the-job behavior. Evaluations increase the likelihood that this will occur.

For example, attempts by terrorists to blow up U.S.-bound airplanes with mixtures of liquids led the Transportation Security Administration (TSA) to implement procedures for checking carry-on baggage to detect liquids and gels of more than three ounces. Training was administered to TSA employees after the new regulations were issued. Shortly after this screening process went into effect, one of the authors was in the airport and his carry-on bag was checked.

A TSA worker searched the bag and found nothing amiss. The supervisor had been watching. She reopened the bag, pulled out a can of shaving cream, and said, “I am sorry. This is too big, and you will have to either check it or leave it with us.” She then explained to the first TSA worker why she had flagged it.

In this case, on-the-job observation of the frontline worker by a supervisor served as the evaluation mechanism to ensure that the worker’s knowledge and skills were adequate to ensure proper implementation of the new regulations. The supervisor was also helping to improve and reinforce the screener’s knowledge.

This is a common business situation where immediate evaluation can act not only as a check but also as a reinforcer of new behaviors that might otherwise not be adopted. This notion is furthered by the common suggestion after training to make sure that employees apply what they have learned as soon as they get the opportunity.

Several issues are critical to proper evaluation. Managers must allow time for new skills to be performed, consider a variety of methods, repeat the evaluation, and consider cost versus benefits.

Allow Time

No evaluation should be attempted until trainees have had an opportunity to use new skills on the job. Sometimes there is an immediate opportunity, as in the case of the new TSA rules. However, if the purpose of the training is to teach a foreman how to handle a grievance from an upset employee, no change in behavior is possible until a grievance has been filed. Assuming that there will have been time for the new skills to be applied, evaluation should begin two to six months after the training session.

Survey Selectively

Anyone who personally observes the behavior or the work that is performed by the trainees may have insight that could help management evaluate the results. To get a variety of data, managers should not only observe trainees themselves, they should also survey others who observe the trainees in the regular course of business. One such source is the immediate supervisor. However, this person may not see the trainees often and may have biases regarding certain workers. Therefore, management should also solicit input from elsewhere, including the worker’s direct reports, peers, and customers.

Sometimes it is difficult to find traditional sources of information due to the nature of the training. For example, a successful airline wanted to evaluate the training of flight attendants and ticketing agents. The managers went down the list of possibilities. Other flight attendants weren’t an option because of union rules; pilots were busy flying the plane; supervisors were ruled out, as they could not spend enough time in the air to supply good data; and Federal Aviation Administration (FAA) personnel, who do conduct their own audits, have different evaluation criteria.

The authors noted that the airline had a large population that was available and could do the job—the passengers. The airline developed a questionnaire to send to customers who could then supply useful information.

Repeat the Evaluation

Some trainees change their behavior as soon as they return to their jobs. This usually occurs because of their strong internal motivation and immediate coaching or evaluation. Others take longer to adapt, while some never adjust to the new procedures. In addition, some of the workers who change immediately may revert to the old behavior after a period of time. Others will simply stop following the new rules if they think no one is watching.

Given that these changes occur along a continuum, it is important to repeat the evaluation at appropriate intervals. Regular evaluation not only serves to provide data on an ongoing basis, but it also acts as continual reinforcement.

There is no set rule for how often to repeat an evaluation. Each organization has to make the decision on its own, taking into account the importance of the behavior, the job climate, and other significant factors unique to the situation. For behaviors that are not critical to the company’s mission, the evaluation could be repeated six months after the first assessment. Then, depending on circumstances and the time available, a third evaluation could be made three to six months later.

Consider Costs

Just as with other investments, managers should compare the cost of evaluating changes in behavior with the benefits that could result from the evaluation. In many organizations, much of the cost of evaluation is in the staff time that it takes to do it. Other costs of evaluation can include hiring an outside expert to guide or even conduct the evaluation. Additional factors to consider include the benefits that can be derived from evaluation, such as changes in behavior and final results. The greater the potential benefits, the more time and money can be spent on the evaluation.

Specific factors that require robust evaluation include new behaviors that are critical to the execution of organizational strategy, pilot programs, highly visible or controversial programs, new skills that are drastically different from traditional methods, and any type of leadership or coaching behaviors that will affect the success of all other training.

It is important to separate the evaluation of a training program and the expected results of a program. Even if a change in behavior does occur, positive results may not be achieved.

For example, one supervisor at a large company was trained to implement the principle of managing by walking around. This program encourages managers at all levels to walk among the lowest-level employees to show that they are concerned and to hear directly from front-line employees. One manager, who we’ll call Charlie, had never shown any personal interest in his people before he began to implement the program. So when Charlie started asking employees about their children or vacations and generally tried to engage them in casual conversation, the staff viewed the new behavior with suspicion. They wondered what the boss was up to, and morale plummeted. In this case, the manager’s change in behavior had negative results.

As that case shows, new corporate policies have to be evaluated to see not only whether they are being implemented but also whether they are achieving the desired results. The first step toward that end is to ensure that evaluations are conducted so that management can assess whether skills and knowledge imparted to employees are being adopted in the workplace as intended.

Donald L. Kirkpatrick is professor emeritus at the University of Wisconsin—Madison.

James D. Kirkpatrick is vice president of global training and consulting for SMRUSA in Rolling Meadows, Illinois.

This article is excerpted from their book Implementing the Four Levels, published by Berrett-Kohler Publishers of San Francisco. To purchase the book call 800/929-2929 or visitwww.bkconnection.com.