ILTA’s Peer to Peer
May 2007

Measuring the Success of Your Training Programs (by Roberta Gelb)

Thousands of dollars are spent on software training at law firms, sometimes even hundreds of thousands or millions each year for larger firms. Firms know they need to train the users to get the most out of their considerable technology investments in legal and business software. However, IT staff sometimes skip over the critical step of measuring the return on investment (ROI) resulting from training, an important process which requires up-front planning as well as after-action reporting.

It’s All in the Numbers

At a law firm, costs for software training are literally coming out of the partners’ pockets. Therefore, when it comes to needs assessment, training course outlines and training delivery, one factor must be considered above all others: how to measure the results of the training and prove that it was worth the expense incurred.

When starting a training program, you need to focus on how to measure and quantify results. Accountability to the firm’s management is of key importance. Through proper planning and collaboration with your training staff and partners, you have the opportunity to look very good — or very bad — to your management. By developing your training project requirements based on the firm’s strategic goals and developing measurable objectives based on those goals, you can quantify return on investment at your firm.

Bridging Goals and Training

Often the training department or consultant (and sometimes the entire IT department) is excluded from some important strategic discussions at the law firm. To provide true ROI for technological projects, it is critical for training to be an integral part of the firm’s strategic plan. By being aware of management’s intentions for new software purchases, you can work with them to achieve the firm’s goals.

For example, we were hired by a firm to design, implement and train staff to use a new document-assembly application. The firm’s management and IT department worked together to create clear requirements for both the document production system and the associated training for the new software. They conveyed to us their strategy to equip their staff with productivity-enhancing technology that would help staff work more efficiently and complete more work in fewer hours.

We were instructed that the new software and corresponding training was to automate document production to effect the following changes:

• • Enable secretaries to produce more documents in the same amount of time
  • Enable the attorneys to create properly formatted documents
  • Shorten the time required for new hires to be able to produce critical documents

With this specific direction, we knew exactly what our objectives were for the project. This focused our training course development and led to outstanding cost-saving results. Prior to system implementation and related training, the secretary-to-attorney ratio at the firm had been 1-to-2. Once the document-assembly software was implemented and the users were trained, the ratio changed to 1 secretary for every 3 attorneys, a 50 percent increase in productivity. The firm functioned more efficiently and profitably because the secretaries were able to support more lawyers. This was a direct result of the software implementation and training.

Ask the Right Questions To Get the Right Answers

The main question any training department needs to ask is: What do the people at our firm need to be able to do with technology to do their jobs effectively? Then, the training department needs to ask itself: How do we ensure our training meets those requirements?

It is critical that the answer to the first question be translated into measurable training objectives. For example, a typical answer might be: Our attorneys need to be able to use our document management system.

The measurable training objective then becomes: Following the training on document management, the participants will be able to demonstrate ability to perform the following four functions:

• • Profile any document within two minutes
  • Find any document within three minutes
  • Add matters to My Matters folders
  • Add documents, searches and matters to the Favorites folder

These objectives are clear, realistic, useful and quantifiable. Through an evaluation form provided after the training session, we were able to assess that many attorneys reported that the most important skill they gained in training was the ability to quickly find documents, and that they had achieved the other goals as well.

Translating the initial requirement into a measurable training objective is not always this straightforward. For example, for a firm migrating from Microsoft Word 2003 to Word 2007, the objective would not be simply to do “Word training.” There are many variables and assumptions that can affect this kind of training. First, if we are to assume that a properly formatted document, using styles and automatic numbering, takes less time to edit than a poorly formatted document, then we can assume that a measurable training goal would be to train users to create documents using styles and automatic numbering. With that plan, both pre- and post-training evaluations should be done to see if the training had any effect on the users’ competency. Another follow-up evaluation could be to measure the number of corrupt documents created both pre- and post-training, since corruption is often the result of bloat from direct formatting. Analysis of these numbers after training would speak to the success of the education program and provide proof that training, for both the users and the firm, was worthwhile.

Analysis — Before, During and After — Proves Success

Training programs can also be designed based upon careful analysis of work volume and content. For example, at one large organization, the helpdesk calls were increasing so dramatically that the department requested head count for an additional person. Before hiring that person, management asked the training department to work with the helpdesk personnel to see if training could alleviate some of their burden and enable them to shoulder the workload without adding additional IT staff. As their consultants, we helped them analyze the helpdesk calls and then develop a training program based on that analysis. In line with our plan, we trained the users on various applications to reduce their need to call the helpdesk for lower-level technical help. For months after the training, the IT department reviewed the helpdesk call logs to measure the impact of the training in reducing helpdesk calls. Following the training, the helpdesk calls decreased significantly, and the need to increase the size of the helpdesk staff disappeared.

To be successful, training should be designed and conducted with a focus on measurable results. Only after determining the standards by which results are evaluated can the real effectiveness of the training be judged.

---

Roberta Gelb is President and founder of Chelsea Office Systems. Under her management the firm provides needs analysis, programming, macro/template development and training for law firms. She writes for Law Technology News and is a member of the Law Technology News Editorial Advisory Board. She can be reached at gelb@chelseaofficesystems.com.

This article was first published in ILTA’s May, 2007 issue of Peer to Peer and is reprinted here with permission. For more information about ILTA, visit their website at www.iltanet.org.