​

JUST A FEW YEARS AGO, IT funding was based largely on fear and doubt at the executive level. But as information security has integrated itself into overall corporate governance, CISOs must justify new expenditures and show how they align IT with organizational goals. To that end, they are increasingly measuring IT performance to demonstrate the results in concrete terms in accordance with other business metrics. While formal programs are still relatively rare, many managers are finding that metrics are expected in at least some situations.

Organizations such as the University of California, Irvine, have been using applications, such as FoundScan, by McAfee’s FoundStone division, to develop metrics. The software scans computers for vulnerabilities and provides results, complete with graphs and a score ranging from 0 to 100. In a number of instances, IT staff have used the scores to show administrators how effective they are, according to Garrett Hildebrand, network planning and security manager. This can help when it comes to requesting more funding, he says.

Hildebrand, who works in the university’s central computing office, cites one example at the School of Biological Sciences. A few years ago the school’s IT staff was receiving a large number of complaints about infected and compromised computers. The staff said that in return for additional help, it would produce measurable results.

“That’s exactly what they did, showing charts and graphs to the administration,” says Hildebrand. The department was able to keep the additional staff permanently, he says, and is now known for its security.

Measuring overall business results means looking at technology along with an organization’s people and processes, says Khalid Kark, a principal analyst at Forrester Research. In designing a program, a central challenge is figuring out what to measure and how to measure it, he says. “There’s no magic formula, you have to be able to figure out what’s important to your organization.”

But defining metrics and thresholds can take years of trial and error, says Kark. The metrics themselves can lead to unintended behavior. Kark cites an example of one organization that tried to measure the time it took to resolve help desk calls. Desk staff started to focus on keeping calls short, rather than resolving problems, frustrating some callers.

Metrics should be consistent and transparent. CISOs should identify and document information sources, assumptions, and calculations. Kark also recommends testing metrics on a subset of users before fully implementing them.

Another major challenge is making metrics meaningful to executives. Kark recommends working with managers to identify metrics that relate to business threats or corporate objectives. That will yield data relevant to strategic business decisions.

It’s also important to translate data related to security metrics into terms that make the business impact clear, which may mean assigning dollar values. These values will have to be based on certain assumptions, which CISOs should document and share with executives.

CISOs should keep their metric programs small and focused. This is a case where less is more. Executives need only a handful of key metrics, notes Kark in a paper on the issue. Too many will dilute their impact.

Aside from risk management, the use of metrics is also driven by the demands of regulatory compliance, says John Pironti, chief information risk strategist at Getronics, a Waltham, Massachusetts-based IT services firm. They can be particularly helpful in cases where self-assessments are acceptable.

“Most regulations just want you to show you have control of an environment and that you’re getting better. If I can prove some of that, I may not have to answer 500 questions,” says Pironti.

One factor hindering metrics’ development has been fear of exposing business weaknesses, notes Pironti. Some managers worry that documentation of vulnerabilities, for example, in an effort to establish a benchmark for security performance, could be discoverable during litigation. “It can make people nervous. As soon as you put something out there, it becomes a liability you need to fix,” he says.

That’s where another of Kark’s recommendations comes in: Organizations shouldn’t set unrealistic goals. But metrics should still be aggressive enough to demonstrate visible improvement. They should be designed to “easily show progress over time.”