Breaking Down the Hard Business of Measuring Security Culture
“Measuring culture is difficult.”
That’s how Rachel Briggs, OBE, led off the section on security culture metrics in her new ASIS Foundation research report, Security Culture: A Strategic Capability That Builds Resilience in a Volatile World (sponsored by Amazon WWOS, Axis Communications, and Thermal Radar). Many security professionals agree with that assessment.
Quoting from a Harvard Business Review article, Briggs noted that “Culture is easy to sense, but hard to measure.”
Some variation of that sentiment came across loud and clear from the security professionals interviewed for this article.
“I think culture cannot be measured directly. It has to be inferred from multiple objective and subjective indicators,” says Samantha Steenkamp-Farrell, senior security specialist with the World Bank.
From Valerie Acree, a security culture strategist and CEO of Elite Prevention Strategies: “You build cultures through relationships and trust, and that can be a hard thing to measure.”
When thinking about how to assess security culture, Eric Smith, CPP, a security management consultant with Business Karate LLC, says, “When we talk about culture, there’s a human side to it. You can’t just pull out the tape measure and easily gauge this or that. It’s going to take real work to figure out.”
In “5 Factors to Help Build a Case for Security Culture,” Briggs wrote that security technologies and policies will not be effective by themselves in protecting an organization. Delivering security at scale must be accompanied by culture: “the everyday actions, decisions, and behaviors of employees, contractors, and partners,” she explained.
With security culture playing such a vital role, finding ways to assess and measure it—even if it’s incredibly difficult—is essential.
Security Management spoke with the experts mentioned above, reviewed sources on the more general concept of corporate culture, and pulled lessons from the ASIS Foundation report to give security professionals ideas on how to approach the daunting task of trying to measure security culture.
Three Categories of Data
Start by heeding Smith’s advice: There is no tape measure for security culture. Measuring culture requires abstract thinking. Even when there are things you can count, such as the number of times employees make a report to security, using the metric requires security professionals to leverage their experience and intuition to interpret the data through the lens of security culture.
The report gives several examples of possible metrics. In the interviews, the experts added some more, as did the review of corporate culture literature. Since there is no single way to measure culture and because different industries, regions, and settings will have different security needs, it can be helpful to categorize different types of metrics that can be used to measure security culture.
Objective, indirect metrics. These are counting metrics. They are objective because they are quantifiable: security patrols reported six cases of propped doors in the last week. They are indirect because they are not specific measures of security culture on their own.
To be used to assess security culture, objective, indirect metrics must go through an interpretation process. Even trends need interpretation: If the number of reports coming into security has decreased, is that a sign that security culture is leading people to follow security controls and so there are fewer things to report? Or is it a sign that trust in security is falling?
Here’s a laundry list of objective, indirect metrics from the report, interviews, and other sources.
|
Objective, Indirect Metrics for Security Culture |
|
|
Propped doors |
Access control violations |
|
Attempts to access restricted areas |
Reports to security from staff |
|
Utilization rates of security services (such as escorts to vehicles) |
Clicks on or reports of phishing emails |
|
Reports of lost or stolen credentials or keys |
Physical or verbal abuse incidents |
|
Security incidents by severity level |
Repeat incidents |
|
Lighting outage reports |
Safety violations |
|
Not following vendor or visitor protocols |
|
Every security department collects some types of counting stats. Briggs advises that security professionals assess those metrics with an eye toward what they may be saying about the organization’s security culture.
Objective, direct metrics. These quantifiable metrics are more directly relatable to security culture. Many organizations conduct staff surveys that include questions on staff opinions on security or specific security controls. Acree, Smith, and Steenkamp-Farrell all mentioned such surveys as a metric—and they all commented on those surveys’ limitations.
“It’s challenging for a survey to capture whether employees believe in security—whether they believe reporting something would actually lead to a change or to some form of action,” says Steenkamp-Farrell.
Acree says she knows several organizations that do a good job with these surveys and get a good picture of whether or not employees feel safe at work and if they think authorities would take reports they make seriously. But it is a picture, and therefore static.
“The annual metrics just give you an overview,” she says. “It’s not going to give you a complete understanding of the culture at a site or in the organization.”
There are plenty of other examples of objective, direct metrics. Acree comes from a retail security background. District or regional managers make official site visits to stores, and when they do, they summarize the meeting in a report. Counting the number of summaries that include discussions of security-related topics would be a direct, objective metric.
Smith jumps on an idea from the ASIS Foundation’s culture report—the idea of micro-trainings, or short 5- or 10-minute, single-security-topic trainings delivered routinely. He says not only is that a great idea, but related metrics, such as how many staff completed the training sessions or the results of two- or three-question assessments, can help inform the strength of an organization’s security culture.
Smith is also a proponent of red teaming and penetration testing, which can be used to test and assess security culture.
If the goal of objective, indirect metrics is to make additional use of metrics an organization is already compiling, then objective, direct metrics are notably different. They require the security team to intentionally collect them. That means the value of the information each metric provides must be worth the time and effort required to produce it.
Subjective metrics. In “Security Culture: Why It’s Easy to Design and Hard to Achieve,” John Rodriguez drives home the point that interpersonal relationships are key to establishing security culture in an organization. As Acree says, relationships build trust. If relationships and trust are an integral part of security culture, then trying to measure them is needed. The problem is that the strength of a relationship or the amount of trust is entirely subjective. The key is taking subjective areas and turning them into useful metrics that can help inform the assessment of security culture in an organization.
In some cases, that can be straightforward. Briggs put forward the idea of conducting interviews with different staff groups as an advanced metric. These focus groups allow for a depth of understanding that is impossible to gather from a static survey. The feedback is qualitative, but an assessment of a security culture focus group discussion can assign a subjective value to it: Did the discussion signify a good, bad, or neutral approach to security? If you conduct focus groups at multiple sites or in multiple departments, you build up a metric of subjective information that can be used for comparison and additional analysis.
Security professionals can use the same idea beyond the focus group setting. Smith notes that every interaction, formal or informal, is an opportunity for security professionals to assess culture.
“I think it’s a good idea to have a check-in schedule with all the different departments in your organization,” he says. “It’s a constant reminder for people, and that is, I think, a big tie-in to setting the kind of culture you want in an organization. How people respond to that kind of cultural sharing will tell you a lot about where security is in the organization.”
The extra step to make Smith’s check-in idea a valuable, actionable metric is to document the interaction and, just as you would for a focus group, assign it a subjective value that can then be compared to other departments and assessed for status changes over time.
Combining Data to Paint a Picture
Examining all of the options above, none of them are great metrics for security culture. In isolation, none of them are even particularly good metrics for security culture. Steenkamp-Farrell had the perfect metaphor to describe it.
“You can’t measure your health with a single number. A doctor doesn’t take your temperature and declare that you’re healthy,” she says. “They look at your blood pressure, your heart rate, your bloodwork, other symptoms, medical history, how you’re functioning overall. That combination tells the story, and I think it’s the same for security culture.”
Security professionals who want to paint a picture of their organization’s security culture will need to decide what metrics they already collect and if and how those metrics can apply to assessing culture. Then the security professionals need to figure out what other metrics are important enough to their situation that the information is worth the cost in time and resources to collect. Finally, the alchemy is to figure out ways to turn what is a collection of isolated, not-particularly-good-on-their-own data points into a cohesive picture that begins to tell the story.
To pull on the medical metaphor again, any particular number or data point may or may not be concerning on its own. But in combination with other information, it might lead to the diagnosis that an intervention is necessary.
Smith gives one example of this when considering security culture metrics. He was a security professional at an organization that had a nice, detailed survey on attitudes toward security, and the information could be aggregated by department. “I was able to take that and go back and correlate it with some observed behaviors, like how often they called security, and it tracked.” He also used the survey results as a guide when interacting with the department.
At the risk of overdoing the medical metaphor, just like environmental changes or specific actions can affect your health—for better or worse, such as exercising more or contracting a virus—the environment of an organization is always changing. What is being measured may not change, but how they are interpreted might. And sometimes, just like new symptoms will lead to new medical tests, some changes in an organization will require security professionals to adjust what is being measured.
Yes, trying to measure security is hard, and anyone who has read this far could be feeling overwhelmed. Fortunately, in the report, Briggs provides guidance that… well, it doesn’t make measuring security culture easy, but it does make it seem possible—manageable, even!
Where to Start
One place to start is to think about security culture measurement systematically. Briggs breaks security culture into six factors: attitudes, knowledge, behavior, communications, compliance, and empowerment. (See this video and the report for more explanation of the six factors.) She proposes thinking about metrics in terms of how they describe these six factors.
As much as possible, try to develop just a few metrics in each category. A security team might have a dozen or more different objective, indirect metrics readily at hand. Nearly all of them are going to fall under behaviors and compliance. Security professionals need to use their judgement to determine which ones have the most relevance to security culture in their organization and use those, rather than all of the objective, indirect metric options. Other categories may require some measures that need to be developed and managed, such as a survey or assigning subjective labels to interactions.
Next, Briggs advises that security professionals take the metrics they have and those they think are important to develop and create a snapshot baseline. It will be imperfect, and that is okay. Over time, compare how the baseline metrics are changing. In the art of culture measurement, security professionals can make adjustments if the metrics do not appear to be painting the picture quite right. Importantly, the baseline is the whole point of trying to measure security culture. It may show mismatches that need to be addressed. Deviations in the baseline over time can signal not only that an intervention is necessary, but if any implemented interventions were successful.
Finally, once you’ve developed security culture metrics, use them as a tool beyond the security team. Briggs noted that in organizations that have embraced enterprise security risk management (ESRM), asset owners are responsible for the security risks to their assets. Security culture metrics can be used to highlight employee behaviors that influence the security of the asset. Even in organizations where ESRM is not embraced, security culture metrics can be used to have conversations with managers and directors.
Security professionals can also consider if it would be advantageous to combine security culture with the aims of other departments. IT and information security is an obvious possible integration, and one that may have redundant or at least similar metrics. Human resources, safety, facilities, and legal are other areas where a combined cultural metric system could benefit an organization.
It’s important to realize there will not be a one-size-fits-all approach. In fact, it’s probably the opposite: Wach security cultural measurement system needs to be unique to the organization.
“The first question might be, what aspect of security culture are we trying to understand,” Steenkamp-Farrell proposes. “Culture isn’t one thing. I think it’s made up of so many different pieces. So perhaps the mistake organizations make is trying to find that single metric for culture. But it isn’t a KPI [key performance indicator]. It’s a pattern. The more independent indicators point in a particular direction, the more confidence you can have that you are measuring it well.
“I think we tend to think of security as just compliance or just protection,” she continues. “There’s so much more that goes into it. The policies we put in place don’t drive it. It’s really the culture that drives it.”
Scott Briscoe is the content development director at ASIS International. He hopes that, at the very least, companies can develop a security culture that will prevent bad actors from bypassing lobby security with a pizza.
