Managing Risk Assessment Program

ATTENTION: This page is intended to be viewed online and may not be printed or copied.

5. Managing a Risk Assessment Program

5.1 General

The risk assessment program establishes a framework for the overall risk assessment steps in the risk management process. The risk assessment program sets the parameters for the overarching organizational structure, resources, commitment, and documented methods used to plan and execute risk assessments. An effective program has a foundation of clearly defined objectives. A competent person possessing the necessary training, skills, and experience should manage the risk assessment program. The necessary resources should be identified and committed to meet the program objectives (including qualified personnel, financial allocations, and sufficient time). Priority should be given to assessing matters significant to the organization’s mission and the achievement of its objectives. The risk assessment program should also consider legal, regulatory, contractual, and societal obligations. A comprehensive risk assessment program should identify opportunities to maximize favorable outcomes as well as minimize the likelihood and consequences of undesirable and disruptive events.

The risk assessment program should define:

Objectives and purpose of the risk assessment;
Scope, activities, areas, and locations to be covered by the risk assessment;
Duration, number, schedule, and frequency of the risk assessment;
Responsibilities and authorities associated with managing and conducting the risk assessments;
Risk assessment criteria (standards, policies, assessment metrics, and other criteria);
Assessor competence and selection of teams;
Business management issues related to risk assessment criteria and the risk assessment itself;
Resources (human, time and scheduling, financial, technology, equipment, travel, etc.);
Confidentiality, safety, and security issues;
Methods of how the risk assessment will be conducted;
Communication of risk assessment findings;
Monitoring risk assessment activities;
Documentation, records, and documentation procedures; and
Risk assessment evaluation and continual improvement.

A goal of a risk assessment program is to review the risk management controls and system, as well as to identify opportunities for improvement. When developing the risk assessment program the following issues should be considered:

The management approach and the management system standard(s) being used;
The size and nature of the organization being assessed;
The complexity and volatility of the operating environment;
The scope, complexity, and level of maturity of the risk and business management system(s) being assessed;
The risks associated with the organization being assessed and its applicable industry sector;
Business attributes and priorities of the organization being assessed; and
Allocation of resources required to adequately evaluate the management system.

5.2 Understanding the Organization and its Objectives

The key task of the persons planning and conducting a risk assessment program is to develop an understanding of the organization to be assessed. This does not mean that the assessor must become an expert in the operation of the enterprise to be evaluated, but must acquire enough of an understanding of how the organization operates to appreciate its complexities and nuances.

Understanding the organization should include (but is not limited to), factors such as:

Organization mission and business objectives;
Nature of the business activity;
Tangible and intangible assets and its value chain;
Governance, authority, and management style;
Current risk control measures;
Types of services provided or products produced, manufactured, stored, or otherwise supplied;
Stakeholders and their objectives;
Types of clients, clientele, and customers served;
Information flow;
Roles, responsibilities, and accountabilities;
Supply chain and critical infrastructure dependencies and interdependencies;
Legal and regulatory environment;
Voluntary commitments of the organization;
Competitive nature of the industry;
Enterprise culture;
Geographic spread of the enterprises;
Any special issues raised by the production, administration and service processes (e.g., environmental waste, disposal of defective goods, etc.);
Type of labor (e.g., labor union, unskilled, use of temporary workers, outsourcing, use of immigrants, etc.);
Hours of operation;
Sensitivity of information; and
Perception of risk tolerance and acceptance (internally and externally).

When evaluating the objectives of an organization, some questions to consider include:

What are the explicit and implicit strategic objectives of the organization and the divisions within?
What is the state of development, size, industry sector, geographical spread, maturity of its business management style, and complexity of the organization and its activities?
What is the nature and extent of the significant risks associated with achieving the organization’s objectives?
What are the boundaries for risk taking, what risks are they willing to take, and which are they not?
What is the attitude towards governance in the organization and in the management of risk?
Is there an organizational structure to facilitate the management of risk?
What is the risk management culture in the organization?
Is the organization progressive and innovative or conservative and adverse to change?
Are there resources and systems to support the risk management processes?
What are the determining factors to consider in risk appetite and risk tolerance?

5.2.1 Enterprise Value of Tangible and Intangible Assets and Services
In order to understand the organization it is necessary to identify the people, assets, and services that provide the enterprise tangible and intangible value. People involved in or affected by the organization include employees, customers, visitors, vendors, patients, guests, passengers, tenants, contract employees, and any other persons who are lawfully present on the property being assessed. Unauthorized persons (such as trespassers) need to be considered in the risk assessment. Property includes real estate, land and buildings, facilities; tangible property such as cash, precious metals, and stones; monitoring, control, data, and communication systems; support infrastructure, instruments; materials (e.g., raw materials, process materials, finished goods, and hazardous materials); high theft items (e.g., drugs, securities, cash, etc.); as well as almost anything that can be stolen, damaged, or otherwise affected.

Intangible assets include the brand, goodwill, or reputation of an enterprise that could be impacted. Another high value intangible asset is information. Information includes intellectual property and proprietary data, such as trade secrets, marketing plans, social media interaction, business expansion plans, plant closings, confidential personal information about employees, customer lists, and other data that if stolen, altered, or destroyed could cause harm to the organization.

Services provided to the internal and external stakeholders are important parts of the organization’s value chain and may be affected. For example, non-availability of IT or accounting services may have an impact on the organization; its operations and assets.

The enterprise value of assets and services should be considered within the context of:

Value relative to critical mission activities, services, and products;
Exclusive possession;
Utility;
Cost of creation or re-creation;
Criticality and competitive edge;
Critical human resources and knowledge;
Operational and business impact (including dependencies and interdependencies);
Cost of lost opportunity;
Shelf life of the asset;
Reputation and brand impact; and
Other considerations important to management or clients.

The value of an asset and service should be considered within the context of how the assets contribute to the organization’s achievement of its objectives. While organizations may have a myriad of assets, products and services, typically not all are mission critical. Therefore, in addition to considering the monetary value of assets, valuation should consider how the asset fits within the value chain of the organization and its relative value in achieving strategic and tactical objectives.

5.2.2 Considering Risk Criteria
The organization should understand and define its criteria to evaluate the significance of risk. The risk criteria should reflect the organization’s values, objectives, and resources. While risk criteria should be established at the beginning of the risk assessment process, they are dynamic and should be continually reviewed. When defining the risk criteria the organization should consider:

Critical activities, functions, services, products, and stakeholder relationships;
The operating environment and inherent uncertainty in operating in specific regions;
The potential impact related to a disruptive or undesirable event;
Views and perceptions of stakeholders;
Legal and regulatory requirements and other requirements (e.g., contractual obligations, human rights commitments) to which the organization subscribes;
The organization’s overall risk management policy;
The nature and types of threats and consequences that can occur to its assets, business, and operations;
How the likelihood, consequences, and level of risk will be defined and determined;
Needs of, and impacts on, stakeholders;
Establish the relative timeframes for evaluating likelihoods and consequences;
Reputational and perceived risk;
Level of risk tolerance or risk aversion of the organization and its clients (define the boundaries for when risk is acceptable or tolerated);
How the level of risk will be determined; and
How combinations and sequence of multiple risks will be taken into account.

When setting the risk criteria, the organization should understand the risk it is willing to pursue, retain, or take (risk appetite), as well as the risk it is ready to bear after risk treatment (risk tolerance), and the risk it is not willing to undertake (risk aversion) in order to achieve its objectives. When setting the risk appetite it is important to understand the nature of the uncertainty and whether the organization is able to manage the risk to the level it is willing to pursue. Risk appetite, risk tolerance, and risk aversion have temporal and environmental components and will change over time as circumstances change. For example, changes in the economic or socio-political environment may be monitored for their effects on how acceptable a risk may be. Also, when evaluating the impact of a risk on the enterprise it is important to revisit the designated levels of risk appetite and risk tolerance to determine if factors (e.g., reputational impacts) were fully understood when making the initial estimates. Risk appetite, risk tolerance, and risk aversions also may vary among different enterprise levels and elements of the value chain, but should be aligned.

Risk appetite, risk tolerance, and risk aversions need to be articulated concepts. Risk appetite has to be set in the context of the maturity of the business and risk management processes of the organization. The organization needs to have the competence and capability to manage risk within the boundaries it sets. Therefore, the boundaries should be tailored and proportionate to the size, nature, and maturity of the business and risk management processes.

5.2.3 Understanding Bias
Biases may sometimes lead to perceptual distortion, inaccurate judgment, and illogical analysis of information. There is a common tendency to acquire and process information by filtering it through one's own likes, dislikes, and experiences. The person managing the assessment should identify and understand the inherent and cognitive biases within the organization and the individuals conducting the assessment. The inherent bias is the effect of underlying factors and assumptions that impact information collection and analysis. Cognitive biases are tendencies to think in certain ways or a failure to imagine plausible alternatives. Types of biases to consider include (but are not limited to):

Social and cultural biases;
Familiarity and confirmation biases;
Perception, observational selection, and memory biases;
Belief and behavioral biases;
Relational, group-think, and tribal biases;
Confirmation and post-rationalization biases;
Information availability biases;
Decision-making biases; and
Illusion-of-control biases.

5.3 Establishing the Framework

Establishing the framework begins with identifying the internal and external context, including the internal and external operating environments, and other factors inside and outside the organization that may influence the risk assessment program. The framework provides the foundation and rationale for designing, implementing, monitoring, maintaining, reviewing and continually improving the risk assessment program.

5.3.1 Context of the Organization
Conducting a risk assessment of an organization requires knowledge of the internal and external factors that can influence an organization’s performance in managing risks. When planning the risk assessment process it is important to consider:

Risks associated with the industry sector and the organization’s processes;
Internal factors affecting the operating environment of the organization;
External factors affecting the operating environment of the organization;
Internal and external stakeholders who are risk-makers and risk-takers;
Internal and external stakeholders that are impacted by risks; and
Factors that influence the acceptance of risk in the organization and by its stakeholders.

Understanding the key factors, drivers, and issues that influence an organization’s ability to achieve its objectives and meet its obligations is an integral part of any strategic or tactical planning process. The context will provide a foundation for risk management activities. It is a complex undertaking, particularly in organizations with less mature systems for managing risk. Therefore, the steps outlined in this Standard should not be viewed as a linear set of sequential steps but rather as an iterative process where the context of the organization is re-evaluated as more information becomes available.

5.3.2 Internal Context
The organization should identify, evaluate, and document its internal context, including:

Strategies, policies, objectives, plans, and guidelines to achieve objectives;
Governance, roles and responsibilities, and accountabilities;
Organizational values, ethos, morale, and culture;
Financial arrangements and restraints;
Information flow and decision-making processes;
Internal stakeholders who are the owners, contributors, impacted parties, and managers of risk (enterprise-wide and by sub-divisions);
Capabilities, resources, and assets (tangible and intangible);
Procedures and practices;
Activities, functions, services, and products including their value streams; and
Brand and reputation.

5.3.3 External Context
The organization should define and document its external context, including:

The cultural and political context;
Legal, regulatory, technological, economic, natural, and competitive environment;
Contractual agreements, including other organizations within the contract scope;
Infrastructure dependencies and operational interdependencies;
Supply chain and contractor relationships and commitments;
External stakeholders who are the owners, contributors, impacted parties, and managers of risk (within the value chain, vested interests, impacted communities, and the media);
Key issues and trends that may impact the processes and/or objectives of the organization;
Perceptions, values, needs, and interests of external stakeholders (including local communities in areas of operation);
Operational forces and lines of authority; and
Brand and reputation.

In establishing its external context, the organization should ensure that the objectives and concerns of external stakeholders are considered in the risk management criteria.

5.3.4 Supply Chain and Subcontractor Mapping and Analysis
Managing risks in the supply chain, including subcontractors, requires an understanding of the organization’s culture and environment as well as the context of the global environment of its supply chain. Each node of the organization’s supply chain involves a set of risks and management processes.

The organization should identify and document its upstream and downstream supply chain, including its use of subcontractors, to identify significant risks and the potential to cause a risk event. Analysis of supply chain risk should be included in an organization’s overall risk assessment program. The organization should define and document the nodes and tiers in their supply chain and subcontractors to include in their risk assessment program.

5.3.5 Risk Managemnt Context
The risk management context of the organization describes the scope, as well as risk control parameters, methods, and plans currently in place for the risk management activities. Before starting the design and implementation of the risk assessment program, it is important to understand the objectives of the risk management program and to evaluate and understand both the extent and efficacy of the current risk control measures and system.

When determining the current state of affairs issues to consider include:

The defined objectives of the risk management programs.
The risk management program objectives aligned with the overall business management objectives of the organization.
What are the nodes in the value chain that are responsible for the greatest measure of value?
What are the identified activities, products, and services considered essential for achieving the organization’s objectives?
What are the identified threats and vulnerabilities?
What are the risk control methods in place, the efficacy in controlling identified risk, the residual risk, and the perceived cost-benefit of the control measures?
Were there specific exclusions to the risks identified and treated?
What are the data, information, and intelligence sources used to determine risks and their perceived reliability?
What are the responsibilities, accountabilities, and resources for the management of risk?
What are the information, reporting, and records management requirements?
What are the interdependencies between the internal, external, and risk management contexts?

5.3.6 Needs and Requirements
The person(s) conducting the risk assessment should understand the reason and purpose for the assessment. There should be a clear understanding between the risk manager and top management as to the purpose of the risk assessment program and intended use of the outcomes. Various purposes of risk assessment exist. Examples are:

Determine if the organization is achieving its overall management objectives;
Provide input for decision-making processes;
Identify actual, potential, and perceived risks and evaluate risk treatment processes;
Protect tangible and intangible assets;
Use of a systematic process to identify weaknesses in the organization’s processes and risk management approaches;
Evaluate risk treatment measures;
Identify opportunities for improvement;
Verify accepted industry practices;
Promote consistency in processes across business units;
Promote and evaluate training and awareness programs;
Provide visible management support for risk management programs;
Conduct due diligence for purchases and supply chain partnerships;
Evaluate and improve the allocation of resources;
Understand risk exposures related to activities, projects, and operations;
Identify business opportunities (including launching new partnerships, products, and services);
Demonstrate regulatory compliance;
Reduce liabilities;
Address consumer and supply chain needs and concerns; and
Demonstrate reliability of product and service delivery.

When developing the risk assessment program, the risk manager should understand the organization’s intended use of the assessment results. The intended use of the risk assessment outcomes may influence the attitude of the participants to the risk assessment process.

5.3.7 Objectives of the Risk Assessment Program
Clearly defined risk assessment objectives are crucial to implementing a successful risk assessment program. Risk assessments will provide more value to the organization if the risk assessment program objectives are aligned with organizational and management objectives. The risk manager and top management should clearly define and agree upon the risk assessment objectives.
When defining the risk assessment program objectives, the following factors should be considered:

Management and decision-making requirements;
Tangible and intangible assets to be protected;
Business management system requirements;
Organizational, business, and operational goals;
Legal and contractual obligations;
Risk management priorities and performance;
Perceptions and expectations of stakeholders and other interested parties, including supply chain needs;
Previous risk events including exercises, drills, minor and major incidents including near misses; and
Level of maturity of the organization’s management system.

Examples of risk assessment program objectives include (but are not limited to):

Perform gap analysis for determining improvements to business and risk management processes;
Verify conformance of a management system to the requirements of relevant standards;
Demonstrate effectiveness of risk treatment measures and identify opportunities for improvement;
Validate organizational risk management for internal and external stakeholders;
Demonstrate consistency with accepted industry practices; and
Evaluate alignment of risk management with the overall business management approach in order to achieve the overall organizational objectives.

5.3.8 Evaluating the Criticality of Decisions
A decision-maker's response to an organizational situation with variable outcomes is a function of perceived risk and perceived decision criticality. It is important to know the underlying psychological, social, and emotional components that influence decision-making (assessor’s decisions or the decisions of others). Some factors to consider:

Accurately defining a problem and its context is a large part of making a good decision, not just solving a problem;
Framing a decision in terms of potential loss or gain will influence the criticality of the decisions and perceived level of acceptable risk;
Timeframes for decision-making will influence the criticality of decision-making (shorter timeframe usually results in higher criticality);
Quickly changing environments require revisiting the relevance of past experience and expertise; and
Uncertainties, not just obvious problems, affect critical decision-making.

One risk may have compound effects on other risks. In the decision-making process, it is important to assess risk so that interaction between multiple risks is understood. The impact of various decisions in the assessment and treatment of risks should be considered throughout the risk assessment process, as well as the potential for unintended consequences when addressing risk decisions.

5.3.9 Establishing the Scope of the Risk Assessment Program
The scope of the risk assessment program should be defined in order to achieve the risk assessment objectives and consider the context of the organization, its needs, and requirements. The scope should define the processes, functions, activities, physical boundaries (facilities and locations), and stakeholders included within the boundaries of the risk assessment program. The scope of the risk assessment program will have a direct effect on the resource and time requirements needed for the individual risk assessments. When setting the scope of the risk assessment program, resource and time requirements are directly proportional to the size of the scope. The risk manager and top management should agree to the risk assessment program scope prior to commencing any assessments. Any subsequent changes in scope should be mutually agreed upon and documented.

The scope of the risk assessment program may consist of one or more individual risk assessments. If conformance to a management system standard is the objective of the risk assessment program, the scope of the program should be in alignment with the scope of the management system with any divergence noted and understood.

Additional factors to consider in setting the scope:

Size and complexity of the organization;
Results of previous risk assessments;
The likelihood and consequences of known undesirable and disruptive events (including consideration of previous incidents and weaknesses of the management system);
Emerging risks and business opportunities;
Reports and concerns of internal and external stakeholders;
Supply chain nodes to be included;
Complexity and maturity of the risk management system; and
Factors related to timing, logistics, communications, and information accessibility.

5.4 Establishing the Program

5.4.1 Roles and Responsibilities
The roles and responsibilities of the parties conducting the risk assessment and the client should be clearly defined and understood, including:

Risk manager (RM) – the person responsible for managing the risk assessment program and assuring the necessary financial, human, physical, and time resources are committed to conduct an effective risk assessment;
Risk assessment team leader (RTL) – the person designated as leading the risk assessment team;
Risk assessor (RA) – a person conducting the risk assessment, individually, or as a member of a team;
Technical expert – a subject matter expert with specific knowledge or expertise supporting the risk assessment team but does not act as an assessor (e.g., a legal or industry sector expert, threat assessor, physical security specialist, information technology specialist, supervisory control and data acquisition, SCADA, specialist);
Observer – a person who accompanies the risk assessment team (e.g., a client’s representative, client liaison, or guide); and
Client – top management or business division of an organization that requests the risk assessment. A client may be internal or external to an organization being assessed.

NOTE: All persons performing functions should demonstrate competence in the roles they are conducting. Depending on the size and complexity of the scope, some or all of these roles may be combined. The combined competence of the team should be sufficient to cover all areas of expertise needed to conduct an effective assessment. See section 7.2 on competence.

The risk manager is responsible for the planning, management, and conduct of the risk assessment program, while the RTL is responsible for the conduct of individual assessments. They are both responsible for the professional and ethical behavior of the risk assessment team members. The RM and RTL are responsible for:

Defining the objectives, criteria, and scope of the risk assessment program as individual assessments;
Communicating and consulting with relevant parties to the risk assessment;
Ensuring the risk assessment team and its members have the necessary competence to successfully conduct the risk assessment;
Ensuring the allocation of adequate resources for risk assessment;
Ensuring the risk assessment program is executed as planned in a timely fashion;
Ensuring the completeness and integrity of documentation;
Ensuring risks to the client and risk assessment team of conducting the risk assessment program are appropriately managed;
Reviewing work product(s) assigned to assessors for completeness and accuracy; and
Ensuring the integrity and confidentiality of information.

The client should appoint at least one representative from top management to interface with the assessment team. The client’s representative should have the authority to provide the assessors:

Authority to conduct assessment and make decisions;
Appropriate organizational, functional, stakeholder, and historical information to evaluate risks;
Access to areas and activities to be assessed;
Access to relevant persons;
Access to information;
Facilities for the risk assessment team use (e.g., private work space, telecommunications, safety and hygiene facilities, etc.);
Support personnel if needed;
Safety, security, and regulatory requirements; and
Information needed for protection of proprietary rights and confidentiality.

5.4.2 Legal and Other Requirements

Assessors should perform professional duties in accordance with the law and the highest ethical principles. An assessor should observe the principles as listed in section 4 to be faithful and diligent in conducting professional responsibilities. Assessors should safeguard confidential information and exercise due care to prevent its improper disclosure. The assessors should not maliciously injure the professional reputation or practice of colleagues, clients, or employers.

Risk managers and RTLs should be mindful of legal and liability issues related to the assessment. Assessors should understand their responsibilities to:

Avoid conflicts of interest and to protect real and perceived impartiality;
Not use information learned during the course of the risk assessment for personal gain or the gain of others;
Not share information beyond a need to know basis or that can be used to restrict competition;
Exercise responsible care and competence to avoid violation of the principle of due care;
Report findings honestly;
Observe environmental, safety, and security regulations; and
Not disclose proprietary information.

Assessors should be apprised of their responsibilities to report illegal and unsafe activities within or outside the scope of the risk assessment, including legal requirements for disclosure. Once discovered, an assessor should not ignore illegal or unsafe activities. Assessors should inform the RTL – who informs the client and risk manager. The RTL should verify and create a record of the condition. If the team is endangered, the risk assessment should be stopped, and not resume until the endangering condition is rectified.

5.4.3 Competence Requirements
Competence and ability to apply knowledge and skills to achieve intended results is necessary of all parties involved in the conduct of the risk assessment. Competence is the demonstrated sum of personal attributes, generic risk assessment knowledge and skills, risk management knowledge and industry sector specific knowledge and skills.

To conduct an effective risk assessment, the RM, RTL, and assessors should demonstrate skills and knowledge in the following areas:

Interpersonal and communications skills;
Systems, PDCA, and process approaches to risk management;
Standards being used, as well as normative documents;
Principles of risk management based on ISO 31000;
Cultural awareness and understanding, including respect for individual’s rights;
Technical knowledge of the activity being assessed;
Risk assessment and management from a mission and operational perspective;
General knowledge of regulatory requirements; and
Industry sector and risk discipline specific good practices.

The RM and RTL should ensure assessors provide risk assessment services only in those areas where they have the necessary knowledge, skills, and experience.

5.4.4 Identifying and Managing Uncertainty in the Risk Assessment Program
Changes both internal and external to the organization may affect risk. Therefore, analysis of the uncertainty related to the risk assessment processes is an integral part of developing and improving the risk assessment program. To effectively assess any organization, it is important to understand the risks related to:

Complexity and dynamic nature of the external and internal environment;
Achieving the objectives of the assessments;
Real and perceived impartiality;
Legal and regulatory issues;
Execution of the assessment on the client’s organization and its activities;
Health safety and security of the assessment teams; and
Perceptions of interested parties.

There is a need to understand the uncertainties related to the risk assessment program to achieve its objectives and assure credibility.

5.4.4.1 Risk to the Organization of the Assessment
Risk assessments involve evaluating inherently sensitive information of organizations. This introduces an element of uncertainty to the risk assessment process. The risk manager should evaluate the potential tangible and intangible impacts of the conduct of the risk assessment client.

The risk manager should consider:

Information security and confidentiality needs;
Protection of information sources;
Background of the risk assessment team;
Clearances;
Exposures of vulnerabilities;
Reporting requirements; and
Disruption to operations.

5.4.4.2 Risk to Achieving the Assessment Objectives
Persons conducting risk assessment should understand the uncertainties that may have an impact on the achievement of the objectives of the risk assessment. It is also important to allocate available time and resources to the areas with higher levels of risk. The planning process should prioritize resources commensurate with the associated level of risk and ensure important risk factors are not overlooked.

In identifying, analyzing, and evaluating risks to the assessment program, the risk manager should consider:

Planning;
Organizational and leadership buy-in to the process;
Overall competence of the assessment team and team members;
Sufficiency of allocated resources;
Implementation of the risk assessment plan;
Communication between team members as well as between the assessment team and client;
Appropriate documentation and recordkeeping (and documentation control) consistent with jurisdictional requirements; and
Monitoring of program outcomes.

5.4.4.3 Risk to Real and Perceived Impartiality
The risk manager should establish and document a procedure for identifying, analyzing, evaluating, and treating (e.g., reducing) risks associated with real and perceived threats to impartiality. Consideration should be given to biases described in section 5.2.3, as well as factors related to criticality of decisions.

5.4.4.4 Legal and Regulatory
When planning the risk assessment program the risk manager should consider the jurisdictional requirements related to:

Authorities and accountabilities;
Security (physical and information);
Safety;
Disclosure and non-disclosure requirements;
Duty of care; and
Contractual obligations.

5.4.4.5 Health, Safety, and Security of the Risk Assessment Teams
When there is the potential for exposure of the assessment team to threats and hazards during the risk assessment, the risk manager should evaluate health, safety, and security-related risks and take appropriate actions. For example, specialized training or protective equipment may be needed or required for specific assignments and tasks.

5.4.4.6 Perceptions of Stakeholders and Other Interested Parties
The perceptions of external interested parties may impact the design and implementation of the risk assessment program. Therefore, during the design of the risk assessments, the risk manager should be aware of and consider the perceptions of:

Key stakeholders (e.g., workers, unions, and labor organizations, customers, investors, etc.);
Supply chain partners;
Government regulators;
Neighboring communities and adjacencies;
Civil society groups and organizations; and
The media.

5.4.5 Program Approach and Procedures
Design of effective risk assessment procedures should consider the adequacy and effectiveness of the risk management controls and identify changes in risk profiles and priorities. The level of confidence in the assessment outcomes will be based on the evidence and facts collected, not perceptions and assumptions.

The risk manager should develop one or more procedures for managing the risk assessment program. When developing the procedures the risk manager should identify performance metrics that will be used to determine if the procedures were effective and successfully applied. Procedures should be developed for:

Planning the assessments to evaluate the organization’s risks and controls;
Identify and maintain the appropriate level of assessor competence;
Selection of assessment team members and appointment of RTL;
Ensure effective communication between all parties involved in the assessment;
Evaluating required resources, logistics, and feasibility of assessment success;
Conduct of the assessment including data collection and sampling techniques;
Evaluation of the assessment data, definition of priorities, and improvement of risk treatment methods;
Performance assessment of the assessment process to identify opportunities for improvement;
Integrity, confidentiality, and protection of information;
Handling, chain of custody, access control, and archiving of records; and
Monitoring, review, and continual improvement of the risk assessment program.

5.4.6 Commitment of Resources
Once the objectives and scope have been established for the risk management program, the risk manager should identify and assure the commitment of resources necessary to conduct a successful risk assessment program. The risk manager should provide resources in terms of personnel, time, travel, and the financial resources necessary to develop, implement, manage, and improve the risk assessment activities (including assuring assessor competence). From the organization’s perspective, the tangible and intangible benefits of increasing the likelihood of achieving organizational objectives should outweigh the costs of conducting the risk assessment.

Personnel resources include the designation of appropriate and adequate full and part-time assessors, as well as accompanying technical experts. The makeup of the assessment team should reflect the objectives of the risk assessment program and the complexity of the organization’s system to manage risk. The risk manager should calculate the personnel hours required to successfully complete each portion of the risk assessment.

Factors that will affect the allocation of resource requirements (particularly personnel and time requirements) include (but are not limited to):

Complexity of risk criteria and range of risks to be assessed;
Risks associated with the organization, its activities, and its context;
Complexity and size of the organization to be assessed (e.g., technologically complex or labor-intense organizations may increase the personnel hours needed);
Maturity of the existing risk management system;
Risks associated with the risk assessment program (including minimizing bias);
Desired timeframe in which the assessment is to be conducted;
Risk assessment methodologies and sampling methods;
Results of prior risk assessments;
Extent of changes in operating environment;
Review of documentation;
Availability and accessibility of information;
Number of sites, multi-site considerations and diversity of stakeholders;
Single or multiple shifts, as well as weekends and off-hours;
Physical size and layout of the organization to be assessed;
Meeting requirements (opening and closing meetings, top management briefings, and assessment team meetings);
Communications (including availability of information and communications technologies and methods);
Safety and security arrangements and equipment;
Travel and logistics (including lodging, meals, and breaks);
Data analysis and report preparation;
Availability of competent personnel to conduct the risk assessments; and
Anticipated scheduling delays.

5.5 Implementing the Risk Assessment Program

5.5.1 Setting Criteria for Individual Risk Assessments
The risk assessment program may consist of one or more risk assessments, the sum of which achieves the overall objectives of the risk assessment program. The objectives, scope, and criteria of the individual risk assessments within the program should be consistent with the overall objectives of the risk assessment program. The objectives of the individual risk assessments should be clearly defined and documented. Examples of individual risk assessment objectives may include (but are not limited to), determining:

Performance evaluation of the risk management system, including consistency of risk treatment measures with the output of the risk assessment;
Evaluation of the conditions underlying the risk(s);
Extent of compliance with legal and other requirements;
Efficacy of the organization’s risk treatment processes;
Adequacy of risk management controls in a changing operational environment;
A basis for risk-based spending;
Awareness and promotion of a risk management culture in the organization; and
Opportunities for improvement.

The scope of the individual assessments should be clearly defined and documented. Examples of individual risk assessment scope include (but are not limited to):

Specific facilities and physical locations;
Individual divisions and organizational units;
A value chain in the organization;
A specific set of risks;
Evaluation risks related to new products and services; and
Specific processes.

The criteria of the individual risk assessments should be clearly defined and documented. Examples of individual risk assessment criteria include (but are not limited to):

Risk management goals established by top management;
Management system standards requirements of one or more standards;
Accepted industry practices;
Headquarters or supply chain requirements;
Legal requirements;
Security requirements;
Concerns and perceived risks of stakeholders; and
Risk management policies and procedures.

The scope and depth of a risk assessment should be determined and documented by the organization. The objectives and decision timeline drive specific types of risk assessments to be applied in different situations. Figure 3 illustrates different scopes and depths of risk assessments based on specific applications.

Adapted from A Cultural Approach to Decision Making PResentation at RIMS 2011 ERM Conferece by Dr. Carl Spetzler. Copyright © 2013. Risk and Insurance Management Society, Inc. All rights reserved.

Figure 3: Formal vs. Informal Risk Assessments

Risk assessments become an automatic and informal part of the decision-making process when risk management is fully integrated into the organization’s culture. When decisions become more significant or complex, a moderate deliberative risk assessment process is needed. In these situations, limited risk assessment techniques may be used in order to reach a decision in a shortened timeframe. When decisions are strategic in nature and complex, a more rigorous deliberative effort is needed. In such cases, multiple risk assessment techniques can be applied when there is a longer decision timeframe.

5.5.2 Identifying Risk Assessment Methods
The risk manager should determine the appropriate methodology for conducting an assessment to achieve the objectives, scope and criteria. The level of detail and complexity of the risk assessment should be tailored to the decisions that it is intended to support. Methods chosen will be a function of the size and nature of the organization as well as risk, human, cultural, infrastructure, and geographic factors. The risk assessment methodology employed will drive the skill sets and competence required of assessors. Additional guidance is given in ISO 31010 and in section 6 of this Standard.

When choosing a risk assessment methodology, care should be given to remaining within the organization’s capabilities. The methodology should follow a logical process by which the inputs into an assessment are evaluated to produce the outputs that inform the decision-making processes. When trying to determine the methodology, previous assessments or an industry accepted approach may be a good starting point, but should be reevaluated for appropriateness and tailored to the current circumstances. Choice of methodology should also consider, data availability, and resource constraints.

When selecting a methodology, it is important to understand the reliability and confidence levels of the available data, particularly estimates of likelihood and consequences. There is no single methodology that is appropriate for measuring the likelihood and consequences of various risks. Each methodology requires independent judgment regarding its design. In some cases, it may not even be possible, or necessary, to explicitly determine likelihood and consequence. As a general rule, simple methodologies are less prone to errors and are easier for stakeholders to understand, as well as more likely to fulfill the principles of transparency and practicality. The methodology that best meets the decision-maker’s needs is generally the best choice, whether quantitative or qualitative.

5.5.3 Competence, Evaluation and Selection of Risk Assessors
The credibility of any risk assessment program is dependent on the experience, knowledge, and interpersonal skills of the assessment team. The risk manager should select team members and a RTL based on the competence needed to achieve the objectives of the risk assessment and with the interpersonal skills necessary to interface well with persons in the organization being assessed. The size and composition of the team will be dependent on the objectives, scope, and criteria of the risk assessment and the size and complexity of the organization being assessed.

The team members are responsible to collect data to support analysis and evaluation of risks and any proposed control measures to treat risk. Team members should be able to gather information efficiently, objectively, and with due consideration of potential disruption to the organization’s normal routine.

The risk manager should establish well-defined criteria for selection of individuals and assigning work. Procedures should be developed to evaluate particular assessor qualifications, including:

Knowledge;
Experience; and
Personal skills and traits.

Factors to consider in selecting members of an assessment team include:

Overall competence of the assessment team needed to achieve the risk assessment objectives;
Nature of the risk management system and what specific risk disciplines have been addressed (e.g., compliance, safety, security, crisis and continuity management – assessors may have a specific discipline focus and bias so discipline balance should be considered);
Knowledge of industry sector and the risks the sector faces, including understanding the specific context of the organization and its dependencies;
Complexity of the risk management activities, including the use of single or multiple management system standards;
Risk assessment methods to be used;
Legal, regulatory and other requirements keeping in mind jurisdictional variations;
Independence, impartiality and avoidance of perceived or real conflict of interest;
Personal, cultural, social and language skills required to deal with diversity in the organization;
Security, clearances, citizenship, and safety requirements of the team members;
Dynamics of the team members and their ability to work together and with the client;
Logistics and availability of personnel; and
Leadership requirements and the need to oversee and train new assessors.

When considering the selection of assessors, the risk manager should evaluate the qualifications, knowledge, experience, personal skills, and traits of the assessors needed to achieve the risk assessment objectives. The risk manager should have a documented process for evaluating and selecting assessors. See section 7 for additional details.

Technical experts may supplement the competence of the team. At all times the technical experts should operate in conjunction with the risk assessors. Technical experts are intended to supplement the overall expertise of a risk assessment team to provide subject matter expertise. Technical experts are not a substitute for assessors having competence in the risk disciplines being assessed.

Assessors-in-training may also be included in the team. Assessors-in-training should have knowledge of conducting risk assessments, the risks associated with the organization, and risk management. They should participate under the direction and guidance of an experienced assessor.

The risk manager and RTL, may make adjustments to the team during the course of the assessment depending on the necessity for additional competencies.

5.5.4 Establishing Roles and Responsibilities of Risk Assessment Team Leader
The risk manager should assign an individual to be risk assessment team leader (RTL) well in advance of commencement of the assessments to allow for sufficient preparation time. Since the RTL is tasked with conducting the assessment, as well as directing and monitoring the team, the individual should be an experienced assessor and familiar with the business and industry sector being assessed, as well as risk-based disciplines being managed. The RTL is responsible for:

Satisfactory performance of all phases and activities of the assessment;
Representing the assessment team with the client and/or organization’s management team;
Initiating and maintaining communication with the client and/or organization’s management team;
Encouraging diversity of views while maintaining professional behavior and harmony amongst the assessment team members;
Developing the risk assessment plan;
Managing risks during the risk assessment process;
Leading, organizing and directing assessment team members (particularly assessors-in-training);
Making effective use of resources during the risk assessment and time management;
Conducting opening and closing meetings;
Conducting regular meetings and briefings with the risk assessment team as well as client and/or organization’s management team;
Protecting the health, safety and security of the assessment team;
Assuring the confidentiality and protection of sensitive and proprietary information;
Preventing and resolving conflicts;
Reviewing the evidence and observations of the assessors and leading the team in determining the findings and conclusion; and
Preparing and submitting the risk assessment report, assuring its factual accuracy and clarity of recommendations.

Specific assessment assignments should be based on the experience and knowledge of the individual assessors and reflect the complexity of the assessment tasks. There should be a balance in the assessment team between technical, legal, industry, administrative, and risk-based discipline management knowledge. The RTL should assign and communicate assessment responsibilities prior to commencing the assessment.

5.5.5 Managing and Maintaining Program Documentation, Records, and Document Control
The risk manager should identify the documentation needs of the risk assessment. Procedures should be established for the use and handling of documents and records created for the risk assessment program by the risk manager. Clear procedures should be outlined for obtaining and handling client and organizational documentation. The client and organizational management must explicitly approve copying of any information or photography. Assessors should not remove, modify, delete, or destroy documents (including electronic files) without explicit written permission to do so.

The risk manager should establish, implement, and maintain procedures to protect the sensitivity, confidentiality, and integrity of documents and records including access to, identification, storage, protection, retrieval, retention, and disposal of records. Documents should be clearly labelled as to their status and version (e.g., draft or final, active or archival) as well as level of sensitivity and confidentiality. Records should be maintained of access to information and documents.

In instances where reports are deemed confidential, the risk manager should establish computer and network controls over files and risk assessment information to prevent access by unauthorized users. When confidential information is collected the risk manager should establish procedures and provide technology to assessment team members to use encrypted storage devices or laptops to secure this information.

Records and documentation should be created, maintained, and appropriately stored for both the overall risk assessment program and individual assessments, including;

Program objectives, criteria, and scope
Risk assessment and treatment methods, and measures;
Evaluation of achievement of risk assessment objectives; and
Risk assessment program effectiveness and opportunities for improvement.

For individual risk assessments, records should include:

Plans and reports;
Assumptions, stakeholders, and information sources;
Risk criteria and risk appetite;
Safety, security, and confidentiality requirements and conditions;
Agenda and minutes from opening and closing meetings;
Non-conformance and corrective action reports;
Modification of risk treatment methods; and
Risk assessment follow-up reports.

Procedures should be established to create and maintain records of risk assessment performance. Performance review records should be used to drive continual improvement of risk assessment process and assessment team competence. Examples of performance records include:

Feedback from the organization and client;
Selection criteria and competence of assessment team members;
Performance evaluations of the assessment team members and team leader;
Effectiveness of time management; and
Needs for continuing training and competence improvement of assessment team members.

5.5.6 Performing the Risk Assessment and Operational Control
The risk manager, in conjunction with the RTL, should identify the documentation necessary to assess risks to the organization and its value chain. The RTL should contact the appropriate internal and external stakeholders to assess the availability of documents related to the risk assessment within the scope of the assessment.

The organization’s risk management policies and procedures are reviewed first to determine if the risk management system has been clearly and completely defined and designed. Organizational and client documentation should be reviewed to determine if it conforms to risk management requirements as well as legal and regulatory requirements. Document review is not a checklist approach, rather it is an examination of how the elements of the risk management system interrelate and integrate to meet objectives. For example, assessors should examine and evaluate the interrelationships and integration of the organizational objectives, value chain, business management, and risk management approaches.

When conducting the initial document review, attention should be given to:

Scope of the organization’s risk management system;
Parameters addressed by the risk assessment program;
Context of the risk environment;
The organization’s risk criteria;
Methodology and key outcomes of previous risk assessments;
Selection and effectiveness of risk treatment measures;
Internal audits and management review; and
Availability of current documents and responsible duties.

The document review should provide input into planning the second stage of the risk assessment: the on-site activities. The document review should provide indications of areas needing more focus and resources in conducting the second stage of the risk assessment, as well as the organization’s readiness for the second stage.

The document review will indicate the likelihood of achieving the risk assessment objectives and may indicate the need for changes in the assessment approach and team composition. Any changes should be made in consultation with the risk manager, RTL, client, and organization’s management.

The second stage of the risk assessment consists of information and evidence gathering to substantiate risk assessment outcomes. It should consider:

Are all requirements of the risk management system effectively implemented and achieving the policy and performance objectives set forth by the organization?
Are issues identified in the risk assessments effectively being addressed and are they consistently reflected through all the elements of the risk management system?
Are legal, regulatory, and contractual obligations being met?
Is management committed and leading by example?
Has the organization acted on identified risks, internal audit findings, exercise results, and lessons learned from events by implementing appropriate corrective and preventive actions?
Is there a change management mechanism?

5.5.7 Decision Models
Decision-makers need to evaluate alternatives in terms of values and uncertainty in assessing risks. Decision analysis provides insight into how the defined alternatives differ from one another and provides a basis for considering new and improved alternatives. This involves understanding the foundation of values used for probabilities, the value functions for evaluating alternatives, the value weights for measuring the trade-off objectives, and the risk preference. The risk manager should evaluate the sensitivity of the outcomes, weigh the reliability for key probabilities, and assess the weight and risk preference parameters.

Scenario analysis is a process of analyzing possible and plausible future events by considering alternative scenarios and outcomes. It provides a basis for making decisions in the context of the different conditions and outcomes. Creating scenarios challenges assumptions about what may and may not happen. Basing decisions and plans on more likely scenarios helps determine if decisions are reasonable even if conditions and circumstances change. Developing and evaluating alternative scenarios reduces uncertainty in decision-making and elucidates unknowns that may occur.

Alternatives can also be evaluated using the Pareto analysis which assumes for risk events that roughly 80% of the effects come from 20% of the causes. It is a simple technique for prioritizing possible changes by identifying the problems that will be resolved by making these changes. This allows the risk manager to focus on the most effective areas of risk assessment while downplaying the rest. For example, Pareto analysis can help organizations identify the proportion of goods and suppliers on which it is most dependent in terms of cost, value creation, production, and failure, and hence the goods and services that can pose the most risk to the organization and its supply chain. Pareto analysis is designed for users to identify which small set of practices, functions, suppliers, staff, etc. have the greatest impact. However, it can be limited by its exclusion of problems which may be small initially, but may grow with time.

5.5.8 Influencing Factors
Stakeholder influence, such as the impact of government regulation, generally is fairly obvious when assessing risks. However, there are individuals or groups within an organization's sphere of activity or geographic space that exert less obvious conforming influences on it.

Using influence diagrams can help identify the strength of these influencing factors and help the RLT determine potential weighting for consideration.

At their simplest and most basic, influence diagrams are a representation of the influencers on objectives and risks. Charting influencing factors and what they impact can offer critical insights. The diamonds in the diagram represent influential variables and the connections indicate varying levels of dependence (see Figure 4). The higher the number of connections reveals a high dependency node. Tracing these dependencies can lead to greater understanding of how multiple influencers may affect performance.

Figure 4 Influence Diagram Example.jpg

Figure 4: Influence Diagram Example

5.5.9 Managing and Reporting Program Outcomes
The risk manager is responsible for review and approval of the assessment findings and the final risk assessment report. For credibility, any recommendation for changes should come from the assessment team and be re-submitted for approval. In addition the risk manager is responsible for:

Appropriateness of corrective and preventive actions for non-conformities in the risk assessment process;
Ensuring the distribution of the risk assessment report to authorized parties only;
Maintaining the confidentiality of sensitive and proprietary information; and
Assuring proper risk assessment follow-up where necessary.

5.6 Monitoring the Risk Assessment Program

5.6.1 Monitoring Measurement Evaluation of Program Performance
The risk manager should establish performance metrics and measure the effectiveness of the risk assessment program. Performance metrics should be used to evaluate the performance of both the overall risk assessment program as well as individual risk assessments. Performance monitoring and evaluations should include:

Response and implementation of corrective and preventive actions for identified nonconformances in the risk assessment process;
Achievement of risk assessment objectives;
Value-added for the organization and client;
Risk-based management;
Time management;
Resource management;
Ability to achieve objectives and implement individual risk assessment plans;
Competence and professionalism of assessment team members; and
Effectiveness of communication between all parties involved in the risk assessment.

5.6.2 Evaluating Program Risk Management Outcomes
The risk manager and RTL should revisit the risks identified during the risk assessment process of both the risk assessment program and individual risk assessments to determine if the identified risks have been adequately managed and if any risk emerged that were not previously identified.

5.6.3 Nonconformity, Corrective, and Preventive Action
The risk manager should establish, implement, and maintain procedures for dealing with nonconformities and for taking corrective and preventive action for issues identified in the conduct of the risk assessment program. The procedures should include:

Identifying and correcting nonconformities and taking actions to mitigate their consequences;
Evaluating the need for actions to prevent nonconformities and implementing appropriate actions designed to avoid their occurrence;
Investigating nonconformities, determining their root causes, and taking actions in order to avoid their recurrence;
Recording the results of corrective and preventive actions taken; and
Reviewing the effectiveness of corrective and preventive actions taken.

5.6.4 Risk Assessor Competence and Skills Improvement
Assessors should enhance their knowledge, skills, and competence through continuing professional development. The RTL should evaluate the performance of all the members of the assessment team, with the risk manager evaluating the RTL. Evaluations should recognize both strengths and weakness to help with assessor selection for future risk assessments.

The RTL and risk manager should provide feedback to assessors, particularly assessors-in-training, to help them enhance and maintain their proficiency. Evaluations should consider:

Personal behaviors and professionalism;
Communication skills;
Interactions with other team members and the client;
Ability to follow instructions;
Strengths and weaknesses at accomplishing specific assessment tasks and assignments;
Knowledge and evaluation skills related to the assessment criteria and any discipline specific management system standards;
Risk-based management knowledge; and
Industry sector expertise.

5.7 Review and Improvement

5.7.1 Adequacy and Effectiveness
The risk manager should review the risk assessment program to assess whether the risk assessment objectives are being met and to ensure the program’s continuing suitability, adequacy, and effectiveness. Reviews should include assessing opportunities for improvement and the need for changes in the risk assessment program.

Risk assessment program review should include a review of:

Appropriateness of objectives, criteria, and scope;
Effectiveness of risk assessment and treatment measures of the risk assessment program;
Conformity to risk assessment program procedures;
Effectiveness and accuracy of risk assessment methods;
Resource allocations (including human resources);
Maintenance of records and documentation; and
Protection and integrity of information.

5.7.2 Need for Changes
The risk manager should monitor the context of the risk assessment program and manage change. Factors that may trigger the need for changes in the risk assessment program include changes in the:

Needs, perceptions, and expectations of stakeholders and other interested parties;
Organizational structure, governance, or business models;
Risk related to impartiality and conflict of interest (real and perceived);
Risk environment of the client and the assessors;
Sector and industry trends, including identification of accepted industry practice;
Legal and regulatory requirements;
Skills required for effective assessing of risk; and
Availability of resources.

5.7.3 Opportunities for Improvement
The risk manager should review the overall implementation of the risk assessment program to identify areas for improvement. Continual improvement and risk assessment program maintenance should reflect changes in the risks, activities, and operation of the program that will affect the achievement of objectives. The risk manager should ensure that any risk assessment program problems and their root causes have been identified and that corrective measures have been initiated to prevent or minimize recurrence. Any changes resulting from implementing opportunities for improvement that will impact the on-going risk assessment program should be identified by the RTL to the client or organizational top management prior to implementation to ensure their understanding of potential benefits and any consequential process changes.

The risk manager should address issues related to improvement of the risk assessment program implementation and the improvement of assessor competences. When appropriate, request for client feedback for possible risk assessment process improvements may be considered.

Next: Performing Individual Risk Assessments

Risk Assessment