ATTENTION: This page is intended to be viewed online and may not be printed or copied.

6. Performing Individual Risk Assessments

6.1 General

This section focuses on individual risk assessments, both the preparation for, and the execution of these risk assessments. Depending on the scope of the assessment, not all provisions in this section are applicable to all risk assessments.

Risk assessments can be conducted by an internal team, external team, or combination depending on the needs and resources of the organization and depth of expertise. A risk assessment often follows the order described in this section. However this is not always the case depending on the circumstances of the assessment, particularly the definition of assessment objectives.

6.2 Commencing the Risk Assessment

6.2.1 Setting Objectives
Objectives of the individual risk assessment should be clearly understood and documented in order to focus tasks, resources, and goals of the assessment activities. All risk assessments should include an analysis and evaluation of the effectiveness of current risk treatment measures and opportunities for improvement. Objectives are set within the context of achieving the organization’s overall business and risk management objectives. Objectives should be anchored in key value drivers. In defining the objectives for individual assessments, consider:

• Nature of the organization’s objectives;
• Events that could affect the achievement of enterprise-wide objectives (positively or negatively);
• Clear outcomes to achieve from the assessment;
• Use of the risk assessment outcomes and dissemination of results;
• Risk categories to be considered;
• How the individual assessment relates to the overall risk assessment program;
• Current control measures to manage risk and to protect tangible and intangible assets;
• Indicators for measuring risk levels;
• Timeframes for the risk assessment; and
• Resources needed to achieve the objectives.

Objectives of individual assessments may be broadly defined to consider enterprise-wide strategic or operational requirements; or more narrowly focused to consider risks related to specific products, activities, process, or functions. The objectives can consider issues related to the organization and/or all or part of its supply chain (however, in today’s world few organizations are not affected by their supply chain and dependencies).

Individual risk assessments may identify, analyze, and evaluate risks related to one or more issues contributing to uncertainty in achieving the organization’s objectives, including (but not limited to):

• Mission and strategic vision;
• Operational aspects (e.g., people, processes, and systems);
• Legal and regulatory compliance and ethical practices;
• Contractual obligations;
• Ability to meet project objectives;
• Product design, development, manufacturing, distribution, use, and disposal (including services);
• Information access, protection, and use;
• Brand and reputation;
• Financial, credit, and market factors;
• Security (tangible and intangible asset protection);
• Safety issues;
• Undesirable and disruptive events (e.g., criminal activities, natural disasters, technology failures, mismanagement);
• Socio-economic and political factors; and
• Supply chain and dependencies (upstream and downstream).

Once defined, the objective(s) of the individual risk assessment should be written in a concise statement and referred to in defining the scope, assumptions, procedures, and outcomes.

6.2.2 Identification of Stakeholders
Stakeholders should be identified who are the internal and external risk-makers and risk-takers. A stakeholder is any individual or organization that is directly or indirectly involved with or affected by an organization’s decisions and activities. Clearly identifying the internal and external stakeholders should be conducted in tandem with identifying uncertainties in achieving the organization’s objectives and protection of its tangible and intangible assets.

Examples of stakeholders include (but are not limited to):

• Internal
  
  1. Persons working on behalf of the organization, such as employees (and their families);
  2. Business owners/partners;
  3. Boards of directors;
  4. Trustees;
  5. Management (enterprise-wide as well as organizational units and functions level);
  6. Labor unions and workers’ associations; and
  7. Onsite contractors/vendors.
• External
  
  1. Customers/clients: present and potential;
  2. iContractors/vendors/distributors;
  3. Investors/shareholders/donors/venture capitalists;
  4. Competition;
  5. Financial institutions and creditors;
  6. Trade associations and international consortium;
  7. Analysts;
  8. Civil society and non-governmental organizations (NGOs);
  9. Media;
  10. Government and regulatory agencies;
  11. Local law enforcement;
  12. Emergency responders; and
  13. Surrounding communities and community leaders.

6.2.3 Identification of Internal Context and Variables
In setting the parameters of a risk assessment, consider the interrelated conditions in which objective(s) exist or occur, as well as what the variables might be. Establishing the internal context involves understanding how the following interrelated conditions apply to assessing risks:

• Capabilities of the organization in terms of resources and knowledge;
• Information flows and decision-making processes;
• SWOT Analysis (Strengths, Weaknesses, Opportunities, Threats);
• Performance metrics and key performance indicators;
• Internal stakeholders;
• Objectives and the strategies that are in place to achieve them;
• Perceptions, values, and culture;
• Policies and processes;
• Standards and reference models adopted by the organization; and
• Structures (e.g., governance, roles, and accountabilities).

The value drivers and varying perceptions of risk-taking held by the internal and external stakeholders should be understood. Variables to be considered in assessing the risks these stakeholders perceive might include:

• Likelihood: The frequency, relative frequency, or probability;

• Severity: The impact of the consequence (may be expressed in multiple terms: financial, human, reputation, property, ability to continue operations, etc.);

• Opportunities;

• Timing: Speed to onset (velocity), when the event/trend occurs (trigger), how long it lasts (duration), and resumption of operations (recovery time);

• Vulnerability: susceptibility related to the entity’s preparedness, agility, and adaptability;

• Expected value: Mean, mode, or median for forecasts, budgets;

• Variability: Range, standard deviation, and probability distribution;

• Ratios: How much of one thing there is compared to another thing;

• Capacity and resiliency (i.e. the capacity of the organization to adapt to a changing environment);

• Controllability;

• Visibility (for monitoring);

• Interdependencies;

• Readiness; and

• Degree of confidence and reliability of the assessment and each of its variables.

6.2.4 Documenting Assumptions
Assumptions are an integral part of assessment and problem solving. When conducting risk assessments, assumptions (both one’s own and others) should be clearly defined and documented. A risk assessment can potentially misinterpret information if the assumptions are not clearly understood. Furthermore, reviewing risk assessment outcomes is not reliable unless they are considered within the context of the assumptions made by the risk assessment team. Interpretation of data and evidence by different stakeholders is shaped by their assumptions. Therefore, assumptions should be identified, clearly stated, justified, and documented.

Assumptions are often linked to an individual’s perspective and point of view. They provide a window into how the persons conducting the risk assessment perceive and interpret the evidence and data gathered. Persons conducting the risk assessment should consider:

• What are the assumptions based on?
• How are the underlying assumptions of risk assessment impacting the outcomes?
• How is the assumption affected by the level of uncertainty?
• Are the assumptions a reflection of the assessors’ biases?
• Are assumptions that something is a “given” based on opinions or evidence?
• How do the assumptions affect the confidence in the interpretation of evidence?
• Are assumptions about likelihood balanced by potential consequences in achieving objectives?
• Could the assumptions be different if made by another individual?
• Would the outcomes be different if they were based on different assumptions?
• Were the assumptions made when setting the risk criteria still valid in light of the evidence and data gathered?

Identifying a potential risk event as a risk provides a basis to put a proactive plan in place to manage the risk. All risk assumptions should be monitored and validated throughout the project to ensure a continued understanding of their nature.

6.2.5 Defining Scope and Statement of Work
The scope may be enterprise-wide or limited to an organizational unit, geographic location, product flow, or a particular activity, function, or source of risk. The scope defines the boundary conditions of the individual risk assessment (what is in and out of the assessment). As with any project, scope is a function of needs, resources, authorities, and time.

Care should be taken not to over-scope or under-scope the risk assessment. When defining the boundaries of the assessment, the scope should be in sync with the objectives and needs of the client, as well as the objectives and scope of the overall risk assessment program. Under-scoping may result in some organizational objectives, assets, stakeholders, or threats being overlooked. Under-scoping may result in tunnel vision with regard to the interaction of sources of risk (e.g., the close interaction between physical and cyber security when assessing the organizational objectives related to information security). Over-scoping may result in a waste of time and resources without being able to provide enough focus to the needs of the client.

A scope statement should be prepared clearly defining the boundaries of the risk assessment. This should include a statement of work highlighting what are the organizational, physical, operational, logical, and risk disciplines included in the boundaries so to explicitly delineate what is in and what is out of the risk assessment.

The RTL should obtain from the client verification or permission and access to conduct the risk assessment within the approved scope.

During the course of the risk assessment, the RTL should notify the client if any significant conditions exist outside of the scope of the assessment that otherwise may impact risk to the organization or constitute an additional risk.

6.2.6 Policy and Management Commitment
Prior to commencing any on-site risk assessment activities, the RTL should obtain the appropriate authorization and support of the client and/or top management in the form of a policy statement. The policy statement should include statements of:

• Risk assessment objectives, scope, and timing;
• Importance of assessment to the organization being assessed;
• Clear authorization to conduct the assessment within the stated scope;
• Need for confidentiality and information integrity;
• Client and/or top management commitment to engage in setting criteria and reviewing output;
• Commitment of persons working on behalf of the organization to share information with assessors; and
• Commitment of the client to communicate the importance of participation in the risk assessment to persons working on their behalf within the scope.

6.2.7 Commitment of Resources
The RTL should obtain the appropriate resources from the client and/or top management to conduct the risk assessment activities. If the RTL determines that there is insufficient time and resources allocated to conduct the assessment, the client should be notified. If additional resources cannot be secured, then the objectives and scope of the assessment should be modified accordingly with the agreement of the client.

6.3 Planning Risk Assessment Activities

6.3.1 Gap Analysis
A gap analysis is a technique that can be used to determine what steps might need to be taken to improve the organization’s capacity to conduct a risk assessment to move from a current state to a desired, future state. Also referenced as need-gap analysis, needs analysis, and needs assessment, gap analysis seeks to answer the questions: "where are we?" – the current state; and "where do we want to be?" – the future state. The gap analysis includes an evaluation of the suitability of the current process for assessing risk and if it is sufficient to manage risks. Gap analysis can also be used within the individual risk assessment.

Gap analysis consists of three steps:

• Noting currently available factors, such as abilities, competencies, time, and performance levels given the current resource situation ("what is");

• Listing success factors needed to achieve future, desired objectives ("what should be"); and

• Highlighting the gaps - that is, the amount by which the need exceeds the resources - that exist and what gaps may need to be filled to be successful (“what to consider”).

6.3.2 Legal and Other Requirements
When conducting individual risk assessments, the RTL should review the legal and other requirements discussed in section 5.4.2 relative to the objective and scope of the individual assessment.

6.3.3 Objectives, Targets, and Strategies

6.3.3.1 General: Objectives, Targets, and Timelines
A challenge in conducting risk assessments in order to achieve the objectives is time. The RTL needs to develop an assessment strategy, or “path”, to collect data in a representative, logical, and methodical manner. Effective planning is necessary to make efficient use of time to provide an informative risk assessment. Depending on the desired outcomes for the risk assessments and whether the scope is enterprise-wide or limited to a specific area, process or project, reasonable targets and timelines should be established within the constraints of available resources and funding.

6.3.3.2 Analysis Approach Using a Risk Portfolio Design Format
An entity’s risk portfolio is a complete collection and range of uncertainties that affect an organization’s future. Risk portfolios sometimes are referenced as a “risk universe”. In essence, it is an “uncertainty” portfolio based on the organization’s internal and external context, timeframe and strategic and operational objectives. It is a generally accepted principle that a risk portfolio is designed according to the entity's risk appetite, risk tolerances, timeframes and return objectives. The expected value of each objective may influence the risk/reward ratio of the entire portfolio. The potential impact of each risk may influence other risks as well as the overall objectives of the planned strategy. Certain risks may hedge other risks naturally which may alter the overall response/control allocation.

In the example given in Figure 5, the risk portfolio for the internal context is categorized into three areas for which the organization may have certain objectives: strategic, operations, and financial. A fourth category is used for the external context. While this type of design format does not address the range or impact of the risks, nor the interconnectedness of the risks within the risk portfolio, it does provide an approach for the risk assessment team to consider the breadth and depth of penetration the assessments should cover given the general objectives, targets, and timelines contemplated.

Copyright © 2013. Risk and Insurance Management Society, Inc. All rights reserved.

Figure 5: Risk Portfolio Design Format

In this type of design, there may be sub-risks within each of the subcategories. For example, under Operations/Infrastructure, one might include Information Technology, which could have further sub-risks such as technology infrastructure, under processes with sub-risks such as data availability, data integrity, data/privacy, systems development, and systems implementation, and under Talent with sub-risks such as skill requirements.

6.3.3.3 Analysis Methodology
Risk assessment methods, definitions and goals may vary widely according to whether the risk management technique is being used in the context of operational, project, strategic, or other risk management environments.

Similarly, outcomes and solutions may differ depending on whether risk assessments are used in operational, project or strategic settings (see Figure 6).

Adapted from 2012 RIMS Conference presentation by Joanna Makomaski. Copyright © 2012. Risk and Insurance Management Society, Inc. All rights reserved.

Figure 6: Managing Uncertainty in Context

For example, operational risk assessments may be limited to uncertainties associated with existing operations and operational plans – the assets, processes, people, and systems in place – in order to deliver a particular outcome from the organization’s operations, such as planned earnings. Project risk assessments typically are used to assess uncertainties and potential consequences related to expected outcome(s) of a particular project or initiative, such as delivering the project within the planned time, budget and scope. Strategic risk assessments, on the other hand, focus on the broader deliberation and actions regarding uncertainties and untapped opportunities that affect an organization’s planned strategy and strategy execution, such as growth (e.g., opening new markets) or contraction objectives (e.g., eliminating certain product or service lines).

Each assessment method has its benefits and drawbacks. When choosing a particular methodology, the assessors first need to consider the goal and objectives outcome that the organization is seeking, view the risks as deviations from that outcome (whether positive or negative) before evaluating available solutions and recommending actions that best suit the entity’s overall risk profile (position) and desired level of risk.

6.3.4 Data Gathering
It is the assessment team’s responsibility to collect factual information/evidence and analyze it against the risk criteria. Based on the factual evidence, the assessment team will determine its findings. Reliable information should be used as evidence. The assessment team should have a well-developed data collection strategy and sampling plan to ensure the gathering of comprehensive information that reflects the scope of the risk assessment. In dynamic, unstable, or uncertain conditions where evidence may be limited, indirect and corroborating information should be sought to proximate information reliability.

Information and data can be gathered from various sources, including (but not limited to):

• Review of documents, performance indicators, and records;
• Websites and databases;
• External reports (e.g., Industry publications, crime statistics, and government reports);
• Interviews with persons (internal and external stakeholders);
• Subject matter experts;
• Physical evidence; and
• Observation of operational processes.

The RTL, in consultation with assessment team members, should determine how much information needs to be gathered. When developing a sampling plan it is important to keep in mind that the assessment is trying to find systematic weaknesses and opportunities for improvement and not just isolated occurrences. Sampling examines selected items and elements from the overall population. The method of sampling should be defined and documented using sampling practice and procedures appropriate for the data collection objectives. If contradictory data is collected or possible systemic problems are identified, the sampling size may be increased to determine if there is a trend or pattern. See Annex C for more information on sampling.

6.3.4.1 Types and Methods
Gathering data is the first step in the risk assessment process of finding, recognizing, and recording risks. The purpose of data gathering is to identify what might happen or what situations might exist that may affect the achievement of the objectives of the system or organization. The process includes identifying the causes and sources of the risk, events, situations, or circumstances which may have a material impact upon objectives, and the nature of that impact. Such methods can include:

• Evidence-based methods, examples of which are observations, interviews, checklists, and reviews of historical data;

• Systematic team approaches where a team of experts follow a systematic process to elicit risks by means of a structured set of prompts or questions;

• Inductive reasoning techniques such as scanning, scenario analysis, key performance indicator(s), and event tree logic diagrams; and

• Other methods that may not fall directly into one of the three methods noted above.

Various techniques can be used to improve the accuracy and completeness of data gathering for assessment purposes. Irrespective of the actual techniques employed, it is important that in the overall risk assessment process recognition is given to human and organizational factors.

Interactions during the course of the assessment between the assessment team and those who best understand the risks facing the organization may take a number of forms. In identifying and evaluating the risks that are relevant and important to the organization’s objectives, the assessment team may gather data by exploring procedures, processes, activities, technologies (including information systems), and the interaction between human and technological performance. Gathering data can be accomplished through direct contact or through indirect review.

Direct contact – between stakeholders and the assessment team, such as:

• Conducting interviews (in-person, telephone, or on-line) including completing surveys and questionnaires with stakeholder participation;

  • Open-ended questions (structured interview).

  • Close-ended questions (checklists).

• Conducting document reviews with stakeholder participation;

• Brainstorming in group sessions, involves stimulating and encouraging free-flowing conversation amongst a group of knowledgeable people to identify potential failure modes and associated risks;

• Facilitated workshops, or Delphi methodology, a means of combining expert opinions that may support source and effects identification, likelihood and consequence estimation, and risk evaluation. It is a collaborative technique for building consensus involving independent analysis and voting by experts;

• Scenario co-development; and

  • What If Questions.

  • Scenario analysis - a process using descriptive models to ascertain and analyze possible events that may occur in the future and their potential outcomes. It can be used to identify risks by considering possible future developments and exploring their implications. Sets of scenarios reflecting (for example) ‘best case’, ‘worst case’ and ‘expected case’ may be used to analyze potential consequences and their probabilities for each scenario as a form of sensitivity analysis when analyzing risk.

  • Exercising (e.g., table-top, war gaming, red-teaming, and adversary path development).

• Envisioning multiple potential outcomes.

Indirect review – assessment team review of available data and documentation, such as:

• Conducting document repository reviews (e.g., loss data and near-miss records, customer satisfaction reports, internal audit, security, and management reports);
• On-line risk survey results;
• Industry and analyst reports;
• Competitor risk factors in business reports (e.g., 10ks);
• Publicly available risk surveys;
• Meteorological and geological data and reports;
• Technical disaster and failure reports; and
• Media sources.

Methods used in analyzing risks may be qualitative, semi-quantitative or quantitative. The degree of detail required will depend upon the particular application, the availability of reliable data and the decision-making needs of the organization. Some methods and the degree of detail of the analysis may be prescribed by legislation. Additional discussion of risk identification sources may be found in section 6.4.4 Implementation.

6.3.4.2 Error Analysis
When conducting risk assessment it is important to take into consideration that certain information and measurements are subject to uncertainties. In order to draw valid conclusions the error must be understood, indicated, and dealt with properly. Error analysis considers the kind and quantity of error that may occur. An indication of how accurate the results and level of uncertainty should be included with the conclusions of the risk assessment.

Errors arise from measurement and sampling inaccuracies as well as inherent variability of complex natural, human, physical, social, and economic factors, some of which are subject to random influences. It is important to differentiate when conclusions are based on objective or subjective probabilities and/or if variability is the result of inherent fluctuations. When events are described subjectively they should be based on best available data, insight, and judgment.

Sampling errors can be either systematic or random. Systematic errors are inaccuracies which tend to shift all measurements in a systematic way so that their mean value is displaced. This may be due to such things as biased questionnaires or preconceived notions of the person conducting the measurements. Random errors are inaccuracies which fluctuate from one measurement to the next. They yield results where the mean value varies. They can occur for a variety of reasons including a lack of focus of questions, imprecise definition of terminology, or lack of sensitivity of the analysis models.

Many statistical methods exist for quantifying error. The key to understanding the reliability and level of confidence in risk assessment outcomes is to clearly understand the sources of error and the extent of their influence on the conclusions.

6.3.4.3 Sensitivity Analysis
Sensitivity analysis is any systematic technique used to understand how risk estimates and risk-based decisions are dependent on variability and uncertainty in the factors contributing to risk. Sensitivity generally refers to the variation in output of a risk analysis model with respect to changes in the values of the model’s inputs. Sensitivity analysis attempts to provide a ranking of the model inputs based on their relative contributions to model output variability and uncertainty.

There are many analytical methods that may be referred to as sensitivity analysis, some of which are very simple and intuitive. At its simplest, the assessor may compare outcomes from an analytical approach using different input assumptions and values. Evaluate the level of uncertainty by using a different plausible estimate for each calculation. Sensitivity analysis can also involve more complex mathematical and statistical techniques to determine which factors in a risk analysis model contribute most to the variance in risk estimates. Complexity generally is due to the fact that multiple sources of variability and uncertainty are influencing the estimate at the same time, rather than acting independently.

When making decisions based on the sensitivity analysis, the following should be considered:

• Most systems are dynamic;
• Previous assumptions and values may not apply to changing conditions;
• Model outputs may be very sensitive to certain parameters and assumptions (particularly subjective likelihood estimations);
• Model parameters may better describe some risks better than others; and
• Complexity of the model may actually be sensitive to multiple variables.

It should be kept in mind that risk analysis models are based on certain assumptions and premises; therefore, the analysis is only as accurate as the reliability of the variables and parameters used.

6.3.4.4 Stress Analysis
Stress tests are a form of simulation used to determine reactions to different situations. Stress tests are also used to gauge how certain stressors will affect a company or industry. Computational analysis is used to test hypothetical scenarios for stress testing to evaluate the effect of uncertainty on systems in a wide range of situations. They are typically used to evaluate the range of possible outcomes and the relative frequency of values in that range for quantitative measures of a system such as cost, duration, throughput, demand and similar measures. Such simulations may be used for two different purposes:

• Uncertainty propagation on conventional analytical models; and
• Probabilistic calculations when analytical techniques do not work.

In general, computational analysis simulations will be used to assess either the entire distribution of outcomes that could arise or key measures from a distribution such as:

• The probability of a defined outcome arising; and

• The value of an outcome in which the problem owners have a certain level of confidence that it will not be exceeded or beaten, a cost that there is less than a 10% chance of exceeding, or a duration that is 80% certain to be exceeded. An analysis of the relationships between inputs and outputs can throw light on the relative significance of the factors at work and identify useful targets for efforts to influence the uncertainty in the outcome.

Performance testing differs from stress testing. Performance testing is conducted within the “normal” operating environment, whereas stress testing occurs at maximum capacity and outside of “normal” parameters.

6.3.5 Review of Documentation
Before performing the risk assessment, the RTL should obtain initial documentation about the organization to be assessed in order to prepare for the assessment activities. The RTL and assessment team should review relevant documents to determine the risk assessment activities and better understand the client and organization. This includes organizational policy documents, mission statements, company profiles, organizational structure, management system(s), and industry practices. It also includes information related to products, services, processes, and activities, as well as understanding the geographic extent, interactions, and dependencies.

The RTL should obtain any risk management system descriptions, including manuals, for study by the team. Previous risk assessment reports are also useful but should not bias current assessment efforts. Proprietary concerns and non-disclosure agreements may need to be addressed. Document review should examine the scope and policy statements of the client’s risk management system to check consistency with the risk assessment objectives, criteria, and scope. Any inconsistencies should be clarified with the client.

Sufficient documentation should be obtained in preparation of the risk assessment to determine if the risk management system is properly designed and if there are any significant gaps that would indicate the risk management system is neither complete or being properly maintained.

6.3.6 Preparing the Risk Assessment Plan
The RTL prepares a risk assessment plan based on objectives, scope, and criteria in the risk assessment program and the documentation and information provided by the client. The risk assessment plan should be reviewed and accepted by the client according to the stipulations of the risk assessment program. The risk assessment plan should be presented to the client prior to onsite activities. Any issues raised by the client to the risk assessment plan should be resolved between the RTL and the client.

The risk assessment plan identifies, where relevant:

• Objectives and scope of the risk assessment;

• Risk assessment criteria such as risk criteria, standards, contracts, regulations, manuals, and reference documents to be used in the risk assessment;

• Follow-up activities from previous risk assessments;

• The client, management representative, guides, and the divisions, facilities, and functions to be assessed;

• Assessment team members (e.g., RTL, assessors, technical experts, observers), their competencies, roles and responsibilities;

• Allocation of appropriate resources;

• Risk assessment logistics including date and place of the risk assessment, travel, lodging, and risk assessment facilities;

• Timeframe and overall schedule of risk assessment activities;

• Communication procedures including meetings with client and assessment team;

• Risk assessment methods including evidence collection and sampling methods;

• Risks identified related to the risk assessment, the client, organization, and assessment team;

• Confidentiality, safety, health, and security measures;

• Conditions that warrant stopping the risk assessment;

• Language of the risk assessment and report;

• Risk assessment report topics; and

• Specific exclusions and assumptions.

The risk assessment plan should:

• Provide the basis for the agreement with the client for the conduct of the risk assessment;
• Consider the effect that the risk assessment activities may have on the client and its functions;
• Facilitate efficient communication, coordination and scheduling of the risk assessment activities to most efficiently and effectively achieve the objectives;
• Take into consideration the competence and composition of the assessment team (including whether technical or security experts are needed); and
• Outline appropriate assessment methods and practices (e.g., sampling and interview techniques).

The complexity and scope of the risk assessment and the confidence level of achieving the risk assessment objective, determines the amount of detail needed in the risk assessment plan. The scope of the risk assessment may differ between initial and following assessments. The risk assessment plan should include some room for flexibility to allow for changes as the risk assessment progresses.

6.3.7 Establishing the Risk Assessment Team
The RTL delegates responsibility to each team member regarding the specific processes, activities, locations, and functions of the risk assessment. When delegating the roles and responsibilities, the individual assessment team members’ competencies, strengths, and weaknesses are taken into consideration, as well as the effective use of resources.

Assessment team briefings are held to best ensure that the risk assessment objectives are achieved. This can be done by allocating work assignments and deciding upon possible amendments. The frequency of assessment team briefings is determined by the RTL.

Throughout the risk assessment, the assessment team should be aware of changed and new circumstances or risks. Assessors should notify the RTL who should consult with assessor team members to address these changes in order to achieve risk assessment objectives. The RTL should communicate to the client any identified significant risks (particularly threats to health, safety, and security of the assessment team and client’s organization) as well as any significant changes from the risk assessment plan.

6.3.8 Determining Feasibility
The RTL should determine the feasibility of achieving the risk assessment objectives. If the risk assessment is considered feasible there should be reasonable confidence that the risk assessment objectives can be realized. If the assessment is not feasible, the RTL should notify the risk manager, client, and organizational management. Risk assessment preparation should be suspended until all parties agree to subsequent changes.

Within the defined scope and objectives, factors that contribute to the feasibility of the risk assessment include:

• Adequate resources committed to the risk assessment;
• Adequate time within scheduling constraints;
• Availability of assessment team personnel with the mix of characteristics, competences, and necessary clearances;
• Cooperation with the client and a conducive work environment;
• Access to adequate and relevant information for preparing and conducting the assessment;
• Logistical expenses; and
• Language requirements.

6.3.9 Documentation and Document Control
The organization should establish and maintain records to support risk assessment activities. All assessment documentation should state when it was conducted, produced, period of time covered, dates of any revisions or addendums, and the assessors who contributed to, produced, and authorized the document.

Documentation may include, among others:

• Records required by the client and organization;
• Objectives, scope, criteria, and assumptions;
• Stakeholders consulted in risk assessment process;
• Asset characterization and identification;
• Risk assessment methodology used;
• Inspection, maintenance, and calibration records;
• Pertinent subcontractor and supplier records;
• Incident reports;
• Records of incident investigations and their disposition;
• Risk assessment results;
• Management review results;
• External communications decision;
• Records of applicable legal requirements;
• Catalog of significant risks, likelihoods, and impacts;
• Personnel screening;
• Training records;
• Process monitoring records; and
• Communications with stakeholders.

The organization should establish, implement, and maintain procedures to protect the sensitivity, confidentiality, and integrity of records including access to, identification, storage, protection, retrieval, retention, and disposal of records. Records should be retained for a minimum designated period or as otherwise required or limited by law. The organization should establish, implement, and maintain procedures to:

• Approve documents for adequacy prior to issue;
• Protect sensitivity and confidentiality of information;
• Review, update as necessary, and re-approve documents;
• Record amendments to documents;
• Make updated and approved documents readily available;
• Ensure that documents remain legible and readily identifiable;
• Ensure that documents of external origin are identified and their distribution controlled;
• Prevent the unintended use of obsolete documents; and
• Ensure the appropriate, lawful, and transparent destruction of obsolete documents.

Organizations should ensure the integrity of documents by rendering them securely backed-up, accessible only to authorized personnel, and protected from unauthorized disclosure, modification, deletion, damage, deterioration, or loss.

6.4 Conducting Risk Assessment Activities

6.4.1 Preparing Work Documents
Assessment team members prepare work documents to facilitate and record their investigation and report its results. Work documents provide a flexible roadmap for conducting the assessment activities and record observations for risk assessment evidence. Work documents should show what was evaluated, how it was evaluated, what was examined, and what was observed. Work documents can include checklists, assessment sampling plans, and forms for recording information including assessment findings and records from meetings.

Well-prepared work documents can help improve assessment time management. The use of checklists, forms, process maps, and log sheets should provide structure for the various assessment activities. However, the use of checklists should not restrict what an assessor needs to do and should be flexible to consider changes that take place throughout the assessment. Through the RTL, the organization and client should be made aware that the use of checklists does not restrict what an assessor needs to do.

When developing the work documents, procedures should be specified for their retention, access, and the need to protect confidential and proprietary information. The integrity of the information should be ensured at all times.

Effective work documents should:

• Be tailored to the user;
• Indicate background information needed;
• Guide the assessor about what objective evidence needs to be examined;
• Record the process of evidence collection;
• Outline the types of questions to ask;
• Provide a means to record which parts of the program were sampled;
• Include space to document samples taken, documents reviewed, as well as record comments and observations;
• Provide evidence of the thoroughness of the assessment; and
• Be reviewed at the end of the assessment for effectiveness and improvement.

An example of an assessment checklist is to build a matrix listing the specific risks that the assessor wishes to verify for assessment and treatment. In the columns, the assessor can record evidence related to the criteria and additional notes:

• Risk criteria used and evidence related to criteria;
• Existing risk controls and their effectiveness;
• Level of criticality;
• Level of threat;
• Level of vulnerability;
• Level of risk;
• Documents;
• Finding of full treatment needs and/or opportunity for improvement;
• Conclusions; and
• Comments.

Checklists should be reviewed before each assessment to determine if they are still relevant and appropriate. When preparing checklists they should be designed to:

• Maintain clarity of the assessment’s objectives;
• Provide structure;
• Help ensure thoroughness;
• Maintain the rhythm and continuity of assessment;
• Reduce the assessor’s bias thereby increasing objectivity in evidence;
• Reduce the workload during assessing;
• Provides formatted evidence collection; and
• Provide a record of assessment and evidence collection.

Checklists should be revisited before each assessment to evaluate if they are appropriate for the job at hand.

6.4.2 Assigning Roles and Facilitating Communication Among Team Members
The RTL should make specific assessment assignments based on the competence of the individual assessors and the complexity of the risk assessment tasks. There should be a balance in the assessment team between technical, legal, industry, administrative, and risk management knowledge. The RTL should assign and communicate risk assessment responsibilities prior to commencing the assessment.

Formal channels of communication between the assessment team, client, and external bodies (where applicable) may be necessary during the assessment. This may be especially necessary where legal requirements require the mandatory reporting of certain risk and regulatory violations.

Communication within the assessment team should occur regularly to assess the progress of the risk assessment, reassign work among the assessment teams, and exchange information as needed. Frequency of the communication should be at least daily or based on the complexity of the assessment and the needs of the assessment team. Team briefings confirm the observations of the day’s assessment and provide the RTL the opportunity to clarify the assessment team member’s evidence and their interpretation. This is particularly important in cases where team members will not be on-site through the end of the assessment. If there is a concern about an issue outside the assessment scope, it should be noted and reported to the RTL. It is up to the discretion of the RTL to communicate the concerns with the client.

The progress of the assessment and any concerns regarding the assessment should be communicated by the RTL to the client preferably on a daily basis, or as needed. The purposes of the updates are to:

• Ensure the client is kept informed of the assessment progress and results;
• Solicit additional input and information from the client; and
• Make sure there are no surprises during the closing meeting.

If evidence collected during the assessment suggests or indicates an immediate and significant risk to the organization, client, or assessment team, the client should be informed of the risk without delay.

The RTL should report and provide an explanation to the client if the available assessment evidence suggests that the assessment objectives are unattainable. The RTL and client should determine the appropriate action (e.g., modify the assessment plan, change the assessment scope or objective, and terminate the assessment). The need for a change in the assessment plan may become apparent through the progression of the assessment and should be reviewed and approved by the client and risk manager, where appropriate.

6.4.3 Conducting a Pre-Assessment Meeting
The pre-assessment meeting typically initiates the data collection phase at the site of the risk assessment. The purpose of the pre-assessment meeting is to:

• Confirm the risk assessment plan – review the purpose, scope and outline of the assessment process and methods;

• Introduce the assessment team and meet counterparts of the organization or client participating in the assessment;

• Confirm and explain risk criteria;

• Confirm communication channels;

• Verify clearances and approval to conduct the risk assessment;

• Verify the feasibility of risk assessment activities; and

• Provide an opportunity for the client to ask questions about the assessment.

The RTL chairs the pre-assessment meeting. A designated assessment team member should record attendance and minutes. It should be held with the client’s management. Those who are responsible for the services, functions, or processes being assessed may be present as well.

The pre-assessment meeting should be as detailed as necessary to ensure everyone present understands the assessment process. The pre-assessment meeting is where, at a minimum, the nature of the assessment is explained. The formality of the meeting is dependent on the type of assessment being conducted.

The following items are appropriate for the pre-assessment meeting (where applicable):

• Introduction of members of the assessment team to client representatives, including experts, observers, and guides. Each of their roles should also be explained;

• Confirm the risk assessment plan - scope, criteria, reference standards, objectives, and methods used in the assessment;

• Confirm the logistics of the assessment including:

  • Schedules – especially site visits and meetings;
  • Communication channels between the client and the assessment team;
  • Language to be used during the assessment;
  • Issues of health and safety;
  • Review security and emergency procedures for the assessment team;
  • Any issues related to information security and confidentiality; and
  • An overall assessment schedule, showing topics, assessors, and approximate times to complete.

• Inform the client how the risk assessment findings will be reported including the method of grading non-conformities and method of presenting assessment findings;

• Confirm how the client will be informed of the progress of the assessment throughout the risk assessment;

• Confirm what resources and facilities will be made available to the assessment team;

• Express the conditions in which the assessment may be terminated;

• Explain possible ways to address the possible findings in the assessment; and

• Give information regarding the systems for feedback from the client on the results of the assessment, as well as the system for complaints and appeals.

The pre-assessment meeting sets the tone for the assessment and establishes a rapport between the client and the assessment team. The RTL should prepare an agenda for the pre-assessment meeting and project both knowledge and confidence in the assessment activities. Assessment team members should participate in the pre-assessment meeting only if called upon by the RTL.

6.4.4 Implementation

6.4.4.1 Risk Identification
Risk identification ascertains the sources and nature of risk and the effect of uncertainty on achieving the organization’s objectives. A thorough risk identification process will consider the myriad of uncertainties that may affect organizational objectives. These may include natural, intentional, and unintentional events such as malevolent, criminal, technical, institutional, logistical, logical, demographic, environmental, or social/political events. It is more than asking “what can go wrong” but also includes asking which risks may be pursued as an opportunity.

While different risk disciplines use a range of techniques for identifying the nature and sources of risk, they all should contain the following components along with an understanding of the interplay between these components for a comprehensive identification and characterization of the risks:

• Asset and service identification, valuation, and characterization;
• Threat and opportunity analysis;
• Vulnerability and capability analysis; and
• Criticality and impact analysis.

Risk identification can be conducted using qualitative or quantitative analyses or a combination of both. Regardless of the method of evaluation the assumptions, level of precision in estimating parameters, and reliability of information used should be noted.

Risk identification is part of a good business and risk management strategy. Therefore, when conducting the risk identification the business SWOT (strengths, weaknesses, opportunities, and threats) analysis should be consulted as a key input.

Identifying risks should answer the following questions:

• Why could something happen?
  
  • A cause or factor creating risk.
  • Effectiveness of risk treatments.
• Who could be involved?
  
  • Individuals or groups associated with threat, control of risk, and/or impacted by risk.
• How could it happen?
  
  • A source of risk.
• What could happen?
  
  • Potential event and likelihood.
  • Potential consequences and likelihood.
• When could something happen?
• Where could it happen?

Information sources related to the risk identification process include (but are not limited to):

• SWOT analysis;
• Business plans;
• Intelligence;
• Threat advisories;
• Meteorological and geological reports;
• Significant fluctuations in the availability and pricing of basic commodities such as food, water, and natural resources ;
• Previous, current, and emerging data and trends;
• Political, social, and economic trends;
• Insurance information;
• Internal and external stakeholders;
• Audit findings and exercise reports;
• Internal crime, loss, and risk event data;
• External risk event and crime data;
• Industry risk data;
• Law enforcement agencies;
• Government/international agencies;
• Industry associations;
• Media, internet, and public reports;
• In-house systems; and
• Informal and personal relationships.

Methods for soliciting input include (but are not limited to):

• Conducting an exercise;
• Scenario evaluations;
• Questionnaires;
• One-on-one structured interviews;
• Incident, exercise or audit reports;
• Brainstorming sessions;
• Group discussions;
• Workshops;
• Stakeholder and focus group discussions; and
• Expert testimonials (including public and private sector sources).

When conducting risk identification activities it should be noted that some risks are continuous and some vary with time. Additionally, threat, opportunity, vulnerability, and criticality levels may be time-dependent. At times, dependencies that will affect consideration of the level of risk and the need for treatment include:

• Duration of an event;
• Cultural context of time;
• Day, week, or month of the year;
• Time of day (e.g., break periods, business hours, shift work);
• Timelines for service and product delivery;
• Supply chain context of time; and
• Time restrictions on travel.

6.4.4.1.1 Asset Identification, Valuation and Characterization
Usually the preliminary step is to identify and evaluate sources of uncertainty in achieving organizational objectives. Asset characterization identifies what assets may be at risk, what is their criticality to the organizational objectives, and what are the potential consequences of those assets being compromised. Questions that should be answered:

• What are the activities, functions, and assets that contribute to achieving the organization’s objectives?

• What is the value chain of the organization and what are the activities, functions, and assets that contribute to the critical value generators?

• What is the tangible and intangible value of the asset for the organization and its supply chain?

• What are the dependencies of the organization’s activities and functions on the asset?

• Is there a potential for significant positive, neutral, or negative consequences related to the asset?

The loss of the most valuable assets or disruption of critical value generating activities and functions may result in unacceptable damage to the organization and/or disruption of dependent activities and functions. The value of an asset is frequently measured relative to more than one consequence. For example, minor harm to people may result in major harm to brand and reputation. Furthermore, when characterizing the activity, function, or asset, consideration should be given to its value relative to the organization and its supply chain, as well as its potential value to an adversary or competitor.

All activities, functions, and assets that contribute to achieving the organization’s objectives, and within the scope of the risk assessment, should be considered. Tangible and intangible assets include (but are not limited to):

• Internal and external human resources;
• Property (e.g., facilities, equipment, materials, products, physical systems);
• Process controls (physical and cyber);
• Financial and administrative processes (e.g., funds, inventory, accounting, and recordkeeping systems);
• Information and telecommunication systems;
• Transportation systems;
• Access to critical infrastructure and support utilities;
• Intellectual property and proprietary information; and
• Brand, image, and reputation.

After identifying activities, functions, and assets that contribute to achieving the organization’s objectives for each activity, function, and asset consider:

• Its contribution to the value chain of the organization and the achievement of objectives;
• The potential for risk to be exploited for the advantage of the organization;
• Severity and timeframes of the consequences if activities, functions, or assets were lost, or offer a potential opportunity;
• Critical infrastructures, dependencies, and interdependencies (internal and external);
• Functions and countermeasures that currently exist for protection and support;
• Criticality to value chain and achieving the organization’s objectives; and
• Priority and critical value relative to other activities, functions, and assets.

6.4.4.1.2 Threat and Opportunity Analysis
Sources of risk and related threats and opportunities should be identified and analyzed once priority and critical activities, functions, and assets have been identified. This will provide a basis for understanding what risk events may contribute to uncertainty in achieving the organization’s objectives. The process should consider both threats and potential opportunities.

Threat analysis considers impacts, timeframes, and factors that may prevent achievement of objectives. Unintentional events look at the possibilities of human error. Threats can be intentional and unintentional and may occur through errors of commission and omission.

Opportunity analysis typically looks at the potential for change that an organization might undergo to improve its overall results. Opportunities might increase the overall demand or discrete price points for its products and services, broaden or restrict its offerings, as well as increase efficiencies through expense reductions and operational improvements. Whatever the goal, undertaking an opportunity analysis helps to provide an understanding of what potential effects, positive and negative, are likely to take place if different decisions are taken.

Threat and opportunity analysis can be conducted using either quantitative, qualitative, or a combined approach. Regardless of the method, a common set of metrics and scales should be defined so that the calculations can be performed and reported using consistent scales and parameters. Comparisons will only be valid if values are determined using the same methods and metrics. All priority and critical activities, functions, and assets should be analyzed.

Sources of risk give rise to potential threats and opportunities. Threat and opportunity analysis sets the boundaries as to the type of threats and opportunities that can be addressed, therefore, the range of risk sources associated with the achievement of organizational objectives should be considered. Threat and opportunity analysis often contains subjective estimates, therefore the confidence in the predictions should be considered within the context of the reliability of the information. Likelihood estimates are particularly sensitive to the information and assumptions they are based on.

Using the output from the asset identification, valuation and characterization, consider sources of risk that create uncertainty in achieving the organization’s objectives. Consider both intentional and unintentional risk events that may affect the achievement of the organization’s objectives (natural and man-made hazards; social, economic, and political factors; as well as actions with mal-intent). Determine what are the threats and/or opportunities associated with potential risk events. The output of the threat and opportunity analysis assessment should be comprehensive list of threats and opportunities focusing on prioritizing the most relevant to the achievement of objectives.

Threats may be identified in terms of “threats from” and “threats to”. “Threat from” is based on the nature and attributes of the threat and how the threat may cause harm and/or uncertainty. “Threat to” considers the locations of the potential assets and services. In assessing the threat, the nature of the threat should be considered (e.g., is it malevolent, naturally occurring, or accidental). For a malevolent threat the assessment should consider “who/why” (e.g., description of the adversary), “what” (e.g., the material used by the adversary), and the “how/when/where” (e.g., the characteristics of scenario and related tactics).

Malevolent threat is assessed by evaluating the combination of motivation/intent and capability of an adversary to impact priority or critical asset, function, activity, or capability. Figure 7 illustrates the interaction of these elements.

Figure 7: Elements of Threat

Threat analysis can be conducted using threat tree analysis. Three types of mapping or matrix techniques include:

• Asset tree – asset, means of access, internal or external threat actor, intentional or unintentional motive, capability, event, consequence;
• Threat type tree – type of threat, act, resultant event, consequence; and
• Adversary tree – type of adversary, motivation, capability, methods, event, consequences.

In order to determine a realistic threat level, consider the following flow diagram, Figure 8.

Figure 8: Determining Threat Levels

The likelihood of threat should be considered as part of the threat analysis. There are many different approaches which can be used. One is a narrative approach which basically uses a qualitative description for the threat level and threat characteristics. Subject matter experts may provide input based on an analysis of events, trends and other indicators or analysis of specific threat characteristics (e.g., intentions, capabilities, and other attributes). Another approach is the threat ranking approach which is generally a semi-quantitative approach for estimating the components of threat and then combining them into some value and/or ranking with some description. The attributes that are rated for each threat should be orthogonal (e.g., should not overlap such that there is double counting). In some cases the rating scores can be used to represent a "risk-like likelihood".

Threat profiles are usually dynamic. Therefore, threats should be monitored on an on-going basis. Specific information for individual facilities is often lacking. When estimating the threat levels it is important to understand the internal and external context of the location being assessed, as well as the unique sources of risk for the location. For example, sympathy in local communities for the acts of violence may influence the likelihood of the threat of terrorism and violent crime. Organizations operating in a cultural setting where there is little sympathy or acceptance of violence would face different threat levels than a society that condones the use of violence as a means to justify perceived wrongdoings.

Threat and opportunity characterization seeks to identify general and specific sources of risk and describe how they manifest themselves. Scenarios can be developed to analyze how the threat or opportunity will materialize and what are the various factors and stakeholders at play. Once a scenario has been identified it can be evaluated for differing magnitudes of the risk event. Similar scenarios may be triggered by events resulting in similar consequences. By evaluating the different possibilities it is possible to identify risk treatment options that focus on both likelihood and consequences.

When evaluating the potential for intentional threats, consideration should be given to the presence and proximity of “hard” and “soft” targets. A resilient and determined adversary will consider the same factors illustrated in Figure 8 in order to successfully carry out a threat to cause a risk event.

6.4.4.1.3 Vulnerability/Capability Analysis
Vulnerability/capability analysis evaluates the efficacy of the risk measures in place (deliberate and/or inherent) that will have an effect on the likelihood of a threat or opportunity materializing and the likelihood and extent of consequences. Vulnerability is dependent on the risk control measures (e.g., countermeasures) deployed to manage a risk event. Capability is dependent on the adaptability of the entity and its ability to respond to negative events and to take advantage of potentially positive ones. Risk control measures can be either physical or virtual (e.g., technologies, physical barriers, administrative procedures, etc.) It should be recognized that some risk treatment measures may reduce the likelihood of an event taking place but do not make the target less vulnerable.

Analysis of vulnerability includes analyzing the attributes of the event and assets, services, and activities. Factors to consider include:

• Efficiency of risk control measures;
• Level of profile, recognition, visibility, and iconic status;
• Value of assets (including symbolic and reputational);
• Understanding which parties support the objectives of the organization and those that don’t;
• Alignment with potential adversaries’ intent and motivations;
• Timing, intensity, and duration of the event;
• Accessibility;
• Interdependencies and dependencies;
• Perceived and actual recovery times;
• Cascading affects (e.g., a toxic release compounded by wind currents);
• Demographics and local culture; and
• Potential for collateral damage.

Steps to consider in determining the level of vulnerability:

• Identify risk scenarios (from asset valuations and threat analysis);
• Define how the risk scenario will be manifested (single or multiple paths);
• Determine the effectiveness of the risk control measures;
• Determine the vulnerability based on attributes of the scenario events and potential outcomes; and
• Determine the level of vulnerability based on severity of the consequences and recovery time periods.

Level of vulnerability is determined based on metrics designed to measure the achievement of the organization’s objectives. Therefore, not only is the value of the asset, service or activity considered, but also the timeframes that asset, service or activity may be unavailable. When determining the vulnerability consider:

• Is the vulnerability due to a single weakness or multiple weaknesses?
• Does the nature of the vulnerability make it difficult to exploit?
• What is the time dependent nature of the vulnerability, cascading effects, and recovery time?
• Is the vulnerability lessened by multiple layers of countermeasures?

Event trees can be helpful tools in evaluating the vulnerability. Although many models exist, a simplified example is to:

• Assume a risk scenario;
• Identify threat actors and methods;
• Identify targets and potential consequences;
• Identify accessibility;
• Identify countermeasures;
• Determine if single or multiple layers of defense exist;
• Determine the efficiency of countermeasures (consider the conditions of deployment); and
• Determine level of vulnerability.

6.4.4.1.4 Criticality and Consequence (Impact) Analysis
Criticality and consequence analysis provide a measure of impact of the risk event relative to achieving the organization’s objectives and the impact of losing a tangible or intangible asset, activity, or function will have on the operations of the organization and its stakeholders, respectively. A well done criticality and consequence analysis will allow the analysis to focus on those assets, activities, and functions that are of most importance to the organization and stakeholders.

It is important to understand the criticalities and consequences in order to develop a cost-effective risk management strategy. The consequences will depend on the nature, location, and other factors of the event. Scenarios are often used in calculating plausible, implausible, and catastrophic consequences. This should be done evaluating the consequences against the criticality of the asset, activity, or function.

The criticality of an asset, activity, or function can be intrinsic or derivative. The intrinsic criticality indicates the direct value of the asset, activity, or function in achieving the objectives of the organization. The derivative criticality indicates the indirect consequences of risk event and how the resultant consequences indirectly related to the asset, activity, or function will affect the organization achieving its objectives. In evaluating the criticality consider:

• The value of the asset, activity or function to on-going operations and value generation;

• The value of the asset, activity or function to internal and external stakeholders including competitors and adversaries;

• Timeframe of criticality – time period an asset, activity, or function can be unavailable before effects are significant;

• Derivative affects – the effect on other assets, activities, or functions;

• Impact on brand, image and reputation;

• Exclusive possession;

• Availability of alternatives for the assets, activities, and functions; and

• Perception of criticality of supply chain partners and other stakeholders.

Many scales exist for grading consequences. The exact scale should be determined by the accuracy of the predictions, whether a consequence is quantifiable, and the intended use of the information. The scales should be determined also based on their utility to the risk managers and decision-makers. Regardless of the scale used, it should be consistent throughout the risk assessment process. When assessing the consequences of a risk event consider:

• Human impact: Physical and psychological harm to employees, customers, suppliers, and other stakeholders;

• Physical asset impact: Property losses and replacement costs;

• Information asset impact: Loss of sensitive, proprietary, or personal information;

• Financial impact: Lost or deferred sales/business, loss of market share, lawsuits, regulatory fines/penalties, overtime pay, stock devaluation;

• Reputational impairment impact: Diminished standing in the community, negative press;

• Community/societal impact: Indirect impacts on the regional economy, reduction in the regional net economy, losses to the tax base of local jurisdictions; and

• Environmental impact: Degradation to the quality of the environment.

An example of a flow diagram for considering the consequences of a risk event illustrating the importance of time considerations is given in Figure 9.

Figure 9: Criticality and Consequence Analysis

6.4.4.2 Risk Analysis
Risk analysis is a process to understand the nature and level of risk to determine its significance. The organization takes the information generated during the risk identification process and evaluates this within the context of its operations and the risk criteria. The risk analysis process assesses the likelihood and consequence to determine the level of risk and prioritize risk treatments. To begin, organizations may choose to rank risk events with varying degrees of detail, depending on the risk, and the information, data, and resources available.

As seen in Figure 10 the output from risk identification provides the input to risk analysis.

Figure 10: Determining the Level of Risk

Likelihood and consequence can be expressed qualitatively or quantitatively (or a combination of methods). The decision on which approach works best for an organization is based on the:

• Availability and reliability of information;

• Scales and level of detail of the risk identification process;

• Methods for determining threats and impacts to tangible and intangible assets, as well as tangible and intangible impacts (intangible assets and impacts may not lend themselves to numeric evaluations);

• Other risk analysis processes and methodologies used by the organization; and

• Most effective method for communicating level of risk to decision-makers.

Regardless of the method used to determine the level of risk, care should be taken to assure a consistent approach and consider the level of confidence, particularly for aggregated data. Units and scales of measuring risk determined during the definition of risk criteria should be used consistently throughout the analysis. The risk analysis method used should meet the needs of the risk evaluation and treatment decision-making process.

6.4.4.3 Risk Evaluation and strategies
Risk evaluation uses the risk criteria and outputs from the risk identification and risk analysis steps, to determine what risks are acceptable with existing risk treatments and which require additional risk treatment. The level of risk determined during risk analysis will indicate the priorities for risk treatment. Evaluating the level of risk before and after treatment combined with value driver analysis provides the basis for determining if the residual risk levels fall within an acceptable level of risk set by the risk criteria. Risk treatment prioritization should also be predicated on an understanding of the risk tolerance. If the level of residual risks is found to be greater than the acceptable level of risk set by the risk criteria, the organization should consider alternative or additional risk treatments to reduce the level of residual risk. Initial treatment decisions will be driven by tolerance, not just addressing residual risk. Risk evaluation considers the cost and benefits of different treatment options. Care should be taken during the risk evaluation stage to make sure treating one risk is not creating another risk.

Risk evaluation considerations include:

• Objectives of projects and opportunities;
• Tangible and intangible impacts;
• Legal, regulatory, and contractual requirements;
• Critical control points;
• Tolerability of risks to others;
• Whether a risk needs treatment;
• Deciding whether risk can be tolerated;
• Whether an activity should be undertaken; and
• Priorities for treatment.

Acceptable risk levels will be unique to each organization and its value chain. They may vary by project, commodity, product, or service, as well as over time. The organization may have varying levels of risk-tolerance for different divisions, subsidiaries and partners. It may not be practical to eliminate all risk due to costs. It may be desirable to accept risk to gain an opportunity (e.g., to increase market share, or pursue labor or location benefits). To achieve as low as reasonably practical risk, a typical target of risk evaluation is to determine the most cost effective treatments.

Examples of reasons an organization may tolerate risk (by informed decision) include:

• The level of the risk is so low that specific treatment is not appropriate within the constraints of available resources;

• The risk is such that there is no treatment available. For example, the risk causes may not be within the control of an organization;

• The cost of treatment, including insurance costs, is so manifestly excessive compared to the benefit that toleration is the only option. This applies particularly to lower ranked risks;

• The opportunities presented outweigh the threats to such a degree that the risk is justified; and

• Organizations may also determine to accept a risk by informed decision-making or to maximize a business opportunity.

Regardless of the method used to evaluate risk treatment(s) to achieve a level of risk as low as reasonably possible, it is important to understand that this is an iterative process where the risk manager can pick multiple layers of risk treatment measures including:

• Eliminating the risk exposure;
• Isolating the risk source or potential targets;
• Technical modifications and substitutions;
• Administrative and procedural controls;
• Protective, preventive, and mitigation measures;
• Risk sharing; and
• Accepting or exploiting risk by informed decision.

During the risk evaluation process, the proposed risk treatment methods should be evaluated to consider the cost/benefit of the measure to reduce risk and whether the risk treatment changes or introduces new risk to the organization and its value chain. Figure 11 illustrates how the output from the risk identification and analysis steps can be represented by a funnel approach where intolerable risk must be treated at any reasonable costs. Treatment measures are applied to bring the risk to a level that is as low as reasonably possible where further task treatments are disproportionate to the cost/benefit. Risks reach a tolerable level where risk is within the level of tolerance of the risk criteria. Contingency measures might be considered for risks that remain after treatment.

Figure 11: Risk Evaluation Funnel

One way an organization may wish to assess its risk tolerance is through a risk “frontier” graph, plotting the likelihood of events by their consequence (Figure 12). Organizations may find some risks to be of such low likelihood or to have such limited consequence, that they do not warrant any further treatment or consideration. For those of greater likelihood or consequence, the organization may wish to use resource management to reduce volatility. Such mechanisms may seek to reduce the likelihood, duration, or consequence of a risk event. Organizations may also determine to accept a risk by informed decision-making to maximize a business opportunity.

Figure 12: Conceptual Risk “Frontier”

A two-dimensional means of representing the risk levels is to use a matrix showing risk events defining likelihood and consequence (sometimes referred to as a heat map, risk matrix, or event matrix). This technique allows managers to easily see the relative likelihood and consequence of differing risks. To use this method effectively it is critical to have well-defined and consistently used criteria for the different likelihood and consequence levels. Various scales are used by different organizations; the gradations, scaling, and terms used should be based on what is best understood by the users and the decision-makers. Figure 13 shows a sample matrix illustrating the concept.

Adapted from RIMS workshop on Risk Management Techniques. Copyright © Risk and Insurance Management Society, Inc. All rights reserved.

Figure 13: Sample Matrix

Figure 13 illustrates a two-dimensional depiction of identified risks related to a fictional organization’s objectives. The sample depiction considers risks from both an opportunity and threat perspective. This type of qualitative assessment assumes that the terms used in the matrix have been defined, and that the assessment team has analyzed the likelihood and potential impacts/consequences of each of the risks in the context of the organization’s noted objectives for placement on the matrix.

The matrix shows how organizations may wish to prioritize risks by likelihood and consequence. Scales for the matrix should be defined when setting risk criteria. The type of scale, parameters, and level of detail will be dependent on the requirements of decision-makers.

A risk registry may also be used to catalog risks. A risk registry is a list of identified risks and characteristics of the risk, the severity of the consequences and the likelihood of their occurrences. Risk registries are often used to compare risks from many different sources. A risk registry should include (but not limited to):

• Name of risk;
• Description of risk;
• Time period for estimates;
• Risk Owner;
• Likelihood or frequency of occurrence;
• Impacts, severity or consequence of occurrence;
• Interdependencies and dependencies; and
• Actions and/or countermeasures to reduce the likelihood and consequences.

Note: For additional risk analysis methodologies, see the ISO 31010:2009 Risk management – Risk assessment techniques.

6.4.4.4 Cost-Benefit Analysis
Cost-benefit analysis provides a method for evaluating and comparing the value and cost of risk treatment options. The analysis should consider both direct and indirect costs and benefits. Examples of these are:

• Benefit:
  

  • Direct benefits—arising from reduction in the likelihood or harmful consequences of the risk; and

  • Indirect benefits—arising from collateral effects of the treatment such as reduced insurance premiums, improved management and staff confidence, and enhanced reputation.

• Cost:
  

  • Direct costs—of implementing the proposed treatment and/or that could arise if the risk eventuates (e.g., loss of an asset); and

  • Indirect costs—arising from the loss of productivity, business disruption, diversion of management attention, loss of reputation or brand value.

6.4.4.5 Risk Control and Treatments
Once an organization understands its context and has analyzed its potential risks, it can begin the process to modify and reduce risk. It is important to keep in mind when developing a risk treatment strategy that risk treatments have the potential to create new risks or modify existing risks.
After an organization has identified and prioritized the risks that it faces, it can devise risk treatment plans. Plans include developing strategies and measures to protect value chains from sources of risks, responding to events that these risks may cause, and continuing operations and recovering from undesirable and disruptive events. Risk treatments seek to:

• Remove the risk source, where possible;
• Remove or reduce the likelihood of the risk event occurring;
• Remove or reduce negative consequences;
• Share the risk with other parties, including risk insurance;
• Accept risk through informed decision or to exploit an opportunity; and/or
• Avoid activities that give rise to the risk.

For organizations to cost-effectively manage risk they should develop balanced strategies to adaptively, proactively and reactively address minimization of both the likelihood and consequences of undesirable and/or disruptive events. Furthermore, the selection of risk treatment controls should be integrated with the overall risk management programs with its priority stakeholders. Such a program should have at least three elements: 1) protecting the organization and its value chain; 2) responding to events; and 3) continuing operations while recovering from events. Plans should also involve determining ways to measure risks as well as testing the effectiveness of the plan itself and its ability to limit risks.
The organization should establish, implement and maintain procedures to prevent and manage undesirable and disruptive events to prevent negative consequences, and exploit positive consequences to the organization, its key stakeholders including supply chain partners, and the environment.

Procedures should be concise and accessible to those responsible for their implementation. Plans and procedures should be accepted across all different management areas and risk disciplines to avoid a silo approach (e.g., a business continuity plan needs to take into consideration how security measures within an incident response will impact continuity of operations). Operations and plans should be examined to ensure appropriate integration and coordination is used to capitalize on limited resources. Examples of risk treatment procedures are provided in Annex F.

6.4.5 Generating Findings and Conclusions
Risk assessment findings should be determined by evaluating the data and evidence collected against the risk criteria. Risk assessment findings indicate the level of risk and the needs for acceptance and/or treatment. Findings should be based on substantiated, documented evidence that classifies and prioritizes the risk assessment evidence to indicate the significance of any risks, as well as identify opportunities for improvement and alignment current accepted industry practices and legal requirements. This will help the client and organization understand the effect of a risk on the organization.

Significance of risks can be based on:

• The needs to exploit opportunities to protect and create value;
• Whether there are single or repeated occurrences;
• How the risk affects the achievement of the organization’s objectives;
• Level of the risk; and
• If singular or multiple risks lead to high risk situations in the organization.

Levels of risk and their priority for treatment should be directly linked to the supporting risk assessment evidence and should be recorded. Risk may be graded on a qualitative and/or quantitative scale. Individual risks can be grouped to provide evidence of systemic issues within the risk management system. This will help identify the issues that indicate the need for the system and processes to be changed. The need and imperative for change is based on the risk to the organization being assessed, as well as its stakeholders. Clear classification and documentation of observations will help identify follow-up actions.

Risk assessment findings are generated by the RTL in conjunction with assessment team members. At appropriate stages throughout the assessment, the assessment team should meet to review the assessment findings up to that point. Aspects which should be considered when determining assessment findings include, requirements of the client and organization, sample size, follow-up actions from previous assessment findings and conclusions, and categorization of the assessment findings where necessary.

When creating records of levels of risk and treatment needs, the assessment team should identify the risk criteria being used, risks assessed, evaluate the assessment evidence to support the level of confidence in the finding, and state whether the evidence is consistent with the risk criteria (particularly risk attitude of the organization). When creating records on specific risks, the assessment team should identify the risk being assessed, show the risk assessment evidence to support risk treatment decisions, and include related assessment evidence to support the findings. Every level of risk determination should be traceable back to evidence gathered for a specific risk.

6.5 Post Risk Assessment Activities

6.5.1 Conducting Post-Assessment Debriefing
The post-assessment meeting ends the on-site activities of the assessment and presents a draft or preliminary assessment report to the client. The post-assessment meeting should be facilitated by the RTL. The purpose is to present the assessment team’s conclusions and findings to the management of the organization, and those responsible for the areas being assessed (where applicable). The post-assessment meeting should present areas of both upside and downside risks, as well as strengths and weaknesses in the risk management system, starting with strengths, providing sufficient information for the client and organizational management to understand the findings. A designated assessment team member should record attendance and minutes.

The level of detail is dependent on the level of familiarity the client has with the assessment process. Also the formality of the meeting is dependent on the type of assessment. In some cases, a formal meeting is necessary with records of attendance and minutes, while in other situations the meeting may be a less formal communication of the assessment findings.

If situations arose during the assessment that might call the results of the assessment into question, the assessment team should advise those present of the situation. Furthermore, any differences in opinion regarding the assessment conclusions or findings between the assessment team and the client should be discussed. The parties should try to resolve any disagreements. If the parties cannot resolve their differing views, that should be recorded.

Participants should discuss an expeditious time frame for an action plan to address assessment findings and adapt the risk management system, where needed. Recommendations for improvements may be presented if specified by the assessment objectives. It should be clear that any recommendations are non-binding, and should be noted that in subsequent assessments these may bias an impartial evaluation.

The following should be addressed with the organization’s management so that they are acknowledged and understood at the post-assessment meeting (where appropriate):

• The assessment findings and conclusions;
• The method of reporting
• Information and report handling and dissemination;
• The handling of assessment findings and possible consequences; and
• Post-assessment activities (where applicable).

6.5.2 Reports and Records
The risk assessment report communicates the results of the assessment to the client and organization, as well as provides a complete and concise record of the assessment.

6.5.2.1 Overview
The risk assessment report is prepared by the RTL, with input from the assessment team, and is provided to the risk manager as soon as possible after post-assessment meeting. The assessment report is approved and reviewed by the risk manager prior to distribution. For credibility, any changes to the report, including findings, should be made by the RTL. The client determines who will receive copies of the assessment report. The purpose of the assessment report is to:

• Provide information about the assessment findings and conclusions;
• Initiate a request for corrective actions to significant risk requiring immediate attention;
• Serve as a basis for identifying opportunities for improvement of the risk management system; and
• Provide a record of the assessment.

6.5.2.2 Distributing the Assessment Report
The risk assessment report should be issued without delay within an agreed timeframe. If the assessment team is unable to do this, the reasons should be promptly communicated to the client, organization, and the person(s) responsible for the risk management program. In compliance with good project management procedures, the assessment report should be reviewed, approved, and dated. Distribution of the risk assessment report is at the discretion of the client and organization. The risk manager should not send a copy of the risk assessment report to anyone unless explicitly approved in writing by the client and organization. The organization conducting the assessment maintains a copy for its records as per agreement with the client and organization. Retention of a copy of the report should be consistent with legal and liability requirements and needs.

In some instances, reports may be required to be submitted and transmitted digitally in a secure fashion. In these instances, the risk manager should control the release and accessibility of this information by using appropriate information security methods. Passwords and encryption should comply with accepted government or industry practices and methods for securing this type of information.

The client and organization should treat the risk assessment report as protected information and provide document handling safeguards.

6.5.3 Follow-up and Monitoring
It is the responsibility of the organization and client to apply corrective, preventive, or improvement actions indicated in the assessment report. The client should implement these actions in a timely manner. The client should keep the RTL and risk manager informed of these actions to facilitate on-going monitoring of risk. These actions should be documented and verifiable so they may be included in a future assessment. Verification that the corrective, preventive, or improvement actions have been conducted and are effective should be documented before the follow-up assessment commences.

The organization should establish a defined and documented risk monitoring and change management program to ensure that any internal or external changes that impact the organization risks are reviewed in relation to the risk criteria. It should identify any new critical activities that need to be included in the risk management program. The change management program should define the frequency at which the risk assessment should be updated as well as the events that would trigger the conduct of a new assessment.

6.6 Checking and Review

6.6.1 Assessment evaluation
The RTL should establish, implement, and maintain performance metrics and procedures to monitor and measure those characteristics of the risk assessment that have material impact on its performance. The procedures should include the documenting of information to monitor performance, applicable operational controls, and conformity with the organization’s risk assessment program objectives and targets.

6.6.1.1 Identifying Opportunities for Improvement
The RTL should monitor, evaluate, and exploit opportunities for improvement in risk assessment performance and eliminate the causes of potential problems, including:

• Ongoing monitoring of the operational landscape to identify potential problems and opportunities for improvement;
• Determining and implementing action needed to improve assessment performance; and
• Reviewing the effectiveness of any actions taken to improve performance.

Actions taken should be appropriate to the impact of the potential problems, and resource realities.

The risk manager and RTL should ensure that actions are taken without undue delay to initiate opportunities for improvement. Where existing arrangements are revised and new arrangements introduced that could impact on the risk assessment program, the RTL should consider the associated outcomes before their implementation.

The results of the reviews and actions taken should be clearly documented and records should be maintained. Follow-up activities should include the verification of the actions taken and the reporting of verification results.

6.6.2 Improvement
The review of the risk assessment should include assessing opportunities for improvement and the need for changes to the risk assessment program. The results of the reviews should be clearly documented and records should be maintained. The organization should continually improve the effectiveness of the risk assessment activities.

Next: Confirming the Competence of Risk Assessors