Print E-mail

The Monitoring Process 2013

Overview

In Policy Governance® the board delegates a broad range of authority and control to the CEO. In addition, there will be a policy in place that states that the only criteria to which the CEO can be held accountable are those set by policy. The standard the CEO is held to is any reasonable interpretation of the policies that have been put in place. How does the board assure itself that the operational organization is performing well?

Monitoring Reports are the key. Through this instrument, the board requires the CEO to provide scheduled and regular reports that monitor the accomplishment of the Ends and the avoidance of the Executive Limitations.

Three Methods of Monitoring
There are three methods of monitoring available to the board. Any, all, or a combination of the three methods may be used for monitoring any policy. These methods are:

  • Internal Report – The CEO produces a report detailing the policy, the CEO's reasonable interpretation, and data that indicates compliance or non-compliance with the policy. In the case of non-compliance, an explanation is typically provided and a plan for return to compliance is often expected.
  • External Reporting – The board uses a third-party inspector with knowledge in the area, such as an accounting firm for the annual audit, who will be objective and brings special expertise.
  • Direct Inspection – The board may wish to physically make an inspection, as a whole or with a task force, to gather the data and make an analysis concerning compliance. An example might be concern about the CEO's compliance with policy regarding treatment of staff. The board may wish to directly talk to employees to confirm compliance. This can only be done, though, if the board as a whole has set policy within the monitoring schedule, indicating that it will do so.

True to the principles of Policy Governance®, the monitoring schedule is set by the board in advance for two reasons: so that the person responsible for producing the monitoring is aware of this requirement, and so that the board is reminded to actually look at those things that it feels are important enough to justify policy creation. This is best accomplished by setting a monitoring schedule that lists each policy, the monitoring method, and the frequency of monitoring. A single policy may have more than one method or date. The majority of the monitoring is usually done by internal monitoring, once per year.

CEO Monitoring
The board defines its expectations for performance in two categories: Ends, to define desired outcomes; and Executive Limitations, to define unacceptable conditions, activities or events, which then limit managerial choices in operational planning.

In all cases the board will end its definition at a relatively broad level. The intent is to capture the values by which decisions shall be made, not to make the actual decisions. It is expected that the CEO will need to make additional lower-level (more detailed) decisions within these areas. Because the board both demands that the policy requirements be met and recognizes that any given word may be interpreted differently by different individuals, the board allows "any reasonable interpretation"; further decisions will be made but they must be consistent with the actual policy under which they are made. The CEO's interpretation then sets the actual benchmark for measuring compliance.

In monitoring, the board requires the CEO to provide a written explanation of his or her reasonable interpretation, justifying why that interpretation is indeed reasonable in an objective way that can be measured and checked. The board also requires that the CEO provide data that actually will allow the board to be assured that the policy criteria, as reasonably interpreted, are actually being met. Promises, pontifications, and efforts are not enough; the board needs to know either that the actual criteria have been met or any reason why they have not.

Operational Definition
So what does an operational definition look like?

It should be:

  • Declarative – a solid, concise, clear statement of what the policy means to the CEO in terms that will allow the board's policy to be put it into "operation." That is, it should define the policy well enough to be the first step in taking physical action.
  • Measurable – in order for the board to verify successful completion they need a defintion which can be measured as well as an understanding of how the measurement will be developed. To be operationally valid it has to not only be specific enough let you take physical action but should also allow you to know whether you are getting the expected results. Definitions that are so abstract, vague, or wishy-washy as to defy measurement are not really operational definitions.
  • Testable – the definition and its measure should be clear enough that the veracity of the report can be tested. The result of the measurement must be replicable by the board or some person other than the CEO.

Justification
The reasonableness of the operational definition should be justified within the report. "Just trust me" or "because I say so" do not give the board the opportunity to test the information, nor does it offer board members any assurance that the CEO's interpretation is indeed a reasonable one. There are different levels of justification that might be offered:

External Source for Standards (Preferred) – using an externally developed and monitored standard from a source that understands your field and is recognized within it.

Internal Research (Next Best) – developing your own standards from your own research. For example, standards that apply to customers could be developed by conducting focus groups or a survey with customers; standards applying to a policy that addresses staff could be developed by conducting focus groups or a survey with staff. The standard may not be statistically significant or recognized outside of the organization, but it is derived from actual research that can be measured and replicated.

Providing the Reasoning or Logic Behind Your Definition (Most Questionable) – outlining for the board members the process of thought between their policy and the CEO's reasonable interpretation of it, including the steps of logic taken and the underlying assumptions used.

Measuring Against the Standards
The board may have addressed a level of performance (e.g. "in the top 10%") in its policy. If not, the CEO should identify how this organization should compare to others, taking into account the resources available and the relative priority of the issue addressed in the policy. For example, if the average or defined best practice in your field is rated at 75, the CEO will want to use as his or her interpretation that average or best practice, something more than that, or something less than that. This determination is based on how the organization compares to others using the standard and how important the issue is to this particular organization. Aiming to be better than others in the field in a particular area probably means a higher standard than average, and an issue more important to this organization than other organizations probably means a standard at or better than average.

Typical Monitoring Process Flow – Internal Report

  • The CEO presents a report that does three things:
  • Illustrates his or her interpretation of the policy
  • Justifies why the CEO feels the interpretation is reasonable.
  • Provides evidence of compliance with the policy (which may be based on performance on one or more indicators identified by the CEO).
  • The board asks first, "Is the interpretation reasonable?"
  • Yes: Go to the next question.
  • No: Ask the CEO to come back with another interpretation.
  • Yes, but it is not what the board meant: Change the policy to reflect what the board intended the policy to mean, either by rephrasing or going to the next level down until it is clear.
  • The board asks, "Is the interpretation justified
  • Yes: Go to the next question.
  • No: The board members will need to consider carefully what follows, making the judgement of reasonableness based on their own internal criteria should give them less confidence in compliance.
  • The board asks, "Will the information/evidence provided indicate compliance?"
  • Yes: Go to the next question.
  • No: Tell the CEO to come back with different or additional evidence.
  • The board asks, "Is the CEO in compliance with the policy?"
  • Yes: The report is found to have a reasonable interpretation and data sufficient to show compliance.
  • No: The board demands corrective action as is appropriate (interpretation and data showing compliance by some date certain, dictated by the board (which may or may not ask the CEO to suggest a reasonable timeframe). The board controls the deadline for compliance regardless of whether or not they consult with the CEO and even if they feel that the next scheduled report meets their requirement for correction. If the lack of interpretation or data raises enough concern it may also lead the board to take more serious action such as dismissal.
  • If the individual board members reach agreement as to interpretation and data showing compliance and the Chair hears no concern, the monitoring report can be voted on as a separate item on the consent agenda

Developing Operational Definitions

Basics
Each definition must:

  • Be a working definition with enough detail that it could be operationalized (put into action).
    • This usually requires narrowing down certain aspects by being more specific in the definition.
    • It also usually includes setting some standard of achievement.
  • Fit within a reasonable interpretation of the board's policy (not extending beyond it or being so narrow as to not really address the fullness of the issue).
  • Be testable. Detail must be provided to an extent that the definition could be put into action and then tested for whether or not it was working.

As seen through an employee's eyes

  • Looking at the policy language, what else would I need to know about (have further defined) from the CEO in order to actually design and implement a system or process that would deal with the policy and meet the CEO's broadest expectations?

As seen through the board member's eyes
There are a number of assessments this person typically must make:

  • Is an operational definition provided (enough detail; how it impacts the organization; testable)?
  • Is the definition reasonable?
    • Does it fit within the range set in the policy, and does it cover a broad enough range of circumstances so as to represent what the policy suggests?
    • Is a source or rationale provided that helps me assess its reasonableness?
  • Is there a way to measure whether or not it is working?
  • Does the data actually show that the standard of achievement is being met?

What the Board Members need

  • A definition that they find reasonable
  • A proposed measurement system that they find credible
  • A standard of compliance that they can judge as justified or rational
  • Data that is verifiable

FREQUENTLY ASKED QUESTIONS

What does an Internal Monitoring Report consist of?
Anything that provides reasonable proof that the CEO has done what the board has asked and avoided that which it set off limits. Typically there will be three parts to the report:

  • CEO's interpretation of the policy being monitored and a justification of that interpretation.
  • CEO's description of the measure being used to determine compliance with the policy.
  • Data that shows the policy is being followed.

What is the CEO's responsibility?

  • To provide her or his reasonable interpretation of the policy in question so the board members can determine whether the policy fits their intention.
  • To develop the measure(s) that would show performance.
  • To develop and provide the data that indicates performance and compliance.

What if we can't find an extremely accurate measure?

John Carver says, "Far better to have a crude measure of the right thing than an exact measure of the wrong thing."

How is the CEO accountable?

  • The CEO must provide reports that monitor the progress made in accomplishing the Ends policies and the avoidance of the situations and methods outlined in the Executive Limitations policies.
  • The CEO is only accountable for a "reasonable interpretation" of the policies actually developed by the board. The CEO is not responsible for what the board "meant" or "said but didn't write down," and is not accountable for meeting unstated expectations or on-the-spot measures.
  • The CEO must develop measured data that shows that their interpretation is actually being achieved.

What situations place the CEO out of compliance?

  • Failure to provide the required monitoring report (out of compliance with the policy requiring monitoring reports).
  • Failure to offer a reasonable interpretation.
  • Failure to provide a measurement system and data which would prove the interpretation is being achieved.
  • Failure to actually do what the policies require or to avoid that which the policies set off-limits regardless of the monitoring report and its information.

What should the CEO do in the case of non-compliance?

  • Report non-compliance as soon as it is predicted or known (as per policy on "Communication and Support to the Board".
  • Provide an explanation for non-compliance.
  • Be able to explain when the situation can be corrected by or why it cannot be.

What are proper board responses to a Monitoring Report?

  • Find it in compliance and approve the report on the consent agenda.
  • Find it out of compliance either because the interpretation cannot be judged to be reasonable or the measurement system and/or data do not convince the board of performance on the interpretation. Then the board may do three things:
  • Set a date by which compliance will must be achieved.
  • Ask the CEO to come back with another report containing an interpretation and data that do show performance by a date certain.
  • Take appropriate and even drastic steps if the area of non-compliance warrants it.
What if the board doesn't agree with the measures or the reasonable interpretation (even if the CEO is in compliance by the board's interpretation and/or measure)?
  • If the interpretation is unreasonable, the board finds it out of compliance and stipulates a date by which this must be corrected.
  • If the interpretation is reasonable, but not what the board meant, the words of the policy must be either clarified or expanded. The next report will then have to meet the test of reasonable interpretation for the refined policy.
  • If the measures are not reasonable or complete, or they do not show compliance, the board finds the CEO out of compliance and stipulate a date by which this must be corrected.
How do we maintain the integrity of the monitoring process?
  • The board must hold the CEO accountable for those things it has said it wants the organization to achieve and for those things that it has said need to be avoided. This means that the monitoring must take place and any necessary hard questions  are asked.
  • The CEO must hold the board members to the words they have actually said and their power to use a reasonable interpretation of them. In turn, the board must hold the CEO accountable for only those reasonable interpretations. Anything else creates vagueness and lack of clarity, two enemies of good governance.
  • If either side is lax, there is a significant chance that accountability will disappear.
Utilizing the Monitoring Process to Reduce Concern and Fear

There are times that the board may feel the need to increase its oversight in a particular policy area. Perhaps board members have reason to believe that they are vulnerable in a certain area due to changing circumstances, or perhaps they have reason to believe that the current CEO requires more stringent watching. Changing the level of oversight is best achieved through one of two changes in the monitoring schedule:

  • Increase the frequency of the monitoring of the appropriate policies.
  • Change and/or add to the methods of inspection (internal report, external report, direct inspection).

Changing the policy is not recommended. Policies should be made to last and should not be changed on a situational basis. These methods allow the board to have flexibility and a firmer grip on the broadest values without needing to change the level of detail that the policy defines.

One Possible Process for the Board

  1. The monitoring schedule is set in policy. The monitoring type, frequency and date is defined for each policy in the areas of Ends and Executive Limitations.
  1. In the Ends and Executive Limitations areas, the Chair (or other board representative empowered to assist the board) assigns each individual policy to an individual board member.
  1. Each board member promises to really understand the policies he or she has been assigned and to become the "internal expert" on those policies.
  1. The board as a whole determines a standard for agreement on the individual assessment that would allow the report to go on the consent agenda or require discussion by the board (for example a super majority find it reasonable interpretation to sufficient data to show compliance). [Alternatively, if the board wishes to spend the time to discuss each one, then when the agenda reaches that item, the person assigned to that policy may give a short summary of what he or she saw in the responses. The Chair can then ask the board if it wishes to consider any action, such as policy changes, a plan to learn more about it, or an improvement plan, based on what the board has learned.
  1. At the meeting prior to the board session at which a given policy or policies will be assessed, the board members will receive all appropriate monitoring reports. They will be asked to make their own judgment as to whether or not the CEO's monitoring report measures up to the criteria defined in the policy. If they accept that the interpretation provided is indeed reasonable, the criterion for the remainder of the report is whether or not it shows compliance with the CEO's interpretation of the board's policy, not the actual words in the policy. An evaluation sheet will be provided either online or via email that will guide the individual board member through the process of evaluating the report.
  1. The evaluation forms are either completed online or completed and forwarded to the relevant policy's "internal expert" to be compiled and summarized.
  1. There is an item placed on the consent agenda for CEO Assurance (or Acceptance of Monitoring Reports).
  1. If the board does have something it would like to take action on, the Chair helps the board determine how and when to do so.
  1. Each year the assignments to policies are rotated so that no one person becomes attached to the same policy year after year.

 

 

Go to Part Two >>