Strategic Plan Evaluation – From IAF Interview

formagatepurpose

Magate Wildhorse and I thank the Association for Strategic Planning, and the International Affairs Forum (IAF) for the opportunity to speak on the following topics ahead of the ASP Conference 2018. The theme of this year’s Conference was “Bridging the Strategy Execution Gap”.  We had a wonderful time in Chicago. We met many strategy experts from various industries and countries; learned and shared during various conference sessions and we came home with a prize.  Ha ha see why you can’t afford to miss ASP Conference.

Following are the full-length scripts from my pre-conference interviews hosted by Dimitri Neos of the IAF.

The interviews are shared in three parts and covers the following topics:

  • Part 1: Pre-Implementation Evaluation of the Strategic Plan
  • Part 2: Monitoring of the Strategic Plan
  • Part 3: Post-Execution Evaluation of the Strategic Plan

We also discussed specific challenges related to closing the strategy to execution Gap.

From our ASPConf2018 pre-conference interviews.

Part 1: Pre-Implementation Evaluation of the Strategic Plan

IA=FORUM What is your approach to Monitoring and Evaluating strategic plans?

MEEGAN SCOTT: Our approach to the evaluation of strategic plans depends, on the stage in the strategy process, the purpose of the evaluation, and the terms of reference for the evaluation. Our role depends on whether we are adjunct internal or external consultants, and who commissioned the evaluation.

I mentioned internal adjunct because that’s a service we provide where we are adjunct to a team and not just be involved for a couple of days.

The evaluation would involve examining the context as well as the basis and logic of the strategy contained in the plan. We would also be, comparing expected to actual results, identifying emergent strategies, making recommendations for corrective actions, and developing recommendations for performance improvement.

How it actually plays out at Magate Wildhorse depends on whether we are doing the evaluation prior to execution. If it’s before execution, we focus on the content of the plan and we look for the typical consistency, balance, consonants, feasibility, advantage, completeness, clarity, and ability. When we look at consistency, we are looking at strategic intent, the framework, and so forth.

Depending on the strategy, the framework and context, we are looking at– the questions that would focus on things like: are the strategic intent and strategy framework consistent? Was, or to what extent, was a value chain considered for improving products or service delivery— or both? Has the strategy identity of the organization been clearly articulated? Are they relevant? Is there a mission statement; vision, values, and culture statements? Is there a value proposition? We also look to answer the question — Were there provisions and powerful messages for communicating the strategy identity?

Does the plan provide for building human resource and leadership capacity in response to internal gaps or a desired future state of the organization?

Another thing we look at is what, and how solid, the planned activities initiatives are. For example, activities and initiatives for retaining or growing membership or customer base as well as those for capturing non-users. We also ask, is there a timeline, or calendared activities and processes for strategy renewal and updates? What is the frequency of updating the strategy plan and version control. What is the efficiency of the plan

development process?

A question asked to help us as well as the client to do better in the future.

You have to learn what you could have done differently, what you have to get the clients to do differently, what they could have done differently.

During the analysis of the processes, we would use methods such as customer satisfaction assessments. We also look at time sheets, schedules and journals for assessing the time and process. We review our own reflexive journal for every strategic planning exercise; I get a completely new journal.

Organizational history and review of assumptions and sharing are important. You have to examine when and why the organization started. How has it changed, what has impacted the change and what the client or organization is looking for in the future?

That process can really aid organizational learning as well as raise flags or clarifies concerns related to mission drift.
We ask clients to answer questions related to history and the processes that they can do away with. We also look for mechanisms for ongoing surveillance as well as balance in accordance with the strategic framework. Key questions here would consider — How well does a plan addresses risk, the assumptions and the contingencies? Or if it addresses them at all.

To see what’s in the control kit or better other things in the control kit we look at: timelines, levels of effort, performance standards, quality standards and performance indicators.

The mix of lead, lagging, SMART and SMARTER indictors, the last being (Specific Measurable Attainable Realistic and Time-bound), plus extending and rewarding.
For ensuring the plan is actionable we also look at work plans and budgets —how well do they support each other?

In the next post we share the discussion on Monitoring the Strategic Plan.

Hosted by IAF ahead of ASPConf2018.

See also:

Bridging the Strategy Execution Gap — Select Challenges

Monitoring strategic plans

Evaluating the corporate strategy plan post-execution (post-implementation)

Magate Wildhorse, The  Noësis & Artificial Intelligence

————————–

ASPConf2018, the Association for Strategic  Planning Conference, an event for your professional development and business calendars.

Copyright © 2018 International Affairs Forum, Association for Strategic Planning, Magate Wildhorse, Meegan Scott
All Rights Reserved

iaf_logo_black magatelogogreen

Post-Execution Evaluation of Strategic Plans – IAF Interview

formagatepurpose

IA-Forum: What is your approach to Monitoring and Evaluation strategic plans?

Magate Wildhorse and I thank the Association for Strategic Planning, and the International Affairs Forum (IAF) for the opportunity to speak on the topic of Evaluating Strategic Plans, Post-Execution  ahead of the ASP Conference 2018. This year’s conference theme: “Bridging the Strategy Execution Gap”.

Below Meegan Scott shares with Dimitri Neos of the International Affairs Forum on the Magate Wildhorse approach to evaluating strategic plans post execution. The pre-conference interview addresses post implementation strategic plan evaluation, a best practice for driving strategy execution success.

Our approach to pre-implementation evaluation of strategic plans was shared in the previous post. In the second post of the series we addressed monitoring the implementation of strategic plans.

IA-Forum:  What about the post-execution evaluation process?

Meegan Scott:  The task at hand in post-implementation is to make a judgement about the strength of the organization at a milestone review period (when we’re asked to do evaluation).

It could be a Mid-term Review of a Plan or at the end of a Plan Period.

We ask if the organization is stronger at the end of that milestone period or the planned period than when the plan was created, and at the start of execution.  Was the strategy executed successfully?  This is an attempt to assess the effectiveness of a plan in guiding the organization towards achieving improved performance.  We look at that in terms of effectiveness, efficiency, relevance, financial viability, cost effectiveness, and for some type entities, we would go deeper into looking at quality aspects. 

For that type of plan, let’s say for a manufacturer of clothing, we may use the Hoshin Planning Model to add those related lines of questions to the evaluation.  In general, we also look at how the plan helps the organization with adjusting to changes in the environment.  These include political factors, social factors, competitor inflation, interest rates, legislation and even ecological factors.  Sometimes, we’ll find that entities do not know all the governing legislation affecting them.  So, we normally place a table up at the front of a plan that includes governing legislations.

We also look at the plan logic and the premises and predictions.  How did those work according to plan?  Were they accurate, to what extent, and what needs to be adjusted?  We look at whether or not the plan helped to improve motivation in a desired culture and advance the mission of the organization.  We also look at its impact on the organization, on its history.

Did it help to create new history?  Did the plan carry out what was meant to be done?  Did it help to strengthen and improve financial management partnerships, program management, leadership at different levels, and HR capacity to support both the present and the desired future?  Analyzing HR capacity is in part to help management retain tacit knowledge in the organization rather than just staying put and waiting to hire staff.

Typically, the evaluation will take the form of a self-assessment.  Even though it may be a request for an evaluation by a donor, we try to make it into a self-assessment so that the client can benefit from owning and growing that culture of performance and measurement and improvement.

The client also benefits from receiving information for decision-making related to their strategic choices for strategy updates and reformulation for the next plan and milestone period. That adds much more benefit than if you did the evaluation with merely accountability in mind.  We therefore approach the evaluation with a view to gathering performance information, meeting accountability requirements, and to guide resource allocation.  Another important thing we look at is the infrastructure for delivering strategy.

If the organization is implementing multi programs as would be in the case of a government department, and many NGOs, the approach would be heavily influenced by the terms of reference.  That comes with a call for proposal versus if it was just the organization that came up with the idea and asked for a proposal.  In the latter case we are left with greater leverage in designing what it is that we will be doing.

Our approach includes a blend of evaluation approaches.  This depends on the competence of the organization in collecting and using performance information and the information needs outlined or that we glean from the call for the evaluation and the intended users.  That blend would involve components of the utilization focus evaluation approach.

It would include consultations for ensuring that the information collected will be of benefit and is what is desired by the organization and its stakeholders.  We may even use a theory-based evaluation approach for assessing the logic for addressing a particular problem, the effectiveness, and the context.

We would want to look at the theory of change, how it’s holding up against what was expected, the participants and their attitude, and how their participation impacts the outcomes for them.  We could also use a more all-inclusive strategic evaluation approach.  That would be a strategic evaluation into the outcomes and the impact of the target population.

Irrespective of the evaluation approach or blend thereof, we would consider planned results against actual results and unintended results.

So, going back to the strategic evaluation— we’d look at the results and service levels as well whatever they are creating, selling or giving away.  For outcomes, our examination would be in terms of their relevance and effectiveness.  For outputs, the focus would be the products or services and how efficient the organization was in producing them (the outputs). Other output related questions to answer would be— along the lines of how cost-effective it was to deliver the solutions, and the quality of those outputs that were delivered.

You also want to analyze the internal management and leadership as it relates to output processes that are involved and for developing them.

So you perform an individual level assessment and review of measures.  This can be an area of challenge or resistance.  The moment you begin to ask for job descriptions and such, expect a break or stop in information flow.  At the plan level, we review measures indicators, strategy identity, et cetera.

A key component of the exercise is the management response session that we’ll lead for discussing the findings, the recommendations, and the judgment.  From that, you’ll get feedback into how management feel about the judgment and the findings.  This may result in some insight about the context and maybe some adjustment.  You will also draw out of that process actions for improvement and try to get some calendar and resource commitments towards that.

A review of external literature and internal organizational documents is part of the process.  External literature includes literature of the external environment.  When looking at internal literature, we examine their reporting and administrative documents, the operations plan, the corporate strategy plan, performance reports, and minutes from board meetings.  Other methods or lines of evidence for data analysis could include conducting surveys, interviews, and consultations.  If some instances we must calculate, develop estimates, or undertake social media searches. At times we even have to look at or conduct lab research or participatory.

The output would be typically an organizational assessment and development report.  This would include the proposed strategic options and choices for informing the development of a new strategy plan, strategy update, or a plan for the next plan period.

Let’s turn to the kinds of questions that we would ask. Different evaluations may have their own unique questions.  But in general, we’d ask questions such as, what were the goals and objectives in the plan?  How did the organization perform based on the strategic intent stated in the plan and its related goals?  We’d also ask how effective did the organization use the plan to manage the delivery of its results?  That is, the priority, the focus areas, the approaches, and accountability.  Another question is: how easily or difficult did the plan make the performance management and measurement process?  Because if it is just a summary of a plan and not fully elaborated and support by measurable indicators, there’s going to be trouble at the execution stage.

An important question to ask is: does the plan include an alignment mechanism for cascading and aligning?  We also look at whether or not the major initiatives and commitments were delivered on time and in budget.  If there were deviations, how wide was the spread and what needs to be changed.  Therefore, if they finished before schedule, were late or on time, we want to understand the reasons.  We also look at the overall workings of the plan logic based on the theory or theories of change or the strategy maps and strategy framework or income output map, or any combination of them.  We also look at whether or not the scope of operations is made clear by the plan.  How suitable was the initiatives or initiatives for building capacity and advancing the strategic direction articulated in the plan— is another area which we examine.

So, to arrive at a judgment we’ll use multiple sources of qualitative and quantitative evidence.  We look at the use of the plan and the process and the annual and periodic review and strategy update.  Are they following that guide and are they updating the plan?  We ask if a priority trade off happened and, if so, why?  We also look at the effectiveness of the plan in communicating to the board, to management, to partners, to funders, and staff.  Do they understand the plan or do they find it to be a burdensome document?  Does it address the value chain and how they’re going to make product relationships and leverage them in terms of the products or services or the supply chain? 

Moreover, we analyze how effective the plan is in articulating the strategic identity.  So that’s how we do that post-implementation evaluation.

See also:

Interview Transcript on IAF

Strategy Execution Challenges

Pre-implementation evaluation of the corporate strategy plan

Monitoring of the corporate strategy plan

—————–

ASPConf2018, the Association for Strategic  Planning Conference, an event for your professional development and business calendars.

Copyright © 2018 International Affairs Forum, Association for Strategic Planning, Magate Wildhorse, Meegan Scott
All Rights Reserved

iaf_logo_blackmagatelogogreen