Evaluating the performance of an organisation

An organisational assessment is a systematic process for obtaining valid information about the performance of an organisation and the factors that affect performance. It differs from other types of evaluations because the assessment focuses on the organisation as the primary unit of analysis.

Organisations are constantly trying to adapt, survive, perform and influence. However, they are not always successful. To better understand what they can or should change to improve their ability to perform, organisations can conduct organisational assessments. This diagnostic tool can help organisations obtain useful data on their performance, identify important factors that aid or impede their achievement of results, and situate themselves with respect to competitors. Interestingly, the demand for such evaluations is gaining ground. Donors are increasingly trying to deepen their understanding of the performance of organisations which they fund (e.g., government ministries, International Financial Institutions and other multilateral organisations, NGOs, as well as research institutions) not only to determine the contributions of these organisations to development results but also to better grasp the capacities these organisations have in place to support the achievement of results.

Example

Examples of Application of Organisational Assessments

• The Multilateral Organisation Performance Assessment Network (MOPAN) is a group of 16 donor countries that have joined forces to assess the performance of the major multilateral organisations which they fund. MOPAN has developed an assessment approach that draws on perceptions and secondary data (i.e., documents) to assess the performance of organisations with a focus on their systems, behaviours, and practices (or capacities). The exercise is used to encourage discussion among donors and multilateral organisations about ways to enhance organisational effectiveness.
• In 2011, an evaluative report was disseminated by the International Monetary Fund (IMF) regarding its performance leading up to the global financial and economic crisis. Among the factors that hampered the organisation’s ability to detect important vulnerabilities and risks, the report highlights the pervasiveness of cognitive biases and groupthink as well as the operational structure of the organisation: on the one hand, it was widely believed in the organisation that a financial crisis could not happen in a large advanced economy and on the other, the existence of a silo mentality prevented information from being shared across units and departments to help predict the crisis. The assessment results are being used by the IMF’s board and executive management to revise how the organisation operates.
• The Center for Effective Philanthropy developed a conceptual framework for assessing the performance of foundations. This framework provides a way for a foundation to infer the social benefit created by its activities relative to the resources it invests, and aims to allow its leaders to understand the performance of their organisation over time and in relation to other foundations.

Frameworks

A number of models or frameworks for conducting an organisational performance assessment exist. The choice of which framework (or combination of frameworks) to use depends on the nature of the organisation, on the purpose of the assessment, and on the context in which the assessed organisation operates. The Reflect and Learn website presents details on the rationale and particularities of various frameworks. As highlighted in a paper presented on the Impact Alliance website, it is important to note that different frameworks are underpinned by different philosophies and theories of organisational change; an organisation should choose a framework that is congruent with its own management beliefs and culture, to ensure that it fully engages in the process and truly benefits from the assessment (EDITOR: The link for this paper is no longer working. Please bear with us while we find an alternate link for this source. July 5, 2016).

One of the most comprehensive frameworks for Organisational Performance Assessment (OPA) is the Institutional and Organisational Assessment Model (IOA Model) elaborated by Universalia and the International Development Resource Center (IDRC). This model views the performance of an organisation as a multidimensional idea, that is, as the balance between the effectiveness, relevance, efficiency, and financial viability of the organisation (see schematic diagram below). The framework also posits that organisational performance should be examined in relation to the organisation’s motivation, capacity and external environment. Indeed, a review of the literature conducted as a preliminary step for developing the framework showed that organisations change: in response to factors in their external environment, because of changes in their internal resources (e.g., financial, technological, human), and as a result of fundamental shifts in values within the organisation, which in turn affect the organisational climate, culture and ways of operating. The book Organisational Assessment: A Framework for Improving Performance by Lusthaus et al. further details the IOA Model and discusses key methodological issues for carrying out an OPA. Meanwhile, Enhancing Organisational Performance: A Toolbox for Self-Assessment by Lusthaus et al. provides tools and tips for organisations wishing to conduct an OPA. Both these resources are publically available (in French and English) on the IDRC website.

Source: Universalia Institutional and Organisational Assessment Model (IOA Model)

Key resource: As highlighted above, the Reflect and Learn website presents a range of frameworks for conducting organisational performance assessments. The site also introduces the process and management of OPAs and provides a database of concrete tools that organisations can use to carry out assessments.

Important considerations

A self-assessment or an external assessment?

A key decision that an organisation needs to make when undertaking an organisational assessment is whether to self-assess its performance, to commission an external assessment, or to use a combination of both approaches. Some advantages of a self-assessment are that it encourages the organisation’s ownership of the assessment, and thereby increases the latter’s acceptance of feedback and commitment to the evaluation’s recommendations. However, drawbacks of the self-assessment approach are that external stakeholders may question the independence or validity of the findings and may fear that hard issues will not be tackled, due to potential sensitivities within the organisation.

Which issues to prioritise?

As underlined by Kathleen Immordino in her book on organisational assessment in the public sector, the questions an organisation needs to ask as part of an assessment depend on the specific context of the organisation. “In any complex organisation, there are innumerable ‘things’ that can be measured and studied. An effective assessment process focuses on those things that have the greatest impact on the way the organisation functions.” Practical considerations which may guide the selection or prioritisation of key questions for an organisational assessment are:

1. the time required and resources available to answer each question;
2. the organisation’s purpose for conducting an assessment (for example, a desire to strengthen accountability or a desire to inform a new strategic planning cycle); and
3. the need to balance the interests of multiple stakeholders.

Once an organisation has a clear picture of what it wants to measure, it will need to identify what indicators (quantitative and qualitative) to use to assess its performance. This can be one of the more challenging steps in the organisational assessment process as a plethora of potential indicators may appear useful, but weeding out the ones that really matter and that answer the assessment questions can be difficult.

Challenges of selecting indicators as part of the organisational assessment process

• Measuring something within an organisation can increase its importance: for example, a social service NGO that chooses to track the number of people it serves within a community may end up trying to increase the number of people it visits and to reduce the time spent with each person (with potential consequences for the quality of the services rendered).
• Simple indicators may not always fit the bill and may need to be combined: Developing adequate indicators to measure the complex dynamics that exist within an organisation can be quite challenging. Organisations may develop a set of carefully considered indicators but need to modify them over time as they analyse their results.
• Indicators may be interpreted differently amongst stakeholders: For example, an indicator that measures the diversification of funding of an organisation to assess its financial viability can be viewed positively by certain stakeholders, as diversification signifies that the organisation is not overly reliant on a single donor. Meanwhile, other stakeholders may view this measurement in a negative light, as dealing with multiple donors can lead to fragmentation and increased organisational costs in order to manage multiple donor requirements (each donor may have its own priorities, expectations, systems, and evaluation and reporting requirements).

What to consider in selecting the options?

Organisational assessments follow the tradition of a case study methodology. A case study requires a research design that focuses on understanding the unit (the organisation) and can use a combination of qualitative and quantitative data. The choice of options depends on the specific circumstance for the organisation and its stakeholders. We have found that observation (site visits), document review, interviews and surveys are some of the most common options used. Site visits and observation provide vital information on the facilities, physical artifacts, and interactions between staff of an organisation. Meanwhile, a document review is used to follow the written record of the organisation: meeting minutes, reports, policies, etc. Interviews are a prime source of data for organisational assessments and should be conducted with a wide range of respondents (both male and female). Surveys are particularly useful for gathering data from a large number of people and for obtaining information regarding people’s attitudes, perceptions, opinions, preferences, beliefs, etc. These four data collection options can be used to triangulate information and validate conclusions: using more than one data source can help identify discrepancies between what people say and what people do, as well as between what the organisation is and what it ought to be.

Factors that support the use of results of the assessment

Organisational assessment results have a wide variety of uses. For instance, they can be used by an organisation to build its capacity, to validate its work, to promote dialogue with funders or partners and to help devise its strategies for the future. However, to ensure that results of the organisational assessment are used, their use must be planned for by the organisation from the onset of the assessment, as well as considered throughout the implementation phase and even once reports have been submitted and disseminated. Some conditions which enhance the utilisation of the results are when:

• The purpose and benefits of the assessment are clear to the organisation’s stakeholders.
• The main focus of the assessment is on learning rather than on accountability.
• Internal leadership is identified to champion the process and results of the assessment.
• The organisational culture is one that supports use of positive and negative feedback in planning and managing change
• Stakeholders are involved in the assessment process (from the negotiation and planning stages).
• Stakeholders see the assessment as relevant, credible, transparent, of high quality, and the findings have face validity.
• The assessment team is able to communicate the intent of the assessment, their approach, and the results to senior staff and board members.
• The report is timely (i.e., produced at an opportune time within the planning cycle of the organisation).
• There is a process in place and resources allocated to implement and follow-up on the assessment’s recommendations.
• Recommendations are realistic and feasible (for example, financially).

Resources

Examples

This is a report by the Independent Evaluation Office (IEO) of the International Monetary Fund (IMF) regarding its performance leading up to the global financial and economic crisis.

Guides

This book, downloadable as e-book or pdf covers all aspects of organisational performance, including the enabling environment, institutional capacity, management, financial viability, and staff motivation.

This book provides a tested process for organizational self assessment. It includes tools and techniques and formative assessment practices to help organisations evaluate if they are meeting their goals effectively.

Websites

This website presents details on the rationale and particularities of various frameworks for conducting an organisational performance assessment.