THE ROLE OF BEHAVIORAL FACTORS IN THE SUCCESSFUL
IMPLEMENTATION
AND USE OF PERFORMANCE MANAGEMENT SYSTEMS
André A. de Waal
- Published in A. Neely, A. Walters & R. Austin (ed.), Performance Measurement and
Management: Research and Action,
Cranfield School of Management, UK. This paper received the Highly
Commended Paper award at the Third International Performance Measurement
& Management Conference, held in July in Boston, USA. The paper is a
summary of the first part of the book Quest for Balance (John Wiley & Sons, 2002). -
Abstract
This paper examines the role behavioral factors play in
the successful implementation and use of performance management systems that
are based on critical success factors, key performance indicators and the
balanced scorecard. Case study research was performed which identified 18
individual behavioral factors to be important. The research also showed that
the use stage in a systems implementation project has to be performed well in
order to assure a regularly used performance management system.
Introduction
Performance management systems are defined as “the formal,
information-based routines and procedures managers use to maintain or alter
patterns in organizational activities” (adapted from Simons, 2000). These
systems focus on conveying financial and nonfinancial information that
influence decision making and managerial action. The recording, analyzing, and
distributing of this information is embedded in the rhythm of the organization
and is often based on predetermined practices at preset times in the business
cycle. These systems are designed specifically to be used by managers.
According to Neely (2000), there is a natural evolutionary cycle at work in the
development of theory and practice in the field of performance measurement and
control systems. During this cycle, managers were first concerned that they
were measuring the wrong things (late 1980s and early 1990s). After struggling
with the adoptation of new and alternative systems, like the balanced scorecard
(throughout the 1990s), they now turn to the issue of how to use the data
provided by these new systems (late 1990s and early 2000s). Zairi and Jarrar
(2000) state that the main reason for managers to use data from the performance
management system is to influence the behavior of subordinate managers and
employees. To do so successfully, these managers need a clear view of human
nature and behavior in organizations. Simons (2000) gives several assumptions
about the nature of human activity in organizations: (1) people in
organizations want to contribute to an organization of which they can be proud
of; (2) people employed by business organizations also know the difference between
right and wrong, and generally choose to do right; (3) people strive to achieve
– even in the absence of external inducements (money, promotion, praise) people
often set a personal goal for themselves; (4) people like to innovate – they
have an innate desire to experiment by
creating new technologies and new ways of doing things; and (5) people want to
do competent work, a job well done allows them to exercise their skills and
receive satisfaction from their competence. Simons concludes that people like
to have and show good performance.
The human element in performance management
Performance can be considered an outcome of both
organizational and human activities. Originally, performance measures were used
as surrogates for these outcomes, and a direct link between performance
management systems, human nature, and outcomes was not made. This omission was
addressed by Argyris (1952) and later on by Simon et al. (1954). They explored
the human behavioral side of performance management system use, looking
specifically at the budgeting system. Both concluded that budgets and budgeting
processes could be associated with important human relation problems. These
included worker–management separation, cross-boundary conflict, and job-related
tension. Their conclusions were substantial departures from the mechanistic
approach to performance measurement found in traditional management theory.
Nowadays, the issue of the “human element” receives more
than before attention in the literature. Simons (2000) states that performance
measurement and control systems cannot be designed without taking into account
human behavior. Holloway et al. (1995) argue that successful implementation of
performance measurement depends above all on understanding and accommodating the
human element. A closer look at the literature reveals that a lot of this
attention for the human element seems to be still focused on its relationship
to the budgeting system. In this respect, Hartmann (2000) remarks that it
should be investigated whether personality factors related to individual
preferences for risk and uncertainty are important determinants of managerial
behavior and attitudinal reactions to budgeting. And Vagneur and Peiperl (2000)
state that individual psychological responses to performance assessment should
be investigated, taking into account research from the fields of psychology,
organizational behavior, behavioral accounting, and systems theory. Next to
this, a lot of performance management research focused on the technicalities of
implementing a performance management system rather than on behavioral issues
(Martins, 2000). In recent years, an increasing number of organizations have
implemented performance management systems that are based on critical success
factors (CSFs) and key performance indicators (KPIs). A frequently used format
in this context is the balanced scorecard (BSC) (Kaplan and Norton, 1996).
Despite the increase in experience gained with these systems, there is still a
lot to be learned about the factors that influence effective use of CSFs, KPIs,
and the BSC (Vosselman, 1999). The influence of users’ characteristics on the
use of a performance management system has been underexposed in scientific and
professional literature (Vagneur and Peiperl, 2000; Krause, 2000).
Two recent studies into the behavioral
aspects of performance management system implementation and performance
management system use aim at filling this void. Lipe and Salterio (2000) found
that managers’ cognitive limitations may prevent organizations to fully benefit
from a performance management system, and that cognitive differences between
managers may lead them to use the performance management system differently.
Malina and Selto (2000) found that positive outcomes from performance management
system use were mostly determined by the effectiveness by which the system is
used as a management control device (defined in terms of effective measurement,
comprehensive performance, and weight of the measurement dimensions), while
these outcomes were not attributable to its use as a communication device.
Positive outcomes are generated by better strategic alignment of employees and
better motivation, which indicates that causal relationships exist between
performance management system design, management control use, managerial and
employee behavior, and performance.
In this paper the line of research
into the behavioral aspects of performance management system implementation and
use is extended by addressing the research question Which behavioral factors contribute to the successful implementation
and use of a performance management system? (de Waal, 2002). A performance
management system is regarded successful if managers use the system on a
regularly (daily) basis. The research question is answered by studying three
organizations that have designed and implemented a performance management
system. The research aims to identify the behavioral factors that are
responsible for the successful design and implementation of a performance
management system.
Since the objective of the research is to identify which
behavioral factors are important to the successful implementation and use of a
performance management system, criteria for regular use have been formulated on
the basis of literature (Bruijn, 1994; Gelderman, 1998a, 1998b). These criteria
denote when use of the performance management system, and its CSFs, KPIs and
BSC is valuable to the organization and its managers. The criteria are a mix of
tangible and intangible benefits but focus more on the intangibles (Mooraj et
al., 1999). In the criteria for regular use the ideas of Lewy and Du Mée (1998)
are included, who argue that successful implementation and use of a performance
management system does not necessarily mean that the organization has its
performance management system embedded in the planning and control cycle with
periodic reporting and discussion. In their opinion, a successful
implementation and use of a performance management system can already be
achieved when the managers have an intensified awareness of the importance of
the performance management system. The criteria for regular use are given in
Exhibit 1, in the format of interview questions.
|
Criteria for Regular Use
|
|
Are the results
of the organization, according to managers, improved through the use of the
performance management system?
|
|
Are the results
of the organization, objectively, improved through use of the performance
management system?
|
|
Has the degree
of performance management system use by managers increased?
|
|
Are there plans
for follow-up projects?
|
|
Is there a
difference in manager attitude toward performance management, from project
start to currently?
|
|
Is there
regular communication about KPI results?
|
|
Are the CSFs,
KPIs and BSC incorporated in the regular management reporting?
|
Exhibit 1: Criteria
for regular use
Classification scheme
The scientific and professional literature studied mentions
many behavioral factors that are potentially important to successful
implementation and regular use of a performance management system. Examples
given are: “Managers accept the need for performance management” and “Managers
accept the promoter”. These factors have been grouped and arranged in a
classification scheme (Exhibit 2).
Exhibit 2: Classification
scheme of behavioral factors
This classification scheme was
developed by linking the factors of effective control as given by De Leeuw
(1990) with the control cycle of
performance measurement as given by Van Tuijl et al. (1995). For effective control,
the controlling system (the superior of a manager) and the controlled system
(the manager) need a performance management system. Through the performance
management system, the controlling system gets information about the
performance of the controlled system and the controlled system obtains
information about its own performance. The internal and external environments
in which the controlling and controlled systems operate also influence the
effectiveness of control. In the performance management system, the development
method part describes the way in which CSFs, KPIs, and the BSC are developed.
The content part gives the quality criteria that CSFs, KPIs, and the BSC have
to meet in order to be relevant to both controlling and controlled system. The feedback
part describes the way in which information about CSFs, KPIs, and the BSC is
conveyed to both controlling and controlled system. Each part of the
classification scheme can be divided into subparts. For each subpart,
behavioral factors can be found in the literature, so that it can become clear
how each part of the classification scheme can be influenced favorably.
Method
To answer the research question, case
study research was conducted at three Dutch organizations: a nonprofit
organization, a profit company, and an organization in transition from
nonprofit to profit. All organizations had, at the time of the research,
extensive experience with CSFs and KPIs. The purpose of the case study research
was to identify the behavioral factors that
are the most important to the implementation and regular use of the
performance management system at those organizations. Generally, in a
performance management system implementation project, three stages can be
distinguished: (1) the starting stage
(S), in which the organization decides to implement a performance management
system; (2) the development stage
(D), in which CSFs, KPIs, and the BSC are developed; and (3) the use stage (U), in which the organization starts to use the
performance management system (Kerklaan et al., 1994; Kaplan and Norton, 1996).
In each stage, identification took place of those behavioral factors that were
the most important to a positive end result of that stage and the overall
project. In addition, the stage that was the most important to the overall
success of the project was identified. In total, forty behavioral factors were
researched (Exhibit 3).
|
Classification Scheme Part
|
Subpart
|
Behavioral
Factor
|
Influence on Stage
|
|
Performance management
|
Development method
|
Managers accept the need for
performance management.
|
S (1)
|
|
system – Development
|
|
Managers have an active role during the development stage of the performance management
system project.
|
D (1)
|
|
Method
|
|
Managers agree on the starting time.
|
S (2)
|
|
|
|
Managers have been involved in decision making
about the project starting time.
|
S (3)
|
|
|
|
Managers are informed about the status of the
performance management system project.
|
D (2)
|
|
|
|
Managers are actively communicating about the
performance management system project.
|
D (3)
|
|
Performance
|
Quality
|
Managers understand the meaning of KPIs.
|
D (4)
|
|
management
|
|
Managers are involved in defining KPIs.
|
D (5)
|
|
system – Content
|
|
Managers have insight into the relationship between KPIs and financial results.
|
U (1)
|
|
|
Registration
|
Managers do not get discouraged by the collection
of performance data.
|
U (2)
|
|
|
Purpose
|
Managers have insight into the relationship between strategy and CSFs/KPIs.
|
D (6)
|
|
|
|
Managers have insight into the relationship between business processes and CSFs/KPIs.
|
D (7)
|
|
|
Targets
|
Managers are involved in
setting KPI targets.
|
D (8)
|
|
|
Balance
|
Managers’ KPI sets are aligned with their
responsibility areas.
|
D (9)
|
|
|
|
Managers have insight into the relationship between cause and effect.
|
U (3)
|
|
Performance
|
Feed forward
|
Managers are involved in
forecasting.
|
U (4)
|
|
management
|
|
Managers trust good-quality forecasts.
|
U (5)
|
|
system –
|
|
Managers’ activities are supported by KPIs.
|
U (6)
|
|
Feedback
|
|
Managers’ frames of reference contain similar
KPIs.
|
U (7)
|
|
Performance
|
Feedback
|
Managers are involved in
making the CSF/KPI/BSC reporting layout.
|
D (10)
|
|
management
|
|
Managers understand the CSF/KPI/BSC reporting.
|
D (11)
|
|
system –
|
|
Managers trust the performance information.
|
U (8)
|
|
Feedback (ctd)
|
|
Managers are involved in
making analyses.
|
U (9)
|
|
|
|
Managers trust good-quality analyses.
|
U (10)
|
|
Controlled system
|
Management level
|
Managers use the CSFs/KPIs/BSC
that match their responsibility areas.
|
D (12)
|
|
|
|
Managers’ information processing capabilities are
not exceeded by the number of CSFs/KPIs.
|
U (11)
|
|
|
|
Managers have enough time to work with their
CSFs/KPIs/BSC.
|
U (12)
|
|
|
Management style
|
Managers have earlier (positive) experiences with
performance management.
|
S (4)
|
|
|
|
Managers realize the importance of CSFs/KPIs/BSC
to their performance.
|
U (13)
|
|
|
|
Managers do not experience CSFs/KPIs/BSC as
threatening.
|
U (14)
|
|
|
|
Managers can use their CSFs/KPIs/BSC for managing
their employees.
|
U (15)
|
|
Controlling
|
Responsibility
|
Managers can influence the KPIs assigned to them.
|
D (13)
|
|
system
|
|
Managers have sole responsibility for a KPI.
|
U (16)
|
|
|
Supervision
|
Managers accept the promoter.
|
D (14)
|
|
|
|
Managers see the promoter spends enough time on
the performance management system implementation.
|
D (15)
|
|
|
|
Managers clearly see the promoter using the
performance management system.
|
U (17)
|
|
|
Relationship with controlled system
|
Managers and their controlling systems
have a mutual trust.
|
U (18)
|
|
Internal environment
|
Alignment
|
Managers find the performance management system
relevant due to regular evaluations.
|
U (19)
|
|
|
|
Managers use the performance management system
regularly during the planning and control cycle.
|
U (20)
|
|
|
|
Managers agree on changes in the CSF/KPI set.
|
U (21)
|
|
|
Organizational
|
Managers are stimulated to improve their
performance.
|
U (22)
|
|
|
culture
|
Managers work in a stable, relatively tranquil
environment.
|
S (5)
|
|
|
|
Managers’ results on CSFs/KPIs/BSC are openly
communicated.
|
U (23)
|
|
|
|
Managers’ use of the performance management system
is stimulated by the reward structure.
|
U (24)
|
|
External environment
|
External environment
|
D (16)
|
|
|
|
|
Managers find the performance management system
relevant because it has a clear internal control purpose.
|
D (17)
|
Exhibit 3: Overview
of the behavioral factors
Results
The research question was investigated by applying pattern
matching, which allows patterns to be discerned between the various scores of
the cases. These patterns tell us which behavioral factors, theoretically
predicted to be important, coincide with the criteria for regular use. Pattern
matching is applied to identify patterns between the scores on the individual
behavioral factors and the criteria for regular use, and between the end scores
for the three stages and the scores for the criteria for regular use. The
assumption in pattern matching is that the behavioral factors are independent.
This is why the factors have not been weighed. For pattern matching, a complete
match between the scores of all cases gives a complete coincidence, indicating
that these behavioral factors seem to have a general similarity with a
successful implementation and use of a performance management system. These behavioral
factors can consequently be considered to be essential. A match between three
or two scores gives a partial coincidence, which means that these behavioral
factors have a partial similarity with the criteria for regular use. These
behavioral factors may be important to the successful implementation and use of
a performance management system. Finally, a match between one or none of the
scores indicates there is no coincidence, which means that these behavioral
factors may not be important to the successful implementation and use of a
performance management system. Pattern matching was applied for individual
behavioral factors and for individual stages.
|
Classification
Scheme Part
|
Areas
of Attention to
|
Behavioral
Factors
|
|
Performance management system
|
Managers’
understanding
–
A good
understanding by managers of the nature of performance management
|
§ D4. Managers understand the meaning of KPIs.
§ D7. Managers have insight into the relationships
between business processes and CSFs/KPIs.
§ U7. Managers’ frames of reference contain similar KPIs.
§ U21. Managers agree on changes in the CSF/KPI set.
|
|
Controlled system
|
Managers’
attitude –
A positive
attitude of managers toward performance management, toward a performance
management system and toward the project
|
§ S2. Managers agree on the starting time.
§ S4. Managers have earlier (positive) experiences with performance
management.
§ U13. Managers realize the importance of CSFs/KPIs/ BSC to their
performance.
§ U14. Managers do not experience CSFs/KPIs/BSC as threatening.
|
|
Controlling system
|
Performance
management system alignment –
A good match
between managers’ responsibilities and the performance management system
|
§ D9. Managers’ KPI sets are aligned with their responsibility areas.
§ D13. Managers can influence the KPIs assigned to them.
§ U9. Managers are involved in making analyses.
§ U15. Managers can use their CSFs/KPIs/BSC for managing their employees.
|
|
Internal environment
|
Organizational
culture –
An
organizational culture focused on using the performance management system to
improve
|
§ U23. Managers’ results on CSFs/KPIs/BSC are openly communicated.
§ U22. Managers are stimulated to improve their performance.
§ U8. Managers trust the performance information.
§ U17. Managers clearly see the promoter using the performance management
system.
|
|
External environment
|
Performance
management system focus –
A clear focus
of the performance management system on internal management and control
|
§ D16. Managers find the performance management system relevant because
it has a clear internal control purpose.
§ D17. Managers find the performance management system relevant because
only those stakeholders’ interests that are important to the organization’s
success are incorporated.
|
Exhibit 4: Overview
of the important behavioral factors
The results of the pattern matching indicate that there are
18 individual behavioral factors that coincide with the final score for the
criteria for regular use. The scores for the use stage coincide completely with
the final scores for the criteria for regular use. In other words, it seems
there is a relationship between a well-executed use stage and a good final
score. The scores for the starting and development stages, on the other hand,
coincide partially or not at all with the scores for the criteria for regular
use. This tells us that there is no relationship between how well these stages
have been executed and the final score. So, even a well executed starting
and/or development stage is no guarantee for a good final score, that is, a
regularly used performance management system. It is possible to group the 18
important behavioral factors together in categories in such a way that an
overview appears of the areas an organization has to pay special attention to
increase the chance of implementing a new performance management system that
will be regularly used (Exhibit 4).
Discussion
The research results indicated that
special attention should be paid to 18 specific behavioral factors. In
addition, the use stage turned out to be the most important to the success of
the performance management system. For the starting and development stages,
such a clear relationship was not found. This does not mean that, during these
stages, an organization should not pay attention to the behavioral factors that
are important to these stages. The three stages are executed sequentially,
which means that the first two stages must be executed properly before the use
stage can be started. The fact that the use stage contributes most to the
success of a performance management system may be explained by the fact that
this stage is, in contrast to the starting and development stages, a continuous
stage. The consequence of this is that the behavioral factors that are
important to the use stage have to be monitored continuously to ensure regular
use of the performance management system. In contrast, the attention for
behavioral factors that are important to the starting and development stages
lies in the past and therefore becomes less significant and visible through
time.
The aim of the research was to identify behavioral factors
that are important to the successful implementation and regular use of a
performance management system. Initially, the research concentrated on
identifying behavioral factors that the literature indicated as being of influence
on successful performance management system use. To keep the scope of the
research manageable, a selection was made of the behavioral factors that were
mentioned in the literature. Consequently, potentially influential factors may
thus have been left out of the study. Therefor, a worthwhile avenue of further
study is to look at additional behavioral factors. Further research is also
needed into other factors, such as environmental or organizational factors.
This research may yield more factors that are of great importance to successful
implementation and use of a performance management system. The research results
show that the use stage is the most important to the success of the performance
management system. This means that further study should concentrate on this
stage in order to discover (further) reasons why organizations do not use a
newly implemented performance management system. Research is also needed into a
“maintenance” system that makes sure that organizations, and its managers,
continue to pay attention to the behavioral factors after the performance
management system is put into use in order to make sure that the performance
management system remains a success. As the case study organizations examined
did not yet dispose of a reward system that was linked to the performance
management system, further study should pay special attention to the role of
the reward system in the maintenance system.
References
Argyris, C.
(1952),The impact of budgets on people, The Controllership Foundation,
Cornell University. Referred to by: Vagneur, K. and M. Peiperl (2000),
“Reconsidering performance evaluative style”, Accounting, Organisations, and
Society, 25
Bruijn, R.P.(1994), ‘De Balanced
Scorecard in de praktijk’ [transl. ‘The Balanced Scorecard in practice’], Handboek
Management Accounting, September, D1815:1–22
Gelderman, M. (1998a), A
performance measurement framework, research memorandum, Limperg Instituut,
Amsterdam
Gelderman, M. (1998b), Usage of
performance measurement and evaluation systems: the impact of evaluator
characteristics, Free University Amsterdam and Limperg Instituut, Amsterdam
Hartmann, F.G.H. (2000), ‘The
appropriateness of RAPM: toward the further development of theory’, Accounting,
Organizations and Society, 25:451–482
Holloway, J., J. Lewis and G. Mallory
(1995), eds, Performance measurement and evaluation, Sage Publications,
London
Kaplan, R. S. and D.P. Norton (1996),
The balanced scorecard, translating strategy into action, Harvard
Business School Press, Boston
Kerklaan, L.A.F.M., J. Kingman and
F.P.J. van Kleef (1994), De cockpit van de organisatie [transl. ‘The
cockpit of the organization’], Kluwer Bedrijfswetenschappen, Deventer
Krause, O. (2000), ‘Management
knowledge engineering – a toolkit to engineer adaptive management systems’. In: A. Neely, ed., Performance measurement – past, present, and future, Centre for Business Performance,
Cranfield University, Cranfield:307–314
Leeuw, A.C.J. de (1990), Organisaties:
management, analyze, ontwerp en verandering: een systeemvisie [transl.
‘Organizations: management, analysis, design and change: a system view’], Van
Gorcum, Assen
Lewy, C.P. and A.F. du Mée (1998),
‘In de kaart laten kijken, de tien geboden bij BSC-implementaties, versie 1.0’
[transl.’Show one’s cards, the ten commandments of BSC implementations, version
1.0’], Management Control & Accounting, 2:32–37
Lipe, M.G. and S.E. Salterio (2000),
‘The balanced scorecard: judgmental effects of common and unique performance
measures’, Accounting Review,75, 3:283–298
Malina, M.A. and F.M. Selto (2000),
‘Communicating and controlling strategy: an empirical study of the
effectiveness of the balanced scorecard’, paper presented at the AAA Annual
Conference, Philadelphia, August 13–16
Martins, R.A. (2000), ‘Use of
performance measurement systems: some thoughts towards a comprehensive approach’.
In: A. Neely, ed., Performance
measurement – past, present, and future,
Centre for Business Performance, Cranfield University, Cranfield: 363–370
Mooraj, S., D. Oyon and D. Hostettler
(1999), ‘The balanced scorecard: a necessary good or an unnecessary evil?’, European
Management Journal,17, 5:481–491
Neely, A. (2000), ed., Performance
measurement – past, present, and future, Centre for Business Performance,
Cranfield University, Cranfield
Simon, H., H. Guetzkow, K. Kozmetsky,
and G. Tyndall (1954), ‘Centralization vs. decentralization in organizing the
controllers department’, paper, Controllership Foundation
Simons, R. (2000), Performance
measurement and control systems for implementing strategy: text & cases,
Prentice Hall, Upper Saddle River, NJ.
Tuijl, H.F.J.M. van, P.A.M. Kleingeld
and J.A. Algera (1995), ‘Prestatiemeting en beloning: contextafhankelijk
ontwerpen’ (transl. Performance measurement and reward: context-bound design), Gedrag
en Organisatie, 8, 6:419–438
Vagneur, K.
and M. Peiperl (2000), “Reconsidering performance evaluative style”, Accounting,
Organisations, and Society, 25
Vosselman, E.G.J. (1999), Accounting
en gedrag: zichtbare en onzichtbare effecten van management accounting [transl.
‘Accounting and behavior: visible and invisible effects of management
accounting’], Kluwer, Deventer
Waal, A.A. de
(2002), Quest for Balance, the human
element in performance management systems, John Wiley & Sons, New York
Zairi, M. and Y.Jarrar (2000),
‘Becoming world class through a culture of measurement’. In: A. Neely (ed.), Performance
measurement – past, present, and future,
Centre for Business Performance, Cranfield University, Cranfield:688–694
No comments:
Post a Comment