Dissertation Examining Performance Evaluation Metrics Relating to Revenue Management: A Case Study of a North-American Hydroelectricity Organization

Examining Performance Evaluation Metrics Relating to Revenue Management: A Case Study of a North-American Hydroelectricity Organization

Performance evaluation metrics are an important part of the performance management system for organizations. In that context, this study aimed at examining performance evaluation metrics relating to revenue management accounting in the context of the North-American hydroelectricity market. The study is presenting the performance evaluation metrics in the performance management system’s context as it is important to look at the metrics in their context rather than in isolation.

A review of literature on the subjects of performance management and revenue management in the context of hydroelectric companies reveal that there is limited linkage between the two subjects. However, it has clearly demonstrated that a proper choice of performance metrics drives performance and that it is important to measure the organization’s goals in order to increase the probability of these goals being achieved.

Through the use of performance management frameworks, a questionnaire was developed and submitted to ten individuals at Brookfield Renewable Power Inc., the studied organization. Semi-structured interviews were conducted with five individuals at the firm who are currently working in the Energy Marketing division, the revenue management division of the organization.

The research reviewed the possibility of using complex models in order to establish benchmarks for revenue management performance in the context of hydroelectricity and energy trading. It also documents the main performance metrics used by the selected organization as part of its revenue management evaluation, metrics that were not found by the researcher in literature. These include the evaluation of peak price management and internal transfer pricing using financial transactions at negotiated market prices. Further research with other participating firms could help determine if these metrics could serve as best practices for the industry. The research suggests that management goals that are evaluated with the use of properly defined metrics will develop the right behavior and increase the performance related to the evaluated goals. It finally suggests that management prefer financial rather than non-financial metrics and that they prefer simple metrics that are easy to explain to non-initiated individuals such as public shareholders, rather than complex metrics that will aim at finding the optimal revenue capability.

Table of Content
Chapter 1 - Introduction
1.1 Introduction to the Research
1.2. The Organization
1.3 Background to the Research Questions
1.4 Research Questions
1.5 Conclusion

Chapter 2 - Literature Review
2.1 Introduction to the literature review
2.2 Performance Management Systems
2.2.1 Frameworks
2.2.2 Maturity Assessment
2.3 Performance Metrics
2.4. Link Between Measurements and the Hydroelectricity Market
2.4.1 Physical Market
2.4.2 Financial Market
2.4.3 Integration of the Physical and Financial Markets
2.5 Balancing between detailed and simple information
2.6 Conclusion

Chapter 3 - Research Methodology
3.1 Introduction to the research methodology
3.2 Overall Methodological Approach
3.3. Case Study
3.4 Sampling Methodology
3.5 Data Collection Techniques
3.5.1 Data Collection Techniques - Questionnaire
3.5.2 Data Collection Techniques - Interviews
3.6 Data Collection
3.6.1 Data Collection - Questionnaire
3.6.2 Data Collection - Interview
3.7 Data Analysis
3.7.1 Data Analysis - Questionnaire
3.7.2 Data Analysis - Interview
3.8 Interpretation techniques and conclusions
3.9 Conclusion

Chapter 4 - Answering the Research Questions
4.1 Introduction to the analysis
4.1.2 Controlled and Uncontrolled Revenues
4.2. What is the current state of the organization’s performance management system?
4.2.1 Analysis
4.2.2 Observations
4.2.3 Assessment
4.3 What performance evaluation metrics are currently used by the organization to measure dispatch strategies, hedging decisions and ancillary services’ revenue management performance?
4.3.1 Spot Market Initiatives - Dispatch Strategies
4.3.2 Price Management and Hedging Strategy
4.3.3 Capacity and Ancillary services
4.3.4 Other marketing initiatives (Contract optimization)
4.3.5 Arbitrage revenue
4.3.6 Final remarks on the metrics
4.4 Are the current performance evaluation metrics aligned with the corporate goals?
4.5 What performance evaluation metrics are available in literature for hydroelectric and energy trading performance assessment?
4.6 Does management prefer simple evaluation metrics or complex evaluation metrics? Why?
4.7 Additional discussion
4.8 Conclusion

Chapter 5 Review, Recommendations and Conclusions
5.1 Introduction to the final chapter
5.2 Review of Results and Findings
5.2.1 Maturity Level
5.2.2 Strong Bias Towards Financial Metrics
5.2.3 Transfer Pricing
5.2.4 Alignment with Goals
5.2.5 Choosing Simple Metrics
5.2.6 Simple Trading Activities
5.3 Recommendations to BRPI
5.3.1 Beware of Likierman’s traps
5.3.2 Increase the number of measured goals
5.3.3 Make the Best of Available Data
5.3.4 Expenditures or Investments
5.4 Limitations of the Research
5.6 Conclusions

Reference List

Appendix A - Questionnaire
Appendix B - Interview Questions
Appendix C – Summary of Questionnaire Answers
Appendix D - Detailed analysis of the Performance Management System
Appendix E – Book Structure and Illustrative Example

Chapter 1 - Introduction

1.1 Introduction to the Research
In the field of management accounting, an important segment of the literature concentrates on cost accounting with specialized subjects such as activity based costing (Narong, 2009) and manufacturing costs (Noreen, 1987, Lowry, 1993). The researcher has observed that management accounting is often referred to in the accounting industry as ‘cost accounting’ due to the emphasis on costs. The researcher also observed throughout his work experience and studies in management accounting that revenue management accounting is not as present in literature and textbooks as cost accounting. As part of the management accounting field, an important subject is performance management. Performance evaluation metrics has been an important focus for many organizations (Daly, 2011) and their upper management teams. However, performance metrics often relate to cost management rather than revenue management.

In performance management, an old adage says that we manage what we measure. In that context, revenue management, with the help of performance evaluation metrics, can have an important impact on how an organization manages its revenues (Pavlov and Bourne, 2011). This is true in the context of the hydroelectricity industry where costs are relatively fixed (BRPI, 2011) and revenues make the bottom line fluctuate. This is mainly due to important market price fluctuations and unpredictability of water inflows (Thompson, Davidson and Rasmussen, 2004).

This dissertation aimed at examining performance evaluation metrics relating to revenue management accounting in the context of the North-American hydroelectricity market. It concentrated on a specific organization within the industry where the researcher has worked for numerous years, Brookfield Renewable Power Inc (hereinafter ‘BRPI’). This dissertation distinguishes itself in its focus on revenue management and also due to its attempt to bridge the gap between the mathematical models presented in theory and the practical application in the corporate setting for organizations in the hydroelectricity industry.

There was little hydroelectricity specific management accounting literature found by the researcher as a practitioner in the hydroelectricity and energy trading field that related to revenue performance evaluation. The researcher observed little readily available or well known publicized information on best practices, benchmarks and methodology that could help the revenue management accounting practitioners in the hydroelectricity sector. The dissertation aimed to increase industry specific literature in that specialized sector. Mathematical and financial models have been published over the years in order to explain the optimal management of hydroelectricity plants and the behavior of financial instruments such as fixed financial swaps and options, as well as the electricity spot market prices. Some of the key financial and physical instruments used in that sector will be presented in the Literature Review chapter.

1.2. The Organization
The dissertation is a case study of BRPI an organization that owns and operates hydroelectric plants (BRPI, 2011). The organization has assets with a high degree of flexibility in generation (reservoirs) which are subject to both hydrology and market price fluctuations. It also owns other assets with little to no flexibility (hydroelectricity and wind power), the output of which is either sold under fixed price contracts or in the open market. The organization attempts to extract as much value as possible from the assets’ flexibility while presenting stable revenues for its shareholders, by protecting itself from current and future market price fluctuations (BRPI, 2011).

In order to keep the flexibility and growth while protecting itself from market fluctuation, various strategies are applied by the organization such as hedging the market price with financial products (electricity swaps), meaning that the organization fixes the future price of its generation while keeping the control of the assets in order to generate at the most opportune time. These decisions are evaluated based on some defined benchmarks that have been developed over the years and that are relatively simple in nature. They will be presented and discussed at length in chapter 4.

BRPI owns and operates more than 170 hydroelectricity facilities amounting to over 4,300 Megawatts of capacity. These facilities are situated in Canada, the United States of America and Brazil.

Source: BRPI (2011)

The organization sells electricity to corporate clients (such as paper mills and municipalities) under long term contracts and also sells directly on the electricity open markets (BRPI, 2011). BRPI is therefore exposed to various operational risks that are managed by one of its subsidiaries. These risks include volume risk (hydrology) and market risks related to both short term and long term market price fluctuations. The focus of the dissertation is on the performance of the subsidiary’s actors in managing the revenues of the organization. Further details of the organization’s operations will also be discussed in chapter 4. It should be noted that the Brazilian operations are not in scope for the current paper as it concentrates on the North-American operations.

The researcher was one of the actors who formed part of the discussions during the development phase of the revenue performance metrics (benchmarks) of the case study organization. The performance metrics were derived from what Lieberman and Raskin (2005) refer to as the administrative approach, whereby the prior year data and budgets formed the basis for the establishment of the performance benchmarks. Based on the researcher’s observations, the mathematical models have not been translated into performance evaluation metrics in the past at BRPI. This was due in part to the complexity of implementation with simple management tools such as Microsoft Excel rather than the use of Business Intelligence software. The subject organization is currently contemplating an investment in a robust financial information and management accounting system that would make the use of the complex models more feasible than with current systems as it would be easier to maintain and update than with Microsoft Excel. As part of this paper, it will be interesting to consider if the models were initially ignored because of mathematical complexity or because of system limitations. This will be investigated further as part of the interview questions presented below.

1.3 Background to the Research Questions
Malmi and Granlund (2009) made the argument that management accounting research shall be practical in nature and assist managers and their organizations in increasing performance. With that in mind, the researcher has focused this dissertation on a case study of a single organization, bearing in mind applicability for other organizations in the same field. To that end, the research tried to explain to what extent some of the factors influencing revenues can be controlled, managed or mitigated and how the decisions can be assessed through proper measurement that are practical and applicable. The research took into account published literature and a questionnaire to review the organization’s performance management system using a framework proposed by Ferreira and Otley (2009). The researcher also used interviews with various employees of the organization in order to properly examine the performance evaluation metrics of the organization as discussed in more details in chapter 3.

1.4 Research Questions
The dissertation aims at examining the performance evaluation metrics (benchmarks) relating to the revenue management decisions made by the actors that can influence the revenues of the organization. Performance evaluation metrics are a component of performance management systems and shall be looked at as part of that overall context (Ferreira and Otley, 2009). Therefore, the dissertation first assesses the performance management system of the organization in order to bring the performance evaluation metrics into context. In that regards, the following questions are investigated:
• What is the current state of the organization’s performance management system?
• What performance evaluation metrics are currently used by the organization to measure dispatch strategies, hedging decisions and ancillary services’ revenue management performance?

In an environment of relatively fixed costs and volatile revenues, an important part of the organization’s focus is on revenue performance evaluation metrics, which is deemed to be a good strategy, because having the performance evaluation metrics that encourage the right behavior shall result in increased performance (Pavlov and Bourne, 2011). In that regards, the following question is answered in the dissertation:
• Are the current performance evaluation metrics aligned with the corporate goals of the case study organization?

The research then examines other potential strategies and benchmarks that can be used to improve and assess performance, based on best practices found in literature. In other word, the following question will be answered:
• What performance evaluation metrics are available in literature for hydroelectric and energy trading performance assessment?

Performance evaluation metrics are often developed using management information such as past performance rather than external models. It is however important to find the balance between simple metrics and more complex metrics to ensure good benchmarking of performance and encouragement of the right behavior, as discussed in section 2.5. For instance, Johnston, Brignall and Fitzgerald (2002) argue that good enough information may be better than detailed models as managers will spend less time feeding the models and more time managing the organization. This provides context for the final question that link theory and practice in the context of the studied organization:
• Does management prefer simple evaluation metrics or complex evaluation metrics? Why? (based on literature and interviews)

In other words, between theory and practice, the researcher assessed if there is a balance to be found in order to increase performance while bearing in mind practicability and simplicity.

1.5 Conclusion
The current chapter introduced the subject or revenue management accounting, and more specifically revenue performance metrics in the North-American hydroelectricity sector. It briefly introduced the case study organization, provided the background to the research question and finally presented the questions that will be researched throughout the dissertation. In the next chapters, the researcher will present the literature review relating to the subject at hand, including literature on performance management and literature specific to the hydroelectricity sector and energy trading (Chapter 2). The researcher will then present the methodology used to gather the empirical data used as part of the research and analysis (Chapter 3). The methodology includes the use of a questionnaire administered to a group of professionals at various levels in the organization and semi-structured interviews with half of the participants in the questionnaires for more in-depth review of the organization’s metrics. Chapter 4 will present the analysis of the data and finally, the researcher will draw conclusions on the above questions being investigated in chapter 5.

Chapter 2 - Literature Review

2.1 Introduction to the literature review
As presented in Chapter 1, the dissertation concentrates on revenue management accounting and on bridging the gap between the hydroelectricity specific literature and management accounting application. This chapter is separated in different sections that review the various aspects related to the subject at hand and is intended to go from general to specific. Firstly, we discuss performance management literature in general and more specifically performance management systems and frameworks. Secondly, we review work related more specifically to performance metrics. Thirdly, we present the various types of revenues of the researched organization and we discuss various published models related to hydrology optimization as well as electricity market prices. Finally, we discuss the need for detailed models versus easy to understand, simple metrics, in the development and maintenance of performance metrics.

Both the subjects of performance management and electricity optimization models have been discussed extensively over the years. However, based on the researcher’s observations, they have not been specifically connected in literature in order to help develop strong performance metrics for practitioners in that industry. Following an assessment of BRPI, the researcher aimed at reviewing the possibility and desirability of linking the two subjects together into comprehensive performance measurements for the entity being studied and for other organizations in the same industry.

2.2 Performance Management Systems
In their effort to clearly distinguish performance measurement and performance management through their review of existing literature, which concentrates on published frameworks, Folan and Browne (2005) defined performance management as the use of performance measurement information to bring about positive change in the organization’s systems, processes and culture. In a similar type of research also based mostly on published frameworks, Ferreira and Otley (2009) described performance management as a holistic approach to the organization’s management and control of the performance. Finally, CMA-Canada (2010) has also defined performance management in their published performance management maturity framework as what organizations due to increase their value. They also are of the opinion that a performance management system has to be in place in order to be in a position to properly measure the performance through the metrics (CMA-Canada, 2010) which is also in line with previous work done by Waterhouse and Tiessen (1978) who also consider that the management accounting system needs to be looked at in the context of the overall organization, not in isolation. The researcher has embraced the view for which to understand the performance metrics used by an organization, a comprehensive look at the whole performance management system needs to be undertaken, which will be presented in chapter 4. It should be noted that the focus remains on the metrics rather than on the management system, which has a wider applicability rather than a focus on a specific organization and industry.

2.2.1 Frameworks
As discussed by Folan and Browne (2005) performance management literature is closely linked to manufacturing strategies, which concentrates on cost rather than revenue management. The current dissertation aimed to contribute in filling the gap in the current literature with a revenue management focus. Folan and Browne (2005) made an important suggestion to link performance management with the external world rather than look only at internal metrics. As discussed in section 2.4 below, this is an important concept for the hydroelectricity industry. In their review of performance management literature, Folan and Browne (2005) pointed out that some of the views contradict themselves. For example, some authors are referring to the need for sticking with quantitative measures versus other authors emphasizing on the use of qualitative information . Given that the current case study has a revenue management accounting focus, the researcher will concentrate mostly on quantitative performance measures.

Following a review Neely’s (1999) work which puts a lot of emphasis on the fact that measures should match strategy and that they should be clearly linked to the business processes, the researcher has decided to adopt the Ferreira and Otley (2009) framework in assessing the performance management system of the studied organization. This framework is in line with Neely (1999) in linking the vision and mission as well as the underlying strategy to the measurements. Also, in the framework, there is an important emphasis on business processes and use of the information, making it a complete framework that is aligned with the views of Neely (1999), Henri (2010) as well as Folan and Browne (2005). Ferreira and Otley’s (2009) framework is presented in more details in chapter 3 relating to its use in the research methodology. The framework is very comprehensive and built on important principles. For example, Ferreira and Otley (2009) emphasize on the importance to consider interdependency between the various metrics and controls in the organization when developing a performance management system. They also present an important risk related to performance management systems to the effect that they are not always designed to serve the organization as whole, but are rather focused on some divisions or internal groups.

An important challenge for the current topic is the fact that, as evidenced by the list of frameworks presented in Folan and Browne (2005), many of these frameworks concentrate on costs rather than revenues. Therefore, the researcher needs to adjust the existing work to revenue management as this is the main focus of the current dissertation.

Paulson Gjerde and Hughes (2009) discuss the fact that performance management systems are often viewed as ineffective by surveyed organizations because the system is used strictly as a financial measuring tool rather than a means to develop new strategies and shape the current strategies. Ferreira and Otley’s (2009) framework helps in assessing if the system in place helps in the organization’s strategies. As discussed above, Neely (1999) made an important point that measures should match strategy. Henri (2010) goes further in its review of performance measurement systems by stating that not only should measures match strategy but that measures have a limited life span before they are no longer relevant to help deliver the strategy .

Some tools have been developed over the years such as Kaplan and Norton’s (1996) famous Balanced Score Card. It was observed by Ittner and Larcker (2003) that the balanced scorecard has been applied in a generic rather than specific way in the past, resulting in a less focused tool than intended by their original authors. Therefore, these tools are not discussed in detail in the current paper as the researcher is focusing on metrics in a given industry rather than on tools that have a wider and less discriminatory application .

As suggested by Ferreira and Otley (2009), their framework is used as a preliminary analysis tool in chapter 4 and appendix D in order to understand the performance management system of the organization and help in the detailed analysis of the organization’s metrics.

The performance management system is an integrated view of the organization (figure 1). The figure is the researcher’s representation of systems discussed by various authors such as Ferreira and Otley (2009), Thompson, Strickland and Gamble (2009) as well as Folan and Browne (2005). The blue circle represents the flow, use and link of information in the organization. The Measures and Targets have been highlighted as they form the main subject of the current work.

2.2.2 Maturity Assessment
The Canadian Certified Management Accountants developed a performance management maturity framework (CMA-Canada, 2010) which the researcher used in Chapter 4 in order to assess the maturity level of BRPI’s system. The framework identifies four major categories which are (a) a list of enablers, (b) levels of maturity, (c) improvement techniques and (d) change management. The framework serves as a guideline to evaluate the level of maturity based on a four point scale, presented in Chapter 4 and a list of improvement techniques that are linked to the levels of maturity. The framework emphasized on the fact that an organization needs to attain a certain level of maturity before implementing some of the techniques presented. It is also important to note that the proposed techniques are a summary of what has been published over the years such as the Balanced Scorecard, Activity-Based Management, etc. (CMA-Canada, 2010 ).

2.3 Performance Metrics
As discussed above, the performance metrics (or measurements) are an important part of the performance management system. This forms the main part of the current paper as the aim is to find measurements that are relevant to the hydroelectricity industry. Finding industry relevant metrics is in line with the view of Paulson Gjerde and Hughes (2009) who presented the idea that performance measurements in literature are too generic and need to concentrate on the organization’s business as well as its key performance indicators rather than the generic key performance indicators that have been published over the years such as return on assets, gross margins and other financial measures. They suggest looking at non-financial measures that will drive the financial performance. However, as discussed in section 2.2 above, in practice, non-financial measures also tend to be generic rather than specific.

Performance measurement is an important part of an organization’s success as a competing entity. To that end, Pavlov and Bourne (2011) explain the effects of performance measurements on the organization and individual performances. They explain that performance measurements result in routines in the organization that will be built in order to achieve the targeted performance. The article provides important insight as to the consequences of choosing various performance measures versus others as this choice will impact behavior. Therefore, the wrong measurements can lead to the wrong routines and decreased rather than increased performance. The researcher has previously observed the creation of routines related to measurements during his work experience within the studied organization. In essence, the researcher is of the view that Pavlov and Bourne (2011) produced an article that puts the work of Ferreira and Otley’s (2009) in context relating to the importance of having the right measures in place as they will shape the actions of those evaluated by these measures.

This links well with Likierman (2009) who presents five common traps when using performance metrics. The first trap, which is in line with Paulson Gjerde and Hughes (2009) relates to the fact that organizations (1) put too much faith in numbers, more precisely they tend to use the same metrics as the competition just because competitors use them rather than because it is aligned to one’s strategy, as also suggested by Ferreira and Otley (2009) and by Neely (1999). Two other traps includes (2) sticking to the same numbers too long and (3) the risk of employees gaming the metrics. These two traps are in line with Henri’s (2010) research discussed above concerning the limited life of measures. (4) Measuring Against yourself and (5) looking backwards are important traps that needs to be looked at in the context of section 2.4 below as organizations often compare to their budgets rather than against the industry’s standards or the published models, in the case at hand. These five traps are relevant both in the evaluation of the current performance management system and the evaluation of interview responses in chapter 4. They provide additional insight into the responses received as part of the questionnaire and interview processes helping the researcher in its critical review of the answers provided. The five traps also helped the researcher develop the unstructured interview discussion.

2.4. Link Between Measurements and the Hydroelectricity Market
This is a research on management accounting and not on electrical generation and electricity market optimization. The researcher will therefore look at literature relating to these subjects with regards to potential utilization of more complex models, not with a critical view of the proposed models.

As discussed above, the performance measures need to be linked to strategy as they will have an important impact on the decisions and the routines of employees being measured. To that end, choosing metrics that are (a) aligned with the corporate strategy, (b) that are often reviewed and adjusted, and (c) that are considered relevant and fair by the employees, are key criteria to strong sustainable performance.

With that in mind, it is interesting to look at what has been published in regards to electricity companies in general. Tsamenyi, Cullen and Gonzalez (2006) who performed an extensive case study over numerous years on the evolution of the financial information system of a Spanish electricity company, and present the impact of both internal and external pressures on management accounting in the context of the studied company. As further discussed in Chapter 3, the methodology used by the researcher is similar in some respect to Tsamenyi, Cullen and Gonzalez (2006). They have used interviews, questionnaires and observations as part of their case study. As well, they have clearly established a link between external (regulatory and market) pressures, internal (political, systems) pressures and the way management accounting systems are shaped. The authors did not go into detail as to the actual measures that were used by the organization they have studied, which limits the usefulness in that respect for the current research.

The following will review the type of metrics that are available for the studied organization. In the electricity sector, there are two distinct markets that need to be considered when evaluating the performance and decisions of the actors (Geman and Roncoroni 2006) as these markets act somewhat differently even though they are inter-connected. These are the Physical and the Financial markets.

2.4.1 Physical Market
The physical market represents the physical delivery of the generated electricity and ancillary services related to an electricity producing unit.
(a) Generation Revenues
The main revenue of the organization is the sale of the generated electricity. There are typically three types of clients available for electricity generators which are Industrial Clients, Residential Clients and the Open Market (Parsons, 2010). In the case of BRPI, they sell to industrial Clients in the form of fixed price Power Purchase Agreements and to the Open Market at the prevailing market price. It was observed by the researcher that the majority of the Power Purchase Agreements sold by BRPI are fixed price for the totality of the generation of a specific unit, leaving little room for further optimization. However, some contracts offer various terms and conditions that can provide for optimization opportunities.

Generation that is sold on the Open market is subject to price fluctuation on a five minute to one hour interval, depending on the regional market (IESO, 2011, NYISO, 2011). Electricity is typically delivered in the open market in which it is physically generated. The open market offers more room for optimization of the generation. One way to evaluate the performance of the Traders and Schedulers is to compare the actual revenues from the generation with a benchmark which is based on the market performance. In the language of the studied organization, the attempt to capture the peak prices of the day in order to maximize revenues is referred to as the ‘peaker adder’ revenues. These revenues are considered to be additional revenues based on decisions of the traders and schedulers in relation with their understanding of the market and its particularities. As exposed further in Chapter 5, there is an important link between the five traps presented above and the chosen metrics of the organization. For instance, the peaker adder metric is based on past years’ performance and compared to the budget on a monthly basis.

Some alternatives to this calculation are presented in De Ladurantaye, Gendreau and Potvin (2009) and in Thompson, Davidson and Rasmussen (2004) where they present optimization models for hydroelectricity plants. These models are fairly complex in nature due to the number of formulas and data involved. Other models are also published in relations with hydroelectricity. They are not presented in this current work as they are for assets outside of North-America and do not have the specificities of the North-American markets and weather patterns.

The researcher observed that the data needed in order to process the calculations is fairly common and would be available in the studied organization. These models can serve as alternatives to the current metrics which fall into the Likierman’s (2009) traps as, amongst other traps, the current metrics only measures against the past performance of the company rather than competition and there is a risk that employees start gaming the metrics as they have been in place for a fairly long period. Even though they are complex in nature, these models are important for the studied organization as they make an important link between the metrics and the organization’s strategy to maximize revenues from taking advantage of the flexibility of its assets.

(b) Capacity and Ancillary Services Revenues
The other physical revenues are capacity and ancillary revenues that relate to the power plant but are not actual generation of electricity. These include but are not limited to the following types of revenues:
b.1) Capacity: Capacity revenue represents the potential generation that can be produced at a given time if required by the market operator. This availability serves as insurance for the market operator in order to ensure that it will have enough generation to supply all of its demand at the highest peak of the month (NYISO, 2011). The generating companies are paid a fixed amount (capacity price) per MW of capability, representing the maximum generation of the generating unit at any given hour of the month. The monthly Capacity price is a market price that fluctuates based on estimated offer and demand (NYISO, 2011).
b.2) Black Start: When a market is in total blackout, some of the generating units are able to restart on their own without the need of external electrical feed. The capability to offer such service is paid by the Market Operator at a fixed negotiated price (IESO, 2011).
b.3) Reserve: This type of revenue represents electricity that is kept aside by the generator in its reservoirs and that is available for generation at a maximum requested time which is typically 10 minutes or 30 minutes depending on the type of unit. This is used to regulate the demand fluctuations that are predictable in a fairly short term (IESO, 2011). A price per MW is also offered for this service for the additional MW kept in reserve by the power plant operator. In effect, the operator is paid for keeping water in their reservoir in case of abrupt changes in demand.
b.4) Renewable Environmental Credits: RECs are credits generated by environmental friendly generation units which meet certain criteria based on the construction or renovation date, geographical position, etc. There is currently a limited market for this type of revenues and not all of the credits generated can be sold in the over-the-counter market (Lomax, 2010).

2.4.2 Financial Market
It has been observed by the researcher over the years that the electricity financial market is mainly composed of fixed financial electricity swaps, whereby two counterparties enter into a financial swap exchanging a fixed price for a market index (Benth and Koekebakker, 2008). There is limited use of Electricity Options in the North-American market. For fixed financial swaps, the difference between the fixed price and the market price at the time of settlement represents the settlement amount. This is multiplied by the amount of MW under contract. A simplified illustrative example is presented in Appendix E. These instruments are used in order to fix the price of future generation without committing the actual generation with that counterparty.

The evaluation metric is based on potential performance of the traders. This potential performance is based on the upper management’s assessment rather than on a model. The calculation of the actual performance is based on a transfer pricing (Schuster and Clarke, 2010) methodology which is further explained in Chapter 4. Traders are transferred financial positions at the current market price of the future deliveries (Fleten and Lemming, 2003, Benth and Koekebakker, 2008) and are expected to generate revenues above that price.

2.4.3 Integration of the Physical and Financial Markets
The spot market is subject to hourly demand changes and market constraints where the financial market relies more on long-term demand (Geman and Roncoroni 2006). As discussed in The Economist (2004) in an interview with a competitor of the studied organization that had a similar strategy (both physical and financial transactions), energy producing firms that are also energy traders are in a unique situation to be able to arbitrage both markets efficiently, and benefit from these synergies. As discussed in section 2.2 above in relations with Ferreira and Otley’s (2009) framework, it is important to look at the interrelations between the metrics and not only at the metrics individually.

The researcher observed that electricity is a regional product whereby the electricity is physically delivered to a location and is subject to congestion of transmission lines and isolated demands. Therefore, the financial electricity market reflects these constraints in the chosen index to exchange for fixed prices, creating liquidity issues in lesser demand regions. This creates a challenge in determining the internal transfer price.

2.5 Balancing between detailed and simple information
In line with the detailed models discussed above, Taylor (2005) discusses the importance of enterprise performance management systems in order to properly report on the “fine grained pockets of potential value and efficiencies” (Taylor, 2005, p. 28). The article discusses the importance of integrated information systems in order for the information to be accurate, timely and most importantly, reliable for decision making. It is important to link the performance measures to recognized models that can be properly measured and reported on in order to create the proper change in routines. This article needs to be looked at in conjunction with other articles that provide a counter-argument to Taylor’s approach. Etzioni (1989) discusses the fact that management is able to make decisions or partial information and that a trade-off between detailed models and simple decision metrics need to be considered. As well, Johnston, Brignall and Fitzgerald (2002) argue that good enough information may be better than detailed models as managers will spend less time with data and more time managing. By ‘good enough’ the authors refer to using simple measures that are linked to the corporate strategy rather than complex calculations that measure detailed performance and get lost in details, while ignoring the big picture. They gave an interesting example coming from their interviews that referred to the use of cost accounting methodology that transferred fixed costs to different departments. This did not affect the company’s bottom line as time was spent on how to transfer the costs rather than on how to manage the costs. In the end it is argued that the use of the data is more important than its accuracy. More accurate data that is not acted upon may have far lesser value than ‘good enough’ data, as proposed by Johnston, Brignall and Fitzgerald (2002). It is however important to determine what is considered ‘good enough’ for each organization. It is based on the needs of Senior Management, capabilities of staff, Information Technology constraints, etc. that all need to be taken into account. Paulson Gjerde and Hughes (2009) are also of the opinion that the chosen measures need to be easy to understand and measured often. As Pavlov and Bourne (2011) point it out, these measures do not need to be accurate, as long as they drive the right routines.

It is important to note that Folan and Browne (2005) have discussed the fact that there is an important part of subjectivity in the choice of performance metrics when it is not done as part of a structured framework. Therefore, simple metrics should be considered in the context of the overall vision and mission of the organization and not only for their simplicity in order to stay relevant.

2.6 Conclusion
The current chapter linked various literatures with the studied organization’s specific industry. Performance management is a broad subject that has seen numerous publications over the years, including different frameworks. As part of the field of performance management, one of the subjects, the main focus of the current research, relates to performance measurement. There is not always a clear distinction between performance management and performance measurement in literature (CMA-Canada, 2010). However, the researcher wants to ensure that a clear distinction is made for the purpose of the current research as the aim and objective of the research is concentrated on the performance measurements as they relate to the revenues of a hydroelectricity company. As emphasized in the current chapter and further discussed in the following chapters, the performance measurements need to be clearly linked to the organization’s strategy in order to result in the right decisions for the organization. Finally, as presented in the current chapter, there is no strong link in literature between revenue management accounting and the hydroelectricity sector. The following chapters seek to help establish such a link through the examination of what is done at BRPI.

The following chapter will present the research methodology used as part of the questionnaire and the interviews. As presented above, the researcher used published work to create both the questionnaire and the interview topics. The link between theory and practice is important and needs to be clear in order to bring relevance to the current paper in the hydroelectricity industry (Baldvinsdottir, Mitchell and Nørreklit, 2010).

Chapter 3 - Research Methodology

3.1 Introduction to the research methodology
The current chapter presents the methodology used as part of the research to collect, analyze, interpret and develop conclusions on the researched subject. The research was based on the case study of BRPI.

The methodology presented below was in two parts. In the first part, the researcher administered a questionnaire with the goal to have a clear picture of the overall performance management system of the organization. This questionnaire was based on Ferreira and Otley’s (2009) framework discussed in chapter 2. In the second phase, the researcher conducted semi-structured interviews (Rabionet, 2011) in order to concentrate on the metrics, management decisions, and improvements. Having access to the overall performance management system’s assessment helped put the performance measures into perspective in order to properly answer the last two research questions presented in chapter 1. As the focus is on performance metrics rather than the overall performance management system, the interview process ensured that the research kept to that focus.

3.2 Overall Methodological Approach
The research was in the form of qualitative research as it aimed at answering research questions rather than prove a hypothesis (Bergsjø, 1999). As discussed above, a questionnaire on performance management was distributed and interviews were performed with the intention to concentrate on the performance metrics and on the research questions established in chapter 1. The researcher decided to adopt the interpretative research approach as it is the most common approach in management accounting research, especially in case study situations (Kakkuri-Knuuttila, Lukka, & Kuorikoski, 2008). This approach consists of an explanatory approach that is both objective and subjective due to the nature of the qualitative research methodology used (Kakkuri-Knuuttila, Lukka, & Kuorikoski, 2008). It is considered objective due to the structure of the research and the presentation of outcomes and it is considered subjective due to the researcher’s subjective views and prior knowledge in its explanatory role.

As this was an examination of the performance evaluation metrics, which was an exploratory research, a quantitative approach was not deemed appropriate by the researcher as it would not have provided for the type of insight and development potential that open ended questions offers, and the flexibility to add new elements during the research process, if necessary (Bergsjø, 1999). Furthermore, as discussed by Waterhouse and Tiessen (1978) it is important to understand the organization’s context in order to properly assess the management accounting system. Therefore, a quantitative approach, which offers a list of potential answers , would have not provided the level of insight necessary for the understanding of the performance metrics. The responses would have been limited to the researcher’s initial knowledge and assumptions rather than on the participants’ insight.

We have determined in section 2.2 that performance evaluation metrics should not be looked at in isolation. The metrics have to be put into the context of the organization being reviewed (Ferreira and Otley, 2009). In order to have a complete picture of the organization’s performance metrics, the data collection was done in two parts. In their article, Folan and Browne (2005) presented an important argument that performance measurement and all of the published frameworks should go to the next level and move towards performance management in order to understand the context and the relation between the metrics and the organization’s goals. This further emphasized the position of the researcher to use Ferreira and Otley’s (2009) performance management framework rather than any of the performance measurement framework discussed by Folan and Browne (2005) in order to perform the assessment of the organization‘s performance management system.

The framework questions methodology is discussed in section 3.5.1 below. The questions aimed at providing a clear picture of the performance management system. Once the questionnaire answered , the researcher used the information to prepare for the interviews in order to develop clear questions that relate to the performance management system as a whole, while concentrating on the performance metrics , which is the main focus of the current paper.

3.3. Case Study
In developing its research methodology, the researcher decided to do a case study of BRPI for various reasons. Amongst others, it included accessibility for the researcher of that entity versus others, the cost associated to a multiple entity study compared to a single organization case study and the timeline available for the research (maximum nine months). It is also due to the fact that the researcher was in an interesting position with regards to the chosen entity as he was an outside observer who although had previous work experience in that organization as he worked for BRPI for five years (2004 to 2009). This gave the researcher the insight necessary to quickly understand the organization’s terminology and understand the overall business needs of the organization, making the interview process more efficient. These criteria choices are consistent with other case studies in the same field, such as the one conducted in Spain by Tsamenyi, Cullen and Gonzalez (2006) which also benefited from inside knowledge of one of the researchers.

This case study, which concentrated on a single organization shall be considered a typical study (White, 2000, Yin, 2009) since the findings should be applicable to other similar entities (organizations that perform both in energy trading and hydroelectricity generation) due to the fact that similar factors will determine the performance measures of these entities (Pavlov and Bourne, 2011). Case studies help document context-specific theories into practical situations and increase the body of knowledge in those areas as discussed by Cooper and Morgan (2008) in their description of the use of case studies in the context of accounting research.

3.4 Sampling Methodology
Although the case study is considered to be a typical case, there are certain limitations as to the usefulness of the answers in other similar settings as qualitative research is difficult to replicate time and time again and is dependant on both the researcher and the sampling of the individuals answering the questions (Ulichny, 1991). To increase potential for replication, the researcher describes in detail the sampling method as well as the data collection technique that was used as part of this research.

The questionnaire and interviews were conducted using a purposive sampling (White, 2000) with chosen employees at different levels of the organization in order to have a varied sampling that would provide the best information on the given topic. In the case of the questionnaire, which aimed to understand the overall system, the sampling was heterogeneous (White, 2000). It was distributed to participants working in various divisions in the organization. The researcher opted not to use a random sampling (White, 2000) due to the risk of resulting with participants lacking the substantive knowledge necessary to bring value to the research.

Due to the fact that the researcher is not residing on the same continent as the researched organization, these questions were sent via e-mail to 14 individuals rather than being administered in a face-to-face setting. The choice of 14 individuals was with the intention to receive at least 10 completed responses. This was done based on the researcher’s experience with the chosen individuals and not on any statistical model. As the focus is on qualitative rather than quantitative research, such strategy was deemed acceptable as the goal is to reach the number of intended participants (Bergsjø, 1999) using a purposive sampling method (White, 2000).

The interviews concentrated on the performance evaluation metrics, the sampling was therefore of a homogeneous nature (White, 2000) and concentrated on individuals working in the Energy Marketing division, which is the group most impacted by these metrics. The choice of a limited number of individuals was due to the challenge related to semi-structured interviews that are typically longer, and the fact that the data analysis stage is more challenging, due to the fact that responses are not as structured either (White, 2000). The researcher has interviewed people at various levels in the organization, from the Chief Operation Officer to a senior analyst, providing for a wide range of experience and responsibility.

3.5 Data Collection Techniques
The data collection part, as discussed above, was in two phases. In the first phase, the researcher prepared a list of twelve questions inspired by the Ferreira and Otley (2009) framework. In the second phase, the researcher conducted semi-structured interviews with some of the organization’s employees in order to review some of the measurements used by the organization and discuss the potential use of theory discussed in the literature review conducted in chapter 2.

3.5.1 Data Collection Techniques - Questionnaire
Ferreira and Otley (2009) are of the opinion that their framework is useful for researchers who want to undertake case study research in the performance management system field. They also refer to other researchers who have used the framework in other industries than the current case study (hydroelectricity and energy trading). Therefore, in the first part, the researcher used Ferreira and Otley’s (2009) framework in order to establish the current status of the performance management system of the organization. The framework consists of twelve topics presented in the theoretical development section of Ferreira and Otley’s (2009) paper. The researcher used the topics to develop relatively short questions to be answered by the participants. When possible, the researcher used the questions that were proposed by the authors as part of the framework. However, in some cases, the questions proved to be too long for the purpose of the questionnaire, which served as an exploratory phase to be followed by the interview process (White, 2000). The questions to be discussed with representatives of the organization are presented in appendix A.

As part of the questionnaire’s instructions, it was clearly requested to keep the answers brief and to no more than three sentences per question (appendix A). The purpose of this request was twofold. Firstly, the exercise was the initial data collection as the interview process formed the main data collection exercise. Secondly, the researcher believed, due to numerous interactions with the participants, that brief answers would ensure that more participants would take the time to do the exercise if less of their busy schedule was needed.

The researcher noticed two main limitations in using e-mail rather than face-to-face questionnaire. First, the individuals did not answer their e-mail as promptly as requested and a reminder needed to be sent to some individuals in order to receive the required answers in time for the interview timeline. Secondly, various interpretations of the same question arose and the answers risked of becoming difficult to interpret. This second observation is further discussed in section 3.6.1.

3.5.2 Data Collection Techniques - Interviews
In the second and main part of the data collection stage, the researcher concentrated on the examination of the performance evaluation metrics through face-to-face interviews with various actors of the organization. These were performed during the month of July, when the researcher was physically at BRPI. The discussions were in relation with the current metrics used to evaluate performance. As part of the interviews, the various metrics that are available to the organization in relation to price management and generation management were discussed with the participants. The researcher prepared seven interview questions (appendix B) that related to the dissertation’s research questions, as presented in section 1.4, and the results of the questionnaire’s analysis.

As suggested by White (2000), the first questions were close ended questions and the last questions were more open ended, helping to probe more sensitive areas at the end of the interview process, once the rapport was established between the interviewer and the interviewee. The researcher aimed at thirty minute sessions with each of the participants, which is another reason why the interview questions were limited to seven. The questions were developed with the goal to develop specifically on the metrics and at ensuring that all of the research questions were answered. A look back at the questionnaire answers provided for an interesting introduction to the interview sessions and therefore served as the first interview question. It also helped the interviewer prepare meaningful questions (Rabionet, 2011) in order to demonstrate to the interviewees that the interviewer has a good grasp of the subject matter. The interviewees discussed the current metrics of the organization. As well, an overview of available metrics from literature was discussed with the interviewees in order to assess the potential use of these metrics and compare with the current metrics used. Amongst other goals, the interviews aimed at understanding the willingness and possibility to adopt complex calculation metrics versus the current simple metrics. Using a semi-structured interview helped narrow down the topic, while providing for the opportunity for the interviewees to develop their ideas further and provide the researcher with meaningful stories, as suggested by Rabionet (2011). It is also important to note that the interviewer informed the participants of the confidentiality of the answers provided as suggested by Rabionet (2011) in order for the interviewees to feel that they will be treated fairly and ethically.

3.6 Data Collection
3.6.1 Data Collection - Questionnaire
The individuals to whom the e-mail was sent consisted of at least two C level employees (i.e. CEO, CFO, and COO level), two VP’s and two manager/trader level staff. As well, in order to have a broader view of the system, the questionnaire was sent to individuals from one of the generation divisions (Canada), from the Corporate division and from the Energy Marketing division. As the main focus of the dissertation is the hydroelectricity related metrics, most of the chosen participants were part of the Energy Marketing division, which is the revenue management arm of the organization. In their paper, Ferreira and Otley (2009) discussed the limitations of their findings as they only had answers from top management and they suggested questioning people at various levels of the organization in order to ensure to have a broader picture of the system. Therefore, the researcher ensured to select individuals at various levels in the organization to be in line with Ferreira and Otley’s (2009) suggestion. The researcher has first circulated the questionnaire to two individuals in order to test its practicability and has then administered it to the rest of the group. The pilot was done in order to ensure that the questions were clear and that the questionnaire was easy to fill out (White, 2000). No changes were done to the questionnaire following the pilot. However, some of the questions were not understood the same way by all participants, which made the interpretation of the data slightly more difficult . With a total of 10 responses, there was significant enough understanding of the questions to provide for meaningful interpretation. Furthermore, the various interpretations have presented interesting findings due to the fact that various visions of the organization and various focuses in the answers were related to the division in which the individuals were working.

The questionnaire was prepared and filled out electronically. All of the answers were then received via e-mail and compiled. A total of 10 out of 14 individuals have participated and answered the questionnaire.

3.6.2 Data Collection - Interview
The interviews were held at the organization’s premises and were face-to-face interviews, permitting a better interaction with the interviewees (White, 2000). The interviews ranged from thirty minutes to an hour, depending on the participants’ availability and their knowledge of the various topics discussed. The allocated time ensured that the interviewees were focused and able to provide fairly detailed answers to each of the seven questions and leave room for unstructured discussion at the end. The interviewer took note of all of the answers in manuscript form. Although the preferred method of data collection is to record the interviews (Rabionet, 2011), the interviewer decided not to use that methodology due to the fact that a strong relationship already exists with the interviewees, as they were colleagues of the researcher in the past years, and also due to the fact that interviewees are generally less intimidated by manuscript note taking than if they are being recorded (White, 2000). The day following the interviews, all of the answers were typed by the researcher to ensure completeness of the data, ensuring that none of the details were left out due to memory gap.

The questions served as a guide to ensure all of the Research Questions were discussed. However, a less structured discussion was encouraged in order to gather additional information and various ideas that were not initially foreseen (Rabionet, 2011). There were five interviewees which were part of the same group of people who previously responded to the questionnaire. The first interviewee served as the pilot for the rest of the interviews in order to be able to ensure questions were clear and precise and that the estimated time for each interview was reasonable (White, 2000).

3.7 Data Analysis
3.7.1 Data Analysis - Questionnaire
The data analysis was done by using the grounded theory methodology (White, 2000) which consists of familiarizing and reflecting on the material and then conceptualizing, cataloguing concepts and linking ideas together, a method that is consistent with the case study performed by Tsamenyi, Cullen and Gonzalez (2006). The researcher has catalogued all of the answers into common themes, in order to assess the current state of the performance management system. It should be noted that the analysis was performed before the interview process in order to help guide the interview questions, resulting in an iterative process (White, 2000).

The answers were first looked at by hierarchic level in order to compare answers of higher level versus lower level employees. The answers were then catalogued into common themes in order to assess comparability. The researcher has used the Chief Operating Officer’s answers and information available on the company’s website as the proxy to compare other answers with. This had for purpose to analyze the degree of both, communication of, and common understanding of, the performance management system at the same level and between staff levels. In answering the questions, the participants were asked not to consult each others nor consult the organization’s website in order to ensure that it was their understanding of the answer rather than the corporation’s official answer that was presented.

The researcher first read all the questionnaire responses, one participant at a time in order to put the answers into the context of the respondent’s other answers. This was important in order to understand the reason for conflicting answers between respondents and to adjust for answers that straddled two questions. The researcher then read all of the answers, question by question, which provided for a global view of the specific questions’ responses. In order to properly analyze the answers, the researcher then separated the answers into specific files and then grouped the answers of each question into common themes. These were then catalogued in order to be presented in appendix C, and analyzed further as part of Chapter 4. The answers were then compared to the literature review presented in Chapter 2 in order to link practice with theory (White, 2000).

3.7.2 Data Analysis - Interview
The same analytical methodology was used to analyze the interview answers as was used for the questionnaire, in order to regroup the elements in common themes and facilitate the interpretation. The answers were also compared with the questionnaire to review common themes and discrepancies, if any. The interview process was somewhat iterative (Bergsjø, 1999) as the interviews were not all held the same day. Therefore, in reviewing the answers to the previous interviews, the researcher was able to ensure that all of the research questions were dealt with enough depth to arrive at sound conclusions. This part of the investigating approach was an iterative approach using the information of the questionnaire and of the previous interviews to build up the data, which is in line with the grounded theory methodology (Denscombe (2010) .

It is well understood by the researcher that the interview process has strengths and weaknesses. The strengths include but are not limited to being targeted and providing insightful information. The weaknesses, on the other hand can result in a bias in the responses due to the way the questions are drafted, a bias of the interviewee and a risk of reflexivity (Yin, 2009). In order to benefit from the above mentioned strengths, the researcher analyzed the data in a targeted manner. The answers to the interviews were first reviewed at each participant level to ensure that there was no bias transpiring from individual interviewees. If this had been the case, the biased answers would have been discarded for lack of usefulness. The questions were then separated into their various research questions’ relevance in order to provide insight on these particular questions (Yin, 2009). In focusing on the questions rather than the data, a more focus review of the data was done (Yin, 2009). The answers were also matched with relevant literature review presented in chapter 2 in order to link theory and practice in a clear and focused manner.

It is interesting to note that the researcher has played around (Yin, 2009) with the data for an extensive period of time before starting the proper analysis. This provided the researcher with various avenues for both analysis and presentation. In the end, it was decided to present the data based on the list of research questions rather than based on the questionnaire and the interview responses independently as it had for advantage to present a more in depth look at each of the research questions.

3.8 Interpretation techniques and conclusions
An important challenge with qualitative data analysis is the amount of data involved and the challenge to choose what is relevant to the case at hand as not all data should form part of the interpretation stage. The researcher had to edit some of the data and present only what was relevant to the case at hand rather than present all gathered information (Denscombe, 2010). For example, answers to question 7 were discarded as they did not add value to the specific research questions.

Using the above data analysis for each stage of the research, the researcher used the findings from the primary research and ensured to link the answers of the primary research with the literature review from chapter 2 in order to arrive at a conclusion and research results (White, 2000) for each of the research questions. The researcher needed to consider potential bias of both the interviewer and interviewee in the interpretation of the data (White, 2000) arising from the interviews as discussed in 3.7.2. The number of years of experience and position in the organization were also considered when comparing answers from the various participants for both the questionnaire and interview questions. Using the questionnaire to confirm or infirm the interview data helped reduce the potential bias (White, 2000) and provide a stronger picture of the overall state of the performance evaluation metrics. In reviewing and comparing the answers, the researcher has taken an important look at interview answers inconsistencies as it is not unusual as part of face-to-face interviews to try to answer what we believe the interviewer wants rather than what we feel is the exact response (Tsamenyi, Cullen and Gonzalez, 2006). The researcher’s past experience in the organization and the information available on the company’s official website assisted in the data interpretation phase. A summary of the findings was used in order to find conclusions in relation to each of the research questions and in order to finalize the examination of the performance evaluation metrics. As discussed by Baldvinsdottir, Mitchell and Nørreklit (2010), a description of findings needs to be followed by an explanation and understanding of what was observed in order for the research to be relevant. These will be presented in the subsequent chapters of the current research paper. This is also in line with the grounded theory approach discussed earlier in the chapter. However, although some recommendations are presented in chapter 5, the researcher was not aiming at providing prescriptions for the organization, as this is the aim of a consultation mandate, not of management accounting research (Baldvinsdottir, Mitchell and Nørreklit, 2010).

3.9 Conclusion
In the current chapter, the researcher presented the reasons why Ferreira and Otley’s (2009) framework was chosen to develop the questionnaire. The iterative process of the questionnaire followed by face-to-face interviews were also discussed. Finally, a review of the data collection, analysis and interpretation methodology were presented.

In the next chapter, the researcher presents the findings from the questionnaire and interview, and presents an overview of the performance management metrics and their usefulness. Links between the findings and the related literature are also presented. Finally, the research questions presented in chapter 1 will be answered.

Chapter 4 - Answering the Research Questions

4.1 Introduction to the analysis
In the following, the researcher presents the results from the questionnaire and interviews in context of published work in order to answer the research questions presented in chapter 1 .

In answering the research questions, the researcher used the data from both the questionnaire and the interviews, as discussed in chapter 3. To illustrate some of the points made below, extracts of the interviews are presented in the text. Using quotes from the interview serves to illustrate the point in a way that makes it clear to the reader that it is the view of the interviewee and not the researcher’s interpretation (Denscombe, 2010). In order to keep anonymity, the extracts from the interviews will not be associated with specific participants.

4.1.1 BRPI Corporate Structure
BRPI is structured into five divisions. The Canadian, United States and Brazil operations (electricity production) form the first three divisions. The fourth division is the Energy Marketing division, which is responsible to sell the energy on the markets and enter into long term physical electricity contracts. Finally, the fifth division is the corporate services division, which is the head office of the organization.

As the current dissertation focuses on revenue metrics, the researcher concentrates on the Energy Marketing division which is the division that has an impact on the controlled items related to revenues.

4.1.2 Controlled and Uncontrolled Revenues
Controlled items are those revenue items that the Energy Marketing division can influence. These are further discussed in section 4.3. In general terms, they include the ability of the Energy Marketing division to capture higher than market prices for energy sold on the physical market and its ability to capture long term financial fixed prices above the price received as a transfer price from the hydrology group (see section 4.3 and appendix E). Items that are not considered controlled are also discussed below and include the hydrology variance and market price fluctuations for the period that it is not actively managed by the Energy Marketing division. The other divisions, or more precisely the legal subsidiaries of the other divisions, transfer their electricity to the Energy Marketing division through legal contracts. The Energy Marketing division is then responsible for commercialization of that energy. The Energy Marketing division is not a pure trading division, which normally has trading capabilities and activities outside of the asset base of the organization (Parsons, 2010). It provides the services to the other divisions based on the assets positions that the organization has, i.e. its potential generation. Therefore, it serves as a support function to the generation of energy (Parsons, 2010) rather than a true stand alone Trading entity, which is similar to what Parsons (2010) described in its review of Constellation Energy Group. Parsons (2010) reviews and comments on Constellation’s Trading activities changes in strategy over the years.

The revenue management metrics used by the organization have for primary purpose to review the way the Energy Marketing division manages the revenue optimization phase. This is once they are released the generation for active revenue management as further detailed in section 4.3.

4.2. What is the current state of the organization’s performance management system?
The first research question refers to the current state of the performance management system. A summary of answers to the questionnaire is presented in appendix C. The answers have been grouped into common themes to reduce duplication and help in the interpretation of the data (Denscombe, 2010). Some of the answers can contradict each others as people in different positions within the organization have different perceptions. This was taken into consideration in the analysis phase. The following presents the researcher’s overall interpretation of the answers and discussion on the common themes. A detailed review of the performance management system based on the both the questionnaire answers and on the interviews is presented in appendix D.

4.2.1 Analysis
The initial information used for the analysis come from the questionnaire answers, which aimed at providing initial findings. The purpose of the questionnaire was to assess the current state of the performance management system using Ferreira and Otley’s (2009) framework. This was further analyzed as part of the interview process in order to compare the researcher’s understanding of the current state of the performance management system, from the analysis of the questionnaire answers, with the view of the participants. In order to confirm or infirm the observations made by the researcher in its review of the questionnaire responses, the researcher asked the interviewees to assess the current state of the performance management system as a whole, based on their recollection of the answers they had provided in the questionnaire.

4.2.2 Observations
In summary , the researcher has observed gaps between the theory and practice. This was observed in regards with the communication of the vision and mission of the organization which have been confused with the organizations strategies to achieve such vision and mission. As well, it was observed that the use of the information is reactive rather than proactive. Finally, there is slow progression to use the information for other purposes than to evaluate past performance.

The researcher also observed that an important focus is on the assets and the steady cash flows they produce as discussed in the organization’s website, (BRPI, 2011) rather than on maximizing the revenues in a given year. This focus may impact the performance, or the expected performance of the team responsible for optimization as it may not be one of the organization’s main focuses, creating a gap between the expectations of management and the Energy Marketing division, both in terms of resources available to perform their task and the level of compensation received. To that end an important corporate focus is on risk management and operating within the risk framework of the organization, which is in line with other integrated trading operations that offer services to their generation divisions (Parsons, 2010).

It is important to note that the corporate legal structure was planned in a way to protect the assets of the corporation and leverage its financing power. The researcher observed that the assets are held in various legal entities that have individual financing agreements and sell their energy to the legal entity responsible for the marketing of the energy. This provides for the possibility to finance assets at the project level (BRPI, 2011) increasing the financial leverage, while isolating risks in separate legal entities, lowering the risk of the whole group in case a particular asset becomes unprofitable. This is in line with the organization’s view that assets are the priority of the organization and not the proprietary trading potential (Parsons, 2010) that can be done around and outside of these assets, making it a risk averse environment.

4.2.3 Assessment
Using the Canadian Certified Management Accountants performance management maturity framework (CMA-Canada, 2010), the organization can be considered at Level 2- Established, out of a four level assessment. The system is considered to be fairly stable, since the management system is not in constant major changes and it is repetitive since it is used for the yearly evaluation of performance and many of the measurements are calculated and published monthly. However, there are issues with efficiency due to the lack of systems and quality of data, which reduces the consistency of improvements.

This assessment is in line with a majority of interviewees who estimated that the performance management system is at a low level of maturity. The interviewees are of the opinion that the “methodology is solid, however, the systems and processes are not very mature” (Anon, 2011). An important observation from one of the interviewees is that the organization has been in discussion with software vendors to help it automate some of its processes and calculations in order to mature from calculating metrics to the next stage, which is to analyze the metrics. There was a consensus amongst the vendors that the level of maturity in terms of methodology was very high (Anon, 2011). However, the technology to support this methodology is insufficient, resulting in a gap in the use of the information.

Finally, as per the researcher’s detailed analysis, there is a lack of ‘big picture’ (Johnston, Brignall and Fitzgerald, 2002) view of the organization, resulting in missed opportunities (Taylor, 2005) and lack of communication between the various groups that can work together to improve the financial situation of the organization (Inglis, 2008). Therefore, it can be concluded that the current state of the performance management system is at a relative early stage, mostly due to the way the information is extracted and used by the different actors in the organization (CMA-Canada, 2010) rather than on the methodology chosen to evaluate the performance.

4.3 What performance evaluation metrics are currently used by the organization to measure dispatch strategies, hedging decisions and ancillary services’ revenue management performance?
The second research question was discussed both at the questionnaire and interview levels. The researcher’s past experience with the organization and confirmation of past observations coming from this experience have also been used to document the following section.

BRPI is an organization with relatively stable costs (BRPI, 2011). As one of its goals is to have steady cash flows, it is therefore important for the organization to stabilize its revenues (Anon, 2011 ). The researcher observed that some of the items affecting revenues are controllable where others are uncontrollable. The two main items that are not controlled are the water inflow, or hydrology factor, and the long term electricity market price fluctuations.

The organization has developed a methodology to estimate its water inflow and the estimated level of generation in a given period (Anon, 2011). There are both short-term and long-term estimations. However, as discussed in the interview process, such models are not evaluated with the goal to maximize the level of generation for a given period and “the hydrology is not a controlled variance” (Anon, 2011) for the organization leaving a gap in the level of hydrology optimization as further discussed in section 4.4.

Long term market prices are also monitored by the organization. However, as a general rule, only the market risk of the next 30 months is managed (BEMI, 2009). Therefore, in the long term, the organization is exposed to market price fluctuations that are not actively managed as part of the revenue management process. Any price fluctuation occurring between the plan date and the release date (for new generation release) is also considered not to be a controlled variance (BEMI, 2009).

Although the revenues of the next 30 months are managed, the measurements are concentrating only on the current delivery year rather than on transactions performed for future years.

It is important to note that the metrics presented below are for management accounting purposes and not for the presentation of revenues in the financial statements. These are metrics used to evaluate the optimization and enhanced revenues generated by the Energy Marketing division, not an actual profit, as there is not a full capital and cost allocation made to the division (Anon, 2011 ). As discussed in Parsons, (2010) estimating the value-added of an Energy Marketing division is challenging due to the fact that some of the value generated by the Energy Marketing division is inherently built in the assets purchased by the organization. One such example is the flexibility of the assets providing for the ‘peaker adder’ revenues discussed below. It should be noted that the organization evaluates the performance of the Trading team similarly to the way a profit unit is evaluated due to the fact that there are transfer prices (see section 4.3.2) and targeted revenue levels (Parsons, 2010).

Finally, it was observed that the metrics are compared to both plan and prior year. The comparative to prior year only serves as an indicative figure, the comparative to plan is the value against which employees are evaluated. The target amount for the year is an aggregation of the metrics presented below. It was determined through various discussions and observations with employees of the organization that the Energy Marketing division is not penalized if it performed less in some areas than others as the total target is the final metric used for compensation.

As we can see from details presented below, the current performance evaluation metrics are simple in nature and do not use models that are available in the literature presented in chapter 2. The reasons behind that will be further discussed in the current chapter. The following tables serve to illustrate the way variances to plan are presented. The dollar figures are for illustrative purposes only.

In figure 5, which represents revenues in millions, the organization had previously established planned revenue of $80M for the month. There are two major uncontrolled variances to plan which total $5M. The first variance is additional hydrology of $7M, which is the multiplication of additional Megawatt per hour (MWh) generated during the month times the planned price for that given month. The second uncontrolled variance is the difference between the transfer price paid by the Energy Marketing division and the planned price on energy that was transferred after the plan date (new generation release). In the above example, the market price decreased between the plan and release date, which resulted in a negative uncontrolled price variance of $2M for the delivery month.
The controlled variances which represent the initiatives, are presented in more detail in Figure 6. In general terms, they represent the difference between what had been planned for each of the initiative and the actual revenues generated.

Figure 6 - Detailed Marketing Initiatives Revenues

Source: The Author

Figure 6 presents the detailed revenues, the plan and their variation for a given month. The actual revenues represent the cash flow generated for each of the marketing initiatives. The plan represents the expectation of management for each of the initiatives at the time that the plan was done. In the example, the Energy Marketing division performed above expectations by $3M for the month. A detailed explanation of each of the initiatives is presented below.

4.3.1 Spot Market Initiatives - Dispatch Strategies
The Energy Marketing division is responsible for dispatching the generation that is not committed under long term Power Purchase Agreements to the market. It is tasked to deliver the energy in the best available hours in order to maximize the organization’s revenues. The division is therefore attempting to capture the daily peaks in order to maximize the revenues. To assess such performance, the price received from the market is compared to the average market price for the month, by peak type (see appendix E for simplified illustrative example). The additional revenue made compared to the average market price is the ‘peaker adder’. In order to assess the performance of the team, the peaker adder is compared to the historical average that the organization was able to capture. It is therefore benchmarked against historical performance, one of the five traps presented by Likierman (2009) discussed in section 2.3, rather than being compared with other market participants with similar assets, as suggested by Lieberman and Raskin (2005). It was also observed by the researcher that the peaker adder calculation is based on the monthly average prices rather than the sum of individual daily decisions. It is however unclear as to the right metric to be used in such case since the decisions of a given day affect the decisions of the rest of the month. As observed by the researcher during its employment with the organization, both methods have advantages and disadvantages and no industry benchmark is published on the subject. In the case of the daily value, it would illustrate the daily decision to generate in the best hours. However, the calculation would exclude the fact that some decisions are multi-day decisions, whereby, the scheduler may decide to generate more on a specific day if he predicts that the price will be lower the next day. The monthly method has the advantage of presenting a longer term decision process. However, ignoring the daily potential may result in the organization not capturing the best possible daily price (Taylor, 2005). As further discussed below, the decision to use the monthly value rather than daily value refers to the fact that the monthly calculation is a simpler method for which the level of detail is sufficient to assess the decision process of the schedulers. As well, it provides for multi-days decisions to be evaluated. It is therefore a closer evaluation of the actual strategy of the schedulers, which is in line with what is suggested by Pavlov and Bourne (2011).

4.3.2 Price Management and Hedging Strategy
The Energy Marketing division is composed of many units. Both the hydrology and the Trading units are part of that division. As part of the hedging decision process which relates to the Financial Market discussed in section 2.4.2, the hydrology unit transfers energy volumes to the trading unit at the negotiated market forward curve, which is defined by Schuster and Clarke (2010) as a market-based transfer price since it reflects the external market conditions but considers internal constraints such as the market liquidity ratio. The two units are in agreement that trying to unload a large position in a given day will make the market price collapse.

The Trading unit receives these positions at various intervals through the internal transfer pricing process, which is similar to other entities in the industry (Parsons, 2010). An illustrative example of the transfer price mechanism is presented in appendix E. As discussed by Schuster and Clarke (2010) using an agreed upon transfer price, rather than a straight market price, increases synergies between the two units and encourages the two units to work together in order to release the generation for trading at the opportune time, as it was observed by the researcher.

a) Release of energy
The following explains how the hydrology unit determines the volume that can be released to the traders in order to actively start capturing fixed financial prices for some of the future generation. Some positions are received at fairly long term, which is 18 months prior to the start of a new year. These volumes are for a period of one year at a time. The traders therefore have a position for the second following year (18 to 30 months away). This energy is the ‘baseline generation’ and represents the volume that has a 95th percentile of probability to be generated (BEMI, 2009). This energy is released based on a certain levels of pre-optimization by the hydrology unit, which estimate how much volume can be generated in each of the months based on the reservoir constraints and historical price fluctuations between months. Additional generation is also released to the traders as the probability of generating becomes high enough (95th percentile). This is referred to as the ‘new generation’ releases (BEMI, 2009) and is transferred using a similar price determination method, consulting with the Market Risk unit, which is also part of the Energy Marketing division.

b) Financial transactions
Once the generation is released to the traders, their goal is to enter into fixed financial electricity swaps (Benth and Koekebakker, 2008) at a higher price than the release price they paid to the hydrology team (appendix E). They have a limited time to perform such transactions as per the organization’s Risk Management Policy (BEMI, 2009), which is in line with one of the goals of the organization to have risk adjusted returns (Anon, 2011). These transactions settle financially, which means that there is no commitment for physical delivery of electricity on these contracts (Benth and Koekebakker, 2008). The sole purpose of these transactions is to protect the organization from market price fluctuations in order to ensure a fairly steady cash flow, which as previously discussed, is one of the goals of the organization.

4.3.3 Capacity and Ancillary services
The capacity and ancillary services are the other services that a power generation unit is able to provide apart from the actual electricity generation. Such services include capacity, reserves, black start capability, etc. These services are either negotiated with the various operators or transacted on the open market, for more detail on each of these, please refer to section 2.4.1. When a market exists, the services are released to the traders at the market price through the internal transfer pricing methodology discussed above. This is the case for the capacity market. However, if there is no active market, upper management will impose a transfer price based on the historical revenues, which is common in transfer price methods as discussed by Schuster and Clarke (2010) and also represents one of Likierman’s (2009) traps discussed in chapter 2. Again, no comparative with other market participants is performed, which would have had the benefit to increase the organization’s knowledge of the potential performance of its employees using comparables (Lieberman and Raskin, 2005).

4.3.4 Other marketing initiatives (Contract optimization)
Other initiatives such as the purchase of transmission and long term contracts with an optimization component are valued at the estimated revenues that shall be generated as per mathematical models. The Black & Scholes (Anon, 2011) option pricing model, past performance and prior year targets are the tools that are used most often to set the current year target. The choice depends on the predictability of the parameters that define the potential for optimization. As discussed with one of the interview participants, “these are relatively simple models with a limited number of variables, which makes them fairly simple to explain to non-initiated” (Anon, 2011).

4.3.5 Arbitrage revenue
It was observed by the researcher that the employees of the 24 hour desk, which are also part of the trading unit, are responsible to ensure that the scheduled energy is actually delivered in the market as was initially scheduled by the scheduler and the operations team. They also have the responsibility to arbitrage between markets when there are price disparities between two adjoining markets by purchasing energy in one market and selling to the adjoining market, capturing that price differential. At BRPI, such activity is referred to as Arbitrage revenues (Anon, 2011). The organization estimates a fixed amount of revenue as the target. This target is based on an historical figure that has not changed in the past seven years. It is considered to be a placeholder in the preparation of the budget as an estimate of Arbitrage revenue is not technically feasible due to market volatility.

4.3.6 Final remarks on the metrics
As raised by one of the interviewees, “the revenue and plan are split into categories (hydro, peaker, financial, ancillaries). We are missing the analysis of the observed performance. Many decisions (or lack of) are lost in the hydrology and market price variances.” (Anon, 2011). Therefore, the performance is presented but not necessarily analyzed.

4.4 Are the current performance evaluation metrics aligned with the corporate goals?
In order to assess if the evaluation metrics presented above are aligned with the corporate goals, the researcher presented the published corporate strategies and goals from the corporation’s website (BRPI, 2011) to each of the interviewees. It has been established in literature that choosing the measures and targets that align with the organization’s corporate goals will help achieve these goals (Franco-Santos and Bourne, 2005). Therefore, in reviewing the goals and strategies, the interviewees were asked to match metrics to the related published goals of the organization in order to identify any gaps.

Each of the goals (see appendix B, question 3) were compared with the metrics discussed in section 4.3 and other known metrics of the organization that did not directly related to revenue management. There was a clear consensus amongst the interviewees that the majority of the short-term metrics are aligned with the corporate goals. However, as discussed by one of the interviewees, the metrics do not capture and measure all of the goals of the organization and especially the long-term goals and the non-quantitative goals (Anon, 2011).

For example, there is no review of the optimal water utilization and no concrete measure to ensure that the organization is ready to integrate new assets (Anon, 2011). Some of the goals relate to revenue management initiatives where other goals are more in line with acquisitions, which are not evaluated with clear metrics, as understood by the participants.

One of the goals of the organization refers to the optimization of ancillary services revenues (BRPI, 2011). This particular type of revenues, as discussed in the previous section, is compared to historical performance of the organization rather than to an optimal solution or to comparable organizations (Lieberman and Raskin, 2005). As well, in order to optimize ancillary revenues, it is assumed that changes to the facilities and to the machinery should occur. However, as it was pointed out by one of the participants, “the trading group works with what it has rather than try to improve the facilities as there is no incentive to increase efficiency in their bonus scheme”(Anon, 2011). It should be noted that, given the fact that the organization is in a very competitive market, the comparable information is not necessarily available, which means that “there are no external benchmarks to compare to” (Anon, 2011) used by the organization.

One important aspect to consider regarding BRPI performance evaluation metrics is that they are based on cash flows which “is not an industry standard” (Anon, 2011) rather than on a trading Profit and Loss (P&L) calculation that would be based on marked-to-market values, the trading industry’s standard (Anon, 2011). However, this raised the concern from one of the interviewees, that because the employees are compensated on cash flows, they are compensated on the current year revenues, putting less emphasis on future year revenues, which is one of the organization’s goals and strategy.

One interesting aspect that was raised by an interviewee is the fact that the organization is in a competitive situation when comes time to purchase new assets, which is confirmed by Parsons (2010) as it is one of the few owner of electricity assets that does not outsource its trading and scheduling activities (Anon, 2011). The researcher observed that it is believed that this provides the organization with a greater understanding of the optimization potential of the assets being contemplated and can result in more competitive bids for the organization. This expected additional income later translates into the expected performance of the team on their optimization capability for the new assets. Therefore, the researcher observed that the input provided by the Energy Marketing division on the acquisition is indirectly evaluated as part of their yearly performance rather than evaluated at the time of acquisition, as part of their yearly benchmarks. It was further observed that this is however unclear in the minds of the participants as they are not informed of the direct link between their participation in an acquisition process and their evaluation and compensation.

4.5 What performance evaluation metrics are available in literature for hydroelectric and energy trading performance assessment?
Section 2.4 presents some available models based on literature. Such models include optimization of on-peak capability and of water storage to achieve the best possible market price sales, which are strategies of the organization (BRPI, 2011). As part of the current research, the model from De Ladurantaye, Gendreau and Potvin (2009) was presented to the organization’s analysts in order to discuss applicability of the model. As pointed out by one of the interviewees, each model is specific to its own river system and needs extensive manipulation to fit each of the assets of the organization into models.

Not only should a model be available in literature, it also has to be practical in order to be meaningful for decision making and performance assessment (Neely, 1999). The main concern of some of the interview participants is in regards with the incremental value that such models would bring to the organization compared to the cost of implementation and maintenance.

Through the interviews, it transpired that software vendors have been in contact with members of the organization in relations with the potential development of some of the models into analytical tools. One interviewee has informed the researcher that it has been determined that the development of the more complex models represent an important cost to the organization and that is more than likely the reason why few, if any organizations are using the models presented in theory for their own practice (Anon, 2011). Another interviewee has expressed the opinion that the organization itself is lacking the internal expertise to develop such models into performance assessment systems or to understand their output (Anon, 2011). The researcher observed that there is an important applicability gap between understanding published models and replicating them for one particular instance versus using the models on a daily basis to assess the decisions. Various models are available from vendors for Natural gas powered plants, coal plants, etc. (Anon, 2011). However, as observed by one interviewee, there are no off-the-shelf tools available for hydro plants, creating a gap between theory and practice.

It is interesting to observe that there is a knowledge gap for the users of the information provided by the models (Anon, 2011). The upper management’s personnel, which are the main users of the data do not necessarily have the technical expertise to understand the models and to criticize the output (Anon, 2011). This comment will further be developed in the next section in regards to simple versus complex metrics.

However, as pointed out by one interviewee, it is important to consider the marginal value of the initiative compared to the total revenues of the organization in order to have the proper buy-in for the efforts deployed. Therefore, the fact that the Net Operating Income is an important measure of the organization (appendix C) to establish financial reward performance may be considered sufficient in the mind of management to deem that they have assessed these strategies as part of the total compensation of the employees.

Finally, as it was pointed out by another interviewee, the models would offer “after the fact targets” (Anon, 2011) as to the best scenarios that were available to the traders and schedulers. This would result in not knowing the targets in advance and in a risk of de-motivation (Gong and Tse, 2010) for the traders.

4.6 Does management prefer simple evaluation metrics or complex evaluation metrics? Why?
In light of the fact that the organization uses fairly simple metrics compared to what is available in literature, the above question was raised and discussed with the interviewees. An interesting trend that was observed by the researcher is the fact that the higher the individuals are in the hierarchy, the more they are interested in the simple metrics as they are interested by the big picture rather than the minute details, in line with Johnston, Brignall and Fitzgerald (2002) as discussed in chapter 2. One interesting comment coming from a participant that was a user of the information and that had the technical knowledge to use and develop complex models is that:
“a quasi accurate model is better than something too complex. It is better to use a model with three inter-related parameters that we are able to understand and explain than a model that is more accurate but for which the inter-relations are difficult to explain and monitor. ” (Anon, 2011).
This observation is in line with Etzioni’s (1989) conclusion on the simpler metrics. Another interesting point that was raised by one of the interviewees is that the models should be used only with the right audience. This was also commented by Brandel (2007) in the data accessibility to be provided to various staff members in a Business Intelligence project because too wide of an access to the wrong employees can result in multiple versions of the truth or erroneous interpretations due to lack of proper technical skills.

For example, if the manager and the employee being evaluated understand the model, this model can serve as a basis for evaluation of the employee’s performance. However, it does not mean that the model has to be added to other models to explain the performance of the whole group. A simpler method than the aggregated models could be used to present the performance to upper management who are more interested in the big picture (Johnston, Brignall and Fitzgerald, 2002).

The cost-benefit of implementing such models has been discussed in section 4.5 above and has to be taken into consideration. It is true, as discussed in chapter 2, that the models could help the organization capture additional revenues (Taylor, 2005). However, the cost of implementation of these models must not be prohibitive. An Energy Marketing division has little capital investment (Parsons, 2010) apart from its investment in information technology to capture data from the industry in order to make informed decisions. Therefore, if the models can be developed using this data, it can be assumed that implementation would be at a relatively low cost.

One of the interviewee presented a current initiative of the organization to the researcher. BRPI is currently in the process of developing a strategy called ‘Data to Information’ (Anon, 2011) which aims at transforming all of the available data into meaningful information that would help in the decision process.
There is a consensus amongst the interviewees that more complex models would be interesting. However, there is also a consensus that there is no real appetite for this type of model from upper management as the simpler models are easier to communicate to the shareholders and other investors, which is in line with Johnston, Brignall and Fitzgerald (2002) findings.

Finally, in the interview discussions it transpired that the main reason used for sticking to simple metrics is communication and the lack of staff to both feed and analyze the models. This is in line with what was found in literature and presented in section 2.5.

4.7 Additional discussion
As discussed throughout this chapter, the evaluation metrics used by the organization are short term in nature. An important concern of upper management is to be able to evaluate the longer term decisions such as acquisitions, capital program to retain the value of the assets, etc. These goals are not specifically evaluated by the performance management system. There are metrics that are being used, such as the return on investment. However, these metrics do not necessarily reflect the decision made by management as they incorporate other factors that are not controlled such as market price fluctuations.

As discussed with one of the participants, the long term decisions are reflected on the corporation’s share price fluctuations. Given the fact that only upper management is offered stock options, they are the only ones impacted by the long term decisions made throughout the organization. As they are also responsible for the individuals making the short term decisions, their performance may not be aligned with their decision process. As reflected in the review of the answers to the questionnaire, there is a perception that lower level employees are left with little room to think strategically and may lack the big picture view.

Through discussions with some of the interviewees that were at the upper levels, there is an understanding that the way the organization evaluates the performance of the trading team that they may have fallen in the Likierman (2009) traps discussed in chapter 2. However, they feel that the fact that the organization is in a dynamic environment and that wrong decisions are easy to observe due to the real time nature of selling electricity, that the traders are not necessarily able to game the metrics. Finally, as discussed by Parsons (2010) an important aspect of the type of energy trading done at organizations like BRPI is related to risk management. It is however unclear how to properly evaluate the contribution of the risk management team as part of the overall Energy Marketing division.

4.8 Conclusion
The above has presented the findings of the questionnaire and interview process in an effort to answer the five research questions. An assessment of the current performance management system is presented. The chapter also includes a detailed explanation of the metrics used by the organization, the goals covered by these metrics and those not covered. Metrics available from the literature and the choice of simple metrics is also presented.

The next chapter will present a discussion on the above findings and grounded theories relevant to the research questions and to the subject matter. Finally, it will present the overall conclusion to this research paper.

Chapter 5 Review, Recommendations and Conclusions

5.1 Introduction to the final chapter
The focus of the current research was to perform an examination of the performance evaluation metrics of BRPI. The main goal of the research was to look into the metrics used by the organization and review the possibility and desirability of linking complex mathematical hydrology models and other potential benchmarks with performance evaluation metrics, into comprehensive performance measurements for the entity being studied and for other organizations in the same industry. The following will review the research results, present recommendations to management based on the results, explain the limitations of these recommendations and finally present concluding remarks on the subject at hand.

5.2 Review of Results and Findings
The research was done using literature review on the topics related to the five research questions. It was followed by a questionnaire with the primary focus to assess the performance management system of the organization, and finally followed by interviews with five participants in order to complete the data gathering. The following presents an integrated review of these various research steps and results.
5.2.1 Maturity Level
Various performance management maturity assessment methods have been published over the years such as by Van Aken et al. (2005) who presented a model based on percentages of maturity. The researcher preferred to use the simpler four scale model presented in this paper as the main focus of the paper is not to do a thorough analysis of the model but rather to assess the maturity of the model as part of the overall examination. Also, during the interview process, it was clearly established that the interviewees were more comfortable with four or five point scale models than a percentage base model.

In reviewing the responses to the research questions, it is clear that the organization is at a relatively low level of maturity in regards to its Performance Management System as the system is established and repetitive but not yet efficient (CMA-Canada, 2010). The assessment based on the interview process and compared to the CMA-Canada (2010) criteria and the assessment level related to the organization’s reduced capability to extract data on a systematic basis to develop this data into strategies or metrics to evaluate strategies, rather than based on its methodology, which is also in line with the challenges presented by Brandel (2007). The researcher observed that the organization has a strong methodology in its performance evaluation based on the interviews and when benchmarked with Ferreira and Otley’s (2009) framework. It however does not have the human resources and financial resources in place that are necessary to adopt more complex techniques, as it is often the case in practice as was earlier discussed in Chapter 2 in the review of the findings of Paulson Gjerde and Hughes (2009).

5.2.2 Strong Bias Towards Financial Metrics
The performance management metrics have been described in Chapter 4. Through the examination of the performance evaluation metrics, the researcher observed that BRPI uses mostly financial metrics or metrics translated into financial measures. This likely comes from an accounting and finance background of most of BRPI’s upper management. They tend to request to be presented with financial figures to explain a performance and they are used to performance indicators that are explained in dollars and can reconcile with total revenues. The researcher observed that this also comes from the cash flow mentality of the organization which ensures an alignment between the organization’s strategy and its metrics as suggested by Ferreira and Otley (2009) but has its limitations as discussed in section 5.3.2 relating to long term goals.

The use of financial measures is also in line with Henri’s (2006) observation that financial measures are the most important metrics used in a majority of organizations. However, Henri (2006) also suggested to have both financial and non-financial metrics in order to be able to move to an integrated performance measurement system as not all of the organization’s performances can be translated into financial metrics.

It should also be noted that BRPI is following an observed pattern documented by Ferreira and Otley (2009) by which the performance metrics are mainly focused on a single division rather than being integrated to the whole organization, which in the case of BRPI, relate to the Energy Management division, resulting in the risk of lesser focus on the other divisions’ performance and lower overall corporate performance.

5.2.3 Transfer Pricing
It was observed that most of the metrics at BRPI are using a transfer pricing methodology in order to evaluate the controlled performance of the organization. This methodology is in line with what the researcher found in practice through observations and discussions with actors in the industry as well as in the literature relating to other organizations in that industry such as Parsons’ (2010) presentation of Constellation’s operations. The case study presentation by Malone (2004) relating to a trading operation that served internal clients in another field than electricity also relates to internal transfer pricing methods as the common method. Some of the metrics are based on detailed market information where others are negotiated with upper management which is consistent with the methods presented by Schuster and Clarke (2010) discussed in chapter 4. The researcher observed that this inconsistency in choosing metrics that are not always based on market prices has resulted in frustrations in the past as the parties were not in a position to assess the reasonability of the metrics with the use of models. However, even though the actual value of the metrics was contested, their relevance and their alignment with the organization’s corporate goals have always been agreed upon.

5.2.4 Alignment with Goals
It is important to note that the current metrics are aligned with the corporate goals. However, not all of the goals of the organization are being assessed using metrics. The organization may not be able to deliver on the goals that are not measured as there is a consensus in literature, such as Pavlov and Bourne, (2011) Franco-Santos and Bourne (2005) as well as Henri (2006), that measuring increases performance and behavior.. Referring back to the goals presented in appendix B, the goal related to capital program in order to maintain or increase power plant reliability and value as well as the goal related to improvement of facility efficiency are currently not evaluated. This is also discussed further as part of section 5.3 below.

This raises an interesting challenge as some of the metrics the organization could use in order to evaluate these goals do not easily translate into dollars, resulting in a more difficult reconciliation and less reliance in the eye of the users. When metrics are difficult to define they are often ignored by upper management (Ittner and Larcker, 2003) which was observed by the researcher when new metrics were presented to upper management that were in the form of percentages of performance rather than in dollar figures.

5.2.5 Choosing Simple Metrics
The researcher has presented various models that are available in literature and could serve as performance measures for the organization and others in the industry. The researcher concluded that there are two important limitations to the use of hydroelectricity models in the assessment of performance. The first limitation comes from the complexity of the models and difficulty to turn such models into easy to apply and replicate models in the daily activities of the organization as observed and confirmed during the interview process. This is a common roadblock to transforming data into information as discussed by Taylor, (2005) as well as Ittner and Larcker (2003). Interestingly, the authors of the hydroelectricity models discussed in chapter 2 have not made specific mention of limitations to practical application of their models in a corporate setting, a warning that would have helped shape the current research. The organization is facing system limitations which have been clearly acknowledged by both the organization and various potential vendors as per the outcome of the interviews (see also section 5.5). The second limitation comes from the observed preferrence of BRPI’s upper management to use simple rather than complex metrics as part of performance evaluations, which is in line with Johnston, Brignall, and Fitzgerald’s (2002) ‘good enough’ theory discussed in chapter 2. This leads the researcher to conclude that the organization is acting in a way that is commonly observed. As transpired in the BRPI interviews, there is no appetite for the use of complex models that are difficult to explain to shareholders and other users of the information. It has therefore been clearly determined that the organization is more interested in finding metrics that drive the right behavior as suggested by Pavlov and Bourne (2011) than on metrics that will help capture the optimal revenues in a particular sector of activity which is what was suggested by Taylor (2005). Johnston, Brignall, and Fitzgerald’s (2002) approach was supported by an interview methodology, which is similar to the current methodology, where Taylor’s (2005) suggestion relied mostly on business products and system implementations experience and observations.

It should however be noted that Johnston, Brignall, and Fitzgerald’s (2002) research was limited to successful organizations rather than also research of organizations that did not have strong performance. Therefore, it is unclear if in times of difficulties or lowered performance if ‘good enough’ information would continue to suffice.

The organization should take advantage of the simple and ‘big picture’ approach by bridging one of the gaps discussed in Chapter 4, which is the fact that many employees lack a ‘big picture’ view of the organization. Therefore, by concentrating on models rather than the business, there would be a risk that this lack of ‘big picture’ (Johnston, Brignall and Fitzgerald, 2002) would continue to increase for employees who concentrate on data crunching rather than management. However, by reviewing the ‘big picture’ and the inter relations between many variables, employees should be in a better position to come up with interesting ideas to maximize the organization’s revenues.

5.2.6 Simple Trading Activities
At has been clear during this research that the organization puts a stronger emphasis on its assets than on its Trading and Marketing opportunities and that it has a low risk tolerance. It was clearly illustrated by Parsons (2010) in the review of Constellation’s rise and fall that such conservative strategy is beneficial in the long term as the energy market is tributary to many variables that cannot all be controlled by the organization, such as counterparty solvability and market price fluctuations that may result in increased collateral calls and liquidity defaults. Therefore, the simpler the type of trading activities, the less likely will the organization be affected by large variations in its earnings and be affected by the solvability of its counterparties. Trading on the assets positions reduces the risk to the organization in fluctuating markets as it has the generating electricity to back its financial positions and receives the same net price as was originally intended on its hedged generation.

5.3 Recommendations to BRPI
The following presents recommendations to BRPI in relations with the examination of the performance management metrics of the organization as it links to published literature and the researcher’s findings.

5.3.1 Beware of Likierman’s traps
Likierman’s (2005) traps are an important and simple indicator of the current situation of the organization in regards to the choice of metrics it employs. It is therefore recommended that upper management reviews these five traps (detailed in section 2.3) with the current strategies and metrics in mind in order to increase performance, especially in areas that they do not feel that they have full control of the situation, such as the hydrology optimization strategies.

As pointed out by Ittner and Larcker (2003) some of the metrics can have a reverse effect if they are not well defined. For example, the number of acquisitions is one of the metrics of the Corporate group. The relative value of such acquisitions, the market versus paid price, etc. are not specifically evaluated, which means that the metric does not necessarily translate into better performance through strong acquisitions. It is therefore important to properly define the non-financial metrics and ensure that they meet the overall organizational strategy.

5.3.2 Increase the number of measured goals
Measurement drives performance. This has clearly been demonstrated in literature, such as in Neely (1999) and Pavlov and Bourne (2011) and in the responses to the questionnaire, which clearly present a concentration of employee’s efforts on the measured strategies such as on the optimization of daily generation through the peaker adder methodology. It is therefore recommended that the organization seeks to measure more of its strategies and goals in order to improve performance in areas that are currently not measured and compensated. For example, employees are compensated on cash flows, meaning that they are therefore compensated on the current year revenues, which puts less emphasis on future year revenues. Long term performance is however one of the organization’s goal and strategy that could be further developed through the use of long term metrics that are going to encourage the right behavior.

Another published goal of the organization is to “Improve efficiency of facilities and reduce costs” (BRPI, 2011). Efficiency is difficult to evaluate with strict cash flow calculations, which was observed to be the preferred type of metrics at BRPI as discussed above. However, finding metrics that would evaluate the increased efficiency of facilities such as the percentage of spillage, time response in the case of changed schedules, etc. would result in a more important focus on this published strategy. These are some of the variables discussed in De Ladurantaye, Gendreau and Potvin’s (2009) model and that could be used as simple indicators of the performance. Preserving the reliability and value of the assets through the capital investment programs could also be evaluated based on metrics related to the lost revenues due to unavailability of assets, relative value of assets over the years compared to similar assets or compared to a benchmark increase in value of the assets.

Some participants referred to Key Performance Indicators (KPI’s) as a way to assess the other goals not measured by the short term metrics. As pointed out by one participant, the KPI’s are not well communicated and lack the same level of focus as given to the financial measures. Although the scope of this paper is on the revenue management metrics, a further analysis could benefit the organization as to the use of a dashboard that would incorporate the KPI’s and be looked at on a more continuous basis.

It was observed that the organization concentrates most of its efforts on two of its metrics, which are the optimization of generation in the short term market (peaker adder) and the ability of the traders to reduce the risk of market price fluctuations through the use of fixed financial swaps. Other revenues have received less attention over the years. An important limitation to the current research is the ability to understand if there is less attention because of lesser opportunities to increase the overall revenues of the organization or less attention due to a lack of understanding of the potential for further revenues.

It is also important for the organization to contemplate looking at the current benchmarks and consider if the prior year performance is really reflective of the future year expectations based on market conditions, competition and staff experience as they may be caught in the Likierman (2009) traps which means that the performance could be improved by increasing the expectations. BRPI should also consider the possibility to benchmark performance based on competitors as discussed by Lieberman and Raskin (2005) and detailed in chapter 2.

5.3.3 Make the Best of Available Data
Turning data into information is one of the current initiatives of the organization. This goes in line with Ittner and Larcker’s (2003) suggestion to develop causal models related to the goals of the organization, to gather the data and to turn the data into meaningful information in order to properly assess and to base the organization’s actions on these findings. They came to these suggestions following an extensive review and surveys relating to the effect of non-financial performance metrics on the organizations performances. Applying these suggestions entails an important amount of effort to properly identify and understand the non-financial drivers of performance that will come out of this data. Revenue management metrics that are non-financial could be derived from these efforts for different purposes, including the evaluation of the organization on the efficiency of facility and value of assets goals described above.. This initiative should also serve to fill the gap that was observed by some of the participants, and discussed in sections 4.3 and 4.4. with regards to the fact that the hydrology variance to plan is considered a non-controlled variance in its entirety whereby parts of the variance is controlled by decisions of the hydrology group and are not reflected in the assessment of performance. As discussed by Brandel (2007) it is important that the users understand the data in order for the information to be meaningful and useful for decision making. As well Brandel (2007) also cautions about the common mistake of not committing enough time, money and change management to this type of initiative.

Communicating the information is also an important aspect to consider for the organization. To that end, the use of dashboards that are visible in real time to all employees should be considered as part of the communication strategy, as discussed with one of the interview participants.

As previously discussed, the financial and human resources dedicated to the performance management metrics initiatives are relatively low compared to the needs of the organization. It is therefore recommended to assess the organization’s needs in those two areas in order to consider these additional resources as an investment in the future performance of the organization.

5.3.4 Expenditures or Investments
Furthermore, it was observed through the review of internal reports and through the interviews that the organization tends to look at the expenditures in isolation of the revenues rather than to make extended analysis of the correlation between some increases in expenditures and revenue increases. This may impact the decisions made by the various actors of the organization if they only consider the increase in costs without linking them to increased revenues. It was observed that the organization seems to be lacking a big picture view or causal approach between cost and revenues.

5.4 Limitations of the Research
The current research was limited due to various factors. One of these factors was time constraint and the fact that both the questionnaire and interviews were performed with limited resources. Additional participants, especially at the CEO, COO levels as well as from additional Corporate division employees would have increased reliance on the data in order to perform stronger conclusions.

Case studies are also limited due to their time bound sensitivity (White, 2000, Otley and Berry, 1998). Therefore, some of the data and analysis presented as part of this research may be outdated and result in the fact that the research may not be easily repeatable.

A limitation to both the current research and to the organization is the fact that too much emphasis is put on the marketing initiatives and not enough on the other parts of the business that can also affect the bottom line. The current research is limited to this subject for two reasons. Firstly, due to the fact that the research had a revenue management focus. Secondly, it is due to the fact that the organization also concentrates most of its metrics, variance analysis, strategies, etc. on revenues rather than costs.

5.5 Future Research
Future research involving other organizations in the hydroelectricity and energy trading sector would provide additional insight on the subject at hand and would also help document industry best practices related to revenue management performance evaluation metrics rather than a single organization’s practices. This could be done using a comparable case study methodology and comparative method presented by Otley and Berry (1998) in their review of usefulness of case study research.

A research that concentrates on the cost accounting aspects of this industry would increase the body of knowledge and provide additional information on the management of the bottom line.

Malone (2004) suggests the introduction of an internal market that creates competition and a more competitive internal pricing. An internal offer and demand is introduced in the internal transfer pricing scheme rather than a dictated price. Such a concept could be interesting for the studied organization as different departments have different views on future prices and would be able to test their views as part of an internal market. Initial discussion was held with the interview participants. However, due to time and length constraints of the current research, and the fact that the subject was not discussed at length with the participants, the findings are not presented as part of the current research. This topic could be researched further in the future as it has the potential to bring stronger dynamics into the overall performance evaluation process and increase performance through the use of knowledge from other divisions within the organization.

Finally, as part of the literature on this and other similar subjects, one important aspect that is often neglected is the practical applicability of theories. This has been discussed by Baldvinsdottir, Mitchell and Nørreklit (2010) in an editorial comment and is in line with the current findings relating to the published models. A research on specific models already published and their application in a corporate setting with the help of current technology could also serve as a future research subject that would help the hydroelectricity industry.

5.6 Conclusions
The main purpose of the dissertation was to arrive at conclusions as to the applicability of published models into performance metrics in the hydroelectricity sector. Based on the findings of the current research, the following suggestions in the development of performance evaluation metrics have been developed and grounded in both research data and literature review:

1- Metrics need to be simple to understand and explain in order to be adopted and to result in changes in habits.
2- Financial metrics are preferred to non-financial metrics due to the fact that they can be reconciled with revenues presented as part of the published financial statements, which increase their validity and understanding.

It can therefore be concluded that published models that currently exist as part of the available literature are too complex to be adopted by upper management. It can also be concluded that the methodology adopted by BRPI is responding to both the technical and practical aspects of Revenue Management Accounting as the metrics are both aligned with strategies and simple enough to be understood and explained. The peaker adder methodology and the transfer pricing methodology used for the financial transactions of the organization should serve as models to develop further metrics both at BRPI and in the hydroelectricity industry.

Reference List

Anon (2011), Interview participants, July 2011, Interviews by Author.

Baldvinsdottir, G, Mitchell, F, & Nørreklit, H (2010) 'Issues in the relationship between theory and practice in management accounting', MANAGEMENT ACCOUNTING RESEARCH, 21 (2) pp. 79-82, [Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (accessed: 8 May, 2011).

BEMI (2009) Risk Management Policy, Brookfield Energy Marketing Inc., Internal document.
Benth, F., & Koekebakker, S. (2008). ‘Stochastic Modeling of Financial Electricity Contracts’ Energy Economics, 30(3), pp. 1116-1157, [Online]. Available from: http://www.sciencedirect.com/science/article/B6V7G-4P4FV58-1/2/b5e22c180... (accessed: 6 April, 2011).

Bergsjø, P (1999) 'Qualitative and quantitative research', Acta Obstetricia & Gynecologica Scandinavica, 78 (7), pp. 559-562, [Online]. Available from: http://dx.doi.org.ezproxy.liv.ac.uk /10.1034/j.1600-0412.1999.780701.x (accessed: 7 April, 2011).

Brandel, M. (2007). The Right BI Tool for The Right User. Computerworld, 41(38), pp. 56-61, [Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (accessed: 7 June, 2011).

BRPI (2011), Brookfield Renewable Power Inc. Corporate Overview [Online] . Available from: http://www.brookfieldpower.com/ (accessed: 4 May, 2011).

CMA-Canada (2010) ‘The Performance Management Maturity Framework’ Consortium for Advanced Management – International (CAM-I) Performance Management Interest Group (PMIG), Publisher: The Society of Management Accounting of Canada and the Consortium for Advanced Management – International, Canada.

Cooper, D & Morgan, W. (2008) ‘Case Study Research in Accounting’ Accounting Horizons, 22(2), pp 159-178, [Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (accessed: 12 August, 2011)

Daly, K. (2011) 'Corporate Performance Metrics To Top Board Agendas. (cover story)', Financial Executive, 27(1), pp. 50-53, [Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (accessed: 28 April 2011).

De Ladurantaye, D., Gendreau M., & Potvin, J. (2009) ‘Optimizing profits from hydroelectricity production’, Computer & Operations Research 36, pp. 499-529, [Online]. Available from: http://www.sciencedirect.com/science/article/B6VC5-4PW05HG-C/2/58428e01f..., (accessed: 7 April, 2011).

Denscombe, M. (2010) The good research guide for small-scale social research projects 4th Edition. New-York, USA: McGraw-Hill.

Etzioni, A. (1989) ‘Humble Decision Making’ Harvard Business Review, 67(4) pp. 122-126 , [Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (accessed: 7 April, 2011).

Ferreira, A., & Otley, D. (2009). ‘The design and use of performance management systems: An extended framework for analysis’ Management Accounting Research, 20(4), pp. 263-282, [Online]. Available from: http://www.sciencedirect.com/science/article/B6WMY-4X5HY6D-1/2/5b512c51b... (accessed: 7 April, 2011).

Fleten, S. & Lemming, J. (2003) ‘Constructing forward price curves in electricity markets’ Energy Economics, 25, pp. 409–424 [Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (accessed: 8 May, 2011).

Folan, P. & Browne, J. (2005) ‘A Review of Performance Measurement: Towards Performance Management’ Computers in Industry, 56, pp. 663-680, [Online]. Available from: http://dx.doi.org.ezproxy.liv.ac.uk /10.1016/j.compind.2005.03.001 (accessed: 7 April, 2011).

Franco-Santos, M. & Bourne, M. (2005) ‘An examination of the literature relating to the issues affecting how companies manage through measures’ Production, Planning & Control, 16(2), March 2005, pp. 114-124, [Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (accessed: 7 June, 2011).

Geman, H, & Roncoroni, A (2006) 'Understanding the Fine Structure of Electricity Prices', Journal of Business, 79 (3) pp. 1225-1261,[Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (accessed: 8 May, 2011).

Gong, M. & Tse, M. (2010) ‘Pick, Mix and Match? A discussion of Theories for Management Accounting Research’ Journal of Accounting – Business & Management, 16 (2), pp. 54-66. [Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (accessed: 7 June, 2011).

Henri, J. (2006) 'Are your performance measurement systems truly performing? (cover story)', CMA Management, 80(7), pp. 31-35, [Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (accessed: 24 May, 2011).

Henri, J. (2010) 'The Periodic Review of Performance Indicators: An Empirical Investigation of the Dynamism of Performance Measurement Systems', European Accounting Review, 19(1) pp. 73-96, [Online]. Available from:
http://www.informaworld.com.ezproxy.liv.ac.uk/smpp/content~content=a9206... (accessed: 24 May, 2011).

IESO (2011) ‘Energy Market Overview’, Independent Electricity System Operator [Online]. Available from: http://www.ieso.ca/imoweb/mktOverview/mktOverview.asp (accessed: 5 September, 2011).

Inglis, R.M. (2008) 'Exploring accounting and market orientation: an interfunctional case study', Journal of Marketing Management, 24, 7/8, pp. 687-710, [Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=..., (accessed: 6 September 2011).

Ittner, C, & Larcker, D (2003) 'Coming Up Short on Nonfinancial Performance Measurement', Harvard Business Review, 81, 11, pp. 88-95, [Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (accessed: 25 August 2011).

Johnston, R., Brignall, S. & Fitzgerald, L. (2002) ‘’Good Enough’ Performance Measurement: A Trade-Off Between Activity and Action’ The Journal of the Operational Research Society, 53 (3), pp. 256-262, [Online]. Available from: http://www.jstor.org/stable/822891 (accessed: 7 April, 2011).

Kakkuri-Knuuttila, M., Lukka, K., & Kuorikoski, J. (2008), 'Straddling between paradigms: A naturalistic philosophical case study on interpretive research in management accounting', Accounting, Organizations and Society, 33, 2-3, pp. 267-291, [Online]. Available from: http://www.sciencedirect.com/science/article/B6VCK-4MY6N19-1/2/b0a9e30ae... (accessed: 7 April, 2011).

Kaplan, R, & Norton, D (1996) 'Using the Balanced Scorecard as a Strategic Management System', Harvard Business Review, 74 (1) pp. 75-85 [Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (accessed: 24 May, 2011).

Lieberman, W, & Raskin, M. (2005), 'Comparable Challenges: A new approach to performance measurement', Journal of Revenue & Pricing Management, 4(2), pp. 174-184, [Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (accessed: 28 April 2011).

Likierman, A. (2009). ‘The Five Traps of Performance Measurement’ Harvard Business Review, 87(10), pp. 96-101, [Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (accessed: 7 April, 2011).

Lomax, S. (2010) ‘U.S. Renewable Credit Market May Pass Before Carbon, Group Says’, Bloomberg, [Online]. Available from: http://www.bloomberg.com/apps/news?pid=newsarchive&sid=a9mYvL4JW1QI, (accessed: 18 May, 2011).

Lowry, J. (1993) 'Management Accounting's Diminishing Post-Industrial Relevance: Johnson and Kaplan Revisited', Accounting and Business Research, 22(90) pp.169 -180, [Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (accessed: 7 April, 2011).

Malmi, T. & Granlund, M. (2009), 'In Search of Management Accounting Theory', European Accounting Review, 18, 3, pp. 597-620, [Online]. Available from: http://dx.doi.org.ezproxy.liv.ac.uk /10.1080/09638180902863779 (accessed: 7 April, 2011).

Malone, T. (2004). Bringing the Market Inside. Harvard Business Review, 82(4), pp. 106-114, [Online]. Available from:. http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (accessed: 7 April, 2011).

Narong, D. (2009). ‘Activity-Based Costing and Management Solutions to Traditional Shortcomings of Cost Accounting’ Cost Engineering, 51(8), pp.11-22. [Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=..., (accessed: 8 April, 2011).

Neely, A. (1999) ‘The performance measurement revolution: why now and what next?’, International Journal of Operations & Production Management, 19 (2), pp.205 – 228, [Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (accessed: 24 May, 2011).

Noreen, E. (1987). Eric Noreen on H. Thomas Johnson and Robert S. Kaplan's Relevance Lost: The Rise and Fall of Management Accounting. Accounting Horizons, 1(4), pp.110-116, [Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (accessed: 8 April, 2011).

NYISO (2011) ‘Understanding the Market’ New York Independent System Operator, [Online]. Available from: http://www.nyiso.com/public/about_nyiso/understanding_the_markets/index.jsp (accessed: 5 September, 2011).

Otley, D, & Berry, A 1998, 'Case study research in management accounting and control' Accounting Education, 7, pp. S105-S127, [Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (accessed: 12 August, 2011)

Parsons, J. (2010) ‘Do Trading and Power Operations Mix? The Case of Constellation Energy Group 2008’ Massachusetts Institute of Technology, Revised version 2010, [Online]. Available from: http://www.mit.edu/~jparsons/Presentations/Constellation%20case%20tradin..., (accessed: 22 August, 2011).

Paulson Gjerde, K, & Hughes, S (2009) 'Racing to success by identifying key performance drivers', Journal of Corporate Accounting & Finance (Wiley), 20 (3) pp. 59-65 [Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (accessed: 8 May, 2011).

Pavlov, A, & Bourne, M (2011), 'Explaining the effects of performance measurement on performance', International Journal of Operations & Production Management, 31 (1), pp. 101-122, [Online]. Available from: http://www.emeraldinsight.com.ezproxy.liv.ac.uk/journals.htm?articleid=1... (accessed: 7 April, 2011).

Rabionet, S. E. (2011). ‘How I Learned to Design and Conduct Semi-Structured Interviews: An Ongoing and Continuous Journey’ Qualitative Report, 16(2), 563-566. [Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (accessed: 10 August, 2011).

Schuster, P. & Clarke, P. (2010) 'Transfer Prices: Functions, Types, and Behavioral Implications', Management Accounting Quarterly, 11, 2, pp. 22-32, [Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (accessed: 8 May, 2011).

Taylor, G. (2005) ‘The Value of Enterprise Performance Management’ DM Review, 15(5), pp. 28-33, [Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (accessed: 7 April, 2011).

The Economist (2004) 'Regaining energy' 2004, The Economist, 372, 8390, p. 60, [Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (accessed: 25 May, 2011)

Thompson, M. Davidson M. & Rasmussen, H. (2004) ‘Valuation and Optimal Operation of Electric Power Plants in Competitve Markets’ Operations Research, 52(4) pp. 546-562[Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (accessed: 3 May, 2011).

Thompson, A., Strickland, A. Gamble, J. (2009) Crafting and Executing Strategy – The Quest for Competitive Advantage, Concepts and Cases, 17th Edition. New York, USA: McGraw-Hill International Edition.

Tsamenyi, M. M., Cullen, J. J., & Gonzalez, J. G. (2006) ‘Changes in accounting and financial information system in a Spanish electricity company: A new institutional theory analysis’ Management Accounting Research, 17(4), pp. 409-432, [Online]. Available from: http://www.sciencedirect.com/science/article/B6WMY-4KBX4KP-1/2/52b91fe8c... (accessed: 7 April, 2011).

Ulichny, P (1991) 'The Role of Hypothesis –Testing in Qualitative Research- A Researcher Comments', TESOL QUARTERLY, 25 (1) pp. 200-202. [Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (accessed: 5 July, 2011).

Van Aken, E. Letens, G. Coleman, G. Farris, J. Van Goubergen, D. (2005) 'Assessing maturity and effectiveness of enterprise performance measurement systems' International Journal of Productivity & Performance Management [online]. May 2005;54(5) pp400-418. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (Accessed September 29, 2011).

Waterhouse, J. H., & Tiessen, P. P. (1978). ‘A Contingency Framework for Management Accounting System Research’ Accounting, Organizations & Society, 3(1), 65-76. [Online]. Available from: http://search.ebscohost.com.ezproxy.liv.ac.uk/login.aspx?direct=true&db=... (accessed: 10 August, 2011).

White, B. (2000) Dissertation Skills for Business and Management Students. Hampshire: South-Western CENGAGE learning.

Yin, R. (2009) Case Study Research Design and Method, 4th Edition, Applied Social and Research Methods Series, Volume 5. California, USA: SAGE.

Appendix A - Questionnaire
Instructions:
The following 12 questions are intended to assess the current state of the organization’s ‘Performance Management System’ as YOU understand it. Please provide your personal answer to the following questions with 1 to 3 short sentences per question.

In filling your response, please do not refer to the company’s website or other documents as this is intended to be your perception of the ‘Performance Management System’ of BRPI/BEMI. There is no right or wrong answer. Your answers will be kept confidential as to your name and position.

Name:___________________________________
Title:_____________________________________
Department:_______________________________
Years of Service at Brookfield:_________________

1- What is the vision and mission of the organization?

2- What are the key success factors to the organization’s profitability?

3- What is the corporate structure and its influence on strategic management?

4- What are the strategies and plans of the organization?

5- What are the key performance measures relating to the strategies and plans?

6- What level of performance is needed to achieve the performance measures (target setting)?

7- What processes does the organization follow to evaluate individuals, groups and organization performance?

8- What rewards are gained to achieve performance target?

9- What specific information flows from the performance management system (feedback and feedforward)?

10- What type of usage are made of the information, including for controls?

11- How are changes to the performance management system occurring? Proactive or reactive?

12- How strong and coherent are the links between the performance management system’s information and they way the information is used?

Appendix B - Interview Questions

Introduction: This interview is performed as part of my MBA’s dissertation. All information discussed as part of the interview is confidential.

The dissertation is aiming at assessing the Revenue Management performance metrics of the organization and comparing with available literature.

1- Reflecting on the questionnaire you have filled out, how would you assess the current level of maturity of the organization’s Performance Management System?

2- What are the current management metrics used by the organization to assess the revenue management decisions of the BEMLP team?

3- Do you consider these metrics aligned with the following corporate goals of the organization as per the official website?

• “Increase and strengthen cash flow from the generation business by enhancing and optimizing our power bases
• Improve revenue from facilities by increasing contract prices
• Maintain comprehensive capital program to preserve reliability and value of facilities
• Improve on-peak capabilities and maximize water storage value
• Optimize ancillary revenues from hydroelectric facilities
• Improve efficiency of facilities and reduce costs
• Expand and diversify hydroelectric generation asset base
• Acquire additional hydroelectric assets capable of generating sustainable cash flows
• Develop new hydroelectric facilities with significant strategic advantages
• Increase consolidated leverage by optimally financing assets at the project level, while ensuring a balanced risk profile.” (BRPI, 2011)

4- Many answers relating to the use of the information presented its use as retroactive and mostly related to the performance assessment rather than proactive and value added. What needs to be improved with the revenue management metrics in order to improve decision making?

5- In the available literature on the subject, there are various hydro plant optimization models, valuation models of electricity financial swaps, etc. These models are currently not used by the organization to assess performance. The organization uses budgets and prior year performance.
a. Why do you believe these models are not implemented?
b. Do you think it would be a good idea to implement such metrics?

6- Some theories on performance management metrics contradict themselves in literature. Some literature tends to go towards models as discussed above. Other tends to consider that simple and easy to understand and explain metrics are preferable. The organizations choose metrics that are good enough. Which of these two views best fits BEMI’s appetite.

7- An interesting article suggests the creation of an internal market to determine the transfer prices between departments. By creating an internal market, other departments than trading would also receive positions that can be traded internally. Groups such as Marketing, Acquisitions, Strategy, etc. could use these positions in their operations and trade amongst themselves and with the Trading group. Could this be a potential avenue to evaluate in the future?

Appendix C – Summary of Questionnaire Answers

Appendix D - Detailed analysis of the Performance Management System
In assessing the performance management system, we need to assess the strength of the links between the various elements discussed in the proposed framework. This starts with the assessment of the company’s vision and mission by the various participants in the questionnaire process. It then reviews the link between performance indicators and strategies and the use of the performance indicators in delivering the strategies. The circle in the Figure 2 (page 26) represents the use of the information by different people in the organization in relations with communicating, strategizing and change management.

D.1 Vision and Mission
The vision and mission of an organization are defined as where the organization wants to go (its vision for the future) and, who the organization is and what defines it (its mission) (Thompson, Strickland and Gamble, 2009). The first question of the questionnaire had for purpose to review the vision and mission of the organization as well as the participants’ understanding of what vision and mission means. In their answers, the participants had to provide their own understanding rather than the official version. This enabled the researcher to assess the degree by which the information is communicated and known across the organization. Working towards a common vision and mission and developing metrics that are aligned with these is key to a strong performance and strong positive behaviors (Pavlov and Bourne, 2011).

There seems to be some confusion as to the difference between a mission, a vision and the organization’s strategies to deliver the vision and mission. The organization presents its vision and mission as being an organization in the renewable energy sector and with a vision of ambitious growth (BRPI, 2011). However, the participants refer to the vision and mission as the strategies to generate that ambitious growth such as entering in long term contracts and optimizing energy (Appendix C). The closer participants were to the delivery of energy and to the marketing of the energy, the more emphasis was put on optimization. As part of the evaluation process, these individuals are evaluated on their ability to optimize the assets. Which leads the researcher to see a link between the evaluation metrics and the emphasis put by staff on certain behaviors as discussed in Pavlov and Bourne, (2011) in relations with the creation of habits and also discussed in Likierman (2009) with regards to the risk of gaming the metrics. These were discussed in detail in chapter 2. The answers provided by the participants are aligned with what the organization presents as their published strategies on their official website (BRPI, 2011). It is however important to note that neither a vision nor a Mission are presented on the organization or its parent company’s website under such heading, which may be one of the reason that increases confusion between vision/mission and the strategies.

D.2 Key Success Factors
The second question in the questionnaire had for purpose to review what the participants believed were the key success factors to BRPI’s profitability. The key success factors are a mixture of both short term and long term focus. They are, as suggested by Paulson Gjerde and Hughes (2009) a mixture of financial and non-financial factors. These key success factors are aligned with what the participants have highlighted as the organization’s vision and mission in question 1. Such answers as “Cash flows are equal or above annual budget” (Anon, 2011) and “Tight Budget Control” (Anon, 2011) refers more to the financial factors where others such as “Knowledge of energy markets and surrounding environment” (Anon, 2011) are non-financial in nature with the intention to drive financial performance (Paulson Gjerde and Hughes, 2009).

There is therefore a clear focus and alignment between what are perceived success factors and the perception of the vision and mission. A further discussion on the success factors is presented in section 4.4 as this was further developed in the interviews.

D.3 Corporate Structure, and Strategies and Plans
The corporate structure as well as the strategies and plans should be developed in order to deliver the vision and mission of the organization. In order to ensure alignment, they should be developed in a way to deliver on the Key Performance Indicators that have been identified by management.

In general, the different activities of the organization are separated into different divisions which have a single purpose. There are three operational divisions based on geographical dispersion of assets (Canada, USA, Brazil). A fourth division is responsible for the delivery of the produced electricity through marketing, trading and optimization. Finally, a Corporate division (head office) has ultimate responsibility of the strategy and corporate services such as financial statement preparation and communication. This structure provides for a clear focus of each division on their Key Performance Indicators. For example, the Corporate division is responsible for the acquisitions of assets where the Energy Marketing division is responsible for the optimization of the energy delivered to the market and for capturing strong long term cash flows. As the focus of the organization is on its assets and their longevity, the legal structure is set up in a way to protect the assets. To that end, many legal entities have been created in order to isolate assets from each others.

In comparing the strategies and plans answers of the various participants to the official documentation coming from the website on the strategies of the organization, it is clear that the organization’s strategies are well understood by most participants. However, there are different degrees of interpretation which should not impact the overall alignment of staff efforts. It is interesting to note that although a majority of the participants to the questionnaire are evaluated on short term strategies, they still discussed the long term strategies in their answers. Therefore, organizational communication does not only relate to what individuals are evaluated on.

D.4 Measures and Targets
Key performance measures are set annually based on historical figures, growth plan and controllable revenue streams. These are based on operating cash flows, working group objectives and individual performances using set criteria. This is in line with Paulson Gjerde and Hughes (2009) who suggest developing industry relevant metrics and non-financial evaluation metrics as well.

The level of performance needed to achieve performance measures is considered to be very high by the participants. However, one participant discussed the fact that in an upward market, the financial metrics are easy to achieve and exceed, where in a downward market, the level of performance needed just to achieve the minimum objectives requires important efforts. As discussed with some of the participants, last year (2010) was a downward market. This may have impacted the assessment of level of performance needed in the participants’ answers.

D.5 Evaluation and Reward
The participants have clearly presented the process being used by the organization in regards with the evaluation process. There is a booklet made available to all employees explaining how the process works. The process is well communicated by the human Resources department and clearly understood by the participants. Some of the participants commented on the choice of measures that are in place and the timing, which may not coincide with the management activities that influence on these revenues. Further discussion during the interview process was needed to understand these comments. It mainly referred to the fact that long term hedging transactions are only compensated after the electricity is delivered rather than when the decision is taken, which can be two years after the decision is executed.

The rewards gained for achieving performance targets are in the form of variable compensation. The variable compensation is based on an overall performance assessment that include corporate, group and individual performance levels. There is more weight given to group and corporate performance in order to ensure that the individuals work towards a common goal.

D.6 Use of the information
As reflected in the answers presented in appendix C, the majority of the information that flows from the system is used for feedback to assess past performance rather than feed forward with a purpose to improve strategy. This is further discussed as part of the interview process in order to enhance understanding of the use of the metrics.

The information is used to prepare staff, group and corporate level performance evaluation and to develop employee trainings, etc. In the analysis of the responses, it transpires that the lower level staff do not see how the information is used apart from what they are impacted on, i.e. the bonus calculation. Communication of what the information is used for may help staff understand where they fit in the big picture. As well, copies of the Monthly Management book, which discusses the performance of the organization and its various metrics is not made available to manager level employees, which reduces their potential use of the information.

There are contradicting responses related to the type of change (proactive or reactive) occurring in the performance management system. Some responses reflect a reactive change where others see it as a proactive change. As well, a long term employee sees little changes in the past number of years, which, as per Likierman (2009) is one of the traps to keep in mind as organizations tend to stick to their numbers for too long. An employee that has been employed for a shorter time is of the opinion that the organization is fairly young and that a slow but proactive change has occurred in the past numerous years.

The link between the information coming from the system and the way it is used received diverse answers. The perception goes from weak linkage and no integration to very strong linkage. The users of the high level data (reports) see a stronger link than the users of more detailed level of information. Also, the evaluators see a stronger links than those being evaluated when the data is used for performance evaluation purposes. The performance management is taken seriously and the information coming from the system is used for that purpose.

Appendix E – Book Structure and Illustrative Example
The transfer pricing mechanism is done through the organization’s Risk Management System. The transactions are recorded in various trading books of the organization that have specific purposes. These are used for both Financial and Management accounting.
Figure 7 - Book Structure
Source: BEMI (2009)
The following explanation excludes any long term fixed price transactions with external parties as they do not form part of the marketing initiatives.

The Long-Term Book of the Energy Marketing division purchases generation from other BRPI subsidiaries at the ‘legal entity’ transfer price. The book sells the physical generation to the Physical book at an internal index representing the monthly average market price. It also sells the released generation under internal financial swaps to the Financial Book at the release price.

The Physical book sells electricity to the market at 5 minute to one hour market prices. The difference between the revenues received for the month from the market and the internal index transaction for that month represents the Peaker adder revenue. This is calculated by peak type (on-peak and off-peak)

The Financial book sells electricity to the Financial Market using mainly fixed financial swaps. The difference between the settlement value of the internal financial swaps and the external financial swaps represents the profit of the Financial book (BEMI, 2009).
Illustrative example
Assumptions:
We are using Tables 1 and 2 from Chapter 4 as the base for this numerical example.
The total revenues from both physical and financial revenues equal $75 USD for the month. The remaining revenues come from the other initiatives.
For simplification, the number of MWh sold physically and financially is the same and equals 1,000,000 MWh for the month.

Transactions:
Year 0
The long term book releases 1,000,000 MWh of energy to financial book at $61.
The financial book sells the released energy to an external party at $69, thus capturing a profit of $8/MWh above the released price.
Year 1
The long term book sells all of the 1,000,000 MWh of energy to the physical book at $56, representing the average monthly price (a $5 reduction since its release to the traders)
The physical book sells the energy on the market at $62, capturing a peaker adder of $6/MWh (or $6/MWh above the average price)
Figure 8 - Book Structure Illustrative Example
In millions (USD)

Source: The Author
In the above example, had the organization not actively managed the generation, it would have received $56 (the market price at generation time) rather than $75.