What is Benchmarking?
The process of benchmarking can be defined as “a structured approach to identify actions that lead to superior performance”. Benchmarking is not merely a comparison of performance data or a creation of rankings or league tables. Performance measurements, for example, deliver little benefit on their own, but they stimulate productive questions and lines of inquiry for more in-depth analysis and research.
In the public transport benchmarking groups, there has been a strong focus on results that can be implemented, performance improvement, and on informing strategy. For this reason benchmarking group members undertake a variety of detailed case studies each year in wide-ranging but focused areas of bus management. Many studies identify best practices in operations and management; others offer key insights that can drive strategy and policy and provide information to support better dialogue with city governments, regulators, and other stakeholders.
CoMET and Nova Objectives
The four main objectives of the Metro benchmarking groups are:
- To share knowledge and identify best practices in a confidential environment
- To support members in achieving their operational and strategic goals – focusing on improvements
- To build systems of measures for use by management and to establish public transport best practices
- To prioritise areas for improvement and support decision-making for management, boards, government, and other stakeholders
Key Benchmarking Principles
Fundamentally, CoMET and Nova provide a forum for metro organisations to share their experiences and share information. The following key principles have led to successful benchmarking over nearly 20 years:
- Collaboration – by giving and taking the good and the bad, members help each other improve
- Confidentiality – completely open data and information exchange within the groups and complete confidentiality to the outside, which overcomes political and commercial sensitivities (anonymisation protocols enabling information to be carefully and properly used publicly where appropriate)
- Continuity – a long-term approach with an annual cycle is required; as one-time benchmarking studies are rarely successful because it can take several years and iterative cycles to achieve comparability and consistently
- Speed – the moderate size of the groups, the carefully managed scope of the work, and the online interactions enable fast results
- Independence – the groups are owned and run by the participants (with the RTSC at Imperial College London acting at the administrator, facilitating the process and providing the research resources), enabling flexibility and ensuring that the focus is directed toward their highest priority needs and the areas that will produce the greatest benefits
Primary Benchmarking Elements
The Key Performance Indicator (KPI) system is the basis of the benchmarking process, and is designed to compare performance and identify lines of inquiry. CoMET and Nova use a KPI system with approximately 30 top-level indicators, which are designed to measure the overall performance of the organisation in six distinct areas: Growth, Learning & Innovation; Financial; Customer; Internal Processes; Safety & Security; and Environment. The design is comprehensive enough so as to represent all of the different parts of the metro business, yet concise enough to be able to be used effectively by management. The data items behind this set of indicators is collected to standard definitions by all members on an annual basis.
The performance of a metro organisation is impacted by factors both within and outside management control. It is therefore very important to understand the effects of city context and external factors on metros’ performance. To account for this, the KPIs are often normalised in multiple ways, such as for the local cost of labour or the density of the city. Statistical analyses are used to provide greater understanding of the results, while time series analyses allow for trends in performance to be identified. This helps to highlight which members are changing their practices and what improvements are relatively achievable. Where clear differences or improvements in performance are identified, detailed analysis is carried out through case studies. Approximately four to six case studies are conducted in CoMET and Nova combined each year, that included detailed data analysis, questionnaires, and interviews with functional experts. Case studies are further supported by expert workshops where appropriate. In many cases best practices may be found outside the metro industry, so other public transport operations and even other industries are reviewed for relevant practices.