It’s not the methods, but the purposes of CBA, that need re-appraisal now

DETAILED COMPLEX ANALYSIS of the rationale for building major infrastructure like transport is a recent phenomenon. Until barely fifty years ago, decisions were either made by private investors on the basis of expecting a profit (or sometimes as a statement of personal ambition), or by public authorities undertaking ‘Civic Works’. The latter were done by those believing they were paving the way to a better, more advanced society, or dealing with obvious problems such as disease and death from poor sanitation and the need for supply of clean water to drink, the provision of gas and electricity for power, and safe and reliable roads on which goods and people could move.

Though the theoretical concept of Cost Benefit Analysis dates back to an 1848 article by Frenchman Jules Dupuit exploring “the social profitability of a project like the construction of a road or bridge”, and was formalised in subsequent works by Alfred Marshall including his book Principles of Economics (1890), it was not applied practically to investment decision- making until the middle of the 20th Century.

There was neither the analytical framework, nor the calculating capacity to bring together all the benefits, costs, consequences and alternative choices in numerical form to make a balanced overall judgement about what was, or was not, a worthwhile deployment of public funds — and especially how to select particular potential projects as being more desirable than others.

In the post war world , the relatively straightforward but simple thinking that previously drove investment decisions significantly changed. There came the arrival of the new appraisal technique from the developing world of welfare economics, which heralded a revolution in the disbursement of public expenditure on infrastructure investment, first in America , then taken up in Britain in the 1960s. Indeed, much cost-benefit analysis around the world still follows a template first set out in a 1962 academic paper by Professors Michael Beesley and Christopher Foster, who studied the construction of the Victoria Line in London. They demonstrated that although the line would generate little additional revenue, the value created in time savings and reduced congestion above and below ground far exceeded the costs. Fifty years later, there can be no doubt that in this project they were right. The all-in cost of less than £100m, equivalent to a little over £1bn today, seems a very low price for what is now an indispensable part of the capital’s transport infrastructure. In 1960, Beesley had also looked at the equivalent outcomes achieved by Britain’s first Motorway , the M6 section that was the Preston bypass and had opened in 1958.

The CBA concept was subsequently more widely promulgated until it is now the go-to governmental mechanism for taking proposed schemes through the public spending approval process into construction — but with a whole raft of consequential and contextual tunes being played on the original theme. Many perhaps not so clearly demonstrating obvious public benefit from the massive sums of money involved, but fulfilling a pro-forma test function nonetheless.

Interestingly, the most thorough cost-benefit analysis ever undertaken was arguably that commissioned in 1968 for the Roskill Commission, headed by a distinguished judge of that name who supervised a dispassionate two-year long inquiry into London’s airport needs. From cost-benefit calculations down to the value of threatened Norman churches, the analysis recommended in 1971 that London should have a new airport at Cublington, about 40 miles northwest of the metropolis. The recommendation was not adopted. Politicians procrastinated, chose a more convenient solution in expanding the existing underused Stansted airport and have continued to debate the consequences for the next 50 years, with controversy after controversy about the adequacy of both Heathrow and Gatwick as London’s main airports. The Commission may well have come up with the right answer, but the politics got in the way, as they have so many times since, affecting not just the final choice of schemes, but they way in which they both get into the queue for consideration and the score sheet they must pass to go ahead.

Pertinently, in view of the impending publication of the Department for Transport’s draft third Road Investment Strategy (RIS3), setting out priorities and an outline budget for National Highways for 2025-2030, where we are now with appraisal is explored in his column by Professor David Metz in this issue. As well as some recent unconvincing appraisals of a number of highly expensive road transport schemes based on the existing criteria, often seemingly adjusted to suit the predetermined preferred outcome, he also evaluates the work of the Welsh Government in re-thinking the basis of appraisal to suit its more general objectives, as set out in the new WelTAG guidance.

The history on this journey has many interesting stopping points including myriad government policy documents and White Papers, committing to programmes of transport investment that then need to go through the necessary appraisal — almost inevitably coming out with a suitable sign-off. Crucial are the ‘Business cases’ that have been a Treasury - sought addition to the CBA concept, seemingly to position public investment decisions as all the better for being close equivalents to the hard-headed ones of commercial corporations. Only time and changing social and political concerns (notably the environment) have seemed to be able to interrupt this pipeline of ‘essential well-justified schemes’. Meanwhile the decision-making tools have been suitably adapted and retrofitted to a an inch of their life to meet the latest critiques and conceptual refinements.

Back in 1969, the first such road programme wish-list for the cost-benefit era (the earlier motorways had not faced such a test) was set out in the Green Paper entitled Roads for the future—A new Inter-Urban Plan published by Richard Marsh, the then Transport Minister, and priced ten shillings.

In driving forward the new roads strategy, an attempt was made to assess the cost of congestion and accidents which would be caused over the next ten years on each stretch of road, to identify those sections where the worst conditions would occur and which, therefore, appeared to justify first priority improvement.

A very large number of schemes were being prepared to a point where a suitable cost/benefit examination could be undertaken. Work was to proceed until an initial pool of £1,000m worth of schemes had been chosen. The Green Paper said the direct economic effects from a road improvement scheme would be measured by calculating the savings in time, operating costs and accidents to road users. The rates of return for most inter-urban schemes calculated in this way were ‘very high’. They included benefits of commercial value from regularity in delivery leading to lower costs, reduced stockholding, the use of larger and faster vehicles suited to better constructed roads, and the stimuli of new industrial development that would not otherwise have occurred.

Meeting the expected demand in 20 or 30 years’ time was the objective, and a comprehensive route strategy of high class roads connecting all important centres of population would provide it. It was also seen as enabling the sort of forward information which industrial and commercial interests would find invaluable in planning the future of their businesses, with particular attention to the needs of industrialists and exporters and the changing pattern of industrial activity.

Whilst not all of the improvements in the Green Paper would be motorways, they would all be at least dual-carriageway all purpose roads. By first adding to the motorway network, and then the access to and between the main core routes already constructed, the wider objective was to provide the country with a main system of high class roads to which all important centres of population - existing and projected - would, or could easily, be connected.

This translated to every city with a population of over 250,000 (of which there are only a couple of dozen in England), being connected directly to the strategic network, whilst any town with a population of over 80,000 (which is a lot more) would be within 10 miles of that network. The network would also be designed to serve every major airport and seaport.

Not surprisingly in this motorway-enamoured time, the Green Paper turned into a White Paper and moved into Government policy with the rather heroic claim that congestion on the inter-urban trunk road system would be virtually eliminated.

No detailed priorities or time-scales were set out, as these were expected to emerge in the course of detailed planning. And publication of the proposals will in no way replace the normal processes of scheme selection and negotiation with local authorities and other interested parties, nor, of course, the statutory procedures, promised transport minister Marsh.

With the 1969 Green Paper and a succession of similar subsequent documents the path was cleared for the continuing road building programme that has barely been interrupted since .These studies laid the foundation for successive government analysis of transport projects to either confirm their inherent value, or justify them, depending on your underlying perspective . In the case of roads it was first formalised by work in the Ministry of Transport’s Mathematics Advisory Unit and in the TAL (traffic and accident loss) procedure, and then in COBA (cost-benefit analysis) which has now been subsumed in the modern day web- based transport analysis guidelines (WebTAG) and the delivery of all important Benefit Cost Ratios.

Since 1998 a new framework for appraisal has been used with the aim of presenting a one-page summary of the analysis (‘appraisal summary table’) for ministerial perusal. The economics are but one component of the formal framework, and most recently ‘solutions’ (projects) have been assessed against five objects: environment, economy, safety, accessibility and integration. In turn each of these objectives has a number of sub-objectives. For the economy objective (‘to support sustainable economic activity and get good value for money’), for example, there are five sub-objectives specified. These have been: to get good value for money in relation to impacts on public accounts; transport economic efficiency for business users and transport providers; to improve transport economic efficiency for consumer users; to improve reliability; and to provide beneficial wider economic impacts.

The Department’s overall approach to economic evaluation of a project has become increasingly controversial over the years particularly with regard to its assumptions regarding average values of a unit of travel time saved and its calculation of benefit-cost ratios in economic appraisals, a subject examined in critical detail by a number of experts over the years, both invited to do so by the Department for Transport and independently, notably Professors David Starkie and David Metz. Travel time savings, which are divided between time saved during the course of work and other travel time savings, are a core feature of the user benefits. For road projects, average values per unit of saved travel time, multiplied by the respective estimated quantities of time saved, are generally said to sum to about 80 per cent of the total benefits. Issues identified by Professor Starkie in 2013 included: whether small savings of non-work time have a value and should count in the appraisal; whether unit values are non-linear so that time savings on long journeys count proportionally more, and the rate at which time values should be increased through time. The Department meanwhile modified its approach for non-working time, and more recently, because of the development of communications technology such as smartphones and the use of personal computers during travel, the practice, particularly in the context of rail projects, of treating all business travel time as unproductive time, has been questioned especially in justification of High Speed rail investment. A more fundamental criticism has centred on the invariance of travel time over historic time and suggestions that the benefits of improvements to transport systems are to be found in the additional access afforded, not in notional travel time savings, and idea long promoted by Professor Metz.

In the seventy years or so in which the cost-benefit concept has been underpinning the major transport investment decisions, there have been dozens of such observations and reflections on its strengths and weaknesses, the things overlooked and mis-counted, the emerging issues that might over-ride the capability of the core equation to embrace, and the questionable adjustments made to the basic framework to address its fragilities. Throughout this dialogue, however, the key surviving paradigm is that scheme proposals, thought up in different political, commercial and social conditions, become sponsored as being suitable for detailed consideration and then seem to get a life of their own as they go down the appraisal conveyor belt. It would either be a most remarkable coincidence that most of these suggestions prove to have been brilliant ideas that pass all the required tests, or to suggest otherwise that with creative thinking and imagination, the numbers can usually be found to get them through.

In 2018 the DfT seemed to be encouragingly consulting on priorities for a new appraisal and modelling strategy. But it turned out to be a very limited review of practice, rather than principles, with a focus on the detail of appraisal inputs and processes, rather than its purpose and structure . The chief analyst, Amanda Rowlatt, set out the core thinking that “ Our aim is to provide appraisal and modelling tools that are robust, flexible and easy to use, to support the policy and investment decisions which will be made over the next five years.” A period, interestingly, now concluded and suggesting a further review is perhaps now due.

In her foreword to the subsequent publication reviewing the review in 2019, Rowlatt concluded that the feedback confirmed that there “was a vital role for our guidance in providing a comprehensive, consistent and robust approach for assessing the impacts of transport investment,” though there was also need to increase the scope and depth in some areas, she acknowledged , notably to meet the needs and ambitions of local and devolved areas. “Alongside this, we need to ensure that analysis can be delivered more efficiently and one of our immediate priorities for the strategy is to make our guidance more accessible and easy to use”.

In the review report it was prominently noted that the Department for Transport’s mission “ is to create a safe, secure, efficient and reliable transport system that works for the people who depend on it; supporting a strong, productive economy and the jobs and homes people need.” Whilst it is acknowledged that it is essential that there is as full a view as possible about the impacts this investment will have on transport users, the economy, society and the environment, it is notable that the commitment to investment precedes these considerations. This overview might thus be seen as informative, rather than a reason to re think any basic assumptions behind the investment-led paradigm.

Indeed this is made further evident elsewhere in the strategy review where it is stated that a key objective is “developing the transport analysis guidance and tools to provide the evidence base for transport investment decisions over the next five years, targeting the most significant evidence gaps and ensuring decision-makers are presented with the information they need.” Not though, indicating how this information requirement is most properly defined, or how the ultimate decision-makers can best consider that information and bring together the challenging multiple dimension in making better decisions.

In this regard The Department’s Transport Analysis Guidance (TAG) is fundamentally there to provide guidance on conducting transport analysis for the economic case, by interpreting the requirements of the Treasury’s Green Book on how central government departments undertake appraisal and evaluation, whilst supporting stakeholders and enabling them to prepare efficient yet insightful appraisals in business cases.

The origins of the Green Book seem to lie in its part in the history of CBA in the United States, with the issuing of a report in the 1940s to address the fact that various agencies were adopting different and inconsistent methods of estimating costs and benefits on water related public projects. In 1946 a subcommittee on benefits and costs was appointed “for the purpose of formulating mutually acceptable principles and procedures for determining benefits and costs for water resources projects”. This subcommittee issued a final report entitled “Proposed Practices for Economic Analysis of River Basin Projects” in 1950, which then became known as the Green Book, since recognised as the first landmark in the history of CBA in the United States. CBA covered completely for the first time its modern subjects, including definition of benefits and costs, general procedure for the measurement, consideration of available alternatives, criteria for comparing alternatives, choice of discount rate, risk allowances, and economic life of projects. One of the strengths of the original Green Book lies in its setting out of the basic principles of microeconomics, although not in highly theoretic terms. It stated that the ultimate aim of development projects is ‘to satisfy human needs and wants by providing goods and services’, which refer to all objects and activities that have the power of satisfying human needs, and may increase or decrease in availability as a result of a project. It was aware of the limitations of the market price system in reflecting values of goods and services from a public viewpoint, but concluded that there is no more suitable framework for evaluating public projects in common terms. Therefore, market prices had to be chosen as the starting point for measuring the tangible effects of a project, whether benefits or costs. It was conceded that some tangible effects that cannot be assessed based on market prices may be derived indirectly from prices for analogous effects or from the most economical costs of producing similar effects by an alternative means.

Here, it can be observed, all of the cost-based methods that are used widely today, have their rudiments, but much changed along the way. For example, the Green Book argued that the benefits of navigation improvements were measured by users’ savings in transportation costs rather than reduction of production costs, which was CBA’s founding father Dupuit’s main point when he espoused the concept in relation to exploring “the social profitability of a project like the construction of a road or bridge “ one hundred years earlier in 1848.

Though this is not now by far the only theoretical issue that confronts cost-benefit analysis, it is perhaps an indication of how the concept has arguably morphed into becoming a way of measuring things to enable propositions to clear hurdles of approval, not a prompt to carefully think about how significant sums of money and resources are best deployed within a wider economic and social policy context. There is now a much wider list of things that should - even must- be taken into account in making prudent, robust and fair decisions - the impacts of transport on the environment, and the severity of climate change being the most notable recent addition. The spotlight should thus be as much on what are the competing considerations and trade offs in these difficult decisions - and their implications, and not just producing a bottom line figure needed to pass a theoretical test. Especially if both the input numbers, and the test itself, are barely recognised and intelligible beyond a small group of rocket scientists.

It is the concepts, purposes and applications of transport appraisal, not the methodologies, that are surely in need of most serious re-examination now. And by a general audience, not just amongst a few experts.

Peter Stonham is the Editorial Director of TAPAS Network

This article was first published in LTT magazine, LTT892, 21 May 2024.