Transport investment appraisal - What are we really trying to measure?

THE ECONOMIC APPRAISAL of transport investment as we know it today can be traced back a little over fifty years Though the theoretical concept of Cost Benefit Analysis dates back to an 1848 article by Jules Dupuit exploring “the social profitability of a project like the construction of a road or bridge”, and was formalised in subsequent works by Alfred Marshall including his book Principles of Economics (1890), it was not applied practically to investment decision making in Britain until the middle of the 20th Century.

In the USA, the Federal Navigation Act of 1936 had mandated cost–benefit analysis for proposed federal-waterway infrastructure. It’s most famous early use in the UK is probably the case made by professors Beesley and Foster for the construction of the Victoria Line in London during the 1960s [ref 1]. Drawing on such practice, the Ministry of Transport had set up the Mathematics Advisory Unit by the 1970s, later to become the Transport Appraisal and Strategic Modelling (TASM) division.

With the originating emphasis on journey time savings by road users and the relief of congestion, the framework of appraisal doesn’t seem to have fundamentally changed much since, despite an awareness and concern about wider impacts from the provision of new capacity having grown massively in recent years – latterly and most particularly in relation to the issue of Climate Change. Unfortunately, such concerns still seem effectively to be treated within TASM its Transport Appraisal Guidance as technical bolt on extras - not core drivers of decision-making.

It was therefore refreshing to see the Welsh government issue its own materially revised version of Transport Appraisal Guidance (WelTAG), earlier this year, as reported in LTT 886 of 20 February. This followed extensive consultation on an earlier draft, driven by an awareness that the decision-making paradigm for roads was out of alignment with wider policy intentions. The central concern of the changes is to tackle the climate emergency, which is not the focus of the DfT’s TAG, for which the impact of additional carbon emissions from road investments seems to be a problem to be side stepped. For the Welsh government, in contrast, every transport programme and project it funds must now help reduce carbon emissions. Thus, additional road provision should no longer be the default answer to all transport issues; more active travel is needed and less car use, under agenda-defining Welsh national transport policies, and wider national social and economic goals including the idea of wellbeing.

One target of the Welsh government is a reduction of car miles per person of 10% by 2030 from 2019 levels. To help achieve this the COM-B model of behaviour change is cited, which posits that to change a behaviour, an individual must have the capability to do it, the motivation to do it and external factors must provide them with the opportunity to do it. A question is, of course, whether sufficient numbers of car users have the motivation to reduce car use, given the convenience of door-to-door travel this mode offers.

WelTAG is claimed to be based on the principles of the Treasury Green Book for delivering value for money in all government policies, a document that has perhaps changed more over time than transport appraisal has fully reflected. However, instead of only calculating costs and benefits in monetary terms, WelTAG designs well-being and decarbonisation into the programme or project from the outset, using the new integrated well-being appraisal framework [ref 2].

Accordingly, the usual economic dimension of the business case has been renamed the ‘well-being dimension’, which is intended to show how the programme or project contributes to social, environmental, economic and cultural well-being in Wales. Calculation of benefit-cost ratios (BCR), a core element of conventional economic appraisal, is not now seen as helpful for many smaller transport projects in Wales. For larger, riskier projects, BCRs should be calculated with and without travel-time saving as a benefit, the new WelTAG requires. In addition, monetised time savings should be separated out to show the distribution of them (for example 0-2 minutes, 2-5 minutes and 5 minutes plus). This is intended to enable decision makers to see the impacts and the distribution of the benefits separately, and aid them in making judgements on the impact of the time savings in achieving Welsh Government policy objectives, with the questionable aggregation of small savings a particular issue of concern.

This treatment of travel time savings reflects the Welsh government’s policy objective to improve journey times for active travel and public transport, to make them time-competitive against the private car and encourage a shift to more sustainable modes of travel. Taking into account travel time savings for car drivers when appraising potential investments would encourage more car use, and run counter to legislation to restrict speeds on most local roads to 20 mph in pursuit of safety and environmental benefits. Indeed, the WelTAG document states that journey time savings for private car users will not be considered as a benefit as this does not support the priorities and ambitions of the Wales Transport Strategy.

It should be noted that the position outlined above was how matters stood in February, since when a change of ministers in the Welsh government has signalled a possibly more favourable attitude to new road construction and a relaxation of the default 20 mph speed limit on local roads (see LTT 890, 23 April). However, there is no suggestion of a major reconsideration of the general approach to transport investment as embodied in the formally-adopted new WelTAG principles. Their creation was an unusual step on which the former Welsh Transport Minister hung his hat – but refreshingly so, from his taking a genuine interest in the ‘black box’ that drives policy delivery, rather than the announcement of new schemes.

WelTAG’s disregard of user benefits in the form of time savings, without their replacement, means that economic analysis is now effectively much downgraded. What we then have is what may be termed multi-criteria analysis, with the BCR as just one criterion among many, or omitted altogether. However, investment appraisal without economic content seems like an oxymoron, and thus there is a gap in thinking here. What is missing is a methodology that values the benefits of an intervention to all classes of traveller, to help judge to what extent the intended mode shift would yield a net benefit in magnitude, and a better distribution. An approach based on valuing access benefits could in principle be attractive to do this, offsetting loss of access to private car users by gains of access to those using active travel or public transport. Regrettably, a methodology for attributing monetary values to access improvements has not so far been developed.

The disengagement from conventional welfare economic analysis of transport investments by the Welsh government is only one of a number of such departures from orthodoxy we have seen in recent years, particularly in respect of rail investment. The government’s Integrated Rail Plan for the North and the Midlands, published in 2021, included consideration of options for improved east–west connectivity between the cities of the North of England, all of which were formally assessed as low to poor value for money [ref 3]. The plan recognised that: ‘Rail schemes in the North are at increased risk of being considered poor value for money when applying conventional cost–benefit analysis. This is driven in part by smaller city populations in the North, different travel patterns, as well as the general high cost of building rail infrastructure’. This admission of the failure of standard economic analysis to confirm the politically desired investment programme necessitated recourse to an alternative methodology. As Groucho Marx said: ‘Those are my principles, and if you don’t like them . . . well, I have others.’

The alternative methodology that informed the Integrated Rail Plan had been developed by the National Infrastructure Commission, whose role is to advise on investment in all national infrastructure. The Commission was asked to prepare an assessment of the rail needs of the Midlands and the North. This proposed priority for regional links as having the highest potential economic benefits, rather than long distance links [ref 4].

The Commission used a multi-criteria approach to assess the options, including quantification of the economic benefits both from improving business productivity in city centres and from connecting people to city-centre services. This approach drew upon a previous study addressing how to capture the value of urban transport investments, which recognised the invariance of average travel time, and so looked for the real benefits arising from increased city density made possible by investment in urban transport [ref 5]. Agglomeration benefits to businesses have long been recognised, even in the Victorian era, with urbanisation facilitating clusters of specialist trades and professions as a feature of commercial activity. The Commission’s novel approach was to replace the orthodox transport user benefits with amenity benefits based on the increase in social value achieved by thriving and densifying cities. The government broadly followed the Commission’s advice in formulating its Integrated Rail Plan.

The possibility of forecasting the actual expected benefits of investment, rather than notional gains in economic welfare, is illustrated by a recent London rail investment - the Northern Line extension to a large brownfield site at Battersea, centred on the former Power Station, preserved on account of its architectural merit. The investment decision for this new two-mile tunnelled rail line followed a standard economic appraisal of transport user benefits for a range of alternative property and transport investments, where the predominant benefits were travel time savings. It was found that (the expensive) £1.1 billion extension of the Underground would have a less favourable benefit–cost ratio than other transport alternatives on account of the higher capital cost. Nevertheless, the decision was made to extend the Tube, the increase in real estate value being the deciding factor, such that the Power Station site developers were willing to contribute a quarter of the capital cost upfront. Thus, the decision was taken essentially on a commercial basis, with the estimated increase in real estate value forming an integral element of the investment decision. This exemplifies the scope for a transport authority working with a developer and planners to take into account the value of real estate improvement, and indeed apply some of this to contribute to the cost of the new infrastructure, Land Value Capture, as it is known.

Another major London rail investment, Crossrail, was renamed on opening as the Elizabeth Line. The original case for investment was based on the value of travel time savings to users (business, commuting and leisure) plus a number of wider economic impacts (mainly agglomeration benefits), but with no explicit reference to the impact of the new rail route on real estate values, or on the economic value of the businesses to be accommodated in new developments along the route, although there was a land value capture mechanism to help fund the scheme from business contributions. The modelling assumption was that the boost to development and employment could be implicitly accounted for by the value of travel time savings plus the wider impacts, an assumption that is scarcely credible to non-economists (and perhaps even to some economists).

Transport for London has developed a framework to evaluate the benefits of the Crossrail investment which should be very useful in demonstrating reality in contrast to the theory [ref 6]. It is envisaged that a study to be published two years after opening will address the transport effects of the new railway, including mode shift from cars to public transport, relief of congestion on public transport and roads, and the implications for air pollution and carbon emissions. A subsequent study is planned to consider the broader social and economic effects, including the effect of improved connectivity on new homes and jobs, changing patterns of employment and land use, and residential and commercial property prices.

Such a detailed approach to evaluation is both relatively rare and admirably ambitious, yet there is inconsistency with the original investment case, which was based on the value of notional travel time savings and of wider impacts inferred from econometric analysis, with no indication of the spatial distribution of benefits. It seems unlikely that it will actually prove possible to compare forecast and outturn by deducing time savings and agglomeration benefits from the evaluation findings, as would be desirable to assess the validity of the modelling in support of the investment decision. There is also the question of whether, with hindsight, the investment appraisal could have usefully been based on projections of the actual benefits that are expected to be achieved. After all, the objectives of the investment were not to increase notional economic welfare, but rather to deliver the outcomes that have now been identified for evaluation.

Arguably, exposure of such a material mismatch between notional welfare benefits and real-world expectations contributed to the Prime Minister’s decision last autumn to truncate HS2 beyond Birmingham. The projected travel time saving and other notional benefits were insufficient to accommodate the increase in capital costs of the scheme. In contrast, an approach that recognised the development benefits to the cities of the Midlands and the North might have had a firmer basis in reality, been more generally understood, and would have engendered more regional support, both political and in the form of contingent development around railway stations on the route, as well as of local transport linkages.

For road investment, many of the problems of economic appraisal have been exemplified in the information recently revealed about the business cases and BCR values of several major highway schemes and upon which I and others have commented in recent LTT articles: my analysis of the A428 widening (LTT 888, 21 March 2024) and Phil Goodwin’s critique of the Lower Thames Crossing [ref 7]. The A66 Trans Pennine scheme has also attracted criticism on account of poor value for money [ref 8].

A further notable example of a questionable BCR calculation is the proposed upgrade to the A303, a road connecting London and the South-East to the South-West of England, used by both longer-distance and local travellers. This historic route comprises a mixture of single-lane (in each direction) and dual-carriageway sections. One of the single-lane sections runs close to the Stonehenge ancient monument in Wiltshire, designated as a World Heritage Site. The plan developed by National Highways and its predecessors is to sink a section of the road alongside the site in a two-mile tunnel costing close to £1 billion, for which a Ministerial Development Consent Order has recently been granted, though subject to challenge. User benefits from this major upgrade from facilitating faster travel were estimated to amount to only £250 million present value. To make the scheme at all economically viable, and achieve a break-even BCR of about 1, ‘cultural heritage’ benefits of almost £1 billion have been invoked from the removal of the road from visual intrusion at the Stonehenge site.

The estimation of cultural benefits was based on a novel stated preference survey designed to elicit monetary values for a hypothetical change in noise, tranquillity, visual amenity and landscape severance within the Stonehenge area, by asking individuals about their willingness to pay for, or willingness to accept monetary compensation for, a hypothetical change in the layout of the A303 road. Based on 3,500 survey responses, it was estimated that the cultural benefits to visitors to the site were worth £25 million present value; benefits to users of the road, £50 million; and to the general population, £1.2 billion. However, it is hard to avoid regarding the very high monetary value attributed to cultural benefits as both unusual and problematic, in that there must be very considerable uncertainty about the basis of such an estimation. The question is whether people might have responded with a higher value than they would have done in a real-life decision-making situation, and with the opportunity cost of the substantial sum of money involved acknowledged. This case could be said to be another example of the economic analysis being devised to fit a prior political decision.

The A303 Stonehenge scheme has been the subject of a welcome review by the National Audit Office (NAO), whose role is to provide independent scrutiny of public spending [ref 9]. This review recognised that the Stonehenge scheme was only one of eight projects aimed at upgrading the route corridor, yet no overarching programme-level business case had been produced, National Highways intending to seek approval for each project individually. However, based on the standard appraisal process, five of the uncommitted projects along the corridor are rated as low to poor value for money, which means there must be uncertainty as to whether and when they might go ahead. Without completing all eight projects, the strategic objectives for the corridor would be hard to deliver, the NAO concluded, casting doubt on the merit of the Stonehenge project.

Perhaps anticipating criticism from the NAO, the Department for Transport commissioned a consultant’s study of appraisal at programme level [ref 10]. This addressed the problem of interdependence: how individual schemes might be complementary, competitive or independent of one another, such that the whole may be greater or less than the sum of the parts. This problem relates to investment in both corridors and regions, with no simple solutions since where both interdependencies and uncertainties exist, there is no longer a unique value for the expected benefit of any scheme. The main conclusions of the study were that interdependency benefits could be material, and programmatic analysis is complex, yet nevertheless, decision makers should have sight of a programmatic appraisal if the scheme they are making a decision about is materially related to other schemes. However, thus far the conclusions of this study appear to have been regarded as too difficult to implement.

The examples discussed above serve to illustrate the real problems in implementing the DfT’s Transport Analysis Guidance theories in the face of practical situations relating to real schemes. In particular, they show the way in which political imperatives can trump the analysis of economic welfare, with the latter leant upon to align with the former. There is also a narrowness of focus on individual investments, disregarding the implications for other developments facilitated by the transport improvement, as well as the interactions between a number of such investments along a route or in an area, and disregarding also what are now labelled ‘transformational’ outcomes. And lastly, there is the disregard of the implications for achieving Net Zero at both scheme level and for the whole programme of road investments, increased carbon emission from which are likely to be significant compared with the reductions from the switch to electric propulsion, now seemingly being banked upon to make the major impact needed to combat transport carbon emissions.

Whatever the shortcomings, the Welsh government has taken a much-needed fresh look at transport analysis guidance, to align this with wider policy objectives. It seems that the DfT cannot bring itself to do the same, preferring to continually add to the more than thousand pages of TAG documentation – like the latest batch just issued [ref 11] - without standing back and asking whether it still serves its intended purpose. Indeed, discussion of that ‘intended purpose’ seems strangely off the agenda. It is as though the established formula has achieved a protected status akin to Stonehenge.

References and Links

1. M. Beesley and C. Foster, The Victoria Line: Social Benefits and Finances. Journal of the Royal Statistical Society: Series A, 128(1), 67-88, 1965.

2. https://www.gov.wales/welsh-transport-appraisal-guidance-weltag

3. https://www.gov.uk/government/publications/integrated-rail-plan-for-the-north-and-the-midlands

4. https://nic.org.uk/studies-reports/rail-needs-assessment-for-the-midlands-and-the-north/

5. https://nic.org.uk/studies-reports/capturing-the-value-of-urban-transport-investments/

6. https://content.tfl.gov.uk/tfl-elizabeth-line-benefits-framework.pdf

7. https://tapas.network/49/goodwin.php

8. https://transportactionnetwork.org.uk/road-bulldozed-through-nationally-important-landscapes/

9. https://www.nao.org.uk/press-releases/south-west-road-improvements-and-the-stonehenge-tunnel/

10. https://assets.publishing.service.gov.uk/media/6163fd0ee90e07197718294c/programmatic-appraisal-stage-5-report.pdf

11. https://www.gov.uk/government/publications/tag-forthcoming-changes/forthcoming-change-modelling-guidance-updates