New Green Book paves way for shake up of road and transport appraisals

BACK IN JANUARY 2013, I started my column in the Local Transport Today magazine (LTT) with the words: “Transport ministers, from time to time, like to say that Britain has the ‘best transport appraisal system in the world’... quite a lot of people are not so convinced, suggesting that either the appraisal methods must be faulty or something must be wrong with the decision-making processes they inform.”

The context was one of those collective activities that happen every few years, reviewing weaknesses in the appraisal of big transport projects. They have a certain pattern, with a consultation exercise, a review, acceptance that maybe current practice is not quite meeting the ‘best in the world’ mark, and then after a year or so, revisions that leave some of critical failings not nearly as much improved, or implemented in practice, as had been hoped.

A recurrent thread has been a barely expressed undercurrent of suspicion that the formal rules of appraisal are a way of demonstrating a pre-determined result. This involves parameters, assumptions, models and rules that allow scope for ‘tweaking’ the results, so that even schemes, which seem quite out of key with the strategic objectives of an area reach their pass-mark and find their way into a programme.

Of course, such unease has to be expressed very carefully, as no-one would want to suggest that their colleagues are in any way ‘cheating’. However, I can recall chats in the bar after the evening session of transport conferences (remember them?) where consultants have spoken about ‘pressures’ that can sometimes influence the difference between the first and final versions of reports. The commitment to professionalism, and alert, informed public scrutiny, are necessary antidotes to such pressures.

But now something quite profound seems to be happening. Last month the Treasury published the results of its review of the Green Book, the generic rules of appraisal on which the DfT’s TAG, and other similar guidelines, are built. It is remarkably open in its critique of appraisal in practice. One of the fundamental issues the review has identified is the common failure of those writing appraisals to engage properly with the strategic context in which their proposal sits.

Specifically, business cases frequently do not demonstrate the necessary understanding of:

• the proposal’s specific contribution to the delivery of the government’s intended strategic goals (such as levelling up or net zero); and

• the specific social and economic features of different places and how the intervention may affect them;

• other strategies, programmes or projects with which the intervention may interact, including in a particular geographical area.

The Treasury stated: “While the BCR is a useful metric for capturing quantifiable costs and benefits, there is a tendency to place an inappropriate emphasis on it, in a way that frames value for money as an absolute concept.... Considerable time and effort is expended to ‘boost’ the BCR that would have been better spent developing and testing the other elements of the business case including its strategic coherence, risk management and the implications of significant unquantifiable factors.”

And lest there should be any doubt about the implications of this phrase “effort to ‘boost’ the BCR”, the Treasury seems to point a finger at both the clients and consultants themselves, who ‘may have been tasked with producing a high BCR rather than a properly-rounded appraisal’.

Effort expended to ‘boost’ the BCR would have been better spent developing and testing the other elements of the business case including its strategic coherence, risk management and the implications of significant unquantifiable factors

A key question is, what would transport professionals actually think of this criticism? Well, you’d expect that local government officials, national agencies and consultants would not take kindly to even a hint that they had not been applying proper professional standards to their work. So, it’s very important to see Keith Buchan’s letter in the 16 December 2020 issue of LTT magazine (see panel below) the first professional response to the Treasury criticisms. He reports that a group of four important professional bodies – TPS, CIHT, LGTAG and RTPI - had been in discussion with the Treasury and DfT, and made their own submission for change. Between them, their memberships must include a great majority of professionals in transport planning. Keith Buchan, writing for the Transport Planning Society, fully supports the findings of the Treasury team, which seem to have taken the combined Institutes’ submissions very seriously. “The Treasury findings are clear in terms of the current system’s poor theoretical justification, lack of transparency, and poor outcomes in terms of failing to bring forward schemes which would achieve our objectives”.

This view is supported by others I have talked to. They do not see this as a Government attack on their professionalism, but as recognition of failings that must be corrected.

The context of these findings is clearly a renewed attention to strategic imperatives, especially carbon targets whose increasing importance is recognised in statements from the Committee on Climate Change and the National Audit Office. The House of Commons Transport Committee announced a new inquiry on ‘Major transport infrastructure projects: appraisal and delivery’, which includes considering the implications of the coronavirus pandemic and decarbonisation objectives for transport infrastructure, which I trust many readers will respond to. And although reading the Prime Minister’s real and sustained intentions is a notoriously subtle skill, his recent announcements on the need for seriously intensifying decarbonisation has been widely welcomed.

Of course, this attention to the rules of transport appraisal has a history. I have kept a running commentary on earlier revisions in these columns, which provides a swift reminder of the sorts of issues that were being discussed at each stage. A few readers will remember, for example, the consultation exercise in 2008 to ‘Refresh NATA’ (as the appraisal rules were called at the time, later becoming Webtag, and now TAG).

In a column in LTT magazine (LTT April 2008) I summarised 20 recommendations for change, which had been submitted to DfT by the Centre for Transport and Society in UWE, where I was then employed.

In some ways it is rather depressing now about how little from those recommendations was actually implemented. It’s true there have been significant improvements in the preambles and general statements of principle, recognition of uncertainty in demand forecasts (but not carried through to the appraisal process which would make a difference). There has been a little progress in recommendations on four of the 20 suggestions, though not in the terms proposed, but hardly any change in the actual outcomes of large project appraisals.

But I am at least comforted by seeing that both the language and content of those 2008 suggestions do not feel at all dated. Carbon, health, employment and equity are all emphasised. Mostly they would fit with hardly a change in the current very welcome ferment about appraisal. Consider, for example, the first two in my list of 20 back in 2008:

The only thing I would change now is an acknowledgement that we have to be careful what we wish for. Even the best ideas can turn out to be implemented in a way that undermines, rather than meets, the best intentions. ‘Greater attention to strategic coherence’ could be an excuse for wordy claims unbacked by evidence. It could see the volumes of engineers’ calculations and modelling replaced by shorter, glossier text worded by PR agencies and management consultants, which defy any attempt at challenge, evidential testing or clarification. That’s a danger. But the Treasury findings are, I think, right, and open the way to enabling appraisal to favour a better mix of policies and projects.

Phil Goodwin is Emeritus Professor of Transport Policy at UCL and UWE, and Senior Fellow of the Foundation for Integrated Transport.

This article was first published in LTT magazine, LTT812, 16 December 2020.