TAPAS.network | 16 December 2020 | Commentary | Phil Goodwin
BACK IN JANUARY 2013, I started my column in the Local Transport Today magazine (LTT) with the words: “Transport ministers, from time to time, like to say that Britain has the ‘best transport appraisal system in the world’... quite a lot of people are not so convinced, suggesting that either the appraisal methods must be faulty or something must be wrong with the decision-making processes they inform.”
The context was one of those collective activities that happen every few years, reviewing weaknesses in the appraisal of big transport projects. They have a certain pattern, with a consultation exercise, a review, acceptance that maybe current practice is not quite meeting the ‘best in the world’ mark, and then after a year or so, revisions that leave some of critical failings not nearly as much improved, or implemented in practice, as had been hoped.
A recurrent thread has been a barely expressed undercurrent of suspicion that the formal rules of appraisal are a way of demonstrating a pre-determined result. This involves parameters, assumptions, models and rules that allow scope for ‘tweaking’ the results, so that even schemes, which seem quite out of key with the strategic objectives of an area reach their pass-mark and find their way into a programme.
Of course, such unease has to be expressed very carefully, as no-one would want to suggest that their colleagues are in any way ‘cheating’. However, I can recall chats in the bar after the evening session of transport conferences (remember them?) where consultants have spoken about ‘pressures’ that can sometimes influence the difference between the first and final versions of reports. The commitment to professionalism, and alert, informed public scrutiny, are necessary antidotes to such pressures.
But now something quite profound seems to be happening. Last month the Treasury published the results of its review of the Green Book, the generic rules of appraisal on which the DfT’s TAG, and other similar guidelines, are built. It is remarkably open in its critique of appraisal in practice. One of the fundamental issues the review has identified is the common failure of those writing appraisals to engage properly with the strategic context in which their proposal sits.
Specifically, business cases frequently do not demonstrate the necessary understanding of:
the proposal’s specific contribution to the delivery of the government’s intended strategic goals (such as levelling up or net zero); and
the specific social and economic features of different places and how the intervention may affect them;
other strategies, programmes or projects with which the intervention may interact, including in a particular geographical area.
The Treasury stated: “While the BCR is a useful metric for capturing quantifiable costs and benefits, there is a tendency to place an inappropriate emphasis on it, in a way that frames value for money as an absolute concept.... Considerable time and effort is expended to ‘boost’ the BCR that would have been better spent developing and testing the other elements of the business case including its strategic coherence, risk management and the implications of significant unquantifiable factors.”
And lest there should be any doubt about the implications of this phrase “effort to ‘boost’ the BCR”, the Treasury seems to point a finger at both the clients and consultants themselves, who ‘may have been tasked with producing a high BCR rather than a properly-rounded appraisal’.
Effort expended to ‘boost’ the BCR would have been better spent developing and testing the other elements of the business case including its strategic coherence, risk management and the implications of significant unquantifiable factors
A key question is, what would transport professionals actually think of this criticism? Well, you’d expect that local government officials, national agencies and consultants would not take kindly to even a hint that they had not been applying proper professional standards to their work. So, it’s very important to see Keith Buchan’s letter in the 16 December 2020 issue of LTT magazine (see panel below) the first professional response to the Treasury criticisms. He reports that a group of four important professional bodies – TPS, CIHT, LGTAG and RTPI - had been in discussion with the Treasury and DfT, and made their own submission for change. Between them, their memberships must include a great majority of professionals in transport planning. Keith Buchan, writing for the Transport Planning Society, fully supports the findings of the Treasury team, which seem to have taken the combined Institutes’ submissions very seriously. “The Treasury findings are clear in terms of the current system’s poor theoretical justification, lack of transparency, and poor outcomes in terms of failing to bring forward schemes which would achieve our objectives”.
This view is supported by others I have talked to. They do not see this as a Government attack on their professionalism, but as recognition of failings that must be corrected.
The context of these findings is clearly a renewed attention to strategic imperatives, especially carbon targets whose increasing importance is recognised in statements from the Committee on Climate Change and the National Audit Office. The House of Commons Transport Committee announced a new inquiry on ‘Major transport infrastructure projects: appraisal and delivery’, which includes considering the implications of the coronavirus pandemic and decarbonisation objectives for transport infrastructure, which I trust many readers will respond to. And although reading the Prime Minister’s real and sustained intentions is a notoriously subtle skill, his recent announcements on the need for seriously intensifying decarbonisation has been widely welcomed.
Of course, this attention to the rules of transport appraisal has a history. I have kept a running commentary on earlier revisions in these columns, which provides a swift reminder of the sorts of issues that were being discussed at each stage. A few readers will remember, for example, the consultation exercise in 2008 to ‘Refresh NATA’ (as the appraisal rules were called at the time, later becoming Webtag, and now TAG).
In a column in LTT magazine (LTT April 2008) I summarised 20 recommendations for change, which had been submitted to DfT by the Centre for Transport and Society in UWE, where I was then employed.
In some ways it is rather depressing now about how little from those recommendations was actually implemented. It’s true there have been significant improvements in the preambles and general statements of principle, recognition of uncertainty in demand forecasts (but not carried through to the appraisal process which would make a difference). There has been a little progress in recommendations on four of the 20 suggestions, though not in the terms proposed, but hardly any change in the actual outcomes of large project appraisals.
But I am at least comforted by seeing that both the language and content of those 2008 suggestions do not feel at all dated. Carbon, health, employment and equity are all emphasised. Mostly they would fit with hardly a change in the current very welcome ferment about appraisal. Consider, for example, the first two in my list of 20 back in 2008:
“ 1. NATA rightly already advocates a broad multi-modal approach including demand management. But in practice it is too often confined to narrower calculation of benefits of road projects providing small time savings or increased car travel.
2. NATA is rather weak on demand management; walking; public transport quality; cycling; land use planning (especially favouring settlement patterns which reduce car dependence); pricing systems reflecting full external costs; smarter choices; the redefinition of the styles of street management including traffic calming and the reallocation of scarce road capacity; better information; recognising reliability and variability, both in conditions and choices.
Yet these are precisely the most critically important issues for sustainable transport systems.”
The only thing I would change now is an acknowledgement that we have to be careful what we wish for. Even the best ideas can turn out to be implemented in a way that undermines, rather than meets, the best intentions. ‘Greater attention to strategic coherence’ could be an excuse for wordy claims unbacked by evidence. It could see the volumes of engineers’ calculations and modelling replaced by shorter, glossier text worded by PR agencies and management consultants, which defy any attempt at challenge, evidential testing or clarification. That’s a danger. But the Treasury findings are, I think, right, and open the way to enabling appraisal to favour a better mix of policies and projects.
Phil Goodwin is Emeritus Professor of Transport Policy at UCL and UWE, and Senior Fellow of the Foundation for Integrated Transport.
This article was first published in LTT magazine, LTT812, 16 December 2020.
Professor Phil Goodwin refers in this article to Keith Buchan’s letter in the 16 December 2020 issue of LTT magazine. This letter is reproduced here with permission from LTT magazine.
Appraisals will still use BCR despite revised Green Book
The last issue reported on the findings of the Treasury review of the strengths and weaknesses of current appraisal across the Government (‘New Green Book emphasises good strategic case over BCR’ LTT 27 Nov). Since the review began, TPS has been working together with LGTAG, CIHT and RTPI to provide detailed input to assist in this process.
In an honest and frank response to an avalanche of criticism from practitioners, the Treasury team does not pull any punches. For example: “One of the fundamental issues that the review has identified is the common failure of those writing appraisals to engage properly with the strategic context in which their proposal sits.”
Levelling up and zero carbon are cited as key objectives being overlooked. This sits alongside other failures, especially the distortions caused by a fixation with a scheme’s Benefit to Cost Ratio – the infamous BCR – and how many factors are excluded from its calculation. Overall, the criticisms are profound and ones that all the relevant professional bodies including TPS have long supported. The promises made for reforms are wide ranging and positive, especially a new emphasis on the Strategic Case, and the importance of carbon, place making and equalities.
The key question now is how this will be reflected in practice and here’s where Déjà vu comes in. In 1997/98 remarkably similar reforms were agreed, for very similar reasons, but the old system was retained to run in parallel. There was no sign up to the reforms from the DfT, and within a short space of time the BCR once again ruled the roost, with all the misdirection of investment, and consequent condemnation by practitioners and public alike, which has been the hallmark of the current system.
This time the initial signs are good; a far greater openness from DfT and Treasury, the understanding that skills and training will be essential, and the clarity and substance of the Treasury report which is of itself remarkable and welcome. Against this must be set the fact that the new Green Book itself (the actual guidance) is not as clear and to the point, and seems to fudge issues such as the relationship between cost benefit and multi-criteria (objectives led) analysis. The key to retaining clarity and strength of purpose will be the changes to transport’s appraisal bible: (Web)TAG.
It is here that the role of the Strategic Case in appraisal must be re-emphasised and re-invigorated. It is the gatekeeper and as the Treasury report says, “only options with a strong strategic case should be short listed for detailed cost benefit analysis”. But it still envisages a BCR being produced, despite the powerful criticisms, apparently accepted, that many key factors are non-monetised and completely missing from the BCR. Some of these are things that matter most to real people in the real world.
The crucial test for these positive reforms in transport will be whether TAG moves fast enough to reflect them, and DfT ensures that the new approach is being followed across the piece. A second failure to implement reform would be unforgiveable.
Keith Buchan
London SW1P
You are currently viewing this page as TAPAS Taster user.
To read and make comments on this article you need to register for free as TAPAS Select user and log in.
Log in