Peak Complexity – Why Things Stop Working, and how that Should Worry the Transport Sector

BE HONEST. How many times, over recent years, have you encountered a failure in the systems of a company or organisation, something which caused you loss, frustration or wasted time? Or impacted on the way you can achieve your professional objectives

Have you ever found yourself trapped in a loop, unable to fix a problem, or dealing with “customer services” who seemed equally powerless? Has it ever occurred to you that these growing frustrations of everyday life might be linked to deeper forces?

Until recently, I worked in the world of transport planning where behaviour change and decarbonisation often mean finding new alternatives to the private car. Many of those alternatives depend on automation, digital systems and increasingly AI. All this brings greater complexity, vulnerability and the risks of unintended consequences.

I was working on these observations when a “global IT meltdown” disrupted flights, trains and many other activities. Though the trigger may have been a fixable software glitch, the disruption was expected to outlast the software fix. Surely we should be urgently asking: how did we become so vulnerable- and are there not other ways of thinking about our societal needs and challenges

The link between these problems is peak complexity, the tendency of human civilisations to increase in complexity over time, until that process starts to create more problems than it solves. As those problems, and attempts to address them, over-burden a society it becomes more vulnerable to environmental, economic or military threats. It may eventually collapse, with the surviving population reverting to a simpler state.

I am not suggesting that our society is about to collapse for that reason. I am suggesting that the concept can help to explain several trends observable in the world around us. Where I refer to “systems” I mean both IT systems and procedures operated by people – the breakdowns often occur at the interface between those two.

Why Does Nothing Seem to Work Properly Any More?

Direct personal experience of system breakdowns prompted my interest in this. I have been unable to find much published evidence, an issue I will return to later. The best I could find was this survey by the UK Institute of Customer Service, showing that customer experience, across a wide range of industries, has been slowly declining for several years. The worst scores are in “complaint handling”.

Many of the problems I have encountered have occurred whilst travelling (I don’t include problems with public transport, which are nothing new.) When cycling across Europe in 2022 my credit cards and mobile phone were stolen. Replacing the phone was easy. Getting a new SIM card from my mobile phone supplier and a working credit card proved to be a nightmare. I eventually made formal complaints to the senior management of both companies, who paid me several hundred pounds in compensation.

Whilst travelling to France this year, I realised that I was simultaneously wrestling with system breakdowns in five different organisations – one public (the council handling my proxy vote) and four private. One of them was a ferry company. I had booked a return ticket with my wife but then needed to return separately. The rules allowed this, but the online booking system did not. I spoke to a customer service agent who said she would cancel the return portion of the ticket and allow me to book two singles. So far, so simple. Then on the day of the outward journey my phone rang. It was a customer service agent asking me where I was.

“Sitting inside your ferry” I replied.

I started the story all over again and the agent promised to amend their system. Two weeks later I received an email saying they were expecting me on board the return ferry with my wife!

Travel is not the only activity affected. Some breakdowns relate to essential services. The Department for Energy Security and Net Zero released figures last year showing that 2.7 million smart meters were not working properly. Ours was one of them. Our electricity supplier installed a smart meter, which cut off our night-time circuit, leaving us without heating or hot water for several weeks. They replaced it with another meter which also didn’t work. When I complained a second time, they refused to believe they had failed twice and told me the problem must be in our property. I had to hire my own electrician to disprove that claim. They eventually installed a third meter, which fixed the heating problem, although the display only works when it feels like it. They paid me compensation for the earlier problems, so I decided to live with the faulty display.

That incident, and the one on the ferry, illustrate a common theme. The customer-facing staff did not understand, and could not control, the systems they were using. Faced with these limitations the more helpful ones sometimes try “work around” solutions, which then create further problems. The organisations seem to be expending a lot of time and effort trying to resolve problems caused by their own systems.

To understand the root causes of all this, these articles and this podcast by Tim Maugham are illuminating. Maugham describes how he set out to follow supply chains “in reverse” from container ships to factories in China, to explore the systems on which our consumerist world depends. He describes how individuals working in those chains cannot see, and do not understand, the systems which control them. The same factors which make the system work, make it vulnerable and more difficult to fix when breakdown occurs. He also draws some political conclusions, which I will return to later.

What is Peak Complexity and What Causes It?

I am now going to make a small step from my SIM card and electric meter to the collapse of the Roman Empire. The phrase ‘peak complexity’ has been used in different ways. The concept explained at the start of this article, goes back to the Collapse of Complex Societies, written by archaeologist Joseph Tainter in 1988, although he never used that phrase. Tainter defines collapse as the reversion of complex societies to a simpler state. He believes that complex societies evolve in order to solve problems. In earlier civilisations the most pressing problem was how to feed a growing population. Adding complexity – by specialising labour, for example – initially produces great benefits. But as complexity increases, more effort and resources are needed to bring each new benefit. Each increase in complexity causes new problems requiring further resources. This is akin to the economic concept of diminishing marginal returns; it weakens those societies, making them more vulnerable to external shocks, such as climate change or barbarian invasions.

Where I use the term “peak complexity” I am referring to that process, not to the moment of the peak, which would be impossible to predict. There is no fixed maximum point of complexity. Overcomplexity is a contributor, not the sole cause of social collapse. So just to be clear, peak complexity means:

the diminishing marginal returns, and proliferating unintended consequences, of increasing social or technical complexity.

Tainter applied his theory to the collapse of ancient and prehistoric civilisations, including the Western Roman Empire. More recent writers have challenged aspects of his work; if you read Collapse by Jared Diamond and Understanding Collapse by Guy Middleton, you will begin to appreciate the uncertainties and disagreements in that field. Tainter’s was an inspired attempt to create a generalised theory and, like all such attempts, it can only provide a partial explanation. For our purposes, the key question is not whether he was right or wrong (he was partly right) but whether his model can help us to understand what is going on in our societies today.

In the final chapter Tainter attempts to apply his theory to “contemporary conditions”, which was probably the main interest of his readers. Although he makes no forecasts of imminent collapse, he points to trends which support this analysis of modern globalised societies. One of these trends is the declining marginal returns on innovation. Over time, inventions and new technology gradually solve the easier problems, usually by making things more complex. For example, think of all the technical and electronic systems grafted onto the motor car, since its invention in the Nineteenth Century. That leaves the more difficult problems, which become progressively harder to solve through innovation ‒ the challenge of decarbonising road transport, for example.

Tainter and others went on to research trends in innovation, using the database of US patents (many of which are submitted from overseas). They found that over time more and more “inventors” are needed to submit each patent. They conclude that increasing, or even maintaining, economic growth through technological progress would require an increasing share of resources for research and development, a process which cannot continue indefinitely.

Another point he could have mentioned is that more complex systems create more potential for unintended consequences. Minor changes in one part of a system may cause major impacts elsewhere. There is a whole area of study around complex systems, most of which is too complex to be of much use to ordinary people. One of the more accessible writers in this area, Sydney Dekker, has studied catastrophic failures, such as the global financial crash of 2008. His approach could equally apply to the global IT meltdown mentioned earlier. I am not aware of any similar analysis of smaller-scale breakdowns affecting everyday life.

Why do already-complex societies continue to complexify? In different ways, Tainter and Dekker suggest two common factors: problem-solving and competition, with neighbouring societies or commercial competitors. These factors make it difficult, if not impossible, for a single nation or a single company to voluntarily revert to a simpler state. Collapse only happens when all are affected together.

There is also a psychological factor which you may recognise if you have ever been a manager – or been managed by someone else. In Subtract and this article, Leidy Klotz describes a series of experiments where most participants faced with a problem, chose to add something rather than take something away, even when subtraction would have provided a better solution. Whether it’s innate or culturally-conditioned, most of us tend to overlook opportunities to remove things and to rate complex solutions more highly than simple ones. Academics are particularly guilty of that.

All of these books and articles have one thing in common:

their analyses of the problems are more convincing than their attempts to propose solutions. I will draw a few conclusions at the end of this article, but my main aim is to pose questions and raise awareness. These trends are too deeply embedded in the structure of our society to be solvable by a series of recommendations.

The Impacts of Growing Complexity

Attempting to predict when the moment of peak complexity might arrive would be a hazardous process – as the advocates of ‘peak oil’ have found in recent years. In this section I will make a more modest attempt, to apply Tainter’s model to the present-day, starting with those small-scale customer-facing problems.

The “cybersecurity arms race” has been responsible for some of these. As scammers find ways to bypass each new security measure, the IT security industry responds by adding new layers of complexity. Some of this takes place behind the scenes but new burdens, such as two- factor authentication, have been foisted onto individual customers. Good luck to anyone who loses a credit card and a phone at the same time. Despite those proliferating security measures, the cost of cyber fraud continues to rise.

The proliferation of apps required for each facet of everyday life is now producing impossible security requirements. These typically include a “strong password” avoiding commonly-used words, a unique password for each app or login, a series of PINs, memorable names or places and “security questions”. To complete this race towards absurdity, many companies warn people not to write any of this down. Perhaps a few gifted neurodiverse people have succeeded in complying with all those rules, but the rest of us will be forced to violate at least some of them. That creates a potential for victim-blaming, and self-blaming, discussed below.

The IT industry has responded to that problem with a further layer of complexity: password managers. If anyone believes those have solved the problem, then read this article. Another layer of complexity, responding to problems created by complexifying systems, is the intrusive “rate how we did” questionnaires, unfortunately aimed at the performance of junior staff rather than company systems.

If you have ever worked for a large organisation (or even some small ones) you will recognise the parallel problem of information overload in the workplace. It has also generated a large, and complex, body of research – this article gives a useful overview. As with the password problem, to survive and achieve some results, most employees are forced to cut corners, to overlook or skim- read what might be important messages, and to second- guess which orders or regulations they can quietly ignore. That also makes them vulnerable to victim-blaming and provides a strong incentive to keep quiet about the dilemmas they face (as I am now semi-retired I can happily say to all my previous employers - I broke many of your rules!)

Another complexifying response is to remove the human input altogether. One company which has taken this to the extreme is Google, which has no human customer service interface. I recently wrote to their CEO about a listing problem which cannot be fixed by the website owner, only by Google. I received an irrelevant reply which could have been written by a bot. One business, so frustrated by problems with Google’s systems, commissioned a series of paid advertisements begging someone in Google to take some responsibility for them.

At the national or international level, the growing burden of laws and regulations on businesses, and other organisations such as local authorities, has been well-documented. Attempts to “cut red tape” are controversial – they are often misused – and have rarely succeeded. What is the “right” type or level of regulation would take more than one article to address. I will simply note here that a growing regulatory burden, justified or not, is an example of diminishing marginal returns on complexity.

In Tainter’s study of the Western Roman Empire, a growing tax burden was a key factor in its ultimate demise. When the empire reached the limits of its profitable conquests it was forced to tax its own citizens more heavily, and debase its currency, to maintain its army and provide the public support its citizens had grown to expect. His description arguably over-stretches the definition of “complexity” – Tainter has been criticised for his fuzzy definitions – but the idea can be applied to present-day western societies.

The ratio of tax to national income in the UK is now higher than it has been since the 1960s, and yet many of our public services – particularly the health service – are in an appalling state. In France, the other country I know best, this report paints a similar picture. The Office for Budget Responsibility (see Chart B) shows tax ratios rising across the developed world, and the IMF’s datamapper shows an international trend of rising government debt. As each crisis pushes debt still higher, the ability of governments to deal with the next crisis is reduced. And the biggest crisis of all – climate change – is only just beginning.

Why are governments and public services becoming more expensive? It is difficult to find convincing answers, free from political or ideological point-scoring. The Institute for Fiscal Studies has pointed to a decline in “productivity” in UK public services, but that does not really explain very much. It simply means we are paying more for less. Whatever the reasons, this trend is consistent with Tainter’s observation that complex societies, which continue to complexify, will experience increasing pressures on government spending, leading to higher taxation and pressure for borrowing, weakening their ability to respond to future crises.

Disenchantment with politics has also been observed across the democratic world, accompanied by a rise in populist parties offering simple solutions. Tim Maugham’s study of supply chains offers one explanation for this. After observing how algorithms were controlling the work of everyone from factory workers to ships’ captains, he drew an analogy with elected politicians. We are no longer electing leaders, he says, we are electing “middle managers” who can’t really change very much. I wonder whether the faceless systems which increasingly control our everyday lives are also increasing frustration with conventional politics.

The spread of AI (which generated the illustration for this article) could amplify all of these trends. AI is designed to operate in ways which humans cannot understand or directly control. Some writers have raised the spectre of killer robots roaming the streets. Perhaps a more likely scenario is a world where no-one understands why things stop working, and no-one seems to know how to fix them.

Why Aren’t More People Talking About System Breakdowns?

Big breakdowns, like the “global IT meltdown”, are big news for a short time, but the only people talking publicly about the smaller-scale impacts on everyday life are comedians, like Mark Steele. The audience reaction to this sketch reassures me that my personal experience is not that unusual.

I started this article with an appeal to “be honest” because I am often surprised to meet people who don’t recognise what I am talking about. I wonder: are we living on the same planet? Sometimes, when asked to think back, people do then remember incidents which they did not consider noteworthy afterwards. Are you one of those people?

Another reason for the consensus of silence around system breakdowns is that people often suspect they might have inadvertently caused them. Experiments, such as this one, involving failures of IT systems, have found that users are more likely to blame themselves than operators or systems. This study confirms what I have observed that self-blaming is particularly prevalent amongst women.

Increasing complexity makes it more difficult and time- consuming to establish who or what did cause a system breakdown, as illustrated by the following incident. For several years I have helped to organise a group on Meetup.com, a platform which has spawned some wonderful groups. However, I discovered some time ago that some members of our group were not receiving emails or notifications of our meetings, which are the main reason for Meetup’s existence – and monthly fees.

When I raised the problem, as organiser, the agent replied that the query must come from an affected user (who might not be aware of the messages they are not receiving!) Fortunately, one of these people was my wife.

She has a stressful job and hates everything to do with IT. So I violated security protocols to log on as her and check all the settings on her account. There were lots of them, many related to marketing or data-harvesting. I could see nothing wrong there, nor could I find anything in her spam folder.

I then contacted Meetup pretending to be her. They eventually replied saying:

“after some investigation, we don’t see any issues with your account email settings. As per our Tech team's suggestion, changing your email address may fix the issue as Meetup works flawlessly with Gmail.”

Gmail is run by Google, whose flawless customer service I have already mentioned.

My wife is one of those people who often suspects that she might be to blame for IT problems. In this example – like many others – it was, and is, a fault somewhere in a complex system, which even the technical staff do not know how to fix. To discover that much took me several days and some subterfuge. If you were in my position, would you have bothered, or would you have thought: life’s too short, I’ll leave it for now?

Throughout my working life, working in different industries, I noticed that more of my time was being spent on trying to fix IT-related problems. In my last job, in a university, I spent endless hours on the phone to the IT helpdesk agents, who always tried to help, but didn’t always succeed. I would sometimes ask my colleagues if they had encountered the same problems. Their responses were often: no, I haven’t got round to that yet, or yes and I’ve put it to one side because I haven’t got time to deal with that on top of everything else! The work pressures which cause people to react in that way are partly caused by complexification.

The IT industry, unable, or unwilling, to fix the problems caused by their complexifying systems, often deflects frustrated users towards “our online community” i.e. other users. Some of these people can be very helpful. However, I have also noticed a reflex amongst some – nearly always men – to tell users that they must be causing the problem. This has become the digital equivalent of hunting prowess. Real men don’t complain about such trivial things; they view them as a challenge to combat. That message reinforces the culture of self- blaming and silence amongst everyone else.

Apart from those big global incidents, the mainstream media does talk about system breakdowns and over-complexity, where these occur in the public sector, particularly health services. But most of the breakdowns I have encountered, including those described above, have occurred in the private sector. Private companies exist to make a profit and it seems, in the case of Google for example, that providing a poor service may be more profitable than improving it. The pressure of competition may prove ineffective where a company exerts a quasi- monopoly or where the underlying problems are industry- wide or economy-wide. This is another reason for the culture of silence around these trends: to suggest that the private sector might be systematically failing runs counter to our prevailing political culture.

At a deeper level, peak complexity, particularly when related to digital technology, challenges some of the fundamental beliefs on which modern societies are based. Although naïve support for “progress” is heard less often these days, the belief that technology should advance as fast as it can, to create new opportunities and resolve human problems, remains as widespread and strong as ever. Of course, complexifying technological advances can and do address many real problems. That is entirely consistent with the peak complexity model. The unanswered questions concern the costs, benefits and unintended consequences of that process. Is that balance shifting in the wrong direction?

Does Any of This Matter? Can Anything be Done About It?

I believe that understanding the underlying mechanisms of our society is a challenge worth pursuing, even if we feel powerless to change them. I have referred to many sources of evidence on specific aspects of peak complexity, such as: customer dissatisfaction, the complexity of supply chains, catastrophic global failures, the psychology of complexity and diminishing marginal returns on innovation. In attempting to bring all this together and apply it to the frustrations of everyday life I have been forced to rely on personal experience because evidence at that micro-level is surprisingly sparse. Perhaps there is more out there, which I have not yet found. The evidence I have found has spanned many disciplines, in which I cannot claim specialist knowledge.

If you recognise the problems I have described in everyday life, then two things we can all do are: stop blaming ourselves and start talking about failing systems. Media of all kinds could do more to reflect people’s frustrations, and hold the worst-offending organisations to account.

As with many important issues affecting real life there is a research gap between the specific operational research of private industry, the deep but narrow aims of academia and the fashion-driven agendas of research funders. Falling between those stools, we could all benefit from learning:

• How prevalent are customer-facing system breakdowns and what are their impacts?

• Are they improving or getting worse?

• To what extent is increasing complexity to blame?

More specifically, companies and organisations need to know to what extent, and why, they are:

• Over-complicating their systems

• Building in vulnerability to breakdowns by relying on over-complex systems elsewhere

• Causing loss or frustration to their customers or users as a result and:

• Expecting junior staff to solve problems caused by systems they cannot control

How to fix those problems will obviously vary in each context. I will simply reiterate the question posed by Klotz: are you overlooking opportunities to remove things, instead of adding more complexity?

If anyone believes that customer-facing organisations are already doing those things, all I can say is: you’re not doing it very well.

The broader social questions raised by peak complexity present many opportunities for academic research across different disciplines, including my own field of transport planning.

Peak Complexity, Climate Change and Decarbonisation

Peak complexity is unlikely to cause the collapse of our civilisation. We are facing that threat for a different reason: climate change. That’s not my personal view, it is the view of climate scientists and the United Nations. Tainter has been criticised (by Diamond, amongst others) for downplaying environmental factors in the collapse of earlier civilisations. Whatever the truth of those claims, the scale and speed of climate change caused by humans would test the resilience of any society, however strong. If peak complexity is indeed at work across our societies, then it may already be weakening our ability to defend ourselves against climate breakdown. This book chapter by Chris Burr identified that problem, although its analysis is more convincing than its optimistic conclusions. I will conclude with just one observation and one example in response.

The shrinking timescale available to reduce greenhouse gas emissions has prompted most scientists and social scientists to conclude that we must act simultaneously on all fronts. That means rapidly adopting new technologies at the same time as reducing demand. However, nearly all the (inadequate) responses by governments and industry have relied on technological change. Simultaneous adoption of new technologies across entire economies is likely to encounter the usual problems associated with peak complexity: unintended consequences, cost overruns, delays and breakdowns.

Everyone involved in that process would benefit from reading Klotz’s book and asking themselves:

• Are you pushing unnecessarily complex solutions?

• Are you overlooking opportunities to simplify?

• Are you overlooking opportunities to subtract something instead of adding something?

Much of my research and writing about transport planning has concerned the challenge of decarbonisation. It’s clear to anyone who has seriously examined that challenge that we cannot achieve it without electrifying vehicles. That technological shift, and manufacture of vehicles in general, is causing negative environmental consequences, which have prompted some non-specialists to attack electric vehicles without proposing any serious alternatives. The only logical conclusion to that dilemma is that we must simultaneously electrify and reduce the number of vehicles.

The traditional focus of transport planning is to improve or multiply the alternatives to private car use in a vain attempt to persuade drivers not to use their cars. I have lost count of those alternatives over the years: mobility as a service, bus rapid transit, ultra-light rail, autonomous buses, electric scooters, demand-responsive public transport etc. Some of these have proved useful. Some of them, particularly around automation, are increasing complexity with all the risks and unintended consequences that entails. None of them have made any significant impact on the trajectory of carbon emissions.

Have we overlooked the opportunities to subtract something: the vehicles we could remove from our urban areas? How would we go about that? I won’t pretend to know all the answers, but what I wrote in Urban Transport Without the Hot Air back in 2015 remains relevant to that today.

The author thanks Glenn Lyons for comments on an earlier draft of this article.