Some Edinburgh City councillors already privately refer to the city’s tram project as the problem that “cannot be named”. Much as actors refer to Shakespeare’s tragedy as “the Scottish play”, superstitions of bad luck now bedevil the production. A dramatic shift from the optimism that initially characterised the development of the Edinburgh tram, towards pessimism.

That which cannot be named is no longer just the failure of a flagship local transport policy. The issue has engulfed the City of Edinburgh Council, and now risks destroying local politics completely: Not only the existing administration, but public trust in local government decision-making.

Political heavy-weights, who normally shy away from the minutiae of local governance, are now offering parental guidance in public: Alistair Darling (local Member of Parliament, and former United Kingdom Chancellor and Secretary of State for Transport) described the option to borrow £231 million ($370 million) to complete the city centre section of the tram line as “absolute madness” – the local population would be saddled with vast debts. Days later, Graham Birse (chief executive of the influential Edinburgh Chamber of Commerce) called the decision to not complete the city centre section, “bonkers” – far fewer passengers would use a tram that did not serve the city centre adequately. Even Alex Salmond (Scotland’s First Minister) has become directly embroiled, struggling to contain calls for an immediate public inquiry to identify who is responsible.

Burn the witches! This Scottish tragedy is rapidly descending into farce. That would be unfortunate, because this particular local difficulty goes to the heart of the Scottish nationalist agenda: A desire for greater devolution of public funds to local level. More localised independent entities have fewer financial resources, so are less able to manage expensive, risky projects. Consequently policy ambitions also need to be scaled back. Such scale isn’t necessarily a problem – small can be beautiful. The problem lies in pretending to be big, when not.

This article introduces the concept of risk in tram (and similarly large public transportation and infrastructure) projects, chronicles the decisions that lead a relatively small local authority to need to find hundreds of millions of pounds to support a single project, and explores the implications for future policy-making, especially in the context of a more devolved Scotland. On this page:

• Tram Cost Uncertainty
• Devolution of Chaos
• Scaling Ambition
• Appendix: Policy History of Edinburgh Trams

Tram Cost Uncertainty

I have previously discussed why estimations of the cost of large transportation infrastructure projects (especially light rail) are both inherently inaccurate, and tend towards optimism. Edinburgh Trams transpire to be an extreme example: From £375 million estimated for the original 3-line network, via £545 million for a more pragmatic 2-line network, to over £1 billion for today’s single line (which is still far from completion).

However, this uncertainty was somewhat predictable: The graph below is taken from Bent Flyvbjerg’s 2004 Procedures for Dealing with Optimism Bias in Transport Planning, guidance issued to central government alongside HM Treasury’s Green Book. It shows the probability that the cost of rail projects (including trams) will exceed the estimated budget.

Distribution of Rail Cost Overruns (Bent Flyvbjerg, 2004)

(Data is based on analysis of 46 rail projects from across Europe and North America – UK has few recent rail schemes, however comparison of road schemes suggests patterns are very similar for all regions.)

The average cost overrun is about 40%. In the planning/appraisal process 40% is literally added to the estimated cost of a project as “optimism bias”. Optimism bias is part of Scottish Transport Appraisal Guidance, to which the Edinburgh Tram was subjected, although the original tram analysis pre-dates formalised optimism bias.

The 40% value should protect the UK treasury:

• Over many separate projects, cost overruns will average to zero. Broadly, in the long-term, the treasury will remain balanced, without requiring individual projects to be micro-managed from the top of government.
• Individual project costs are in the hundreds of millions (£). These are still a tiny proportion of UK Gross Domestic Product, taxes collected, or ability for the UK government to take loans. The national economy will not be thrown into a crisis if one specific project goes badly wrong.

Unfortunately, merely adding 40% adds a sense of certainty to an individual project which remains uncertain. For the immediate funders of a project, the important question is, can we fund a cost overrun of 80%? For the City of Edinburgh Council, the answer to that question was effectively, “we cannot”.

A quick glance at the council’s budget puts the tram in perspective: The council’s total annual capital budget (for investment in everything) is only £235 million, less than a quarter of the capital cost of one tram line. Typically transport accounts for 20% of local government capital expenditure, so we might expect a local authority like Edinburgh to be investing around £50 million each year in transport projects. Placing over 10 or 20 years worth of investment into just one project suggests a tram scheme was far too ambitious to ever be a local government responsibility.

Not only do councillors appear to be losing a high-stakes casino game, but they seem to be playing with all theirs chips on the table.

However, the tragedy is not so simple, because while the City of Edinburgh Council are responsible for the tram project, they are not the only funder:

Devolution of Chaos

Conventionally in Britain, higher tiers of government act as financial guarantors for low tiers. A local authority has statutory (in law) responsibilities, and hence cannot “go bankrupt”, however badly it manages its budget. This hierarchical structure is not accidental: It is rooted in currency (money itself), for which central government is solely responsible. In practice central government imposes strict financial controls on local government, which limit the scope for mis-management. Historically important cities, such as Edinburgh, also tend to own a lot of local assets (like property), which can be sold in a crisis.

1990s Scottish devolution did not devolve money, it just altered the hierarchy slightly, with an extra decision-making tier (the Scottish Parliament and their civil service, the Scottish Executive) in between Edinburgh’s council and the United Kingdom’s central government.

In addition to keeping a watchful eye on how the council manages its budgets, the Scottish Parliament is part-funding the Edinburgh Tram – providing extra money beyond normal budgets or spending approvals. The Scottish Parliament’s contribution (currently via Transport Scotland) is limited to £500 million [PDF]. And herein lies the problem: When that funding package was agreed, the local council’s contribution was the remainder, just £45 million, a value broadly attainable by the local authority. But since Scotland’s contribution is fixed, every time the estimated price rises, the City of Edinburgh Council’s contribution rises out of proportion: Without other sources of funding, a doubling of project cost to £1 billion is actually a ten-folded increase in the cost to the council…

By attempting limit the Scottish Parliament’s exposure to the tram project, all the risk has been transferred down the hierarchy, towards the tier of government least able to raise large amounts of cash in a crisis.

Brinkmanship of the worst kind? In the interim, the council’s response is to control escalating costs by reducing the length of the route to be completed, with apparent disregard for whether the resulting tram track connects places large numbers of passengers might want to travel between. Underpinning their decision appears to be one of the worst assumptions of “modern” accountancy: That the asset value of a (tram) service is defined by the cost of constructing the infrastructure on which that service operates. Strategically, the value of the (tram) service to the city of Edinburgh is linked to the number of passengers that use it. Cynically, to the number that see it. Even commercial assets that are technically transferable (like land and tram vehicles) tend to sell for a lot less than they were bought. Scottish government’s reaction to the council’s decision was to withhold part of Scotland’s funding contribution, effectively forcing the council to reach a different decision [the following Friday, 2 September, the city centre section was added again].

With Scottish national government substantially funding the Edinburgh tram, and the project appearing too risky for local government, perhaps the Scottish level of government should have conceived and manage the project from the outset? The appendix at the bottom, Policy History of Edinburgh Trams, will help explain.

Haymarket Tram Terminus? Existing bus to Edinburgh Airport, with new tram station “under construction” on the left (August 2011)

Scaling Ambition

Broadly, there are 2 viable methods of containing risks on the scale of the Edinburgh tram:

• As a Scotland-level public project: Even £1 billion is only around 1% of Scotland’s Gross Domestic Product, a magnitude of (over-) spending that should be manageable by a truly devolved Scottish government. The Scottish Parliament has subsequently established an organisation (theoretically) capable of managing transport projects at this level, Transport Scotland.
• As a primarily private project: Most modern tram systems built in England transferred risk to the private sector (rather than Edinburgh’s model of merely contracting private sector suppliers). Scotland’s past record on private sector transport projects is poor (most obviously on the Skye Bridge), because policy changes too fast.

Unfortunately neither of these methods complement good local public governance: Decision-making and responsibility becomes remote from local politicians. Ultimately, this would dilute the role of local authorities: Logically to organisations that deliver (statutory) local services and act as “community council” talking shops, with no ability to actively shape or invest in the long-term future of their own areas.

That may be where Scotland is heading: Far greater centralisation of powers at Holyrood (the Scottish Parliament), drawn simultaneously from both Westminster (UK government) and Scottish local authorities. Such a structure might even work, and would address one of the major post-devolution issues – “too much democracy”.

However, the most important lesson of the Edinburgh tram is not that such projects should be managed more centrally. Rather, that capital investments should be scaled to match the scale of the government trying to deliver them: The tram’s fundamental failing is that it is too expensive (and consequently risky) a mode for a local public transport project. If an inherently local policy can only be enacted by a non-local government, then (I argue) it is the wrong policy. Local policy ambitions need to be scaled down to what is reliably achievable at local level.

The issue will become critical for a future Scottish government with financial independence from the rest of the United Kingdom (a distinct possibility, given the current dominance of the Scottish National Party). Scotland represents approximately 10% of the UK population and economy. This implies that a truly independent Scottish government could only safely manage 1/10 the amount of risk that the UK government is capable of managing. Meaning, the most ambitious project an independent Scotland could achieve would be 90% less ambitious than a UK-backed project.

Scotland’s current transport projects are still reasonably modest, like reopening short sections of old railway line. But it is easy to imagine the Scottish Parliament’s ever-increasing ambition proposing grandiose projects that the UK might contemplate, but which could bankrupt Scotland – like a new high speed rail network: A core route serving Scotland’s largest 5 or 6 cities, with an under-sea tunnel to Lerwick in phase 2…

The challenge for Scotland is not to pretend to be big, to still be part of the British Empire, or a modern-day China. Quite the opposite: To deliver the same (or better) policy impact, while the maximum cost of individual projects is limited to a tenth what the population has learnt to expect. Overall expenditure would not be reduced – there would simply be far more, smaller, projects. Fortunately, there’s a lot more scope to improve the humble bus stop than you might think!

I suspect Scotland’s history already contains the best example of what can happen when a nation stops pretending to be big, and instead focuses on itself: The original Acts of Union (with England) removed colonial and military distractions, allowing Scotland to develop domestically as an intellectual and commercial 18th century power-house.

The management of chaos (of which financial risk is a part), may become one of the great challenges for structured 21st century government, regardless of size. Paradoxically, the easiest way to manage chaos is not to have to: To foster a system where no one part is too important to fail.

Appendix: Policy History of Edinburgh Trams

The idea for the Edinburgh tram was originally developed by the Scottish Office (the pre-devolution civil service responsible for administering Scotland), as part of a package of transport measures for south-east Scotland, to be funded by road pricing (tolls on car journeys within Edinburgh).

3 conditions made Edinburgh a perfect target for such ambitious transport policy initiatives:

1. Transport was (is) a constant source of annoyance for many city residents – a “political hot topic”.
2. Edinburgh’s geography – densely populated, but hilly – historically lead to above-average local public transport use, with bus travel remaining socially acceptable among Edinburgh’s “middle classes”.
3. Minimal democratic interference (the Thatcher/Major governments contained almost no Scottish Members of Parliament) allowed Scotland’s civil servants to focus.

The first Blairite UK government (1994) heralded both devolution in Scotland, and a shift in transport policy towards public transport. That combination should have been perfect for Edinburgh, except that the Scottish Executive (the devolved equivalent of the Scottish Office) became considerably more focused on managing opinions. Which in transport, has a tendency to result in nothing…

Instead, policy momentum was picked up by the recently liberated (from Lothian Regional Council during local government reorganisation) City of Edinburgh Council: Local ambition steadily grew, from “Greenways” (networks of bus lanes) in the early 1990s, through guided busways and radical parking policies, to strategies based around trams and road charging.

Charging proved too radical: Scottish government compelled Edinburgh to hold a referendum – but only on road pricing, not the trams that pricing had originally been intended to fund. Edinburgh’s population naturally rejected road pricing in isolation – Turkeys don’t vote for Christmas – and the tram project proceeded with traditional forms of funding.

In spite of (initially) primarily being funded by Scotland, the trams were local in scope (with no tangible benefit to anyone outside Edinburgh), and hence construction was “managed” by the council. This was done through Transport Initiatives Edinburgh, a council-owned company established to deliver Edinburgh’s light rail scheme – something which it has since struggled to achieve.

Do you like the sound of £100 million ($150 million)? Can I tempt you with £160 billion? Expressing these figures per person in the population can be useful. The first figure is one bar of luxury chocolate for everyone. Doesn’t sound so big now, does it? The second figure is like everyone having a £2,500 bank overdraft (loan). Strange that, because indirectly, we do.

Unfortunately, applying the economics of household groceries to major items of government expenditure introduces certainty. The idea that one can visit a store where luxury chocolate bars are sold for precisely £1.70. Yet many large elements of government expenditure are akin to ordering a chocolate bar years before it can be eaten, for a price that transpires to be somewhere between £1 and £5.

Larger businesses will be familiar with this concept. It’s called risk. Such businesses are often far more interested in what “it might cost” (£5) than what “it will cost” (£1.70), because what it might cost might lead the business to bankruptcy.

The national economy is chaotic in its complexity, but overall, things should average out. So long as all the assumptions are broadly reasonable: Ultimately some will earn/cost more, some less. Short-term in-balance can be solved by (basically) printing more money, and then down-grading future assumptions until everything is back in balance.

However, this breeds a form of arrogance. A sense that government doesn’t need to consider the possibilities. That we can deliver a radical new policy – that has never been done before – and, in spite of it never having been done before, we know precisely how much it is going to cost. Just like a bar of chocolate.

Unfortunately, assumptions tend towards optimism. On average, projected costs are less than actual costs. This isn’t just a problem for accountants. It means that decisions are taken which do not reflect reality. Potentially leading to a Disneyland scenario, where everything is affordable until after the decision is taken, when suddenly everything has become too expensive. It ultimately challenges the validity of decisions, and in doing so, the moral authority of those that take them.

This article uses the Edinburgh Tram project to demonstrate the inherent uncertainty of large government infrastructure projects. It discusses the role of optimism in planning, and the methods used to reconcile planned optimism with subsequent reality. The article describes how the involvement of the private sector in public projects has evolved over the last 20 years, and the highlights the different time-scales applied to private investment and public choices. It concludes that optimism is not only unavoidable, but necessary. Rather, the true problem lies in tendency of people to demand certainty from the public sector, while accepting uncertainty in the private sector. On this page:

• Digging Holes
• Optimism Bias
• The Downside of Up
• Evolution of Public-Private
• Paradoxes of Uncertainty

Digging Holes

Edinburgh’s Haymarket used to be one the busiest transport junctions in the city. Used to, because for the last 2 years, the highway has played host to a unusual urban art form: Holes.

The exact location of these holes changes from month to month. An elaborate game to confuse pedestrians and drivers, as they fail to navigate the shortest path to their destination. Like the best urban art, the artist is rarely seen – although the presence of various pieces of mechanical digging equipment indicate that the work is somehow still incomplete. Yet the most intriguing part of this sculpture is it’s name, “Edinburgh Trams“. Because in spite of the preponderance of construction activity in the locality, the only track to be seen at Haymarket is the same mainline railway that has run under the road junction for the last 170 years. No new stations, no over-head power cables, and certainly no trams.

“Light rail” requires slightly heavier engineering than road traffic. So laying tram tracks often involves rebuilding almost everything underneath the road surface before the rails can be laid. In a large British city, this means modernising about 200 years worth of poorly (that is, not) planned subterranean infrastructure. A maze of pipes carrying all those important things that normally remain unseen – water, power, communications.

That doesn’t just cause the appearance of a lot of track-free holes. It means that we aren’t entirely sure what’s down there until we start digging. This is one example of what makes the construction planning process especially uncertain for trams: How many holes should we plan to dig, and, hence expect to pay for?

Now that most of the utility work is complete, we know that almost 50 kilometres of utilities were diverted, almost double the 27km originally estimated. Estimation which appears to be fairly typical of the way the rest of the project is progressing:

The Edinburgh tram project was originally conceived as part of a package of transport measures, funded by a new road pricing scheme. The road pricing was abandoned by popular vote, but the tram scheme proceeded anyway.

Before construction had started, a perceived shortage of funds reduced the 3 proposed tram lines (and some extensions), to 2 core routes. Yet by the time the first hole had been dug, the estimated cost of this (smaller) network had already risen above £500 million. That’s a thousand pounds for every city resident – including those that don’t have £1000. By the start of 2010, the (generally accepted) estimated cost had risen to £600 million, with some private estimates closer to £1 billion.

This in a period when (money) inflation has been almost stationary, especially in an otherwise recession-hit construction industry. And all the while, the completion date continues to slip backwards – curiously, always 3 years from now. The Edinburgh Evening News provides a timeline.

This inherent uncertainty doesn’t just make a mockery of announcing something “will cost £375 million”, when the only thing we can be reasonably sure of, is that it won’t cost precisely that much. It creates scope for optimism in planning: To bias all the cost assumptions down. Nobody can convincingly dispute an estimate, because nobody has accurate figures about the cost of building trams through Edinburgh. Estimates are biased down because securing popular (specifically political) approval for a project is easiest when the cost is lowest. Naturally, once a proposal has political support, there is considerable reluctance to abandon it. So everyone keeps digging.

Optimism Bias

The tendency for costs to be under-estimated has been apparent for several decades. Indeed, it has been formalised in the planning process as “optimism bias“. Literally, add n% to the projected cost, because, on average, past infrastructure projects have cost n% more to finish than originally estimated.

For light rail schemes, n is at least 40%. That’s quite a big error for a sector that claims to employ professional people for forecasting and planning.

Of course, adding n% still doesn’t solve the problem: We merely proceed to make even more optimistic assumptions. Claim that that much optimism bias shouldn’t apply to us. Like a gambler, convinced that their luck is somehow above average. Yet (unfortunately) secure in the knowledge that British government doesn’t “go bankrupt”: However bad the bet, those gambling debts will always be funded from somewhere. And you’d be amazed how few people working in government can explain where somewhere actually is.

This isn’t a grand conspiracy. It’s the same kind of optimism that concludes parents to believe their children are somehow “better” than they actually are at school. It’s inbred in planners and politicians alike: They’re fundamentally the kind of people that want to affect change. To get things done. They have a natural disinclination to believe proposals are unachievable.

Extend the planning horizon over (often) decades, and, frankly, almost anything can be made plausible.

Consider an operational, rather than construction example: Optimism fills our new trams with people that were previously causing road traffic congestion (policy objective), and who happily pay a premium over bus fares (operational objective). Yet pessimism would fill our new trams with former bus passengers (minimal policy impact), and who are paying bus fares (unfortunately trams have higher operating costs than buses). Only a couple of inherently uncertain assumptions separate these 2 scenarios. Separate success from failure.

The Downside of Up

There are several methods of managing the financial crisis that (invariably) occurs once reality challenges optimism.

For example, we can always do less. Build 1 tram line now, and another one “later”, maybe. Commission a single aircraft carrier, rather than a pair. It’s a terrible fix:

• 1 tram line clearly delivers less benefit than 2 lines. If the prime objective is perception, not transport (as I contend), then running trams past the Royal Bank of Scotland headquarters and on through the city centre, may be sufficient. But it is not always so easy to scale back. A single aircraft carrier reduces the solution to farce: If one wishes to maintain an active sea-based air capability, then you can’t do it with one ship: Wars rarely adhere to maintenance schedules. Strategically, the choice is between 2 ships or no ships.
• An initial decision to build 2 tram lines triggers an immense amount of preparatory work, the cost of which tends to be lost. Even if no work has been done, contracts have been signed, organisations have been scaled up to deal with the initial (larger) proposal. In defense procurement, the naval architecture associated with a one-ship fleet is rather similar to a 2-ship fleet.

We can also try and pass costs on to commercial contractors. Transfer the risk from government to the private sector. When the price doubles, it is the private contractor that picks up the bill. Sounds great, except:

• Short term, relations between contractor and client can become exceedingly unpleasant. Not least because government is a less aggressive environment than (most) commerce, so public officials often aren’t prepared for unpleasantness. In Edinburgh, work has been stopped completely by disputes with the main contractor, Bilfinger Berger. Disputes that rumble on.
• Medium term, a commercial contractor may default on the contract. The largest businesses struggle to manage hundreds of millions of pounds of unexpected cost. The result can be a half-finished thing, eventually completed late, by a far less optimistic contractor – who has inflated the cost still further.
• Long term, the risk associated with all government contracts rises. Potential contractors increasingly regard public sectors clients as liabilities. Or more likely, greater liabilities.

Evolution of Public-Private

That final observation contributes to the view that it is more cost-effective for the private sector to do everything, with the public sector reduced to buying the result like a service. That the private sector is better at managing risk, and the public sector can only be trusted to purchase at fixed prices – like chocolate bars. It’s one of the logics behind the 1990s-era Public Finance Initiative, Design-Build-Finance-Operate, Public Private Partnerships, and. There’s another problem:

The public sector isn’t sure what kind of chocolate bar it wants. Not just that difficult choice between Almond and Toffee. Rather, today it is buying luxury chocolate, but at some point in the near future its political objectives will change, and the public appetite will be for… somewhat cheaper milk chocolate. Or maybe… biscuits. Feeling hungry yet?

The private sector process is akin to trying to order groceries a decade in advance. A nonsense to consumers that expect to decide while standing in the store, seconds before they buy.

The catch is that, unlike chocolate bars, physical infrastructure lasts decades. The private sector can only invest in, and fund, such infrastructure over decades. Faced with the certainty of political uncertainty – whatever it builds likely won’t be wanted for some of its lifetime – the only sensible solution is not to build.

Box: Over the Sea to Skye
The Skye Bridge connects the Isle of Skye, in Western Scotland, to the mainland. Although technically sea, the crossing traverses a narrow channel of water, about a mile wide. The bridge was constructed as a “Public Finance Initiative” project. The contractor both built and then continued to own the bridge, charging road users a toll. The bridge had replaced a ferry crossing, so rational (economist) planning assumed that bridge crossings could be priced much like the old ferry. Unfortunately people perceive a ferry more like a “cruise”, while a road bridge is considered more like a (“free”) road. A situation that was further confused by the economics of the original ferry operation: In the best tradition of public sector transport operation, the Skye (Kyle) ferry had been cross-subsidising other ferry routes in the Western Isles. So where paid, the Skye ferry fare had been excessive for the distance travelled, and hence bridge tolls became rather excessive. Where paid, because local people had tended not to pay to use the ferry, yet found the privately owned toll-booth less accommodating. Opposition to the toll was initially small (the area is sparsely populated), but intensely passionate. As Scotland gained devolved government (including transport responsibilities) in the late 1990s, the toll was subjected to increased political pressure. Under the previous Conservative administration, Scotland was governed much like an imperial province – devolved government was a very dramatic change to the political landscape. Tolls were first subsidised by government – at significant public cost – until complete capitulation occurred: The bridge was bought (effectively at the price of the contractor’s lost future earnings, not the price of construction), and tolls removed. This political climate then spread beyond Skye – a belief that any Scottish bridge toll was unacceptable, apparent in current discussion about the Forth Bridge. In less than 20 years, the politics of bridge tolls have been inverted. The legacy of the Skye Bridge is not just funding inefficiency (George Monbiot calculated that the bridge should have cost no more than £25 million, but actually cost over £90 million). It also clearly demonstrates how the commercial construction/investment time-scales (in this case, originally a 27-year operating contract) can be completely at odds with far shorter public/political timescales.

Uncertainty can be reduced by what civil servants used to nervously refer to as “the P word”: Partnership. By close communication, shared understanding of each others’ problems, and, ultimately, strong personal relationships between the individuals involved, both the public and private sectors can better predict and adjust to future changes in objectives. Demands that might otherwise appear “from nowhere”. In a stable political environment, this can work – that is, one with rather limited democracy. But it rarely survives the sudden and dramatic shocks to the political system, that are associated with elective democracy: 15 years of gradually evolving, but broadly similar policy by one government, may be completely reversed by a newly elected government. In the British system, literally overnight.

This big uncertainty – the risk of a sudden change in direction – makes everyone cautious. Government officials are reluctant to commit to processes their future political masters may not be committed to. Commercial organisations become reluctant to invest in a process that might be “taken away” at a later date.

The result is a move back to formal contracts, as a way of protecting everyone against this big uncertainty. Except that a “partnership-contract” looks rather too cosy. At best uncompetitive. At worst outright corrupt. And, as the Skye Bridge box above shows, contracts still won’t stop a determined politician. Such contracts merely confuse the strategy, and inflate the final cost.

Now consider that, having merged public and private into one almost seamless project-completing entity, we logically start to wonder why we ever bothered to maintain any distinction? Why not just let government do everything itself?

Why – well, among other things, because the public is too optimistic. Yes, that again. We’re back where we started.

Paradoxes of Uncertainty

The underlying problem is one of time. The paradox, that people excel at evolving and adapting from day-to-day, but struggle to predict their own evolution and adaptation. They cannot conceive in the future, what they can do in the present.

Of course, people tend to live in the present. This isn’t just reflected in the inherent difficulty of trying to order groceries a decade before they are consumed. There’s a fundamental disconnection between the present self and the future self, even though the 2 states of being are logically related.

William Hazlitt is attributed with the observation that our present self views our future self the same way as it views other people. This starts to explain why we tend towards future optimism – we’re applying the same sense of optimism to the future, that we do to other people. That hints at how deep optimism runs through the human condition.

(Optimism could be equally well described as empathy. Which, in turn, probably reveals why depression – arguably a lose of human empathy – tends to make people more rational.)

So:

• Clearly, we are not going to solve the problem by solving optimism. It’s a part of what it is to be human.
• Indeed, managing optimism is no easier than managing people – a largely empathic process that tends to fail when applied rationally, or into the future.
• Meanwhile, optimism appears fundamental to the types of decisions that push society forward. Optimism is what biases human endeavour away from the currently accepted average. Possibly a necessary condition of “trial-and-error” evolution.

I reluctantly conclude that optimism is not only unavoidable, but entirely necessary.

So why is it such a problem for the public sector? Simple: Government doesn’t do uncertainty. Much as it doesn’t make mistakes. Of course it does, it just has terrible difficulty acknowledging that it does.

An investor in a commercial business accepts that some of the money they invest will be spent on unprofitable activities. They judge the success of their investment on the overall return of the entire business. In contrast, public investors in a government enterprise – or taxpayers, as they are called – expect every penny (cent) to be “well spent”. Anything the population deems to be “waste” can approximate to a moral right to “demand their money back”, with almost no consideration of overall performance.

Culturally this makes it hard for the public sector to acknowledge misjudgements, consequently harder to learn from them, and hence, exceptionally hard to learn from them while they are happening – at the time when something still might be done to improve the situation. In practice, either the situation gets so bad that it dissolves into scandal and farce, or once the dust has settled, the National Audit Office (or Scottish equivalent) reports about the woes of British tram procurement. The current generation of local politicians and officials will learn not to try that again (I suspect they already know). But nobody will address the underlying structural problem.

Which leaves an important question: Why do people demand financial certainty from the public sector, while simultaneously accepting uncertainty from the private sector?