Lord Butler: "If David Cameron loses, he has to go – and quickly"

Posted On: 
13th June 2016

From Ted Heath to Tony Blair, Lord Butler spent three decades inside 10 Downing Street. We talk to the crossbench peer about European referendums past and present 

Lord Butler, pictured in 1988, served as an aide to five prime ministers dating back to Ted Heath

After all the sound and fury of the campaign, the polls still seem all over the place – not that we should probably trust them anyway, after last year’s general election debacle – but plans must be made. So the morning after the referendum night before, what kind of Britain will we be waking up to if the Leave campaign wins?

One of the most dispassionate but experienced observers is the UK's former top civil servant, a man who advised five prime ministers – stretching back to Ted Heath, who took Britain into Europe, and Harold Wilson, who held the last referendum on it in 1975. Now a crossbench peer, Robin Butler suggests the country would face a serious political crisis if we vote to Leave on 23 June, especially if it were by a narrow margin of victory on a low turnout.

Lord Butler, who served as cabinet secretary to Margaret Thatcher, John Major and – briefly – to Tony Blair, seems in no doubt on one immediate effect, though it is one strongly denied by David Cameron:

“His position as prime minister would be untenable if we vote to leave. Having committed himself so very strongly to the Remain campaign, his authority as PM would have been shattered... I think he would go and fairly quickly”, he says.

But the former top mandarin argues that the Brexiteers could not expect to see any more significant benefits of their triumph for some time. “Assuming the government triggers Article 50 of the Treaty to begin negotiations to leave the EU, nothing would really happen for at least the two years those discussions would take. We would still be bound by EU rules in the meantime – on immigration as on other issues. And the concessions David Cameron negotiated would fall immediately," he says. 

“Indeed, there might actually be a surge in immigration to the UK in that period, though I am not convinced on that." 

Butler reminds me that the referendum is only advisory: theoretically, Parliament could choose to ignore it. Some think this a serious possibility if a Leave vote is a very narrow one on a low turnout, and this is not something he discounts:

“One argument of the Brexiteers is that they want to restore powers and sovereignty to our Parliament – but all three main UK parties officially favour Remain. So it seems paradoxical to give powers back to Parliament to do something it does not want to. There might be pressure on parties to hold a second referendum.

“It is also possible that the process of withdrawal would be so awful that people would want to think again, but this would be a major trauma, a major political crisis. The Brexiteers are not going to roll over so there would be the mother and father of all fights. There would be a major political crisis in this country.”

But Butler qualifies that dire scenario by stressing that he thinks no-one will really want “the horrors” of another EU referendum on whether to Leave or Remain “for a very long time”.

Significantly, he echoes the grim warning from Sir John Major that the Conservative Party could even split under the pressures of what the former Tory Prime Minister called a “squalid and deceitful” Brexit campaign. His former cabinet secretary told me he thought this was a real possibility, and it would be something he would welcome, personally.

"I am not a member of any political party but I think British politics might be a great deal healthier if the Conservatives, and Labour, did split into pro and anti-European parties. You could just possibly see a realignment of the political institutions. The Conservative Party would split: let us say, the Brexiteers join UKIP and say we are going to maintain our campaign, and leave a pro-EU Conservative Party. Similarly you could have Labour sceptics joining an anti-EU conglomeration.

“Or pro-Europeans could split if Brexit wins. If there was a showdown between Brexiteers and pro-Europeans in Parliament. So you could have a realignment of the political parties on what is the real faultline in British politics at the moment, which is pro and anti EU. There is more chance than there was.”

It is a view also backed by Britain’s leading constitutional historian, Professor Vernon Bogdanor of King’s College, London, who taught David Cameron politics at Oxford. He argues that the Labour Party could also split, potentially, due to its current problems with a left-wing leader alienated from most of his MPs.

Butler says the major political parties as institutions have become “sclerotic". “They are rigid on the basis of divisions that no longer exist in the way they once did, between labour and employers. There is convergence on a lot of major issues.”

I suggest that the example of the SDP splitting from Labour in the 1980s can hardly inspire confidence over such a leap in the dark, given the UK’s unforgiving winner-takes-all, first past the post voting system. Butler suggests the situation has changed a lot since then, and that some of the old fixtures are now breaking down.

He said he thought some of the trade unions would no longer support Jeremy Corbyn's Labour Party, while some business donors might no longer back a strongly eurosceptic Conservative Party. So there is a chance that a split, and a new party, could succeed. “I think the things which have held the solidarity of the parties together have rather weakened... that the fixture of financing that has prevented realignment would not work in these circumstances quite so strongly as in the past. And if the financiers are split, that could provide a more solid new structure.” 

Lord Butler is almost uniquely well-placed to assess the similarities and differences of the two referendums on whether the UK should remain or leave, first the EEC in 1975, now the EU more than 40 years later. And he thinks today's political leaders might benefit from absorbing some of the lessons of history, if only they knew any: "I am astonished all the time at how little history politicians know," he sighs ruefully.

He reveals that he has personal testimony, private information, that he says proves Labour’s wily prime minister Harold Wilson really wanted the UK to stay in the EEC, while insisting that Wilson“took care to conceal it very effectively" and kept his main eurosceptic Cabinet colleagues, including Tony Benn, Peter Shore and Michael Foot, guessing about his own views,“playing his cards very close to his chest”.

“Wilson had strikingly similar objectives to Cameron. A referendum was the only way David Cameron could see of resolving differences in his party. It was the same for Wilson, though the economic circumstances were, of course, very different.”

“When Wilson came back to power in 1974, some of the Downing Street private secretaries went to him and said that we had been involved in Heath's negotiating team for entering the EEC and some supported entry. Wilson replied, privately: “You will have no trouble with me on that.”

Wilson also toyed with his senior eurosceptic ministers, Butler recalls: lulling them into a false sense of optimism when he told the Cabinet that if he could not get agreements on the terms of trade with the Commonwealth, and on measures to relieve Britain of its Budgetary contribution, reflecting the economic difficulties the country was going through, he would personally recommend a No vote.

 "And I remember very clearly that this made Benn and Foot and Varley and Shore – the antis in the Cabinet – really very cheerful, because these seemed really rather ambitious objectives that they thought it unlikely that Wilson would achieve at the Council of Ministers meeting coming up”.

But in a speech just before he flew off to the Council in Paris, Wilson adopted a very different tone, Lord Butler recalls, standing his comments to the Cabinet on their head.  ‘These are my objectives with which I am going to Paris,’ Wilson said. ‘If I can achieve these, I will recommend the British people to vote Yes in the Referendum.’

“I was the Number Ten private secretary, fielding calls that weekend. I vividly remember that the following morning the telephone was hot with Foot and Shore and Benn ringing up and saying 'who authorised the prime minister to say this?’ And I was able, on instructions, to point to the Cabinet minutes where he had said if he could not get these terms he'd recommend a No vote, and they had to agree that one thing was the corollary of the other. At that moment they could see the trap!”

The majority of the Labour Cabinet supported the eventual deal Wilson negotiated, but most Labour MPs actually voted against and only the support of the Conservatives secured victory in the Commons.

One final thought: Robin Butler is irritated at the frequently heard argument that, in 1975, voters and politicians thought they were just voting for a trade deal with the other eight EEC countries and so were misled.

“People were not sold a pig in a poke in 1972 and 1975, or told that this was entirely a commercial arrangement,” he insists. “Margaret Thatcher said in 1975”– and he reaches for the quotation – “‘I believe that the paramount case for being In is the political case for peace and security.’

“And my own view is that those are the decisive arguments today too. This is not a time to cause Europe to splinter,” he adds.