One of the great pleasures of last week was hearing Jack Straw speaking on the Today programme in that patient, reasonable way of the true autocrat, and suddenly realising that I never have to pay attention to him again. Nor for a very long time will I have to listen to Mandelson, Campbell, Clarke, Smith, Reid, Falconer, Blunkett, Woolas or Blears: they're history and the New Labour project to extend state control into so many areas of our lives is incontestably over.
This is not merely a hopeful inference drawn from the change of government. The coalition agreement makes an explicit commitment to liberty and privacy, rolls back state intrusion, restores freedoms and puts a brake on the erosion of rights. There are omissions, of course, but overall this is a moment to cherish because along with specific guarantees, it is clear that the tone of government has changed and that the influence of the new administration may extend right down to the rude and officious exercise of petty authority that flourished under New Labour.
The Queen's speech, now being drafted, will establish a Freedom or Great Repeal bill – the title has not yet been chosen – as a major part of the coalition's legislative programme. All the areas detailed in the agreement between the Liberal Democrats and Conservatives, such as the abolition of ID cards and the children's database (ContactPoint database??), the further regulation of CCTV and the restoration of right to protest will be in it. Measures that weren't in the published agreement will reassert the right to silence and protect people against the huge number of new powers of entry into the home allowed by Labour.
Separate from this will be a complete review of terror legislation that will assess 28-day detention, control orders, section 44 stop and search powers, the harassment of photographers, the Regulation of Investigatory Powers Act, and its amendments, which sanctioned 650 agencies and local authorities to carry out undercover surveillance operations on, for example, people suspected of making dubious school applications for their children, eel fishermen in Poole harbour, punt operators in Cambridge, depressed police officers and malingering council workers.
The strategy is to keep debate on the terror laws apart from the Freedom Act, which in one fell swoop will repeal all the anti-libertarian laws that have accumulated on the statute book in what was described to me as "an absolutely comprehensive fashion". The government does not want discussion of terror laws to obstruct the swift repeal of Labour's attack on liberty in other areas.
Clearly, this all has to be watched very closely indeed – a lot has yet to be decided and there will be pressures from the civil servants, police, GCHQ and MI5 on such things as internet surveillance and phone intercepts. European plans for data collection and surveillance are a particular worry. But the essential point is that this exciting turn of events would not have been possible under a Labour-Lib Dem coalition or a Conservative minority government. It is a rare stroke of luck for the interests of liberty that the coalition allows the prime minister, David Cameron, to embrace this Lib Dem policy with open arms and ignore the reservations of the law-and-order nuts on his right.
This is a palpable benefit of the new coalition government, which should go some way to changing the minds of all those who protest that Labour was qualified to retain power by forming a "progressive alliance" with Liberal Democrats. Seems to me these people are still in the slow lane; or they haven't come to terms with the way Labour savaged liberty, while claiming support from progressives as its moral entitlement. At the end of an historic week, that looks very much like the old politics, because it involves a denial of the facts.
Four years ago, the day after an email exchange about liberty between Tony Blair and I was published in the Observer, the bad cop home secretary, Charles Clarke, set about briefing against this newspaper's campaign and made a speech at the LSE in which he accused me, and others, of spreading "pernicious, even dangerous poison". That was OK – Clarke's invective had the opposite of the desired effect – but it occurred to me then that I should think about what was behind the attack on liberty.
So I wrote a list of reasons, which I still have.
First of all, people such as Clarke, Reid, Straw and Blunkett had been involved in far-left politics in their youth and from those days they retained a belief in the inherent wisdom of the state and saw it as the only real force for good. This accounted for the largely undetected seam of authoritarianism that ran through New Labour.
Second, New Labour was a quasi-revolutionary movement that had fallen in love with the market and had no coherent ideology other than a belief in modernisation. These were people who were impatient with history and the great achievements of such things as Magna Carta, the Bill of Rights and so forth. The struggle for liberty seemed to mean nothing to them. All that was new and modern was good: the old had nothing to tell them.
Third, New Labour displayed a deep pessimism about ordinary people, which had quietly replaced the traditional compassion of the old Labour party. How this was allowed to happen is difficult to say –perhaps it was the brutalising effect of absorbing so much from Thatcherism. At any rate, that jaundiced view inspired ministers and civil servants to draft legislation that removed defendants' rights, to abandon faith in rehabilitation and redemption, and to assume ordinary people would always act in their own interests rather than those of society.
If you want an explanation for the looming mistrust behind so many of the databases and all the other expensive paraphernalia of suspicion it is this pessimism, which has spread through the media and society to the point where we think much less of ourselves and are disposed to believe the worst will always happen.
In that 2006 speech, Charles Clarke ridiculed this sentence of mine: "The presumption of innocence is no longer a fixed legal principle." The point I was making was that in narrow terms the government had undermined legal rights in such things as control orders, where a suspect is held under conditions of house arrest without being found guilty of a crime, or being told the evidence against him, and that more widely, everything from football banning orders to 28-day detention without charge, the use of Regulation of Investigatory Powers Act (Ripa), terror laws and mass surveillance of communications and movement offended the principle that law-abiding citizens should be respected by the state.
Convictions were made easier and we reached a point where mere accusation seemed to become an indicator of guilt. Mark George QC – a left-leaning lawyer from a Manchester chambers – wrote passionately this year about the introduction of hearsay evidence and effects of introducing civil standards of proof (the balance of probabilities) to replace requirements in a criminal case (sure beyond reasonable doubt) in anti-social behaviour orders, serious crime orders and gang-related offences.
The culture of suspicion and of the state always knowing best led not simply to a weakening of justice, but to a process in which the public was gradually infantilised: we were to be protected by health and safety regulations and chivvied by various new manifestations of authority in high-visibility jackets. People returning from the supermarket in Brighton had their wine confiscated for being in possession of alcohol in a public place. A 75-year-old blind man whose guide dog fouled in a field was handed a £40 fixed penalty notice. The UK Borders Agency started asking people where they were travelling to and why, and whether they had anything in their luggage that would be of interest to it. The same agency started locking up thousands of innocent children in detention centres for asylum seekers, a seriously damaging practice that the new government – thank God – is going to abandon.
This mass disrespect for the public played out in parliament, where elected representatives were treated as an inconvenience, sidelined or bullied by Labour whips to pass laws that were clearly against the interests of transparent government. The Inquiries Act 2005, for example, changed the law so that official reports were presented to a minister, not parliament, and enabled ministers to suppress evidence, change the terms of the inquiry and dismiss the chair. I cannot see this law as being anything other than deliberately hostile to good government.
Thinking about the five years of this campaign, I realise that the most agonising part of it was the sense that few people were really paying attention, and that Labour politicians could dismiss those who were as cranks and hysterics. This fear was behind the setting up of the Convention on Modern Liberty last year by Anthony Barnett and me, and the commission of a survey of all the laws that attacked liberty by University College law students, which we published as he Abolition of Freedom Act. The same fear compelled David Davis MP to resign during the previous summer and campaign against 42-day pre-charge detention and the database state, a gesture that cost him politically but which was critical in the defence of liberty.
This vindication is almost as much fun as not listening to Jack Straw. The programme of measures listed in the agreement between the governing parties "to reverse the substantial erosion of liberties under the Labour government" proves for once and all that we weren't making it up. We are very fortunate that the election played out the way it did.