Confirmation of a top-secret programme designed to spy on the British public was revealed earlier this month by Home Secretary Theresa May.

Introducing the draconian Investigatory Powers Bill to the House of Commons, May announced that MI5, the UK’s spying agency, has engaged with the approval of the government, in the bulk collection and analysis of citizens’ personal date for almost 15 years.

In her attempt to justify an even more far-reaching surveillance dragnet, May stated, “The Bill will make explicit provision for all of the powers available to the security and intelligence agencies to acquire data in bulk. That will include not only bulk interception provided under the Regulation of Investigatory Powers Act and which is vital to the work of GCHQ [Government Communications Headquarters], but also the acquisition of bulk communications data, both relating to the UK and overseas.

“This is not a new power. It will replace the power under Section 94 of the Telecommunications Act 1984, under which successive governments have approved the security and intelligence agencies’ access to such communications data from communication service providers.”

Successive governments have allowed the bulk collection of phone calls, text messages as well as email records of British citizens, using obscure “national security directions” contained in an obscure clause in a three-decades-old act. As the BBC noted, “the existence of the clause was public but few understood how sweeping its use could be.”

The clause was utilised by the Labour government of Tony Blair prior to the invasion and occupation of Afghanistan and Iraq and was sanctioned by other home secretaries, including May. British telecoms companies have routinely passed the personal data of their customers to MI5 for analysis and have only been worried that this would one day become public knowledge.

In 1984, the year the Telecommunications Act became an Act of Parliament, a huge wave of working class militancy erupted across Britain against the Conservative government of Margaret Thatcher. The miners who struck for an entire year were dubbed by Thatcher as “The Enemy Within.” Miners faced state-orchestrated strike breaking, police brutality and mass arrests—all facilitated by the state surveillance carried out against the National Union of Mineworkers and miners themselves via the counter-subversion operations headed by MI5 with the active support of GCHQ.

Both MI5 and GCHQ have operated since the Telecommunications Act came into force without any statutory accountability. This was only introduced in February of this year because of the revelations made public by Edward Snowden that GCHQ was operating a massive illegal spying network in collusion with the US National Security Agency.

May never explained the domestic political origins of the Telecommunications Act’s provisions, insisting instead that MI5 and British telecoms companies have only conspired with each other since the 9/11 terrorist attacks in the United States.

May made these operations public only because the government is seeking a new framework to legalise its existing spying operations and to extend them in order to deal with the growing social anger over its policies of war abroad and austerity and cuts at home.

The routine handing over of the data of everyone in the UK is a long-standing demand of GCHQ. In November 2013, within hours of taking up the post of GCHQ director, Robert Hannigan published a statement in the Financial Times, prior approved by Prime Minister David Cameron, demanding that British and US telecom and social media firms deepen their collaboration with the security agencies.

Another Financial Times article noted that “although he [Hannigan] does not mention it by name” he “wants to see the UK government’s communications data bill—the so-called snoopers’ charter —passed into law.”

Within the space of a year, this charter, in the name of the Investigatory Powers Bill, is to become law.

Following Snowden’s revelations in 2013, the government argued that GCHQ collected “only” metadata, not “content” and such data had been collected and passed on for “national security” and anti-terror purposes. No details were given by May about the longstanding programme undertaken by MI5, or what they are capable of doing with metadata.

The bulk collection of metadata is what makes sweeping state surveillance of citizen’s possible. Metadata is the source of state surveillance programmes. It was Snowden who revealed, through GCHQ’s Tempora programme, that the vast amount of metadata swept up was fully computer-readable and searchable.

Simon Phipps, president of the Open Source Initiative (OSI), describes the practice of the “bulk collection” of data as follows:

“They accumulate data from any source that’s public or which they have a legal principle that they believe makes the data fair game, storing it for long periods in huge ‘data lakes’. They then use various justifications such as warrants and notification to secret courts to ‘go fishing’ in the data lake.”

Philips explained how metadata is used for surveillance purposes:

“The metadata remains readable no matter what we do, making it possible to triangulate even on encrypted messages. Triangulation means using apparently innocent data from other places to disclose hidden data that they and your message all have as context. For example, if I know someone's location is at a clinic, that they have recently purchased goods from a high-street chemist and that the web sites they have recently visited are about pregnancy, I don’t actually need to *read* the text of the e-mail to their boyfriend to guess what it’s probably about.”

Responding to May’s statement Snowden stated that the metadata being collected is the “comprehensive record of your private activities. It’s the activity log of your life.”

During 2014, some 517,000 requests were made to the home secretary by the security services, with the aim of securing a warrant, to grant them full access to the content of emails or to listen to calls of those whose metadata was sifted. Given this fact, just how much metadata was analysed to lead the security services to make such requests?

According to the Guardian the security services have been creating huge “bulk personal datasets containing millions of items of personal information.” The BBC’s Gordon Corera reported that the practice was “so secret that few even in MI5 knew about it, let alone the public.”

Under the Investigatory Power Bill, data retrieval, together with the hacking of citizens’ computers, web cameras, mobile phones, microphones, wiretapping, the planting of viruses and spyware on a target machine by the security services is to be legalised. Internet service providers are also required to record each website visited for 12 months and make such data is available upon request.

The government already spends several billion pounds annually on its spying operation against British citizens, but even more financial and human resources are now to be handed over.

This week Chancellor George Osborne announced a further £16 billion in spending cuts. The Treasury is also slashing a further £12 billion in welfare spending. At the same time as he announced these cuts Osborne said, “Over the next five years we will substantially increase the number of people across all three secret intelligence agencies [GCHQ, MI5 and the UK foreign spying force MI6]…”.