But a breakthrough was in sight, he said. Experiments had been taking place for two years at Bude, and had been crowned with success.
The problem was not so much to tap into the internet pipes – both the US and the UK could do that. It was to find a method of reading and analysing the torrents of data within the tapped cables, as they rushed past at speeds of at least 10 gigabytes per second.
GCHQ’s achievement was to be able to build a gigantic computerised internet buffer. The buffer could store traffic. Analysts and data miners would then be able retrospectively to sort through this vast pool of digital material. Full content, such as email messages, could be kept available for three days, and the less bulky metadata, such as email contacts and subject lines, for as much as 30 days. Uninteresting material such as peer-to-peer downloads of movies would be filtered out.
From the residue the spy agencies would, with luck, glean usable intelligence about targets of interest. The system was analogous to a gargantuan catch-up TV service where you could go back and watch any broadcast you’d previously missed.
At Bude, several key transatlantic fibre-optic cables made landfall close by. They could therefore be tapped into relatively cheaply and the data diverted the short distance to RPC-1 – a new ‘Regional Processing Centre’ secretly constructed on-site by a consortium of private firms, led by Lockheed Martin with BAE Systems’ subsidiary Detica and software company Logica. The process of furtive extraction had its own acronym: SSE, for special source exploitation.
By March 2010, analysts from the NSA had been allowed some preliminary access to the Bude project, initially codenamed TINT, then christened TEMPORA. It was described as a ‘joint GCHQ/NSA research initiative’. It uniquely ‘allows retrospective analysis’ of internet traffic.
Soon GCHQ was boasting of major achievements. ‘We are starting to “master the internet”. And our current capability is quite impressive.’ One document spoke of 2 billion users of the internet worldwide, with over 400 million regular users of Facebook, and a 600 per cent increase in mobile phone traffic from the year before. The agency believed it was on top of these developments. The report claimed the UK now had the ‘biggest internet access in Five Eyes’.
Not everything was perfect. The memo noted that American service providers were moving to Malaysia and India, with the NSA ‘buying up real estate in these places’ in a scramble to keep up. ‘We won’t see this traffic crossing the UK. Oh dear,’ the author said, suggesting Britain should follow suit and ‘buy facilities overseas’.
But the general tone of GCHQ’s 2010–2011 mid-year review was cheery. It stated that in one 24-hour period the agency had been able to process and store ‘more than 39 billion events’, ‘increasing our capability to produce unique intelligence from our targets’ use of the internet’. Apparently this meant GCHQ had managed to collect 39 billion pieces of information in a single day.
The NSA was impressed with British efforts. In a 2011 ‘Joint Collaboration Activity’ report it said that the UK now ‘produced larger amounts of metadata than the NSA’. By May 2012 it was reported that a second internet buffering centre had been constructed at Cheltenham, within the vast circular state-of-the-art headquarters complex its 6,000 staff generally referred to as ‘the doughnut’. A third overseas processing centre was also successfully organised and built at a location in the Middle East. The whole program was capable of collecting ‘a lot of data!’ Using TEMPORA, more than ‘300 GCHQ and 250 NSA analysts’ now had access to ‘huge amounts of data to support the target discovery mission’.
Snowden’s files show just how closely British and US intelligence personnel work alongside each other. While working for the CIA in Geneva, Snowden himself visited Croughton, the CIA communications base 30 miles north of Oxford in rustic Northamptonshire. Writing as TheTrueHOOHA, Snowden said he was struck by the large number of sheep grazing nearby in green fields – a classic English scene.
The NSA has had its own operations branch at GCHQ Cheltenham since the 1950s, as well as in London; GCHQ staff work at MHS. With some advance warning other GCHQ employees from Cheltenham can visit the heavily protected US outpost.
The NSA has a senior US liaison officer attached to the UK intelligence community known as SUSLO; his British counterpart operating in Washington under diplomatic cover is called SUKLO. Lesser GCHQ employees are assigned to practically all NSA facilities; they are called ‘integrees’. There is even a GCHQ staffer at the NSA’s tropical base in Hawaii, where Snowden worked.
Typically GCHQ employees do at least one stint at an NSA facility. The agency provides a helpful glossary for the Brits on American life; it gives tips on car hire and points out that in the US a boot is a ‘trunk’. There are joint meetings, training courses, exchange visits, cryptological workshops and celebratory dinners. And, one suspects – though Snowden’s documents don’t tell us this – the odd inter-agency romance.
This intelligence-swapping arrangement dating back to 1947 has been a success story. One document speaks of ‘another fine example of NSA and GCHQ working well together’. The Anglo-American SIGINT partnership is often warm on a personal level, beneficial to both parties and historically enduring. You might call it a marriage.
The files, meanwhile, offer a rare insight into the cloistered world of British spying. Salaries of GCHQ staff may be low but the organisation offers its linguists and mathematicians lots of leisure activities: pub quiz nights, cake sales, trips to Disneyland Paris and an internal puzzle letter called Kryptos. It even has its own social networking site, SpySpace. The main drawback to a GCHQ career is the agency’s provincial location. ‘Be prepared to describe where Gloucestershire is,’ a GCHQ recruitment guide says.
One particularly sensitive aspect of TEMPORA is the secret role played by telecoms companies which own or manage the fibre-optic cables. GCHQ calls them ‘intercept partners’, liaison with whom is handled by ‘sensitive relationship teams’. They include some of the world’s leading firms. BT, the main intercept partner, is codenamed ‘REMEDY’, Verizon Business ‘DACRON’, and Vodafone Cable ‘GERONTIC’. Four smaller providers also have codenames. In 2009, Global Crossing was ‘PINNAGE’, Level 3 ‘LITTLE’, Viatel ‘VITREOUS’ and Interoute ‘STREETCAR’.
Between them these companies help intercept most of the cable links touching the UK. They have British landing points at Lowestoft, Pevensey Bay, Holyhead (linking the UK to the Republic of Ireland), Whitesands Bay, Goonhilly and other seaside towns.
The company names are classified even higher than top secret, as ‘Strap 2 ECI’ – ‘exceptionally controlled information’. Exposure might presumably lead to customer unhappiness. One leaked document warns of potential ‘high-level political fallout’ if the firms’ identities become public. Intelligence sources stress that the companies have no choice. As in the US, they can use the excuse that they are compelled by law.
Thanks to this corporate co-operation, for which the telecoms companies are paid substantially by the British taxpayer, GCHQ was handling 600 million ‘telephone events’ a day by 2012. It had tapped more than 200 fibre-optic cables which touched the UK. It was able to process data from at least 46 of them at a time. This is indeed a lot of data – more than 21 petabytes a day – and the equivalent of sending all the information in the British Library 192 times every 24 hours.