Unredacted Wikileaks archive leaked

The entired unredacted collection of Wikileaks cables was leaked on 2011-08-31. There were three ingredients.

(i) The fact that sufficient information was available on the net. This fact was broadcast by Domscheit-Berg, and published by Freitag and Der Spiegel.

(ii) The encrypted file. Looking at the various wikileaks file archives, the directory xyz with four files x.gpg, y-docs.gpg, y.gpg, z.gpg, looked like a good candidate to investigate, and in the end z.gpg turned out to be the one.

(iii) The password could be found in a book on Wikileaks written by journalists from The Guardian. This book is available on-line.

Consequences

Wikileaks has spent quite some effort over the past year redacting the cables before publication. This effort is wasted now - all unredacted cables are available.

After this lesson everybody will have to assume that what is told confidentially and in private might later be published all over the net. A bad day for trust.

Blame

Three parties are involved. Two made a mistake - stupid of them - and one wilfully publicised this state of affairs. I think this was an extremely bad idea, but Domscheit-Berg might claim that security by obscurity is false security, and that if he could notice this, any number of others could also. In view of the circumstances, many will consider this an act of malice or spite.

Leigh, the journalist from The Guardian who published the password in his book made a bad mistake, but probably he is not a professional. Publishing obsolete passwords is a bad idea: People tend to use passwords in a certain way, with certain patterns. If it is Asterix17 on one occasion, it might be Obelix631 some other time. Therefore, if you reveal someone's old password, you are revealing potentially sensitive information. Of course, as it turned out, this password was not old and obsolete, and publishing it had catastrophic results.

Assange is a professional, and should know that a password is something between you and the authority that is going to check it. You never involve a third party. If you want to give away something that is encrypted, first reencrypt it and then give the reencrypted copy together with its password (and then delete this copy). Anything else is bad security practice.

(It is sad to see the web filled with people pointing at each other. Everybody made mistakes.)

There is also a fourth party involved: The encrypted file was part of a collection of Wikileaks files that was taken by Daniel Domscheit-Berg after he broke up with Wikileaks (Fall 2010). According to Der Freitag Domscheit-Berg returned these files to Andy Müller-Maguhn, president of the CCC (Chaos Computer Club). Afterwards, someone then put this entire collection on the net, probably unaware of the contents of individual encrypted files.

The Wikileaks website contained

2010-11-28
Due to recent attacks on our infrastructure, we've decided to make sure everyone can reach our content. As part of this process we're releasing archived copy of all files we ever released - that's almost 20,000 files. The archive linked here contains a torrent generated for each file and each directory.

From privetbank we learn:

WikiLeaks has released an archive with a huge set of torrents with files from original non-CableGate WikiLeaks on Dec 03 evening. Magnet wikileaks_archive.7z (42827 torrents, 3164 KiB, MD5: 48ced268e846505ecde6618057fa5ca7, SHA-1: 385e46576c8e3adb143e40e4738a6bd83e5a38a8). Link source is #wikichat. Content is not fully validated.

and these 42827 torrents were seeded. A single torrent version (Compleat_Wikileaks_Archive) was made available on piratebay on Dec 15. Among these 42827 torrents was z.gpg.torrent.

My copy of the file z.gpg.torrent has a creation date of Fri, 03 Dec 2010 18:25:35 GMT. (And the other files in the archive have creation dates from 18:11:15 to 18:48:14.) My copy of the archive wikileaks_archive.7z.torrent containing them has creation date Sat Dec 4 06:07:44 2010. The site www.torrentdownloads.net says Torrent added: 2010-12-06 19:25:13.

So, this file has probably been available since Dec 3, and certainly since Dec 6, 2010, and had been released already when The Guardian published its password.

In http://193.198.207.6/wiki/file/ we see

xyz-magnets.txt (09-Jun-2010 01:50) 306 bytes
xyz (06-Dec-2010 15:17) dir
x.gpg       09-Jun-2010 00:32    390M
y-docs.gpg  09-Jun-2010 00:55    8.0M    
y.gpg       09-Jun-2010 00:55    84M      
z.gpg       09-Jun-2010 00:56    352M

that these files (plus magnets) were created on 09-Jun-2010 or before.

File

There are systematic, rather minor, differences between z.gpg (or cables.csv) and the cables published by WL and the various media partners. That suggests that z.gpg is not precisely the file that The Guardian got. This is also what people from that paper stress. James Ball (jamesrbuk) tweeted Sep 3, 7:34 PM:

The file posted in several locations, "z.gpg", containing the 251,000 unredacted WikiLeaks cables is NOT the file sent to the Guardian.

So, this was a different file, perhaps a temporary working copy, but used the same password.

Details

I retrieved cables.csv roughly simultaneously with and independently from several others. Out of historical interest, let me recall the ingredients.

(i) On Aug 13 Domscheit-Berg was kicked out of the CCC. On Aug 25, the magazine der Freitag wrote:

der Freitag hat eine Datei, die auch unredigierte US-Botschaftsdepeschen enthält, im Internet entdeckt. Das für die Entschlüsselung der Datei notwendige Passwort lässt sich ebenfalls über das Internet recherchieren. Die Datei mit dem Namen „cables.csv“ ist 1,73 Gigabyte groß und enthält schon veröffentlichte Botschaftsdepeschen sowie zahlreiche unveröffentlichte Berichte, unter anderem über Gespräche von US-Botschaftsmitarbeitern mit namentlich oder anderweitig identifizierbaren „Informanten“

(Der Freitag has discovered a file on the Internet that also contains unedited U.S. embassy dispatches. The password required for decryption can also be found on the net. This file "cables.csv" has a size of 1.73 GB and contains already published cables as well as numerous unpublished reports, including ones on conversations by U.S. embassy staff with named or otherwise identifiable "informants".)

Interesting. But no cables.csv to be seen.

(ii) On Aug 29, Der Spiegel confirmed

In the summer of 2010, Assange stored the password-protected file containing the cables in a concealed location on a WikiLeaks server. He gave the password to an external contact to allow him access to the material contained in the file.
When Domscheit-Berg left the organization in September 2010 together with a German programmer, the two men took the contents of the server with them, including the encrypted file containing the documents. As a result, Assange no longer had access to the file.
At the end of 2010, Domscheit-Berg finally returned to WikiLeaks a collection of various files that he had taken with him, including the encrypted cables. Shortly afterwards, WikiLeaks supporters released a copy of this data collection onto the Internet as a kind of public archive of the documents that WikiLeaks had previously published. The supporters clearly did not realize, however, that the data contained the original cables, as the file was not only encrypted but concealed in a hidden subdirectory.
Then, in the spring of 2011, Assange's external contact made public the password that he had received from Assange without realizing that this would allow access to the unredacted US cables. The slip-up remained undetected for several months. Members of OpenLeaks, the rival whistleblower organization recently set up by Domscheit-Berg, have now drawn attention to the lapse. They say it proves Domscheit-Berg's allegation, which he has been making for months, that data held by WikiLeaks is "not secure."

(This is what this page says today. When I read it first, the details were slightly different.)

Aha, this secret file can be found in the WikiLeaks Archive. And it is hidden. (In what way? I did not see any names starting with a dot. A directory without search permission?) This WikiLeaks Archive is old and big, but was published as a lot of small torrents. Some searching yields copies of such torrents that are still seeded, and various potentially interesting files, including x.gpg, y-docs.gpg, y.gpg, z.gpg. The time stamps are 09-Jun-2010 and the sizes are reasonable and these are the only files with obscure names. Maybe we have the right file.

(iii) Yes, what about this password? I recall reading this book by The Guardian journalists, or at least reading fragments, but I don't seem to have it any longer. But Google finds "Wikileaks. Inside Julian Assange's War on Secrecy. David Leigh and Luke Harding" as wkileks_secu.pdf in several places. On page 148 the password is a chapter subtitle. It fails on x.gpg, y-docs.gpg, y.gpg but works on z.gpg.

Release

This was 2011-08-31 and the web was buzzing - many people had found or were on the way to finding the complete set of cables. Shortly afterwards this complete set was easily available on piratebay and elsewhere, and Wikileaks decided (after a poll) to also release all remaining cables. (First encrypted as sDgo3FDksdGwsrkrS.enc, later as a plain cablegate-201108300212.7z torrent that unpacks to 60GB of data, mostly thin air.)

Note however that this complete release by Wikileaks is not the same as what is found on piratebay: the cables that had already been redacted are still given in redacted form. For example, in 01PRETORIA1173 we read

HIS LEGAL ADVISOR MOJANKU GUMBI, RUMORED TO BE MBEKI'S MISTRESS (OR ONE OF THEM), OFFERED HIGHLY QUESTIONABLE LEGAL ADVICE ON THE CONSTITUTIONALITY OF INCLUDING HEATH IN THE ARMS INVESTIGATION.

but the WL version (first released on 2010-12-08) redacts this.

AMM

Who is Andy Müller-Maguhn?

On Aug 19 10:05:19 GMT the web page http://www.wikileaks.nl/Media.html contained the text

Andy Müller-Maguhn
Concentrates on technological and social developments in the area of electronic networks. He has been a member of the Chaos Computer Club since the early eighties and serves on its board. The main part of this work lies within the future-compatible structures and lifeforms and in the attempt to support those developments through transparency
andy@ccc.de

and it still does. On Aug 9 19:14:26 GMT the web page http://www.wikileaks.nl/media.html was a contact form, and it still is. On Aug 29 00:13:02 GMT the web page http://www.wikileaks.nl/Press.html was a copy of Media.html except that this text about AMM was removed, and it still is.

On Aug 17 anked asked on twitter:

Conflict of interest? Andy Mueller Maguhn, CCC board member who initiated kick out of DDB is member of #Wikileaks http://wikileaks.org/Media.html

On Aug 19 mrkoot asked on twitter:

.@wikileaks Why did you rename Media.html to Press.html and removed Andy Müller-Maguhn under "Wikileaks"? (Google cache) /c @ioerror