[api]

Okay, I'll re-state it technically:

How do we design a system that is anonymous and un-censorable where users can opt out of being relays for certain types of data?

Hard problem. Trying to solve it would be interesting. Not trying to solve it would make you identical to FreeNet and Tor and all the other efforts in this area, and thus less interesting.

I agree that there is no 100% solution to this problem, since all data can be converted to any format. There is also no 100% solution to pollution in a city, for example, or public health, or usability of a GUI. But there are 90% solutions that could make the problem marginal rather than severe.

BTW, on my "emotional vomit:"

Tell me. If these networks are for real human beings to engage in open communication, what happens when one of these real human beings comes across... say... a picture of a little girl being cooked over an open fire like a pig. (I didn't see this, but I was discussing the Tor .onion network on Reddit and someone claimed they came across this. I believe them.) Do you really think that person is going to return to this network to discuss... say... politics or economics or their local election?

It is a problem. To strip away the "emotional vomit," let's call it a usability problem. How do we make a freenet that is usable for non-psychopaths?

Edit: what I'm really saying is this:

Freenets have been done. It's a solved problem. Add some PK crypto and some hashing and some onion routing and shake.

What isn't a solved problem is: make a darknet/freenet that your mom would feel comfortable using. Make one that your average person -- maybe one with kids and thus really turned off by CP -- would want to one-click install from the Mac app store and browse.

THAT would make a serious political impact. Now you'd have hordes of average people using an utterly uncensorable chat system that was also hard to data-mine and tie to identity.

Right now, most people are going to start browsing the offerings that already exist (Tor is pretty easy to set up) and see stuff like "world's largest archive of hard-core lolita!", close the app, delete it, and never return. That's why these networks are not very popular, and it severely limits their political impact.

[derefr]

> How do we design a system that is anonymous and un-censorable where users can opt out of being relays for certain types of data?

If you do really want true anonymity and un-censorability as guarantees of the system design, then no, I don't think users can decide what they don't want to store or transmit. For, if they can, then their governments can coerce them into making the same "choice." Any preference that can be set by a user, can also be forced upon said user by a system administrator, operating system vendor, etc.

My real question is, do we need cryptography and anonymity built in at a protocol level to have something that's useful for political activism? It seems to me that there are only two real "innovations" these networks bring over, say, pushing encrypted blobs to people over SFTP drops (these, by coincidence, are both factors I've only really seen on Freenet):

1. That you have the ability to "push" content into the network, such that it will then replicate and spread through the network as it is accessed, without the possibility of an audit trail leading back to the source peer (even though the original source may know which client uploaded it, each peer only knows which other peer they got it from, so all you need to ensure anonymity is an internet cafe);

2. That content cannot be removed from the network easily--as there can always be dark peers who have copies of your data block, who will come online later and repopulate the network even if it has been seemingly purged of a block (by, say, all involved homes and data-centers being raided by the feds)--and that this happens pretty much transparently to the people involved, since people are always joining, leaving, and re-joining the mesh/swarm/whatever-it-is.

Encryption need only happen on a layer above this system, where and when it's desired. Anonymity need only happen at the end-points: the users can just access the system over Tor if they don't have the requisite internet cafe/seven proxies handy.

As long as you're just passing cat pictures around, why not just throw them onto a simple, infinitely-sized, everyone-can-create-files-but-nobody-can-delete-them DHT-based "disk"? And if you're passing political activism around, just encrypt and sign it like you were going to send it over email, then drop it in the mesh and email the URN instead. (This is presuming a stable PKI key-publishing/querying infrastructure as well, of course.)

And if you want to make it convenient for end-users, just make a browser extension that can load those URNs through the mesh as if they were regular HTTP URLs, and does the decryption and signature-validation automatically--and have the mesh software install that browser extension--and then you'll have something.

[lmm]

>As long as you're just passing cat pictures around, why not just throw them onto a simple, infinitely-sized, everyone-can-create-files-but-nobody-can-delete-them DHT-based "disk"? And if you're passing political activism around, just encrypt and sign it like you were going to send it over email, then drop it in the mesh and email the URN instead. (This is presuming a stable PKI key-publishing/querying infrastructure as well, of course.)

The latter is basically equivalent to this system, and so your system would have exactly the same problems: the only way you could avoid being a relay for child porn is to refuse to relay any encrypted content, at which point your node is not helping the political activism. Allowing unencrypted content also leaves you much more open to traffic analysis (if only a small fraction of data is encrypted, it's much easier to find the nodes that are inserting the political activism data).

[derefr]

Right, I'm not claiming that my suggested alternative is able to avoid what I might provisionally call the AUE theorem†; I was just suggesting the alternative as a way to separate out the "dumb" everyone-shares-one-infinite-distributed-hard-disk block-transfer layer, from the "smart" only-certain-people-can-see-certain things layer. The lower layer is an infrastructure protocol, with about as much Ethical impact as the Internet itself (to be specific, it could be compared to, say, BitTorrent.) The upper layer is where ethical responsibility comes into play.

We can probably convince your mother to download an app from the App Store that integrates just with the lower layer--hey, it's just like Dropbox, but bigger! [Well, as long as anyone and everyone can read random samples of your data if they like...]--because the upper layer, with the encryption and signing, will siphon off all the stigma of not-so-above-board usage of the protocol and attach it to itself. It's no different technically, but it is very different socially.

The advantage of having one reviled app on a larger infrastructure is that that reviled app gets to "hide" its blocks among all the above-board usage of the infrastructure. Like another poster in the thread said, if you go onto Freenet or the Tor Directory, the links to CP sites are plain and obvious, because it's a large part of what's going on there. But if you could look at your own disk usage as a node in this network, I imagine the number of encrypted blocks as compared to, say, plain-old MPEG frames of TV shows, would be vanishingly small. (And it's be relatively impossible to define which is which, either, since this infrastructure has no "index" or metadata; it merely is a big bucket of blocks named by their content hashes, of which most--not just the encrypted ones--are meaningless unless you have another block giving the order in which to string them together to make a file.)

† "Anonymous, Uncensorable, Ethical: pick two."--named after the CAP theorem of database design. Well, it would really be the AUE conjecture for now--but I'd love to see someone prove it either way; it seems like the sort of thing that is amenable to that.

[lmm]

> it's be relatively impossible to define which is which, either, since this infrastructure has no "index" or metadata; it merely is a big bucket of blocks named by their content hashes, of which most--not just the encrypted ones--are meaningless unless you have another block giving the order in which to string them together to make a file.

If you can't tell whether a given block is encrypted data or just part of an mpeg, how can you choose to store only unencrypted data? I suppose you could make an argument for building this system on top of a nonencrypting distributed data store, like bittorrent, for the sake of looking like that nonencrypted protocol to anyone intercepting the traffic. But there would have to be some metadata that let the encrypting protocol know where to find its stuff, and if the user who's downloading it can tell, so can anyone intercepting the unencrypted stream. Wouldn't you just end up with a situation where the upper layer is to the lower layer as freenet is to the internet?

[derefr]

> If you can't tell whether a given block is encrypted data or just part of an mpeg, how can you choose to store only unencrypted data?

I didn't say you could :) The point of this alternative is that it separates the stigma 1%-99% toward the upper layer, but puts the implementation 95% into the lower layer--and therefore we get a stable, un-censorable distributed storage network on the lower layer with the "abuses" of the upper layer (CP and political activism both) being an unavoidable free rider, but not something "visible" (in the sense of seeing CP sites listed in your index directory) to people only using the lower layer.

This situation, of course, also describes the Internet as it is today: protocols like HTTP and SMTP are used by everyone, and also by some unethical people who send their stuff over those same protocols in encrypted containers using anonymizing proxies.

The difference here is that the two big hurdles--of identity-diffusion over time after initial data seeding, and of guaranteeing data persistence as long as there continue to be consumers of the data becoming persistent-caching peers--are taken care of by the lower layer, allowing the upper layer to just handle transparent encryption in whichever way it sees fit.

(And thus can we also replace the upper layer if we come up with a better way to anonymously and securely get the right metadata into the right hands, without having to throw out the network effect of all the extant peers. They simply start transmitting-and-caching blocks representing the new kind of metadata exchanges along-side the blocks representing the old kind.)

[user49598]

The problem is thinking that there is a way. You can't. Data can easily be obfuscated. As long as bits are flowing through your machine from sources that you don't control, you could be middle man to anything.

[davorak]

API stated it was a hard problem. Hard problems dealing with cryptography can take years and some times produce PhDs when they are cracked. It is easy to tell it is a hard problem, it is hard to say for sure it is not possible. If you have done the hard work of show it impossible please share and save others from repeating your work.

[api]

There is at least one free Ph.D thesis topic in this thread. I think solving this problem and creating a darknet that most people would want to use would be at least as impactful as the development of Bitcoin. People said that was impossible too.

[javajosh]

I see a lot of people saying that you cannot analyze the data flowing through a system - and yet, I have to wonder, how then do people find the data they are looking for? It seems to be a rather fundamental theorem that if you know a URL to a file, no, there is no way to stop you passing whatever you want. But if you only have search, then surely the search itself implies that you have a way to classify the data!

Personally, I think this is enough to stop the spread (if not the storage) of horror. In other words, someone might safely store their cache on my computer without my knowledge (heaven help me) but I refuse to store anything that is searchable as a horror.

[lmm]

I don't think this system is searchable. You'll be storing a bunch of encrypted blobs, each of which can be retrieved and decrypted by anyone who knows a short key. One of them might be a CP picture (if you store enough of them, statistically one of them will be a CP picture). One of them might be an index full of other short keys, pointing to a whole host of CP pictures spread across the mesh.

If you're fine with that, that's good enough; you can run this system, and political activists and perverts alike will be able to stick their blocks there, accessible to anyone to whom they can pass the relevant short keys. But many people will be uncomfortable with even this much.

[davorak]

Yeah this system does not appear to be the quick glance I looked, but api's disired system does not need to model this one.

api's requirements:

> How do we design a system that is anonymous and un-censorable where users can opt out of being relays for certain types of data?

As long as we are reasonable by taking "un-censorable" to mean very difficult to censor and "users can opt out of certain types of data" to mean highly limit traffic of data type <x>, it seems like a hard problem until proven impossible.

[javajosh]

>I don't think this system is searchable.

How then can it be used to share anything? I can see how it could be used as a secure, distributed backup (which itself is rather handy) but I'm not sure how it can be used to distribute data.

[lmm]

It's (AIUI) meant as a replacement for pastebin etc. - you host something on this mesh, and then you only have to spread a short hash key around. It also lets you do the wikileaks thing of publishing a bunch of encrypted data which you could later release the key to.

[darklajid]

I'll bite.

In the OMG, think of the children case: A number of cases that went public (some even linked here) agreed that legally, child porn is 'i recognize it when I see it' kind of subjective. I'm obviously talking about teenagers here and different moralities or a missing context (such as those 'taken for fun' or 'sent to a friend, privately and deliberately' cases).

Api might, from his subjective view, decide that this as-yet-never-encountered image is bad/evil/perverse. How would you ever create an algorithm for that, other than 'api, please press a button that says "fine by me" or "no way hell", right next to the image in question'?

[davorak]

It is not clear to me that a system could be created that would be fine grained enough to take into account individual preferences with out general AI. I can imagine a rough grained system were none of them match perfectly but some get close. A user would have to pick a standard and live with the good and the bad that came with it. Not perfect but more choice then what you have if you sign up to be a tor relay or run freenet now.

[user49598]

This isn't about cryptography, this is about what data can be pulled out of bits transfered through your computer. There is no requisite that they be encrypted, they can be obfuscated, hidden, or just not recognizable as illegal to you. There is no way for you to verify that a collection of bits, put on your computer by a 3rd party, does not in fact represent something illegal. You would have to have access to every existing and theoretical encoding, encryption and obfuscation technique, and use them in every theoretical combination to verify such a thing.

[ktizo]

The hard work has already been done and it is easy to demonstrate the problem and the impossibility of telling 'good data' from 'bad data', assuming the system is cryptographically secure, as by definition a cryptographically secure system resists analysis of content.

Consider the problem of one time pads. If I have two messages the same length, one made of 'good data' and the other consisting of 'bad data' and I encode them both with different one time pads, then it is possible for the resulting ciphertext version of each message to be identical. Another way of putting this is that for any given ciphertext that has been properly encoded with a one time pad, the only information available about the plaintext is the length of the message (assuming you know already that a one time pad was used) and nothing else.

[davorak]

api: > How do we design a system that is anonymous and un-censorable where users can opt out of being relays for certain types of data?

So not necessarily following any of the specifications of the system in the article.

I read his specification to mean that users are anonymous, they can post data and it can not be tracked to them. I do not see this necessarily requiring the data be filtered in a encrypted state only that it can not be tracked back to a submitter who took reasonable precautions.

[ktizo]

The aim of - un-censorable where users can opt out of being relays for certain types of data - seems possibly paradoxical.

[drostie]

It is. What «api» has been proposing is literally, "I want a no-censorship network which I can censor."

That it is paradoxical does not necessarily make it impossible, though. The goals are certainly contrary but I am not certain that they are contradictory.

If you think about community-based censorship, this could probably be arranged even in an anonymity community, as long as it had active-enough participation. A popular search engine like Google can have tremendous ability to censor others even on a network like Tor where people cannot easily be censored.

The chief problem is that «api» faces is that his/her aspirations are too individualistic and unimaginative. You could always put the to-be-censored material in an encrypted archive and distribute the link to the material with the password to it -- this sometimes happens with BitTorrent (and then you'd have to click on ads to get the password and it becomes a nightmare). Then nodes cannot inspect the content. So what are you going to do, limit content-types? This was done by Napster, where only MP3s would be shared -- but a piece of software quickly came out called Wrapster which "wrapped" other files in MP3s. There exist JPEG steganography tools as well, both hiding files within the least-significant bits of the image data as well as in parts of the JPEG which do not get interpreted by a normal JPEG reader (e.g. appending a RAR archive to the end of the JPEG image).

I say "too individualistic" as well because any sort of relay net where the nodes themselves inspect the content that they trade is going to expose itself to a possibility of systematic censorship. "I know that you know what you were sending me" is a horrible way to start your cryptosystem.

Nonetheless, there might be hope for a sort of global data-store which the nodes collectively take responsibility for, which nodes collectively trade and where nodes can vote to "veto" certain indexed files. The idea would be that you can't take down the data store by taking down individual nodes, you can't prove which node "uploaded" a file, and you can't necessarily fault the nodes for failing to down-vote a file tracked by the community since hosting the file is a collective decision, not an individual one. It would have to use central aspects of the design of BitCoin alongside central aspects of anonymity networks, but I don't see why it would be impossible.

[davorak]

Well if no one opted out of any data type then it would be just like the some systems we have today. If most relays opted out of a data type x then the result would probably be that data type x would be less anonymous then other data types. It would take fewer conspirators to subvert the system for data type x. Similar to anonymity from government Z is effectively lost if they control m% of the nodes on tor.

Though I do not study cryptograph professionally that would be my current guess.

[Tloewald]

Forcing such data to be obfuscated is a step in the right direction. At least we won't accidentally happen on it.

[neilk]

I doubt this is easily possible, with Tor as designed.

Maybe you could establish a blacklist of CP sites, and that could be applied at the entry and exit nodes of the Tor network. This blacklist would have to be public and checked by many that it didn't contain non-CP sites, so in effect it would be a public directory of CP, which is problematic already.

Then, those that run entry and exit nodes could voluntarily apply the blacklist. In this way the Tor community could have its own values, while still being independent of any authority.

But this assumes that CP will remain restricted to certain domains in the .onion system, or the traditional DNS system. Which of course they won't. Maybe there will be one .onion domain per picture. Maybe there will be a Flickr of .onion where it's not so easy to figure out who's doing what. Then you'd have to lean on that service to police its own members' content.

I can imagine various messy and imperfect ways to limit the amount of CP in the world, or at least make it harder to find, but we just don't have good legal models for dealing with true freedom of speech. And our institutions today would rather persist in the fantasy that they can completely control speech, than accept that their role might just be to advise the citizens on how to police themselves.

[knowtheory]

Regarding the social implications and imperatives...

I think that groups like LulzSec provide a public service (see "Why the Joker and Not Batman is the Savior of [sic] Us All" http://thisorthat.com/blog/why-the-joker-and-not-batman-is-t... ) in that they show the importance and the need for everyone to be security conscious. I wish there were more groups like this out there raiding and dumping stuff periodically.

I wonder about whether it'd be political or legally feasible to have a law enforcement agency which just trolled around the internet and attempted to crack services that citizens depend upon.

The reason why this is relevant and important to dark nets, is that currently the only folks who use dark nets are folks who have something to hide. That might be folks who are illegitimately persecuted by governments, or folks who are legitimate criminals. These two groups are functionally indistinguishable, even if their intents and causes are different. They both have data they're trying to hide and communicate, without exposing themselves to authorities.

[erichocean]

You could require receivers to calculate a difficult problem where the difficulty was based on the "hate" people had for the content, similar to what Zed Shaw tried to do awhile ago with IRC (?).

Basically, each person that didn't like the content would spend some CPU time to up the difficulty of transferring it. After awhile, it'd take someone who wanted the content so long that they would give up, in which case the content wouldn't be transferred any more.