| > If you want to trust it you have to be able to audit its workings. This is the fundamental problem; we provably cannot even know if the black box will even halt[1], we obviously cannot audit any it for behaviors far more complex than simply halting. Even worse, if the black box is equivalent to a Turing machine with more than ~8k states, it's behavior cannot be described in ZF set theory[2]. > There is no magic sauce from a network layer As long as the black box is Turing complete, any noisy channel[3] to the network it can influence can be used as a foundation for reliable digital channel. The solution is to limit the black box to such that it is not Turing complete. The decision problems about its behavior must be decidable. A good example of a limited design is the original World Wide Web. HTML with 3270-style forms (or an updated version that is better looking with modernized form tags) was decidable, with the undecidable complexity sandboxed on the server. The instructions to the client (HTML) and the communications with the server (links/URLs, POSTed form field) are understandable by both humans and machines. Today's web requires trusting a new set of undecidable software on each page load. We're supposed to trust 3rd parties even though trust is not transitive. We're supposed to accept the risk of running 3rd party software even though risk is transitive. Now these problems are leaking into other areas like DNS, and the Users lose yet another battle in the War On General Purpose Computing[4][5]. [1] https://en.wikipedia.org/wiki/Halting_problem [2] https://www.scottaaronson.com/blog/?p=2725 [3] https://en.wikipedia.org/wiki/Noisy-channel_coding_theorem [4] https://boingboing.net/2012/01/10/lockdown.html [5] https://boingboing.net/2012/08/23/civilwar.html |