[pdfernhout]

@aekotra You wrote a strawman point of "When bad actors are made more capable they are made more destructive, therefore, no one should be made more capable." That is not what I said.

Also, I have worked and continue to work on FOSS tools related to intelligence augmentation as on my GitHub site (mostly under the names of "Pointrel" and "Twirlip"). So I believe overall in the value of such tools if widely distributed. As I said here: https://web.archive.org/web/20130514103318/http://pcast.idea... "Now, there are many people out there (including computer scientists) who may raise legitimate concerns about privacy or other important issues in regards to any system that can support the intelligence community (as well as civilian needs). As I see it, there is a race going on. The race is between two trends. On the one hand, the internet can be used to profile and round up dissenters to the scarcity-based economic status quo (thus legitimate worries about privacy and something like TIA). On the other hand, the internet can be used to change the status quo in various ways (better designs, better science, stronger social networks advocating for some healthy mix of a basic income, a gift economy, democratic resource-based planning, improved local subsistence, etc., all supported by better structured arguments like with the Genoa II approach) to the point where there is abundance for all and rounding up dissenters to mainstream economics is a non-issue because material abundance is everywhere. So, as Bucky Fuller said, whether is will be Utopia or Oblivion will be a touch-and-go relay race to the very end. While I can't guarantee success at the second option of using the internet for abundance for all, I can guarantee that if we do nothing, the first option of using the internet to round up dissenters (or really, anybody who is different, like was done using IBM [tabulators] in WWII Germany) will probably prevail. So, I feel the global public really needs access to these sorts of sensemaking tools in an open source way, and the way to use them is not so much to "fight back" as to "transform and/or transcend the system". As Bucky Fuller said, you never change thing by fighting the old paradigm directly; you change things by inventing a new way that makes the old paradigm obsolete."

The concern that I, and many others, have raised is essentially that technology is an amplifier of the best and worst in us. So, as we amplify our desires, we need to be ever more sure that we are using them towards "good" ends (where "good" is itself open for debate). Essentially, we need to use more powerful technologies (our "head") informed by even more by wisdom and compassion (our "heart").

The movie "Forbidden Planet" is a cautionary tale in that direction given (spoiler) the Krell were wiped out by the unbridled emotions of their own Ids when they made a planet-scale system that could materialize their every desire. A sequel with Robby the Robot called the Invisible Boy has a single malicious AI which has an individualist drive to survive, expand, and control and take over the world for nefarious ends using the latest military technology -- and would have succeeded if not for the more compassionate AI that was embedded in Robby.

These are age-old themes of healthy balance (including of control versus community) -- like the seven deadly sins that are all exaggerations (or amplifications) of healthy impulses as the extreme opposite of the seven virtues.

Yes, as you suggest, collaborative technologies can be a good thing. And yes they are likely to be more of a good thing if distributed broadly given notions of the value of democracy and decentralization (as part of a balance with needed hierarchies, see Manuel de Landa's essay on Meshworks, Hierarchies, and Interfaces).

But, all that does not change the fact that there is an increasing risk from that potential amplification. We need to be conscious of that risk and ideally put resources into managing that risk. And we are putting resources into such risk management to some extent as a global community -- but certainly not to the degree we could.

And one reason we don't put more resources into managing that risk is a discounting of that risk by many who stand to make money by creating proprietary technology they can use to centralize wealth in their direction. There is a lot of money to be made in "picking up pennies before a streamroller" (as in Nassim Nicholas Taleb’s book The Black Swan). Various academic and economic cultures have become very good at providing intellectual justification for making money that way while putting other people's money at risk -- and even putting other people's lives at risk like with war profiteering as with the Iraq war "cakewalk" that has cost several trillion dollars and hundreds of thousands of lives but made a few people very wealthy while destabilizing a whole region and leading to much blowback.

See also, from Wikipedia, "The Best and the Brightest (1972) is an account by journalist David Halberstam of the origins of the Vietnam War published by Random House. The focus of the book is on the foreign policy crafted by academics and intellectuals who were in John F. Kennedy's administration, and the consequences of those policies in Vietnam. The title referred to Kennedy's "whiz kids"—leaders of industry and academia brought into the Kennedy administration—whom Halberstam characterized as insisting on "brilliant policies that defied common sense" in Vietnam, often against the advice of career U.S. Department of State employees."

Our Earth has a certain scale which protected human survival in the past because people could always walk away from a bad city or bad region and live off the land. And people have done so for millennia as civilization after civilization has atrophied and collapsed (often under environmental stress or social corruption). See Daniel Quinn's "Beyond Civilization" writings for example or Wikipedia on societal collapse. In our interconnected world we now have nukes and engineered plagues. We also have total internet-based mobile surveillance (worse than "1984" surveillance) linked to massive compartmentalized "efficient" bureaucracy and corporatism that despots of the past could only dream about. And soon we may have "Slaughterbots". And most people have lost the knowledge to live off the land so they can't just walk away (and there are too many people and too little land for the old ways to support most of us that way anyway). So, this time it is different because we have a lot less potential resiliency in the face of mistakes.

But the fact is, the very same technologies that made it possible for humankind to act as a disruptive geological force (e.g. climate change,mass extinction of species globally) also make possible amazing responses. We could create ocean habitats making mid-oceans into productive fisheries, build space habitats supporting trillions of people and thousands of Earth's worth of biosphere across the solar system, make machines to remove the plastic from the oceans, proceed on insights into how humans can live healthy lives without massive factory-farmed meat consumption, expand our use of indoor agriculture, deploy more carbon-neutral renewable energy, put more R&D into hot and cold fusion energy, and so on.

To increase resiliency we could also actually study the topic more widely and shift our modes of manufacturing and agriculture and other aspects of living and economics. But that would require an ackowledgement of the risk and a deicison to prioritize managing that risk over short-term gains for a few (a complex political topic).

As I said in a Slashdot comment yesterday (made right after the one here, and drawing on similar sources) on the issue of Google and its ethical policy towards AI ( https://slashdot.org/comments.pl?sid=12172580&cid=56708188 ), an alternative for security principles for the USA in particular is to focus on mutual security through having friends and agreements and intrinsic security through having resilient hardened decentralized infrastructure and an educated capable affluent populace. But one difficulty is that those saner solutions are at odds with having a few financially obese people becoming even more financially obese through profits from the war racket and other monopolistic centralized rackets on the backs of uniformed disempowered impoverished workers and consumers -- and so there is fierce well-funded opposition to true security for the USA and the world (whether physical security or information security or progressive taxes or universal healthcare or a social safety net other than prison).

My concern is the "best and the brightest" putting 100% of their effort into pouring ever more gasoline onto the fire while putting 0% of their time into reflecting on what they are ultimately trying to accomplish for their community with that fire.