[sjayasinghe]

This is coming close to the way I envision we’ll be programming in the future.

This goes from traditional 1-dimensional text-based programming to 2-dimensional visual programming.

What I envision is a 3-dimensional augmented-reality programming language where program elements will be floating around in 3d space around us and we use an augmented-reality interface to interact with them.

It’ll also enable us to do IoT programming in a very literal sense.

For example if you look at an air conditioner, light switch, or some other network-connected appliance that’s in front of you in the real world then the augmented-reality display will overlay the interface exposed by that device.

Let’s say that you’re looking at your phone (not the screen but the actual object). Since this is a device with a GPS chip your AR interface will indicate that you can do location-based programming with it.

Then you create the equivalent of an if-condition specifying a 10m radius around your current location.

if (phone is within 10m of current location) {

}

Now you look at a lightbulb in the room and your interface shows you that the lightbulb has a method for turning on.

You draw a line from the then-branch of the if condition to the light bulb.

You’ve created a program that turns on the light bulb in your room whenever you’re within 10m of this room.

[sytelus]

I've been using Unreal Bluescript since past few months. It's probably as practical and large scale as any visual programming system ever have been designed. There are many good things about it, for example, it "compiles" instantly. There is even some visual debugging. You can change a variable name and it gets refactored automatically. It gives you definitive errors right away instead of waiting for compilation. It has nice abstraction of subroutine and macros.

However, I have to disagree that visual programming will actually become more mainstream than programming mechanism for less savvy users. As soon as Blueprints gets bigger (equivalent to may be 200 lines of code), it becomes absolutely unwieldy. You will find yourself clicking all over places all day long. It becomes very hard to parse giant graphs. It becomes hard to keep layout of what is where. It simply doesn't scale. Compare this to even lousy speed of typing 40 WPM, ability to write 1000s of lines of code and be perfectly at peace with everything. Ability to quickly copy paste, refactor, move around by blazing fast keyboard navigation as opposed to just two buttons on mouse.

I was using Hololens other day and created bunch of objects around my room. It became overwhelming just after dozen of objects around me and my hands were getting tired by expensive gesturing all over place. A code that would fit in to 13" display probably takes significant portion of 3D space because each "if()", "while()" etc must be represented by space consuming graphic objects and forest of connections between them. Humans are good at absorbing small graph but as soon as nodes and connections starts climbing they become frustrated. This is why small toy examples look good in visual programming but no one seems to write 10,000 lines of code in those systems.

So keep your expectations accordingly. Visual programming is good for people who don't want to be full time developer but whose job entails them writing may be 100 lines of code every other week.

[sjayasinghe]

So I think programming will eventually become a tool used by everyone rather than something for only for programmers and other tech-savvy individuals. It'll be something that everyone uses the same way basic arithmetic is a tool that everyone uses these days. These days programming is reserved for the few because in order to program you have to learn a language with very specific grammar and the level of abstraction is still quite low. The underlying operations being expressed by current languages are not themselves intuitively hard to understand but the average person is prevented from doing so because of the language barrier. The level of abstraction that the average programmer is working with is moving continuously higher. In the past there was a time when the average programmer was working at the level of assembly language but today the average programmer is working at much higher levels of abstraction. There are obvious disadvantages to current visual programming systems as you've described but it may be the case that we just haven't developed the right abstractions and interfaces that work best for visual programming.

[kpil]

I doubt that will happen.

There is a distinction between incidental complexity - complex tools and languages - and inherent complexity - the problem itself, it's domain and the emergent properties of it all combined with a Turing complete machine.

You can't really do much to reduce the inherent complexity, and even professional programmers are struggling with the inherent complexity in all but the smallest problems.

Some programmers are even struggling more than others which suggests to me that programmers need to be a bit above average IQ to be efficient, and it might even be a lower limit were nothing useful happens at all.

What could potentially happen is that we could invent AI agents that could help us tackle some of the inherent complexities, and maybe non professionals could string together intents that will be interpreted by AI, but that won't happen soon.

Edit:

The insights about incidental and inherent complexity is not mine. They were popularised in the great book The Mythical Man-Month by Fred Brook that have many insights that are applicable to software development in general apart from those insights that are applicable to writing large operative systems.

As the book is 40+ years old and the follow-up essay "There is no silver bullet" - that expand on the complexity topic - is 30 years old, there is little reason that we are still trying to invent the silver bullet.

I guess it could even be described as a failure of the educational systems, because I'm sure there is a lot research in information theory that have been done or at least should have been done, that touches these topics.

[DonaldFisk]

Things have changed since Fred Brooks wrote his book: accidental complexity has increased considerably and is now at least an order of magnitude more than essential complexity. I estimate you can take any non-trivial system and reduce it to around a tenth of the size, without removing essential functionality, without changing languages. Chuck Moore goes even further: http://web.archive.org/web/20071009222710/http://www.colorfo...

I've written on the topic here: http://web.onetel.com/~hibou/blog/NoSilverBullet.html

[sjayasinghe]

In many cases what differentiates a programmer who is better at dealing with inherent complexity than another has a lot to do with the ability to apply the right patterns, tools, or strategies under the right circumstances.

This ability is largely influenced by having been exposed to similar problems in the past and having applied various strategies to solve those problems. One reason why a lot of programmers struggle is because our current educational systems do not expose students to computational thinking at an early age. They've simply not have had enough exposure to that kind of thinking.

Even though computational thinking draws on computer science as a formal discipline the insights are applicable to various other domains and virtually everyone will benefit if they're able to apply the same kind of thinking to other problems in their personal and professional lives. Without having to solve any problems specific to computer science people can still acquire the ability to effectively apply computational thinking by solving other problems in their lives because the inherent complexities of those other problems are going to have some similarities to the inherent complexities in designing software systems.

Now some people may already have highly developed computational thinking abilities without ever having touched a computer because intuitively they've understood how to solve analogous problems in a different domain. Such people may be far better at dealing with inherent complexity than some or most professional programmers but they could never apply those abilities in this domain due to the barrier to entry posed by the incidental complexity associated with the tools and languages.

A friend of mine, a physicist, is an example of the above. Her approach to dealing with complex problems is already far better than many developers I know, but she wouldn't be a good programmer simply because she doesn't understand the tools. However, if she were to invest significant time to understand the tools then she'd be a better developer than most. Not only that, if she had familiarity with the tools then her already developed computational thinking skills would mature much further. If she were to have access to tools that allowed her to apply her ability to deal with complexity without getting in her way then that would be the ideal outcome. What I'm trying to say is that we are a very long way away from minimizing the incidental complexities introduced by our tools and languages.

EDIT: Sal Khan gave a relevant Ted talk on the limitations of our educational systems: https://www.youtube.com/watch?v=-MTRxRO5SRA

He mentions that if you were to ask a literate individual in a past society with a 10 percent literacy rate what would be the maximum possible literacy rate they would've said something like 20 percent (80 percent of the population is incapable of overcoming the inherent complexity associated with gaining literacy). But our educational systems have progressed far enough to achieve a 99 percent literacy rate, which would've been impossible for someone in a society with 10 percent literacy to even consider as a possibility. Our estimates of where the boundary between incidental and inherent complexity lies is generally going to be strongly biased.

[kpil]

I wholeheartedly agree to almost everything you say and especially that everyone is probably able to learn some programming in a suitable language, just as they can learn to read and do math - and it would be good for them if it was taught well, if not because it gives you a tool for solving problems that you might not otherwise acquire. But I am certain that not all, or perhaps not even the majority, have the skills required to pursue a professional career.

I doubt however, that there is much to gain from simplifying the languages, and especially not by going the graphical programming route - it's a chimera occluding the actual problems.

Graphic programming could be used as learning tool, just like a set of training wheels, but just as them it would eventually become a hindrance as both make simple problems simple and everything else much more complex or virtually impossible.

The tools and languages can be improved of course but I do not think there will be any ground breaking improvements, just like there probably will not be any ground breaking simplifications in mathematics, or the process of learning it.

As an side note, I personally wonder why not more work have been done in incorporating first class support for entity-relation and/or graphs in languages - which is probably useful when modeling actual real world problems. But as it is still possible to build using generic language constructs, no one seems to care.

Or maybe the problem is that the educational system is so bad that too many are actually struggling to not topple over with their bicycle even after some years in university.

[DonaldFisk]

> Graphic programming could be used as learning tool, just like a set of training wheels, but just as them it would eventually become a hindrance as both make simple problems simple and everything else much more complex or virtually impossible.

For visual languages such as Squeak, and Skov, the idea is indeed to make programming easier for people.

However, visual programming has another important raison d'être. MIMD programs algorithms are more naturally expressed as dataflow, so that task scheduling and load-balancing become almost trivial, and dataflow is more naturally expressed graphically.

Now we have multicore machines and it is difficult to make optimal use of resources with mainstream imperative programming languages.

Graphs are, as you point out, a very flexible data structure, and there ought to be something to be gained in representing programs as graphs, and having programs operate on graphs. Lisp and Prolog benefit from homoiconicity.

[sjayasinghe]

Developers who've significant experience with functional programming understand that it would be easier and more reliable to solve many (or most) problems using a functional language over an imperative one. But the majority of developers have little to no experience with functional programming. In almost every conceivable way widespread adoption of functional programming would make everyones jobs significantly better. And these days we are beginning to see mainstream imperative languages starting to incorporate functional concepts slowly because people are starting to realize the value of it.

Right now we have far superior tools available to us than the ones used by the average programmer, but due to a combination of historical reasons, poor education, and sheer inertia of past commitment to inferior tools, we don't yet see widespread adoption of the superior tools that are already available.

The visual languages that I've seen so far are a kind of adaptation of existing languages to make them simpler and more accessible to beginners. What I'm talking about is not merely simplification of existing languages, but more a shift to an entirely different kind of paradigm. A shift that would require the development of new abstractions that are uniquely suited for visual programming that are not just adaptations of existing abstractions.

But as with all tools there are limitations. Certainly visual languages will not be best tool for all jobs, but there are certain classes of problems that are going to be easier to tackle with a visual language. There is also the possibility of solving a particular problem (or different parts of the problem) with both visual and text-based programming concurrently. First class support for data structures like graphs are easier to do with a visual language. I use Neo4j for graphs and the cypher query language is quite good, but it's effectiveness comes from the fact that it incorporates visual elements within the text. Something like MATCH (n)-[:REL]->(y) is a query for matching a part of the graph but you're still limited by the one dimensional structure of the language. You could imagine a visual query language for graphs being vastly more powerful in ways that a text based language couldn't possibly be.

[leovander]

There are a few other games out there like this, but what you described reminded me of Glitchspace[0]. There was a post about zachtronics some time ago that listed similar games.

[0] http://store.steampowered.com/app/290060/ [1] http://www.zachtronics.com/

[DonaldFisk]

While in my optimistic moments I imagine we'll be programming visually in the not too distant future, and am doing all I can to bring this about, I still would prefer to turn on light bulbs by flicking switches, which simply close electrical circuits rather than call methods. For that, some might consider me a Luddite.

[sjayasinghe]

Right, I'm totally with you on this. I only used the light bulb to illustrate the basic idea.

This need not be limited to network-connected devices either. You can define interfaces for any object even if they aren't electronic.

For example you can define the interface for a bottle as something that can be opened. You can define processes based on the interfaces of multiple objects. These process definitions can then be transferred to a robot that will then be able to interact with the real world based on those definitions.

[dirkk0]

One way to do this would be the Reality Editor from MIT. http://fluid.media.mit.edu/realityeditor