The Future Of Programming?

[zhiel]

One of the things I've been thinking about lately and in the past years is the direction programming is heading as a whole. I don't think I'm alone in thinking that the field has been getting more chaotic since its birth and the speed of this process is accelerating.

Computer science is a relatively new field of science and the modus operandi has almost exclusively been a sort of a cowboy, wild wild west way of doing things. The one who gets there first claims the prize and defends it. As computer science is half about doing real science and half about the art of applying that science, I won't be touching the former part. Science works and the scientific method is unquestionably valid. The more interesting and as I said chaotic part is the art of programming.

Chaotic is a strong word to use, but I really do think it describes best whats going on. I feel wrong every time I see a new serious programming language introduced or a new serious library being announced. Comments like 'this new tool is great for my purposes' or 'this new thing solves my problem better than the old thing' send shivers down my spine. What is wrong with new and better technology being put out to use? Nothing. The problem lies in the way the process is done. It doesn't matter if the new technology gets used or whether it shrivels away – it always introduces more negative diversity. At least in the case of programming.

The worst are new programming languages. Think of programming languages as a forest of trees that each represent a language. Each time someone plants a new tree (creates a new programming language), it leeches energy from all the other trees in the forest thus not allowing them to grow to their full potential. Each tree looks a bit different than the others, but they all still do the same thing in the end. One might have functional leaves, one might have object-oriented leaves. One has imperative branches and one has a declarative trunk. The big problem with this in my mind is that there is no cultural value attached to these languages, only practical value. Programming is not poetry, the end result is the same no matter what language you describe the solution with. The true problem comes down to the issue of verbosity vs. terseness of the languages (which is an interesting subject, but out of scope for this rant.)

Languages are a tool of communication. Natural languages and human communication is endlessly complex but programming languages are needlessly complex. Humans communicate emotions (which there are almost an infinite amount), computers communicate data (exactly two things: ones and zeros) and programming languages communicate exactly three things: algorithms, data structures and input-output processes. To top this off, programming is the only thing in the universe which us humans have created from nothing. Once we cross over from hardware to software, we enter a sort of a Matrix -style virtual world which is in our total and complete control. Why is it then that we solve the same problems over and over again?

Pretty much the same exact thing is happening with libraries. Think of any problem that can be solved by a library and you'll find in most all cases different implementations in different languages and usually many implementations in the same language. I think this is total madness. They all solve the same problem. The true differences are in the algorithms and data structures chosen but all other differences are just cosmetic. The forest analogy fits here as well as to the programming languages.

All this negative diversity creates incredible amounts of overhead and a huge necessity for plumber programming – making different inputs and outputs fit to other inputs and outputs. This plumbing is subject to the same problem as well; it's being done over and over again with different cosmetics. With all this overlapping programming going on, is the following really such an huge leap of faith? One programming language, one set of libraries and one operating system.

While that may sound like something Hitler would proudly say in a Nazi-rally, the positive implications are almost mind blowing. Jokingly I think it's the grand unified theory of programming, in a sense. Think about writing a program in this utopia: almost no need to worry about backwards compatibility, almost no need to write plumbing, excellent quality libraries to solve almost any problem in existence, almost all code looks the same and you are accustomed to reading it fluently, language and library updates are immediate and global, almost all bugs become really shallow as all programmers are using the same tools, anything new is easy and fast to learn as it follows the way everything else is written and the list goes on and on.

Now I know that there most definitely is no silver bullet solution to rule them all. This is not what I am talking about. I'm talking about unification and globalization of programming. Think of it as one huge global repository. It's not about discrimination either as there are pros and cons to every programming style and paradigm. One way might be better for a certain problem but many times worse for another. The point being that the one language should be a multi-paradigm language, combining the best ideas of all languages into a single language that doesn't force you to write code in a certain format. If you think I'm contradicting my earlier thoughts, think it from an interface viewpoint. The way a black box works on the inside doesn't matter if it works.

This system has positive anthropological effects too. Programmers would stop thinking about producing code that has their own personal stamp on it and start thinking about contributing to the programming world as a whole. Democracy over the best solution wouldn't be an argument over cosmetic differences anymore but a straightforward statement of facts and the problem of verbose vs. terse would balance itself to an equilibrium over time.

Life would be peachy for us programmers, but it's not without fault. The biggest problem is that all this needs something to base it on, like a constitution, and constructing this would be a massive undertaking. Maybe even an impossible one. The amount of ethical, moral, sociological, technical and political questions this constitution should answer is immense. Other issue is how to divide and bridge the computer science advances into this system. New experimental languages and ideas should be developed and encouraged, but kept out of this system until they are proven to have real value instead of being just what new programming languages mostly are.

These are bold suggestions and hard questions, but I'm certain that there are people who think the same way. I would love to hear some comments and criticism on my ideas and maybe even start some further discussion about it. I think its time to start thinking about the big issues more sensibly. The amount of software and code in the world is going to increase at a faster pace the more we advance and at some point I think what I'm proposing is going to be inevitable.

[polyfractal]

So if I could summarize your post, it's basically "There are too many alternatives/variations/duplications in the software world...harumph"

Really? You ask why we "solve the same problems over and over again?" It's because no problem in software is ever truly solved. Were data-storage algorithms and structures solved twenty years ago by Oracle? Nope. Was search solved ten years ago by Yahoo? Nope.

The field is constantly evolving and no tool is ever perfect to meet the demands of modern needs. Honestly, who could have imagined that even small companies routinely work with hundreds of gigabytes of data? Large companies are dealing with petabytes of data.

That obviously requires a different toolset - whether it be a new language or a new platform or a new paradigm.

Your rant is basically saying "We have Model T automobiles dammnit, why would you ever want to build any other type of transportation?"

[codgercoder]

It would help if you grasped the difference between "computer science" and software development.

[kls]

There are a few archetypes of programmers, there are those that see the science of the computer, there are those that see the art of software creation and there are a few in between that see the philosophy in the middle. I would assume by your writing that you are in the first camp. The problem is those is the first camp favor languages with a different feature set than those that view it as a art. One cherishes their language because it enforces structure and rules, while those that see it as the art of human expression favor languages for their expressiveness and their ability to bend the languadge to their creative desires.

I like to use Lisp as an example because it is so hated by one camp and so loved by the other and that is the crux of the issue, language are developed for humans to express their ideas. We all think and abstract differently, I do not believe that their will ever be one language that rules them all, because it supposes that it can encompass the way all programers think, by nature it would be the most convoluted language ever designed, as such it would die in the design phase.

Conversely I see concepts like the JVM and the .NET VM as the future. I look at Java, Scala and Clojure as examples of innovation on all sides of the fence on a platform where they can interopt and leverage the creations by all languages that are supported on the VM. At the core, the issue is the differences in thought process and expression of that thought, until direct thought can be translated into software there is just no way to design Utopia++.

[zhiel]

Then your assumption would be wrong. I consider myself to be more artistic in writing my code. But thats besides the point I was trying to make.

While I enjoy the feeling of creation and I like doing fancy stuff on my computer, I still recognize that all code boils down to those three things I listed. No matter how artsy you get about it. I argue that creativity and expressiveness would be enhanced by having a common language that we could all understand and share our own skills through it. I like to think that if we had this utopistic language, the teaching of programming would become easier and it would be started earlier for students. Also the quality of the teaching would improve significantly.

Language is but a tool to transfer ideas out of your head to others, the better you are at one the easier it becomes. QuickSort works like it does because thats the algorithm. A linked list is like that because thats the data structure. I don't think it matters how you learn these concepts and how you visualize them in your head. My native language is not english but still I can think in it, through practice. I think the same applies to programming languages, just as long as the language has that expressive power. If it doesn't confine itself to a singular style or paradigm but lets you choose, isn't that the most expressive of languages?

[kls]

Then your assumption would be wrong. I consider myself to be more artistic in writing my code.

I apologize for the assumption but you still fit one of the archetypes, as do I.

QuickSort works like it does because thats the algorithm. A linked list is like that because thats the data structure.

Algorithms are very different from the core constructs of the language and they are very different from say UI programming or system integration programming. Algorithms are closer to Mathematical proofs in it's certainty to be efficient. So long as it is the most efficient implementation. That being said, developing systems is a long way up the stack from algorithms.

I agree that there would be a lot of advantage to everyone adopting the same language, we reinvent the wheel constantly and many times poorly. I don't contend that fact, I just honestly don't see how it is going to be resolved any time soon due to the nature of languages. As a thought experiment lets say that someone was given the authority to make all other languages disappear and they chose COBOL to be the one language. There would be a lot of individuals upset about that decision. I personally would choose a Lisp, which would assuredly set more than just a few people off. I personally would not be happy if it resembled COBOL, Perl or PHP, I am not very dogmatic about languages and go with the flow, but my mind does not work the way those three do, and would not like it if a language resembled any of those. What if someone chooses a visual languadge where we have to drag and drop logic? Those languages are almost university hated by developers.

If one of the currently languages is not the candidate for a utopian language then we get into a worse situation where we now have to write a new language and reinvent the wheel yet again to get everyone on board. We are committing the same error that we are arguing against assuming that this time it will be different. If it is not different this time, we have compounded the issue.

I am generally an optimist, but when a broad category of humans are involved in a conceptualization I become a realist as the human element seems to be the one that foils the best laid plans. Some people choose technology for the specific reason that other people do not use it, they like to be in the know about something the broader population is not. That demographic alone means that there will be a group that will not use the one language, even if it holds superior advantage. I just don't see everyone being accounted for, I believe by it's nature a single language cannot account for everyone.

This is why I see ideas like the JVM as so powerful and immediately beneficial because they allow us to leverage the past efforts while still providing the assortment of languages that we need as a vastly different demographic. The main issue that we are trying to solve via a single language is the duplication of effort, I content that we can solve that through technologies like the JVM. Where a lib written in Java can now be used in more modern languages like Scala and Clojure, to me this fixes the issue in a more pragmatic way but even with that solution we have individuals that do not like the JVM platform for various reasons.

[wavetossed]

Lots of people just like to build things and it is often easier to build a tool than it is to find a pre-existing tool that you can get, you can learn and you can use to solve your problem in less time than building something. All too often when you choose the reuse path, you end up going around in circles with stuff that may be good, but it doesn't fit your problem very well.

Also, humans are builders. It's what we do.

If you look at this problem closely, like I have over the past 15 years or so, you can already see evolution and natural selection happening. So I don't really see diversity as a big problem, and I do expect that in a generation or two, there will be a lot more consensus around which tools to use. However, even then, I do expect to see a fair amount of diversity.

Even today, where Google dominates search, they are dwarfed by Baidu in China. And in Russia, many people prefer Rambler. Orkut is a big social network in Brasil, Vkontakte in Russia, etc. Programming tools and libraries are the same, i.e. a lot of the choices are made for cultural reasons and that will never go away so some diversity will always be here.

Hopefully more people can gain the perspective that many years of experience bring, and can contribute to the process of natural selection (which implies improvement too) in order to get more people choosing and using better tools.

[dromey]

TL;DR but I think the future is verified correct programming. It's been around a while but been too hard to do right, or do fast. I think though that the needs for greater security and correctness will reassert themselves.