Here's an IDE idea: create an industry standard plugin protocol. Most of the cool stuff an IDE does is code analysis and transformation e.g. refactoring, error analysis, and autofill. The other half is "makes coding pretty", "provides good shortcuts", and other UI considerations.
Now for the latter - that is great, and everyone has different strengths and weaknesses. And that is just a UX issue.
For the former however, I don't know why I need to use different IDEs for different languages, or even different IDEs for different transforms to the same codebase in the same language (again, for the automated bits).
It would be so much better for the world if there was some common protocol between user environment and code transformer. That way I could buy some of JetBrains awesome plugins and have my trusty Vim or Emacs environments use them. I could just keep using IntelliJ with MS' plugins for C# (and resharper of course). And so on.
For stuff that really needs a special UI, we are in the age of the web browser. Provide a localhost server in your tool and make the reports etc live in a browser window, big GUI IDEs can provide a webkit that can link back to their own editors, but text editor users could still get the useful output.
This "everyone does their own thing" without even pretending interoperability is getting old. We've been doing this coding thing for a little while now - we should be better at tooling that plays nice.
I feel that Software Engineering/Science is lacking a body of knowledge: something which is basically required in other professional industries (architecture, medical, physics, electrical engineering), etc.
Software Engineering/Science is at a disadvantage because there is no limitation placed on us by nature: an EE can't simply change the brightness of the sun to meet a requirement. We can.
To create an industry standard plugin is really saying we need to create a common way to communicate information between systems and sub-systems in software. I feel we have not really figured out the best way to do this yet. Our industry is still young.
I've put a lot of thought (about 6 years now) into a way to standardization communication between software systems and even parts of software. It's a hard problem to crack.
> common protocol between user environment and code transformer
I feel that once we crack the problem of communication between software parts, this will come naturally.
"I've put a lot of thought (about 6 years now) into a way to standardization communication between software systems and even parts of software. It's a hard problem to crack."
HTTP?
It's not perfect, but it seems to be what our industry has settled on as the universal way for pieces of software to talk to each other.
If you are thinking in terms of the payload, JSON seems to be filling that role.
Unix communicates via text, stdin stdout stderr. That's pretty much a standard.
Windows PowerShell communicates with typed objects. COM components and .NET components are a Windows standard way of communicating between software parts. COM has been around for a long time.
Between computers, TCP/IP is a standard way of communicating. But without good documentation of the protocol adding an HTTP layer on top makes it easier to understand. But without good documentation of the protocol RESTful APIs makes it easier to understand. But without good documentation of the protocol WADL...
However I do agree about HTTP. Because I imagine there are more devs that understand HTTP than higher or lower protocols.
Http is a protocol we use to communicate between systems, that is true, but not the parts of software. For example, to print "Hello World" on the screen, we use a sub-routine like WriteLine. The standard we use to communicate between software sub-systems, in this case, is a function with parameters. So, WriteLine ("Hello World") pushes the text "Hello World" into the sub-routine WriteLine.
Parameters are, for basically all languages, the means by which information is communicated between sub-systems.
The thing is, it is really hard to standardize the communication between sub-systems because sub-routines can vary widely in how they are defined (the possible combination of parameters and types is quite high). In fact, you can end up with thousands of different sub-routines: even in the same program. Each one unique and thus "non-standard".
This is the problem we need to crack. How do we create frameworks that don't lead to thousands of specialized interfaces: each one making the framework that much harder to use.
Four months ago, Bret Victor published his "Future of Programming" Talk. The most inspiring part to me was his 'prediction' of discarding brittle api`s for systems that negotiate a communication protocol dynamically. (Toy example: modem filter negotiation) http://vimeo.com/71278954 Relevant explanation @ 13:30-16:30.
> api`s for systems that negotiate a communication protocol dynamically
I think the first step is to find a very easy way to describe communication between sub-routines/processes. Once people can hook things up and compose the interaction of software with foreign/unknown systems (all in real time while the software is running) then we can get systems to start doing it dynamically (run on sentence but ya).
> BTW, there are quite a few limitations placed on you by nature: uncomputable numbers, NP-completness, ...
Not really, NP-completness is only important if you want perfect solutions, which aren't necessary. Uncomputable numbers have no usage in real world scenarios.
Finally it is important to note that while a physical turing machine cannot exist due to lack of infinite tape, real machines are also more powerful than turing machines, for instance by having a real time clock and communication abilities.
For these reasons saying what we can and cannot do is quite complicated. For almost every situation possible, the only answer that is negative about capabilities is "it will take a while".
In fact, a lot of games play with physics to get cool behavior. I can adjust the "constant" G in my game to be something other than 9.8 m/s to make it interesting.
One moment the G vector may be down and then suddenly it flips up. Why not?
Haskell community already standardizes through the use of library APIs and command-line applications like ghcmod. Any editor/IDE can invoke these tools.
The haskell-suite exists as standard of sorts, but the IDE specific tooling generally just involves invoking them like command line applications and using stdout in your editor/IDE.
So you are saying Haskell is still stuck at the command-line non-IDE world of tooling integrated via primitives pipes and such? Unix was great in the 80s.
Considering the alternative in the Java ecosystem where none of the IDE tooling is reusable, it seems okay to me. The basic stdout-oriented nature of it means virtually any language/editor/stack can make use of it.
That it's a command-line application also means it's "start up and throw away" which is easier to get working for most users than daemon based stuff. Typically once it's configured you don't touch it again. This allows people to build on each other's work which can lead to higher levels of abstraction and more human-oriented tooling.
The alternative is everybody having to write Haskell parsers and analysis tools over and over again before getting to the parts that actually make their editor/IDE different. The Clojure community has a similar ethos of tool sharing and reuse.
End users do not generally invoke these command line tools themselves, their editors/IDEs integrate them and offer the native trimmings the users would expect.
Why are you being so hostile? You're not making me want to share anything I know. I'm talking about the kinds of ecosystems that let people like you (that work on live-editing environments) focus on the parts that matter to you and you're behaving like a prick.
I guess someone could make a platform for plugins then also provide a translation plugin for IntelliJ, Eclipse, Emacs, Vim, VS etc. So they can all use the same plugins.
While it's probably possible, I can't imagine it would be an simple task to generalise that level of code analysis across languages.
If you look at Xcode, it saw a significant improvement when it started using clang to build its indexes. That is purely a C, C++ and Objective-C front end to LLVM.
Xcode basically requires a significant portion of a compiler to provide a decent coding environment.
What you are suggesting is almost to create a compiler "protocol" that can handle all languages. I am not sure how much value you would get from such a protocol.
I'm already pretty happy we're starting to see PL tools that aren't coupled to a specific IDE like f.e. Jedi for Python. Of course an older example of this is ctags.
I'm with you on the interoperability, if anyone in the world should understand and be 100% completely behind that, it should be coders and makers of code tools. If we don't demand it and require it, then it's pretty hopeless for the rest of the tech world.
tl;dr: Idea 1 is Vim (or any other programming editor). Idea 2 makes no sense, projects have too many symbols for them to be meaningfully encoded in rememberable icons, plus I don't really see how it would condense information. Idea 3 is LabVIEW. Idea 4 is Visual Studio's code map.
These ideas are all basically the first thing anyone comes up with when dreaming up a new IDE.
The big problem of IDEs is that they're a lot of work to make and polish very well. Most developer environments are either in a terminal, to get rid of the hard UI stuff, are an editor with good scriptability, to get rid of the hard IDE stuff, or are backed by huge slow moving corporations.
The one really big innovation that is going to blow life into innovative IDE development is scriptable GUI, aka Javascript+HTML. LightTable isn't the first and isn't going to be the last of awesome new IDE environments for which implementing ideas like this is going to be a piece of cake.
So hold on to your hats, we're definitely going there.
I think for right now the main idea in the area of IDE features is to have an "open compiler" so that there is programmatic access to the abstract syntax tree of your programming languages (aka, "compiler as a service").
This has been a neat-o feature which slowly became nice-to-have, but it is fast becoming a requirement. For example, the JetBrains products (Idea, Resharper, etc.), and also what Google is doing with clang to help them manage their large C/C++ codebase.
We already compile/interpret our languages. So we need to open up that "machine understanding" to programmatic access, which IDE's and other tools can then use to do all kinds of nice things.
With regards to the idea that just the current function should be on screen: At one point I saw a reference to a study (though of course I cannot now dig up the link!) that suggested an inverse correlation between the amount of code visible on screen and the number of bugs written by programmers, with the conclusion that programmers should buy big screens, use slightly smaller fonts, and avoid using too much whitespace. It is exactly when you are thinking only about the few lines of code you're writing, and not about the context in which those lines are going to run, that you're most likely to introduce a bug.
In an ideal project, of course, your code be so beautiful--so loosely coupled, so free of side effects--that you could in fact focus on a function in perfect isolation. But few of us are lucky enough to live in such a world.
I started implementing a few of these (and lots of others) ideas, and the issue isn't coming up with what to do, it's doing it intelligently. I have a small side project where I'm playing with AST rendering and the details are usually where you get tripped up.
I completely understand what you say, about the problem being in the details. For what is now one year, I have been working (on the side) on graphical code + special UI. Currently I am really happy with the prototypes that I have, and plan to continue working on it. I expect to have an actual product in some time, perhaps two years at my current work rate. I am also pondering if it would be possible to extract commercial benefit from such a product. My current plan is to release a free/open base version of the product with little restrictions. Over that, offer special enhanced versions. Perhaps a simplified educational version for kids to learn to code, a game programming version with openGL integrated, mostly targeted to young programmers, etc.
I think that for now the best way for me, to continue developing my product, is to continue working isolated from the world. At this stage is probably still early, but if anyone is interested to know more, and think that can provide something, I am open to considering possible partnership. Unfortunately at this point, I am not willing to share online any of the progress done.
I also have a few offshoot tool prototypes and a current side project into code visualization that I'm building, also not ready for public viewing. Get in touch if you're interested.
If you want testers, let me know or drop a comment with a link in the post. Don't bother trying to get the details right yourself, let the masses help make the decisions for you.
For #1, I think the Mylyn plugin for Eclipse does something vaguely similar:
"Task context management in Mylyn is based on the idea that your interaction with a system can be transformed into a degree-of-interest model, where each element in the system is weighted according to its relevance to the task at hand. This degree-of-interest weighting forms the context of all the elements relevant to the task. The task context can then be used to focus the UI by highlighting the most important elements, filtering the unimportant ones, and allowing you to perform operations on elements of interest "
Here's an idea: the world doesn't need more ideas, just more people implementing, so throwing out ideas as if they're original and have innate value is unhelpful.
Throwing ideas around is almost always helpful. Often when someone points out a solution to a problem or presents a good idea for something new, someone else, who hadn't thought of it themselves, will be inspired and go do the implementation. What's wrong with that?
I think your point makes sense for throwing out specific ideas to specific problems, but I don't think throwing out a general solution (lacking any concrete implementation) to a general problem is going to make someone who is capable of the implementation suddenly realize they overlooked a giant area.
This article can be boiled down to:
Project idea: better IDEs.
It's in the same vein as comments like:
Business idea: nuclear fusion.
They just aren't productive / valuable.
If say Henry Ford said:
Project idea: Use energy from chambered oil combustion with pistons to convert direction of force into a spinning axis to create an horseless vehicle. (I'm obviously not a car expert)
Then we would be talking about a valuable idea.
My viewpoint is simple: Ideas are not innately valuable, of all the ideas in the world, only a subset of them are valuable. Obviously you don't know what current ideas will be valuable in the future, but I have a heuristic: for an idea to be valuable, it must be a specific (implementation details included (batteries included)) idea.
I agree with the others. I think it's still useful to share, because in order to implement I need to build an entire IDE just to implement my features. That's an awful lot of work to see a feature request implemented.
I do agree that implementers are good though. As I mention in the post, I do have one more idea that I didn't share that I am going to build. I'll comment here again when I'm ready to announce it.
For the nth time, I encourage people to look at flow-based DSP programming tools like Reaktor and Flowstone (or its earlier incarnation, Synthmaker) where many of these ideas have already been successfully implemented. For the life of me, I can't understand why IDEs don't come with flowchart generation/editing built in by default.
Neat stuff. I'm fascinated by the potential in this field, and working on implementing some ideas too. You see some of my efforts at https://github.com/shurcooL/Conception#demonstration, if you haven't already.
So what happens if someone wants to embed C++ in the HTML in your C++ comments? (or just in general, having an early */ in your embedded HTML may cause problems.)
I think it would be interesting to see some design mockups of these concepts, which would make it more clear to the people behind prominent IDE's what they would look like ideally.
I thought about doing mockups for the post, but decided it was already long enough as it was. I may break each one down into mockups still though and post them.
Oh! You just noticed that IDEs suck? Where have you been living, under a rock?
Computer programmers are one of the most dismal professionals on that planet. How many so called programmers/coders can even write a simple parser or a compiler? Less than one percent I suspect. I should know because I am one of them.
After dabbling away with Aho and Ullman somewhere in the 80s or 90s I haven't approached the subject since.
Ever since Smalltalk came out it has been downhill all the way, and it is only now a lot of the current IDEs have caught up with it and in the last few years have begun to extend the concept.
I believe that:
1. Learning to understand and write parsers and interpreters should be one of the first things taught in computer science even before the whole gamut of data structures and algorithms have been studied.
2. Being able to develop IDEs should part of every computer science course. They may even be developed to help illustrate the data structures and algorithms mentioned above.
Unlike most professions computer scientists are the ones most capable of building the tools for creating, analyzing and illustrating their work, so why are they the ones that as far as we can tell use the shittiest tools available to most professions? Honestly I blame Sun and IBM for this. The moment Java came out everything went down hill. Then the web came, then 'scripting' , then everybody decided that they could and should 'build' everything using Notepad, VIM or Textmate.
3. The tools are there, Smalltalk guys have been using and developing their own tools for ages, only the targetted graphical environments have been split up and the OS owners, Microsoft in particular keeps blowing hot and cold as to which one system developers should target. IDE and graphical tools development is so interactive that the idea of using traditional compiled languages to develop them is simply absurd unless you use very fast compilers and computers. It is time they went 3D as well may be using some Blender, Unity3D or some other gaming toolkit to bring them up to date. 2D text and graphics are boring.
4. The bottom line is IDEs should be something developers ought to create or collaborate to create themselves. The big corporations should stay out of that, ie the decisions shouldn't be commercial decisions. It seems that a sad immaturity of computer programming as a profession has led to development tools being dictated by clueless profit seeking corporate bosses. They should be the decisions of developers guilds which should not be linked with corporate interests of any kind. You know what you want, just do it yourselves.
5. What else? Yeah, get your parsing skills together, and don't develop it in Javascript unless you plan to develop your own version of Typescript or Dart later on when subtle bugs begin to creep into your cherished IDE.
PS. Sorry for the trolling at the top. I just couldn't help myself.
I sort of agree with you, but I'd tweak it a little. I think we need to move to a situation where we refuse to use languages unless they offer "compiler as a service." Ideally, this would be open source for each programming language.
As a result, every developer wouldn't need to know how to write a parser/compiler, but rather would need to master the usage of the API the compiler-as-a-service offers. (For example, imagine JetBrains Idea for Java or Resharper for C# as a suite of open source, api-driven libraries).
Then, we could have IDE's with all kinds of features. In addition, we could write programs that transformed or modified our code bases.
This lets the "hard work" of compiler/parser building be centralized to a few developers, but spreads the benefits around to most everyone.
Since this is already happening in some languages (JetBrains, what Google is doing with clang for C/C++, and apparently Rosyln from Microsoft), it seems like only a matter of time before its power becomes apparent and it becomes a requirement for a language to be taken seriously.
I see no reason why Python and Ruby couldn't expose the compiler's AST as a service as well (obviously it would be limited by metaprogramming and dynamic typing ... perhaps in those cases we'd need "interpreter as a service" ... hmmm).
>As a result, every developer wouldn't need to know how to write a parser/compiler, but rather would need to master the usage of the API the compiler-as-a-service offers. (For example, imagine JetBrains Idea for Java or Resharper for C# as a suite of open source, api-driven libraries).
>Then, we could have IDE's with all kinds of features. In addition, we could write programs that transformed or modified our code bases.
>This lets the "hard work" of compiler/parser building be centralized to a few developers, but spreads the benefits around to most everyone.
Oh no you are not wriggling out of that one. Every developer should know how to write a parser and a compiler, not full fledged ones, just enough to implement a small language. That is in essence what computing is about, composing and instantiating formulae, with a full grasp of its semantics. To call your self a computing professional without knowing about parsing is like calling yourself a medical professional without knowing anatomy. This can only happen in the world of computer programming. It is the very essence of what we do, and how such a situation exists is an indictment of the computing profession, its educationists and educators.
It is the very soul of computing.
We might get away with it in the eyes of the public who don't know better because we produce the results they want, internally we shouldn't kid ourselves.
Ok, ok. If you want to define that as your minimum requirement, that's fine. I don't agree, but I'm sure many do. I know how to write a parser and a compiler anyway, so it doesn't step on my toes directly.
What I really meant to say was, everyone would not need to actually write a compiler/parser (regardless of whether they know how to or not). To achieve the goal of having a bunch of nice new IDE features, we don't need everyone writing parsers and compilers. We can achieve that goal by having a few developers write really good compiler-as-a-service offerings.
Now for the latter - that is great, and everyone has different strengths and weaknesses. And that is just a UX issue.
For the former however, I don't know why I need to use different IDEs for different languages, or even different IDEs for different transforms to the same codebase in the same language (again, for the automated bits).
It would be so much better for the world if there was some common protocol between user environment and code transformer. That way I could buy some of JetBrains awesome plugins and have my trusty Vim or Emacs environments use them. I could just keep using IntelliJ with MS' plugins for C# (and resharper of course). And so on.
For stuff that really needs a special UI, we are in the age of the web browser. Provide a localhost server in your tool and make the reports etc live in a browser window, big GUI IDEs can provide a webkit that can link back to their own editors, but text editor users could still get the useful output.
This "everyone does their own thing" without even pretending interoperability is getting old. We've been doing this coding thing for a little while now - we should be better at tooling that plays nice.