[dgreensp]

I get the opposite message, which is that programming tools and the process of programming are so bad, it's hard to get a computer to do a simple task for you without tearing your hair out, never mind building a UI. A better IDE (basically a better text editor) doesn't even scratch the surface.

[mkozlows]

The thing is, it's easy to say something is hard. Programming is definitely hard!

The question is: Is it hard essentially, or is it hard accidentally? That is, can you remove unnecessary complexity from programming and suddenly it'll become easy?

That was the proposition behind first LightTable, and then Eve as originally conceived. But neither of them really found a satisfying answer, a way to say "hey, for making your web app or whatever, if you throw away your existing stuff and use this tool/process, now it's super-easy."

That implies heavily that a lot of the complexity and difficulty is essential. Not all of it -- things will get easier and better over time, as they have over the last ten years -- but enough so that blowing it all up and starting from scratch isn't likely to lead to wins.

[ibdknox]

> But neither of them really found a satisfying answer, a way to say "hey, for making your web app or whatever, if you throw away your existing stuff and use this tool/process, now it's super-easy."

Actually, we think we did. We're just not choosing that as the primary focus of the workflow in Eve. Now with a better version of the editor and with a bit more work on the UI builder, I suspect we could rebuild the entire foursquare clone in under a week. If we had VCS so that multiple people could work together on it, it might only be a couple of days. This foundation for programming has lots of implications for building "real software" - it's actually based on research for making distributed systems much easier to build. [1]

We'll see more of that as we go since we're bootstrapping bits and pieces. One of the first things that will transition over is the compiler, if that gives you any indication of the level of sophistication you can achieve with this programming model.

[1]: http://db.cs.berkeley.edu/papers/eurosys10-boom.pdf

[skybrian]

There's certainly a lot of accidental complexity that comes from lack of standardization. For example, think about how complicated character sets were to deal with before Unicode. Then compare the mess of different kinds of Unicode encodings to standardizing on UTF-8. Or take file formats before XML/JSON/Protobufs. Or, going further back, floating point before IEEE 754.

I agree that blowing it all up is not a win, which is why accidental complexity goes away only gradually, and it's partially a process of hiding it rather than removing it, with lots of politics along the way.

[dunkelheit]

In its essence programming is transforming data (and code is data too). Everything else is incidental. But if I think of the things I do in my day-to-day work as a software developer it is 99% logistics (getting data in the right place and in the right form) and 1% related to actual meaningful transformation. If my understanding is correct (I only had a cursory glance) eve attacks this problem from the promising angle by making all data available in a ready-to-query database.

Still I think there must be some hard lower limits on amount of incidental complexity. Nature just can't allow you to get rid of all of it (impossibility results from distributed systems theory come to mind) just as in manufacturing transport costs can't be zero (goods and materials can't be transported between factories faster than the speed of light after all). It will be interesting to see how eve team works around these issues.

[agumonkey]

> But if I think of the things I do in my day-to-day work as a software developer it is 99% logistics (getting data in the right place and in the right form) and 1% related to actual meaningful transformation. If my understanding is correct (I only had a cursory glance) eve attacks this problem from the promising angle by making all data available in a ready-to-query database.

Reminds me of the old:

"It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures." —Alan Perlis

[gcv]

> Is it hard essentially, or is it hard accidentally?

Not exactly either, but certainly a bit of both. Programming is hard primarily because it is so poorly understood. The entire field is in its infancy. Comparing it to art, I'm pretty sure we haven't even reached the "stick figures scrawled on a cave wall" stage yet. As Alan Kay pointed out we sure didn't invent an arch yet: http://squab.no-ip.com/collab/uploads/61/IsSoftwareEngineeri...

It's nice to see the Eve team trying to do something at least slightly different from the same-old, same-old. Even if it looks a lot like some horrors of yore (FoxPro) when I squint.

[bunderbunder]

I think a lot of it is essential. At its core, programming is about designing processes and procedures, and I'm not aware of any sphere in which the state of the art inspires much hope. Certainly not law, business or accounting.

But I also think there's a lot of value in continuing to try new things. This complexity is so expensive that the payoff from a successful project could be enormous. And even when projects fail, we can often learn a lot from understanding how they fail. See that recent HN discussion on literate programming for an interesting example.

[chipsy]

I've also concluded that it's essential: Programming is the solution for the broadest space of problems in computing.

If you can narrow the problem space to some finite number of goals and workflows, you have a viable application. Application environments with sufficiently many workflows always reincorporate programming as a way to let the user build their own solution - and most things are implicitly programmable, whether through a defined API or through tricky reverse-engineered methods.

What I think muddies the picture is the line between "design" workflows and "engineering" workflows - in the first, you're piecing together the existing technology in a different way, while in the second, you are transferring math and science knowledge into original technology. As individuals we experience personal bias as to which side of programming is more "necessary," which is reflected in the resulting choices of tooling, code style, and preferred problem domains.

Sometimes you want design-heavy programming - e.g., you add some business logic and a UI on top of a database. Other times you want to add engineering to an existing design - you're writing hardware drivers using a common protocol, a data formatting plugin for an application, etc.

Library code acts as a way to expose units of engineering, while a framework defines a broad, but still configurable design space. Sometimes you have overlapping design spaces - you can have client code that works with a GUI framework, but also talks to an internal model and remote data sources.

One of the things that is exciting about programming's evolution is how much it is based on an ecosystem of technologies. Outside of some embedded fields, the era where you are given a hardware manual and are told to come up with your own development environment is over. Successful technologies tend to act parasitically on prior ones. This leads to a lot of compromises, but the general direction remains toward "better fit."

[gavinpc]

They are not "starting from scratch" if they had such a focus on past work and research. I hope they publish something on that.

[jamii]

https://github.com/witheve/Eve/blob/dev/design/bibliography....

http://incidentalcomplexity.com/

:)

[webjprgm]

Yeah. They said for doing simple things rather than building complex systems the current tools are not very convenient.

I find it really useful to use Ruby's built-in CSV library to process data for spreadsheets then visualize it with a graph in Numbers. I pull data from copy-paste tables, extract from Sqlite databases embedded in applications, etc. But for something like scanning Facebook friends as they suggested I'd have to first figure out the API or how to scrape the data. Once I figured that out it probably wouldn't be too hard to write a script for it. I could probably even put it in a crontab (LaunchAgent plist on a Mac).

Anyway, I think there's a huge opportunity to empower more people who are inquisitive and maybe somewhat technical but are not programmers. Make it easy for these people to solve problems rather than trying to teach them to "program". The end goal in my mind is the StarTrek programming model where you discuss with the computer what you want to do to solve a problem. Those crew members are scientists and engineers but only a few of them are "programmers". (StarTrek is a useful yardstick because we seem to like copying technology out of it. E.g. StarTrek communicator which is the late 90s cell phone and StarTrek PADD which is the SmartPhone/tablet.)

[andy_ppp]

And you end up with Moriarty taking over the ship? ;-)

[jaredklewis]

Perhaps, but my intuition is that the bottleneck between most people and programming isn't the tools. It's that programming requires the programmer to conceptualize the problem formally, and then to formally define what he/she wants the computer to do.

We can come up with better and better formal languages and ways for editing these languages (and probably should), but I would guess even a visual formal language, will still pose many of the same difficulties for users that existing formal languages do.