Hacker News new | ask | show | jobs
by drewcrawford 5177 days ago
It's not as if Python or Ruby have set out to be deliberately difficult. They've grown difficulty because they're used to solve real problems, and real problems are hard.

Implied inside your question is the idea that we can, in advance, choose the right level of abstraction. That we have a library / language / set of verbs that is neither too high level (like Automator) nor too low-level (like your bash script example) to solve the problem. While I think we can improve a little bit on the set of abstractions that we have, I think a broad solution is naive.

As a point example, I used to be frustrated that so-called 3D engines tended to grow until they included the kitchen sink. Why do I need a menu system as part of my 3D engine? Why do I need a sound engine? Can't I just use system libraries like a sane person? etc. All I want is a set of primitives that let me render models to the screen. How hard can it be?

It turns out that the problem of rendering graphics to a screen quickly is so hard that it actually infects the hardness of other problems. Doing elastic collisions is a trivial thing that any first-year physics student can do. Until, that is, your game loop skips six frames because it's busy rendering graphics code, at which time you need to retroactively handle collisions that have occurred in the past. Now we're doing rocket science, and so your graphics library needs its own physics system, because you're not going to derive that stuff. And that, basically, is the story of every feature added to every serious game engine that exists today.

You see this problem everywhere you look, for example, at one point people thought POSIX and C were reasonable building blocks that everyone would use, and at another point people thought that about Java. At first, BSD sockets was the primitive, now we're starting to think about HTTP connections as the primitive, maybe we should work to make REST requests in Python not suck so damn much? But at this point SPDY looks like it might be turning into a thing, so maybe we should just build primitives for that instead? If you look at this staggering tower of abstractions you start to see things that seem crazy, like SPDY including its own flow control even though TCP already has it, because TCP has some bad assumptions that should be really be fixed at the network level, but will never be fixed there as a practical matter. I hope you followed Inception, because if not you're gonna have a bad time.

I'm doing a bad job of explaining this, but I had this insight first after reading an article by Kamp, the author of Varnish: https://www.varnish-cache.org/trac/wiki/ArchitectNotes. He makes the argument with a specific example that as we have moved to higher-level primitives, we've built an overly simplistic mental model of how the lower-level ones like OS and hardware work. As a result, we implement our own caching code, because we've forgotten that OSes were designed to do that for us. So in a sense we have more power because we have this nice set of high-level primitives, but in practice we've forgotten all the things that those primitives wrap, we've started to believe that they are simple because they look simple. And so, we muddy up our application code with all this stuff that has already been solved.

I doubt very much that there is any software program that people actually use that is not either undergoing a big refactor right now or has one scheduled, either to make things "simpler" (less abstraction) or "more powerful" (more abstraction). Just thumbing through the projects that I work on, I am adding features to a mental backlog for a future refactor for every single one of them. This seems to be like a fundamental problem that all software projects have, and creating a new set of primitives and saying "no seriously, that should do the trick" is at best going to yield a nice little language like Python or Ruby that somebody will post articles about on HN saying why it's so hard for casual use.

4 comments

"It's not as if Python or Ruby have set out to be deliberately difficult. They've grown difficulty because they're used to solve real problems, and real problems are hard."

The problem of getting from the East coast to the West coast has not changed in 100 years (move body until done), yet when was the last time you had to set the spark and choke and hand-crank a car? Cars solve real problems, and today's cars are sophisticated enough that essentially anyone can use one. To solve real problems.

Certainly the problems solved by a car are in a more narrow domain than programming languages. But cars themselves are in a narrower domain than internal combustion engines.

Edit: stealth.

I'm sorry, but this analogy is awful. Getting from coast to coast certainly HAS changed in the last 100 years. 100 years ago, you had railroads or mostly dirt roads. A crap ton of infrastructure has been put into it (interstates and the intensely complex logistics of maintaining them). A pile of services exists around this problem (airlines, and the even more complex logistics of keeping them running safely and profitably). Going broader, you are ignoring all types of "get between coasts" that apply to stuff rather than "me" - what is the best way to get a pile of widgets from LA to NYC? Airplane? Train? Some trucks? -- the answer is "well that depends on $logistical_considerations". Going narrower, choosing the right car for you is a pretty complicated process, trading off features, options, gas mileage, capacity and so on - just like any set of programing options, only internalized by the culture".

Further, a car is simple to use out of the box, but customization beyond do-dads and bling? Nope, not really easier than it ever was. Different tools? sure. Different base skills? not really. Anyone can do it? Not so much.

And before I go into tech equivalencies here, if you think cars are pretty much the same and can be abstracted to "the car" for the purposes of this discussion, go buy a van or pick up truck. Watch your social graph get all buzzy and see the request pour in for help, turns out your form of tradeoff in "car" are very very useful to people, and suddenly a car isn't just a car.

Further, I would argue that we have done wonders for the tech equivalent of "car". We have amazing computation devices in our pocket that just do what people never knew they wanted to do in the form of "phone". We have awesome abilities to have web presence unthinkable (outside of internet famous blog machines) 10 years ago, in the form "social graph". These are great and just work for most people, until they need something different...

In the infrastructure department: the ability to do semi-custom stuff has grown in ways unimaginable in the car world. Need a website to do your magic? Here are a bunch of platforms like app engine or heroku, all you need to do is get your custom stuff in place, don't worry about the machinery to feed it. Up and down the customization level there are more entry points as well. Going back to the car world, how easy is it really to get a ricer put together? How easy is it to put in crazy hydrolics or a good thumping stereo system? Need to rip off the doors, reshape the body and put in a different engine? Good luck getting a doctor to just do that without serious learning curve.

Basically, my point here is that hard stuff is hard. If it isn't your specific field, it is easy to just blow off the stuff that caters to the masses as "they made it easy", and at the same time, it is easy to blow off the stuff in your field which caters to the masses as "not done well".

Final thought: a lot of people look at cars and computers the same way, as soon as the littlest thing goes wrong, they freak. or they take it to the mechanic who asks "why haven't you changed the oil in 7 years? (c.f. our IT guys who ask why they don't have anti-virus installed).

"Further, a car is simple to use out of the box"

Exactly my point. They didn't use to be simple. Crank, choke, throttle, water in the battery, where's the next fuel repository, where are my goggles, where is the rug for my legs, etc. Now they are, and you just think of moving your body from here to there, and almost not at all about how to make your car help you do that. Now people use a car casually.

It's that kind of change, only much more profound and difficult to achieve, that will be needed before people can "program" (if we still call it that) computers casually.

The difference is that it was easier in the past for people to program casually; it hasn't gotten easier. Consider the microcomputers of the 80's: it was very difficult to own one without knowing at least a bit of some BASIC dialect. Consider microcomputers now: most people get away without ever knowing what it's like to write a program. People who actually want to learn how to program must go out of their way to do so (and that's if they even realize programming is a possibility).

So, perhaps computers themselves have progressed much like cars. It is now possibly to use a computer casually. But programming per se is significantly less casual than it once was.

You used to have to type in 1s and 0s on a keyboard, or cut little square holes in a piece of cardboard with an Exacto knife. You had to learn what opcodes were represented by those 1s and 0s, in which contexts, and reverently hand your stack of cards full of holes to the Keeper of the Card Reader. You had to really, really want to program, with books open on the desk and your lap, and a grilled cheese sandwich holding open the one on your lap. I don't know if that's formal, but it sure doesn't sound like a casual drive around the block to me.

I wouldn't know an opcode from a gerund, and I don't want to know.

Now I can Write a Program for Dummies, or I can write Python the Hard Way, or I can <script>alert('hello world')</script>, all as close as https://encrypted.google.com/search?hl=en&q=how%20do%20I... while holding a sandwich in one hand. That's not as casual as the OP would like, but it's pretty darn casual compared to The Day.

Edit: vocabulary.

Just because it's not difficult doesn't mean it's casual. There's a higher barrier to entry now than there was in the 80s, and that makes programming much less accessible to the casual computer user. The average user perceives coding as a feat attainable by only wicked smart, socially inept nerds.
My point was that a car is an application. Facebook is also simple to use out of the box, and is far more "a car for the web" than a "transportation logistics engine for the web". There is NOT an intuitive interface out there for designing a car or a robust transportation logistics system.
But there is an interface for cars that has evolved into something that is intuitive, and all new cars are designed with that in mind.
> The problem of getting from the East coast to the West coast has not changed in 100 years

But computing problems _have changed_. Not necessarily in areas like problem size (although that has happened) but also in areas of usability and convenience.

A much more apt analogy would be: the problem of stopping at the grocery store after work is 50 years old, why can't I get an optimal set of directions that accounts for rush hour traffic in 2012? I can give you any number of reasons: we've built more roads, there are more drivers, travelling salesman turned out to be a hard problem, game theory is hard, etc. In some dimensions (cranking the car, proclivity of gas stations, etc.) the situation has improved, but solving some problems re-emphasize the ones we haven't solved.

Cars aren't used to cross the continent today, airplanes are. And most people have no idea how to use (pilot) an airplane, they pay someone else to do it for them.
For now.
They never will. Before that happens we'll have fully automated planes operated by -- guess what? -- software.

Software is not really comparable to cars because it's not a solution -- it's a tool to build solutions. Comparably, we have devised easier ways to drive nails into wood (nailguns instead of hammers), but nails still exist and have to be driven into wood. So I don't think software will ever really be casual, just like building construction will never really be casual. Just too many variables to take into consideration.

Certainly the problems solved by a car are in a more narrow domain than programming languages.

You do realize that this is like saying "certainly two times N is smaller than two to the Nth power"? ;)

The 3d-engine example is interesting, because I think there really have been huge improvements there in the past 5 years, through a mixture of UI and architecture. There are a wide range of "should be relatively simple" things that used to be unreasonably hard, but which with the advent of Unity3d are now much easier. Doesn't solve everything, but has given a several times improvement for a lot of low- to mid-hanging fruit.
I have started to look into Unity3D the past couple of weeks and I'm impressed. Deliver to Android/iPhone/PC/mac/xbox etc. I have never felt so close to make my dream game.
The languages are mostly fine (until some next-gen language surprises us all); I think the current problems lie generally within tooling and infrastructure, which in turn come from our mentality and the existing workflow that we're used to.

That said, I enjoyed reading your very insightful post :)

No, you did a very good job of explaining this. Just wanted to let you know.