[luu]

I have no idea how to answer this question. There have been some problems where someone was stuck for weeks, and I came in and coded up a solution in a day. That seems, in some sense, good evidence of being hard, but those problems never seem hard to me. The reverse happens to me too, where someone else solves a problem that was hard for me, and it's easy to them. Is that a hard problem? It was hard to me, but the problem was easy if I saw it the right way, which I didn't.

Last year, I came up with a solution to a problem that we'd been solving sub-optimally for years. My solution is arguably optimal (given a certain set of assumptions) and requires multiple orders of magnitude less code than the previous solution. The solution is the core part of a paper that was recently accepted to a top conference in its field. That sounds like it might be good evidence the problem was a hard problem, but in fact the solution just involved writing down a formula that anyone who was exposed to probability in high school could have written down, if it had occurred to them that the problem could be phrased as a probability problem (that is, the solution involved multiplying a few probabilities and then putting that in a loop). When I described the idea to a co-worker (that you could calculate the exact probability of something, given some mildly unrealistic but not completely bogus assumptions), he immediately worked out the exact same solution. It wasn't an objectively hard problem because it's a problem many people could solve if you posed the problem to them. The hardest part was probably having the time to look at it instead of looking at some other part of the system, which isn't a hard "technical problem".

Another kind of problem I've solved is doing months of boring work in order to produce something that's (IMO) pretty useful. No individual problem is hard. It's arguably hard to do tedious work day in and day out for months at a time, but I don't think people would call that "technically hard".

I'm with you. Maybe I never work on "REAL hard things"? How would I tell?

[sheetjs]

It's all relative. Sometimes solutions require insights. Other times solutions involve a ton of grinding. Many people have a tendency to overemphasize the first and greatly underemphasize the second, even though the grinding may actually be harder than devising clever solutions.

We build tools that read and write Excel files (open source library: https://github.com/sheetjs/js-xlsx) There are plenty of very difficult problems involving ill-specified aspects of the various file formats and errors in specifications, but it is largely a matter of grinding and finding files in the wild that capture the behavior you want to understand. Those are "difficult" in the sense that people still get these things wrong (related: recently a bug in the Oracle SmartView corrupted US Census XLS exports, which boiled down to an issue in calculating string lengths with special characters) but they don't feel difficult since most of the work didn't involve any really clever insights.

IMHO the hardest problem is now fairly straightforward: How do you enable people to test against confidential files? The solution involves running the entire process in the web browser using FileReader API: https://developer.mozilla.org/en-US/docs/Web/API/FileReader , and that is an obvious technical solution in 2017 but few thought it was even possible when we started.

[aidos]

That's actually my default response when people intimate the work I do must be complex / I must be clever - it's really just hard graft. Sometimes you just have to be willing / stubborn enough to chip away at a problem that initially seems insurmountable. Sure, the more knowledge you accumulate, the faster you can figure out when to look, but often enough you just need to roll up your sleeves and bisect your search space.

I can imagine you get some pretty warty excel files. I mostly get PDF for my sins. I'm sure, like me, you've spent hours taking bits out of files until they work as expected and then figuring out what the difference is :-)

[LrnByTeach]

I totally agree with these gems ...

> It's all relative. Sometimes solutions require insights. Other times solutions involve a ton of grinding.

> Many people have a tendency to overemphasize the first and greatly under emphasize the second, even though the grinding may actually be harder than devising clever solutions.

> No individual problem is hard. It's arguably hard to do tedious work day in and day out for months at a time, but I don't think people would call that "technically hard".

[sigmaml]

The incidents you describe are examples of what Alan Kay means when he says that 'outlook' is more important than 'IQ'.

[newUser_istaken]

You left us hanging without linking to your paper. I would be really interested in it.

[jasonlotito]

You are over analyzing it. The question isn't about defining what a hard problem is. It's about sharing how you overcame problems. And the interviewer is looking for stories that demonstrate how you overcame those problems. Times when you lacked the necessary skill to achieve something.

Not being able to answer this question is just a telling as an answer itself.