[alanctgardner2]

In the next year I can imagine most people deploying Spark will be doing it on Hadoop, since Cloudera 5 will support Spark. It's a natural fit, most people don't hate HDFS but their use case doesn't naturally fit the MR programming model.

[dundun]

We've been using it on HDP2 for about a month now. Everything works fairly well and it was super easy to setup because of YARN (and the work the Spark team put in).

[gclaramunt]

yeah, I mostly complain about trying to fit everything to the Map/Reduce model

[TallGuyShort]

The "Hadoop ecosystem" is getting less and less about MapReduce, and more about other execution frameworks that can share HDFS with it.

[alanctgardner2]

You don't happen to work at Cloudera, do you? I noticed you have some submissions about Impala and Oracle being evil, which seems to be a pretty common view among the ex-Oracle DBAs there

[TallGuyShort]

I do happen to work at Cloudera (hence the Impala submissions), although I'm neither an ex-Oracle DBA nor a huge believer that they're evil. I really don't have a lot of first-hand experience with Oracle as a company - which as you'll see, is why my submission was actually a question about the community's perception and why that's a common view.

[bcbrown]

Hey, I've recently begun the interview process at Cloudera, would you mind sending me an email? My address is in my profile, and I'd love to ask you a couple questions.

[enigmo]

I'd like to see other distributed filesystems catch on too. HDFS has a lot of room for improvement.

[monstrado]

Which areas would you say HDFS needs improvement the most? Just so you know, HDFS is still very actively developed, and keeps introducing features / improving functionality (e.g Native NFS, In-Memory Caching, Short Circuit Reads, High Availability, Namespace Federation, etc) on a pretty regular basis.

Feel free to suggest new features, or contribute to the project yourself.

https://issues.apache.org/jira/browse/HDFS

[enigmo]

I know of the JIRA, thanks. I'm aware that it is being actively developed, but I don't necessarily believe that activity is progress. Keeping a few different players in the mix helps keep everyone focused on progress.