Hacker News new | ask | show | jobs
by atrettel 311 days ago
I am quite happy that this post argues in favor of subject-matter expertise. Until recently I worked at a national lab. I had many people (both leadership and colleagues) tell me that they need fewer if any subject-matter experts like myself because ML/AI can handle a lot of those tasks now. To that effect, lab leadership was directing most of the hiring (both internal and external) towards ML/AI positions.

I obviously think that we still need subject-matter experts. This article argues correctly that the "data generation process" (or as I call it, experimentation and sampling) requires "deep expertise" to guide it properly past current "bottlenecks".

I have often phrased this to colleagues this way. We are reaching a point where you cannot just throw more data at a problem (especially arbitrary data). We have to think about what data we intentionally use to make models. With the right sampling of information, we may be able to make better models more cheaply and faster. But again, that requires knowledge about what data to include and how to come up with a representative sample with enough "resolution" to resolve all of the nuances that the problem calls for. Again, that means that subject-matter expertise does matter.

3 comments

This all reminds me of this really interesting book "The Inevitable" by kevin kelly.

It had a fascinating look into the future, and in this case one insight in particular.

It basically said that in the future, answers would be cheap and plentiful, and questions would be valuable.

With AI I think this will become more true every day.

Maybe AI can answer anything, but won't we still need people to ask the right questions?

https://en.wikipedia.org/wiki/The_Inevitable_(book)

I agree that the ability to ask the right questions is a rare skill. I had a supervisor once with that ability. I tried to learn as much as I could about that from him.

That said, I think ultimately there are some questions that have no answers regardless of how we try to answer them. For chaotic systems, even small uncertainties in the inputs result in large differences in the outputs. In that sense, we can always ask questions, but our questions sometimes can never be precise enough to get meaningful answers. That statement is hard to wrap your head around without taking a course in chaos theory.

Aaron Swartz
> This all reminds me of this really interesting book "The Inevitable" by kevin kelly.

I'm fine with a bit of speculative fiction, but I prefer it to be less dystopian than "The Inevitable". Got any good solarpunk recommendations?

> I am quite happy that this post argues in favor of subject-matter expertise

The funny part is that it argues in favour of scientific expertise, but at the end it says they actually want to hire engineers instead.

I suppose scientists will tell you that has always been par for the course...

Without the actual SME's they'll be flying blind not knowing where the models get things wrong.

Hopefully nothing endangers people..