[Xenoamorphous]

> Scientists are actually preoccupied with accomplishment. So they are focused on whether they can do something. They never stop to ask if they should do something.

From Crichton's book Jurassic Park, which like most of his books is about the perils of technological advancements.

They used the quote in the movie, slightly tweaked.

[throw4847285]

I would replace the word scientists with engineers in that quote. People often conflate the two, but in my experience, scientists tend to be more cautious and there are built in checks and balances in the process (however flawed).

Engineers/technologists tend to have no such guardrails, and are also usually embedded into entirely profit motivated environments, whatever their own values might be.

[velcrovan]

This sounds like you don’t have much exposure to actual professional engineering disciplines. I’m sure civil, electrical, structural and mechanical PEs would be quite surprised to hear there are no guardrails on their professions.

[mulr00ney]

Engineers versus "engineers".

I have fond memories of a boss who was an actual, licensed engineer while the rest of us were very much normal software devs. Boss was pretty chill except when someone someone suggested we should be called "engineers" rather than "developers", at which point they said "if you guys were building bridges, people would be dead." (I don't think all software needs to be built to rigorous engineering standards but man... I think about that line a lot.)

[rectang]

I prefer the term "software developer" and that's what I use when I don't need the prestige of the term "software engineer". It's disadvantageous for organizations to do that with actual job titles, though.

Absent US government intervention to codify the term "engineer", probably the only way out of the "engineer" trap is through further title inflation, where the developers all become "vice presidents". :)

[mulr00ney]

Yeah, it's 100% the better term. We've got rules against using engineer here in Canada though several companies I've worked for have called me an engineer. Apparently Professional Engineers Ontario sometimes goes after people for calling themselves engineers but I've never heard of it actually happening, and I don't know that they have any real teeth given that the places I worked that called me an engineer were Canadian-owned. (In fact, the only place where they checked if I could use the title was the one multi-national. Go figure.)

[jimbokun]

People meeting the definition of Software Engineer while having that title may be rare, but we certainly need more of them.

[bombcar]

In my experience “engineers” and builders are often quite “conservative” and really don’t like pushing the envelope, and they often only do it under protest.

The most famous example may be the perpetual war between architects and engineers/builders.

[fuzzfactor]

Researchers need to go wild and sometimes far off-the-rails to increase the odds of coming up with something that is both new, and potentially popular enough, if they want the option to attract marketers who can only thrive on mass-consumption.

With luck, one out of 100 inventions will show promise on those points.

There's always a lifetime wake where the overwhelming vast majority of the work remains undeployed no matter what. The more undeployed milestones and inventions that some scientists have under their belt, the more accomplished they often are whether anybody knows it or not.

OTOH, equally active engineers more often need to have most of their time engaged in actual deployment of some kind or another, otherwise not as much progress will be able to reach as many people that could benefit. So many times nothing would be accomplished without a long-term focused engineering effort once an objective has been identified. But it can be hard to stop a train when it's already coming off the drawing board at full steam.

It does seem a lot more likely for a judicious researcher to cast off some major progress in what could very well turn out to be an unsavory development, such as likely misuse, even if it could be marketed as the most popular thing they have so far. Just add it to the pile of other things that best remain undeployed. There's plenty more where that came from, and the best is yet to come.

Perhaps popularity alone is not always the best measure of progress.

[throw4847285]

I actually originally wrote "technologists" but thought that the word sounded kind of odd. Now I realize it better captures what I was trying to convey.

[Otterly99]

Honestly, scientists too. I did 10 years of research after deciding that it was not worth it. There is a vast amount of research with no direct application that gets published under the assumption that more knowledge is always better, but in my experience scientists rarely question the usefulness of their research (because most of the time, they find it interesting, which is motivating enough).

Granted, I am talking about harmless subjects, but there is also the dimension of resource usage that almost no scientist considers (the amount of plastics and chemicals used in biochemistry and cell biology is... concerning).

[tom2026hn]

I worry that companies like Anthropic, which are moving toward RSI, may prioritize speed over the timely identification of irreversible risks.

[sdenton4]

Every time I prioritize speed over risks, I, too, end up with repetitive strain injuries.

[dbalatero]

Did you mean AGI? I'd think whatever else they are doing, LLMs are reducing RSI...

[moogly]

Maybe. I am not convinced I type fewer keystrokes being a Markdown Monkey compared to writing code largely via autocomplete. Fewer Tab keystrokes, for sure.

[dbalatero]

I was mostly making a joke.

[FarmerPotato]

Yup. "I wonder how long the codebot will be combobulating over that prompt?" I'll just rest my hands for a sec..."

[ascorbic]

Recursive self-improvement

[jimbokun]

We need to force the executives and engineers working at Anthropic and other AI companies to read this encyclical and pass an exam demonstrating that they read it.