[psi75]

This, 100%. That said, most data scientists don't do what you would consider real work (meaning, I assume, interesting work with significant mathematical/analytical meat). There just isn't a lot that's both interesting and useful to private-sector rent-seekers whose opinions of your work determine whether or not you advance.

Most of the people doing real ML in industry are prestige hires--they're hired because their names draw people in, but basically get to work on whatever they want--and you need a top-10 PhD at an absolute minimum to be eligible for those.

The ugly truth about industry is that 99.9997% of it is flow capture based on power relationships, found artifacts (i.e., corruption opportunities) within the state, and the implementation of very simple processes but in a way such that the threat to executive reputations as a first priority, and profit as an important second one, are minimized. This doesn't exactly make a market for ML innovation, unless your boss for some weird reason still cares about being a co-author on your papers (which his bosses will pressure him not to let you publish, because after all, this publishing is a distraction from your paid work).

On the other hand, if you want to be able to afford a house in the Bay Area, and to be tapped for (indeed, most likely forced into, both due to losing interest in and being unhireable for IC work) management in your mid-30s... then go for industry. The poison carrot will make you sick but it will kill you more slowly than poverty, so that ain't so bad, now is it?

[urthor]

Strongly disagree.

There's a vast amount of work that doesn't involve unethical recommendation systems.

Expand your horizon outside the Bay Area.

The plurality of work I see is straightforward computer vision/NLP applications.

[natch]

I suspect the work you're talking about could be easily handled by an intern working with Core ML and a MacBook.

The landscape is varied. There are companies doing real actual big leading edge stuff, there are companies where ML is sprinkled onto projects as a buzzword but no real interesting work happens, and companies that just need a practical small solution like the ones you mentioned, and could get by with Core ML, but don't because they hire a PhD who isn't aware of Core ML.

[carom]

What does productionizing coreml look like if I wanted to stand a model up as an rpc service?

[cassepipe]

I really like your style. I think you should write a book (this is not sarcasm)

[mrcartmeneses]

I’ve kinda developed the view that large organisations come to mirror the Russian Communist Party.

I’m interested in “flow capture based on power relationships”.

Do you have any recommended reading on this?

[jokethrowaway]

Every large organisation tends to be like a small government. Inefficient, drown in politics and unable to change.

There are exceptions - where someone principled dictator impose a VC style model where teams basically become independent startup and die or succeed. 100 fails, one becomes the next revenue maker for the company. That's how AWS was born.

[mrcartmeneses]

There’s way less inefficiency and way more accountability in the public sector. Look at how efficient publicly funded schools are, for example, or publicly funded rail or healthcare. You could literally pick almost any industry.

Accountability comes from elections. If managers in companies had to be re-elected it would be interesting.

[psi75]

This. Corporations are great at imposing mean-spirited personal accountability (i.e., if you're perceived to have fucked up, you get fucked) but that doesn't actually solve problems or change anything. People get fired, careers end, new faces replace the old, nothing gets learned. Of course, once you get into middle management you're exempted from the stack-ranking bukkake, and executives write their own performance reviews and almost never face consequences for their actions.

Companies are fantastic at making it look like accountability exists, because people at the bottom get punished for even the smallest mistakes, but avoiding any consequences that would affect high-ranking members or force the organization to change how it does business.

[Kaapeine]

That sounds a lot like the DARPA model too.

[badpun]

> I’ve kinda developed the view that large organisations come to mirror the Russian Communist Party.

Only the ones which have an unkillable cash cow. So, I suspect Google or large banks are mostly like that, but places like SpaceX or even large consulting firms (Delloitte, IBM etc., where managers essentially eat what they kill) cannot allow themselves to degenerate into a Chinese court.

[bsenftner]

Now this is interesting. I've always found it fascinating that when profit is on the table, democracy is nowhere to be found. I've looked, not too hard TBH, for essays and literature discussing the correlations to business model management structures and government/nation political hierarchies - not education level (propaganda), but critical analysis. I've been an employee of several of the top corporations on our planet, and the idea that corruption is not rampant is a farce. One simply lives within the environmental constraints and leaves when it gets to be too much. Does caring about corporate (and the larger realm of ethics) cast one incompatible with a modern corporate hierarchy?

[psi75]

Unfortunately, the only way to prevent hierarchy is to create a limited hierarchy (this is the purpose of constitutions) a priori; hierarchically naive organizations fail on this account. External parties will demand hierarchy simply because they want to know your organization (or nation) isn't wasting their time--no one wants to deliver a sales pitch to people who can't authorize purchases. If they're not careful, a group of people can end up in a state where the necessary-for-external-relations hierarchy becomes a total one. You see this with startup founders; the one who talks to the investors the most ends up in charge, and the ones who deal with employees or low-status counterparties lose power. This is why "flat" organizations can't really work; people who need things from the organization demand to know who to talk to in order to actually get things done, and eventually those "who to talk to" people end up with informal, then formal, power and it's very difficult to get them to give it back.

The large-scale failure of democracy that's happening all over the world is something different, though. Regulation is struggling to keep up with technology, and it doesn't help that nation-states have already been doing a piss-poor job of protecting people from their employers. If the US falls in the next 20 years, it won't be due to Covid or Trump or nation-level adversaries; it'll be due to the obscene power given to employers, who can literally ruin an employee's life--not just fire him, but anally ravage him in perpetuity with bad references--for any reason or none. Eventually, unless national governments start dropping serious lead pipe on employers' heads, people are going to tire of paying 30+ percent of their incomes to a government that lets bosses get away with this shit.

[mrcartmeneses]

Interesting take. In British history the first positive step towards freedom that I note was the creation of law courts. These gave surfs some power over their lords and provided some level of fairness rather than everything being about favour.

The US does seem to lag the UK and Europe in terms of employment law in some cases (no formal employment contracts for most employees, can be fired without notice, little statutory holiday, maternity or paternity leave entitlement, etc. etc.)

It has been argued that the union movement — while susceptible to corruption — was a hugely positive force in economic and political terms for American workers. Unfortunately thatcher and Reagan saw this as such a threat that they attempted to destroy their own manufacturing base in order to smash the unions.

[badpun]

> If the US falls in the next 20 years, it won't be due to Covid or Trump or nation-level adversaries; it'll be due to the obscene power given to employers, who can literally ruin an employee's life--not just fire him, but anally ravage him in perpetuity with bad references--for any reason or none.

How do you define "US falls"?

>Eventually, unless national governments start dropping serious lead pipe on employers' heads, people are going to tire of paying 30+ percent of their incomes to a government that lets bosses get away with this shit.

People endured much worse in medieval times, and endure much worse right now in China.

[bsenftner]

> People endured much worse in medieval times, and endure much worse right now in China.

Really? The USA has hollowed out portions of the country equal/worse than the worst 3rd world countries.

[mrcartmeneses]

Interesting you say that because there’s some critical analysis I’ve come across in the past that states that taxation is a key driver towards democracy and that captured wealth (such as from oil) promotes oligarchy and dictatorship. There are outliers in either direction, naturally.