[jasobake]

As someone who works at big tech and spends countless hours in meetings hoping to get some small feature coordinated for deployment across two teams, I can't imagine the amount of meetings and 6-pagers that were involved in running these models on bedrock's hardware.

[33MHz-i486]

at this level they just decide and spin up a swat team to execute it in a couple weeks without politicking. the bureaucratic ways, reviews are just for the low levels, to keep them busy with feature scraps while they mostly do operations

[aab99]

yup, I think there are few public articles on aws mantle so you can look it up, but internally this is pretty common knowledge. The entire inference engine of bedrock is built and maintained by a handful of ec2 engineers (all principals and above). Judging by the commit history of the project they are able to just build independent of any of the traditional bureaucracy.

[33MHz-i486]

the way in which Mantle was built is highlighted internally by PEs as some sort of triumph but really its a fairly tone deaf indicted of AWS’s engineering culture ... “To achieve a meaningful result in a reasonable amount of time we had to break nearly ever constraint that we force all other engineers to work under. good luck to you plebs of L6 and below”

[darkwater]

I'm going to play devil's advocate here: _what if_ most of lower level engineers are actually not able to self-organize themselves like this dedicated, I bet hand-picked group of PE can? I'm pretty sure AWS ruthless culture would gladly use less middle-managers and be swifter in time to market if it were so easy, no? What works for a single, highly-focused project (or a handful of similar situations) doesn't scale when you have to take care of bazillion customers, do boring/smaller tasks and keep the machine ticking.

[coredog64]

Having seen how the sausage is made, Amazon's internal engineering culture is a textbook example of the principal/agent problem. You only get a raise from a promo, and you only get a promo from scope. It's way easier for management to demonstrate scope and so those folks build out empires of crap. In the best case they're just consuming oxygen, but I have definitely seen orgs created that were a negative influence on productivity.

[darkwater]

Never worked for BigTech, so I'mm really not in my territory, but I wonder what is the purpose of all those interviews to get "the best of the best" and then let these things happen. What are the incentives for the management layer?

[jjmarr]

[deleted]

[spelunker]

Check out this all of this stuff we can build with a room full of PEs and no rules!

[avereveard]

For better and worse tho.

[o10449366]

Lol, spinning up swat teams because someone high up decides "drop everything this is my pet priority now" is politicking. It looks good for the leaders, meanwhile its the engineers pulling the all nighters and dealing with having to maintain systems that are operationally compromised from day 0 because there's no proper planning/scoping involved other than "Big Man says this needs to be done in 2 weeks"

[tt24]

Okay but I’ll be able to use OpenAI models on bedrock now

Why would I care if AWS asks their engineers to work a little harder on a project

[phillipcarter]

Sure, but also...

...anyone with a brain at AWS knows that supporting OpenAI's latest models on Bedrock is simply good for AWS. That context is rather important!

[ignoramous]

> engineers pulling the all nighters

There's always some carrot with the stick, even if an imaginary one!

[giancarlostoro]

Depends on how its implemented, but Amazon already did add gpt-oss-20b so if the model is similar enough to the OSS variant of GPT, it might not have been as complicated as you might think.

[londons_explore]

I imagine there's lots of custom kernels and optimization...

Openai hasn't been publishing innovations for quite a while.

[NamlchakKhandro]

Neither has anthropic.

They're both just stealing ideas from pimono extensions

[ignoramous]

Some say they have been stealing ideas from Chaebols [0] on circular deals [1], government reciprocation, & monopolisation.

[0] https://business.columbia.edu/sites/default/files-efs/imce-u...

[1] https://www.reuters.com/business/autos-transportation/compan...