[cylentwolf]

I asked a few of my friends that are ML engineers this question and all of them said to run the LLMs in the cloud with their infrastructure because it was going to be way faster. If you just want to tinker around I would look at @JSR_FDD's comment.

[throwawaytea]

Next you're going to tell me that car salesman recommend just leasing new cars, doctors recommend just following the standard of care, construction workers just recommend subbing it out, and your tech friends say just use AWS.

[ramraj07]

If your car salesman friend gives you stupid advice it either means he is stupid or he is not your friend.

[throwawaytea]

I sold cars for a decade and wouldn't even take the insanely cheap priced company demo car, just driving my 10 year old car instead.

On that note though, most car salesman are somewhat stupid, although the best ones are atleast normal. (I was a cal mechE grad and top 0.2% nationwide car salesman). They also slowly brainwash themselves into believing most of what they say.

[derwiki]

There are more factors than speed though, like: privacy