[benreesman]

Indiscriminate scraping is a dick move.

But if you’re going to do it, do it properly. I would have hung it off the Like button with an ungodly ZooKeeper ensemble and trained a GBDT on which parts of which URLs I could just obliterate with Proxygen.

We’d have it all in about 4 days. Don’t ask me how I know.

The second worse thing about the AI megacorps after being evil is being staffed by people who use Cursor.

Edit: on the back of the valued feedback of a valued commenter I’d like to acknowledge that I made a sloppy mistake and have corrected in haste, making no excuses. It would be super great if the largest private institutions in history of the world took the care with give or take everything that I do with trolling on a forum.

[vasco]

> But if you’re going to do it, do it proerly

Top shelf unintentional irony.

[Rinzler89]

Exactly. That line of reasoning just feels like established players kicking the ladder from under them in order to maintain their moat, when competitors start to catch up: "Hey, web scraping and data mining should only be allowed the right way, where the right way = our way."

"Free market" to them = the market where they get to write the rulebook.

[earth-adventure]

Pretty sure he meant the typo.

[vasco]

Exactly.

[viraptor]

> is being staffed by people who use Cursor.

Any specific reason?

[trunch]

Not OP but other than what core functionality they can demo to investors, every AI company seems to have extremely lacking:

- web design (basic features take years to implement, and when done break the website on mobile)

- UI/UX patterns (cookie cutter component library elements forced into every interface without any tailoring to suit how the product is actually used, also makes a Series C venture indistinguishable from something setup in a weekend)

- backend design (turns out they've been hemorrhaging money on serverless Vercel function calling instead of using Lambda and spending a minute implementing caching for repeat requests)

- developer docs (even when crucial to business model, often seems AI generated, incomplete, incoherent)

And this usually comes from hiring much less developers than is needed, and those that are hired are 10x Cursor/GPT developers which trust it to have done a comprehensive job at what seems like a functional interface on the surface, and have little frame of reference or training for what constitutes good design in any of these aspects.

[benreesman]

dawg I ChatGPT’d that license, busy building rn.

[benreesman]

I was the guy trolling, downvote me.

Don’t downvote the person who submitted a substantial comment far more valuable than it’s GP.

[raverbashing]

> (turns out they've been hemorrhaging money on serverless Vercel function calling instead of using Lambda and spending a minute implementing caching for repeat requests)

Oh but why can't the AI do basic backend programming anymore? /s

[benreesman]

Plenty of smart people use Cursor I shouldn’t have been dismissive.

I meant people who don’t work at Cursor.

[sshine]

> people who use Cursor

What's Cursor?

[ro_bit]

AI based code editing tool (https://www.cursor.com/)