Hacker News new | ask | show | jobs
by Hasu 449 days ago
> But let me tell you…last month I sent several hundred million requests to AI, as a single developer, and got exactly what I needed

There are 2.6 million seconds in a month. You are claiming to have sent hundreds of requests per second to AI.
3 comments
iambateman 449 days ago
That's exactly what happened – I called the OpenAI API, using custom application code running on a server, a few hundred million times.

It is trivial for a server to send/receive 150 requests per second to the API.

This is what I mean by instincts...we're used to thinking of developers-pressing-keys as a fundamental bottleneck, and it still is to a point. But as soon as the tracks are laid for the AI to "work", things go from speed-of-human-thought to speed-of-light.

link
hadlock 449 days ago
A lot of people are feeding all the email and slack messages for entire companies through AI to classify sentiment (positive, negative, neutral etc), or summarize it for natural language search using a specific dictionary. You can process each message multiple ways for all sorts of things, or classify images. There's a lot of uses for the smaller cheaper faster llms
link
crazygringo 449 days ago
Yeah I'm curious now.

If you have a lot of GPU's and you're doing massive text processing like spam detection for hundreds of thousands of users, sure.

But "as a single developer", "value for me and my team"... I'm confused.

link
iambateman 449 days ago
I'm NDA'ed on the specifics, sorry.

In general terms, we had to call the OpenAI API X00,000,000 times for a large-scale data processing task. We ended up with about 2,000,000 records in a database, using data created, classified, and cleaned by the AI.

There were multiple steps involved, so each individual record was the result of many round trips between the AI and the server, and not all calls are 1-to-1 with a record.

None of this is rocket science, and I think any average developer could pull off a similar task given enough time...but I was the only developer involved in the process.

The end product is being sold to companies who benefit from the data we produced, hence "value for me and the team."

The real point is that generative AI can, under the right circumstances, create absurd amounts of "productivity" that wouldn't have been possible otherwise.

link