[minimaxir]

And what are those hallmarks in this case? There is no heuristic that is applicable for every programming language.

Unfortunately, ratatui requires a lot of verbose code that may be indistinguishable from LLM generated code: https://ratatui.rs/examples/apps/demo/

[joshka]

https://ratatui.rs/examples/apps/demo/ is pretty much the oldest untouched remnant of code from tui-rs days (pre-ratatui fork in Feb 2023).

Ratatui itself has a lot of much nicer AI generated code in it since then ;)

We've also done a bunch of things to help drive down some of the boilerplate (not all of it mind you - as it's a library, not a framework like other TUI libs)

[acedTrex]

Easiest one is ridiculous comments, always a dead ringer for an LLM.

ESPECIALLY when its from a plan and comments '// STEP 2: ...'

Like here in this posts repo https://github.com/huseyinbabal/taws/blob/2ce4e24797f7f32a52...

This a dead ringer for LLM slop that someone didnt even care enough to go through and clean up.

Edit: Just to keep it going, you ever seen a human write dependencies like this for a small toy tui? https://github.com/huseyinbabal/taws/blob/2ce4e24797f7f32a52...

Edit2: Sure why not, lets reimplement randomly a custom jsonpath when serde_json_path exists https://github.com/huseyinbabal/taws/blob/2ce4e24797f7f32a52...

[viraptor]

> Easiest one is ridiculous comments, always a dead ringer for an LLM.

> ESPECIALLY when its from a plan and comments '// STEP 2: ...'

There are people who actually program that way. The most extreme I know was Bogdan Iancu from OpenSIPS who I've seen create functions, write step-by-step comments for what they will do, then fill out the implementation.

It's just a signal, not a certain thing.

[minimaxir]

That's not a smoking gun. I've definitely seen pre-2023 open source code mentioning steps in their comments. Even though that file also has a lot of tautological comments which are more indicative of LLM coding, it's not a smoking gun either: the frequency of comments is an editorial decision which has pros and cons.

It's the equivalent of calling something an AI generated images just because the fingers are weird, and requires a judgment more concrete than "I have eyes."

> you ever seen a human write dependencies like this for a small toy tui?

Yes? That's just TOML syntax. I'm not sure which dependency in that list is excessive, especially for something that has to handle HTTP requests. If you mean adding a comment header for each section, then that's a better argument, but see argument above.

[mahogany]

The more you see and review LLM-generated code, the more you can detect its fingerprints. Obviously you're not going to prove this is LLM-generated. I wouldn't bet $1M that it is. This could be 100% human made.

But read the same link from above: https://github.com/huseyinbabal/taws/blob/2ce4e24797f7f32a52.... LLMs leave temporal comments like "// Now do X", or "// Do X using the new Y", as responses to prompts like "Can you do X with Y instead?".

or below: "// Auto-refresh every 5 seconds (only in Normal mode)". I would guess this comment was during a response to a prompt like: "can you only auto-refresh in Normal mode?"

Sometimes there are tautological comments and sometimes not: https://github.com/huseyinbabal/taws/blob/2ce4e24797f7f32a52...

``` // Get log file path

let log_path = get_log_path(); ```

This is another signal to me that there is less human influence over the project.

No, none of these are a smoking gun. Also none of this means it was completely vibe coded. To me personally, the worrying part is that these patterns signal that perhaps human eyes were never on that section of the code, or at least the code was not considered carefully. For a toy app, who cares? For something that ingests your AWS creds, it's more of a red flag.

Edit: changed the language a bit to sound less sardonic. My comment is more about LLM signals than a judgment on LLM usage.

[true_religion]

I recently had the pleasure of reviewing some of my oldest production code from when I had first left college.

It worked, no issue there, but the amount of commentary I included definitely surprised me.

I guess I really needed the support structure of comments to keep my logic on track back then, whereas now even convoluted map-reduce one liners are things I see as just obvious literate programming.

I did go a long while in my career still writing code that way when I had to share it with people. I don’t think I stopped until the only people reading my code were senior engineers with way more qualifications than I had.

So, I wouldn’t say just from this code that the creator is an LLM.

[acedTrex]

I mean if you want bury your head in the sand and try to play semantics over if its ENOUGH proof or not go right ahead.

But it's more than LLM enough for anyone who has experience with them to conclude the LLM drove the majority of the output. Hence, slop

[minimaxir]

To be clear, I agree that there was likely AI assistance with the code (as it will be a safe assumption going forward in SWE given the progress agentic LLMs have achieved in the past few months), but a) that doesn't intrinsically mean it's slop and b) the "all AI is slop" bit is intellectually lazy at best especially without objective evidence and doesn't lead to constructive HN discussion.

[acedTrex]

> a) that doesn't intrinsically mean it's slop

If it can quickly and easily be identified as LLM code then yes, it is intrinsically slop and of no value. The person who submitted it did not even value it enough to look at/clean it up. Why would anyone else care to look at it.

If it is LLM generated but then HAS BEEN cleaned up then you cant immediately see the LLM and it passes the check anyways.