[insanitybit]

I just tried ChatGPT and it ran this code.

    from datetime import datetime, timedelta

    # Current date
    current_date = datetime(2023, 11, 1)
    
    # Calculate the date 140 days from now
    future_date = current_date + timedelta(days=140)
    future_date.strftime("%Y-%m-%d")

Result: '2024-03-20'

The ability to execute code is kinda insane for these models.

[block_dagger]

Here’s the python ChatGPT-4 produced and ran with the title prompt:

# Number of elephants total_elephants = 10

# Elephants with legs elephants_with_legs = total_elephants - 2

# Number of legs for an elephant legs_per_elephant = 4

# Total legs total_legs = elephants_with_legs * legs_per_elephant total_legs

Its actual response was correct.

[insanitybit]

Yeah, ChatGPT gave me the correct answer without even needing Python in the case of the title.

[edanm]

Interesting, I copied the text into gpt 4 and it did run python as well.

[londons_explore]

Remembering that the correct answer is 40 because 'legless' is slang for 'drunk'?

[TheCleric]

Is that regional? I’ve never heard that euphemism.

[londons_explore]

It is commonly used in British English. In fact, there is a famous comedy sketch about it that also uses the word 'caravaned'

[pvorb]

Calling a drunk person legless is hardly an euphemism.

[karateka]

It’s kind of funny that they can more reliably spit out code that will give an answer than actually output the answer as text. I guess it’s a workaround that works well for many cases

[bmicraft]

Humans can also more reliably use a calculator (which is basically what python is) for big numbers than doing it in their heads. I think it makes sense.

This reminds me, I've had an alias for calc='python -ic "from math import *' for a long time now. It comes handy more often than you'd think.

[fragmede]

bc

or octave if you need some real power

[runarberg]

Doesn’t that just mean it was trained with inputs and outputs from simple programs?

[insanitybit]

No, it's actually executing that Python code. This is what allows an LLM (or an 'llm based system', I guess) to do something like "reverse <some uuid that has never been observed before>" - it can't just memorize the output and map it to the input because the output has literally never been observed. Instead, if it knows the algorithm for reversing a string, it can just use that and offload the execution to Python.

[runarberg]

That is even less impressive. I was thinking—like normal linear models—it would be capable of interpolation.

[throwup238]

It is. It doesn’t even need an existing language. You can define your own psuedo language in the prompt and have ChatGPT “execute” it (works best with 4 nonturbo).

You can even combine your pseudo language with natural language. See the OP’s custom GPT and the comments here: https://news.ycombinator.com/item?id=38594521

[insanitybit]

That looks totally different. In the case of the Python code it is literally executing it by calling out to CPython.

[no_identd]

…got a source for that claim?

[insanitybit]

Why is it less impressive?

[runarberg]

I would say creating a model which is able to interpolate from training data in a way which produces an accurate output of a new input is a little impressive (if only as a neat party trick), however anybody can run a python interpreter on a server somewhere.

I’m sure there are use cases for this. But in the end it is only a simple feature added onto a—sometimes—marginally related service.

[insanitybit]

Hm, I don't think of it that way I guess. What the LLM is doing is generalizing a problem based on previous problems it has seen and then offloading the execution of that problem to a machine with some defined, specific semantics.

This is a lot more than a party trick. The model is able to describe the program it wants to execute and now it can accurately execute that - that it 'offloads' the work to a specialized program seems fine to me.

It's way more than a simple feature, this is enabling it to overcome one of the biggest limitations and criticisms of LLMs - it can answer questions it has never seen before.

[nomel]

There a code interpreter built in: https://www.pluralsight.com/resources/blog/data/chatgpt-code...

[amelius]

So it only works if the code is 100% correct. Which according to my experience doesn't happen often.

[nomel]

Sure, don’t expect perfection, but it still works great.

[insanitybit]

Most of the time it's just doing really simple stuff like this.

[leereeves]

It even correctly adjusted for the leap year.

But is that actually output from a LLM, or is ChatGPT just running the Python code through a Python interpreter?

[tim333]

Interpreter.

[nurettin]

Still not worth $20/mo