[olooney]

I tried your prompt 5 times in a row using the Try Again button. The results:

    x = 10 and y = 5 (adding equations together)
    x = 10/33 and y = 15/11 (substitution method)
    x = 5.33 and y = 2 (substitution method)
    x =5 and y = 2.5 (algebraic manipulation)
    x = 5 and y = 5 (elimination method)

However, when I asked it to write a python program to solve the problem, it did much better:

    Write a Python program using numpy to solve the following problem: suppose
    we have two unknown quantities x and y. If three time x plus two times y is
    twenty, and 2 times x plus three time y is ten, what are x and y?

It produced the following program:

    import numpy as np

    # Define the matrix of coefficients
    A = np.array([[3, 2], [2, 3]])

    # Define the vector of constants
    b = np.array([20, 10])

    # Solve the system of equations
    x = np.linalg.solve(A, b)

    print(x)

Which is basically correct. (The only nitpick I can see is that `linalg.solve` will return a vector containing both x and y, so a better answer would be `x, y = np.linalg.solve(A, b)`.) If you copy-paste the above program you do in fact get "[8. -2.]", which is correct.

However, ChatGPT, after providing the correct program, also claimed that it's output would be "[5. 5.]" which is not correct.

My impression is that ChatGPT being a large language model, is excellent at translating from English to Python, but terrible at actually performing calculations. Which is fine. We have programs which can efficiently run numerical programs. ChatGPT fills the role of a programmer, not a calculator.

I want to emphasize how impressive I think ChatGPT is. Even the above examples, where it gets the "wrong" answer in the end, are impressive. Most of my interactions with it were very positive. But we need to understand its strengths and weaknesses to be able to use it effectively.