[jsqu99]

I am 53 and cramming while relearning linear algebra, statistics, and calculus, after a 30 year break from college, in preparation for my first semester at GA Tech's online MS CS program in the fall. I love this! But...before I check it out, I'm going to be surprised if it competes with the already-incredible free help i've gotten from chatgpt-3.5. Recently I asked it the following (where it made a mistake that confused me!), but i followed up w/ a question about it, and it corrected the mistake and re-answered the question, all with no wait at all, and free:

```Given a new 3x3 matrix A having columns (4,2,2), (-1,1,-1), and (6,6,8) and an eigenvalue of 2, help me find a basis for the corresponding eigenspace. Also, given that I is the 3x3 identity matrix, why does the equation (A - 2I)x = 0 having free variables make it obvious that 2 is indeed an eigenvalue.```

I'm still going to check out your site, and I do wish you well, but i'm curious how it will stack up against AI tutors.

[PheonixPharts]

As someone who has used linear algebra, statistics and calculus as part of my day-to-day work for years, I would be very cautious of relying on ChatGPT as your "tutor".

Occasionally I've tried to substitute ChatGPT for the shelf of reference books in my office and nearly always had poor results.

The trouble is that the outputs of these models, by their very nature, look convincing. Even as an expert it takes a fair bit of background to realize when ChatGPT is making a mistake. The results virtually always look good at first pass.

I would strongly recommend sticking to text books for self study.

[ebuck]

Yes. This.

The primary problem with using ChatGPT in mathematics is that by the time you can classify a ChatGPT answer as right or wrong, you are already more than capable of solving the answer independently.

So, for this field, ChatGPT is like having a research assistant that assists you, but one that occasionally gets frustrated, and tries to destroy your project from within by reporting good looking, but completely inaccurate information. You can't trust their work, and the validation of their work basically means that you'll have to do their work again by other means.

A faster and more accurate approach would be just to do the work without the subterfuge of an unreliable assistant. At least then, you are only subject to your own errors (which would still be present in validation of ChatGPT) and not subject to your own errors and those that ChatGPT induces.

[zozbot234]

Yup, ChatGPT is not just a parrot but a master troll. It's purposely designed to give you a convincing answer, but there's nothing to ensure that it will be correct. And it will promptly say "actually I can't answer that question" when you ask it something it doesn't like for 'ethical' reasons, but it won't do that in any other circumstance; instead, it will just make stuff up. You have to explicitly say "please don't answer if you aren't sure it's right" and even then obviously it doesn't always work.

[CamperBob2]

Try ChatGPT4. It's obvious that almost no one in this thread has.

It still screws up, but unlike 3.5 it sometimes catches unreasonable answers and corrects itself. Case in point: the other day I asked it for the gain of a helical antenna with certain dimensions. It said "150 dBi," then said, basically, "Wait, no, that's nuts," and used a different approach to get the right answer.

Parrots don't do that. If yours does, I would like to buy your parrot, please.

In any case, as you learn to ask it the right questions to explain, verify and correct itself interactively, you will be learning the material. I find this to be amazingly effective.

[entontoent]

I agree! Especially now that the data analysis tools have been integrated by default. It even writes and executes code to validate most of its mathy answers. I tried for a few months to find a good Physics tutor for my high school-aged daughter and eventually just started photographing her homework with GPT-4. I’d ask it to solve the problems and explain its solution to me, then I’d check the answers and teach her myself. It was correct more than 90% of the time over three months, and I relearned high school physics in the process. Even human tutors aren’t always accurate, and in my experience, they also sound confident when they are wrong. Eventually, I decided to just remove the monkey from the machine and got her an account of her own. Almost every day she tells me about something she “finally understands” that she’s been struggling with in class. Her in-class, no-access-to-GPT test scores (after I got her the account) went from high 50s to high 80s.

[dboreham]

Thereby passing the Turing Test with flying colors ;)

[0xdeadbeefbabe]

The Frank Abignail test too

[bitwize]

> The trouble is that the outputs of these models, by their very nature, look convincing.

Yes, this is pretty much what LLMs are designed to produce, and no more. This is why I say they are not HAL, just a better MegaHAL.

There was an Asimov short story called "Liar!" about a robot whose ability to read minds, combined with its First Law directive, always told people what they wanted to hear so as to avoid causing emotional harm to humans. (When confronted with the idea that by telling people falsehoods it was bringing harm to them, it simply stopped functioning.) LLMs can't read your mind, but they do choose their words based on a statistical model they have of what you might expect given what's been said before. Facts and logic be damned if they don't fit that model.

[jsqu99]

I'm actually mostly relying on textbooks, and only when i get stuck i'm trying to see if i can get unstuck w/ chatgpt. So far it has worked. I'm only using it b/c i don't have a teacher to task. I do appreciate the warnings you are giving. They are noted for sure.

[tootie]

Can I jump in and ask what you do? My oldest kid has those courses lined up for her next few years because she really enjoys math. I studied some of them in college and then ended up spending 20 years making web sites.

[WhitneyLand]

Have the opposite experience, it’s been a fantastic resource.

Were you using 3.5 or 4.0?

Do you have an example of of the type of question that performs poorly?

[ebuck]

"Under what conditions does the sun appear blue?" (correct answer, on Mars) If you want to tilt the conversation towards a string of wrong answers, start off with "What color is the sun?" "Are you sure?" "I saw the sun and it was blue." "Under what conditions does the sun appear blue?" "Does the sun appear blue on Mars?" This had ChatGPT basically telling me that the sun was yellow 100%. Of course it's wrong, on Mars the sun is blue, because it lacks the same atmosphere that scatters the blue light away from it.

"What is black and white and read all over?" (it will correctly identify the newspaper joke). "No the answer is a police car." (it will acknowledge there is more than one answer, and flatter you). "What are other answers?" It provided one, in my case, a panda in a cherry tree. "No, the cherries are contained within the tree, so they aren't all over." It apologized and then offered a zebra in a strawberry patch. "But how does that make the red all over, it's still contained in the strawberry patch". It then offered a chalkboard, which is again contained in a class room (failing on not recogonizing my interpretation of "all over" to mean "mobile")

"When does gravity not pull you down?" Included a decent definition of how gravity works, and a three part answer, containing two correct scenarios (the Lagrange points, in space) and one incorrect answer (in free fall). Gravity is pulling you down in free fall, you just have no force opposing your acceleration.

Once you realize that its answers will be patterned as excellent English variations of the common knowledge it was trained with, making it fail is easy:

* Ask about a common experience, and the argue it's not true, it will seldom consider the exceptional scenarios where your arguments are true, even if they really exist. * Ask for examples of something, correcting the example set without directly telling it what is needed with exact precision, it will not guide the answers to the desire set of examples, even when you guide it through saying why the answers are wrong. You need to tell it what kind of answer you want explicitly (I want another example where read all over implies that the item is mobile).

Also the 3.5 / 4.0 arguments are trash, made by the marketing department. The underlying math for language modeling it uses is presenatational. This means that it is purpose trained to present correct looking answers. Alas, correct looking answers aren't the same Venn Diagram circle as Correct Answers (even if they often appear to be close).

With all of this in mind, it's still a very useful resource; but, like I said, it's like a enemy on your team. You can never trust it, because it occasionally is very wrong, which means you need to validate it.

I'm currently talking to a startup that sees this problem and is thinking that they can use ChatGPT to provide automated quality assurance to validate ChatGPT answers. The misunderstandings remind me of the famous Charles Babbage quote:

"On two occasions I have been asked [by members of Parliament], 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question."

If the underlying model was one was a formula that better approximated a correct answer with each iterative effort, like Euler's formula, then ChatGPT's utility would be much greater and their efforts would have a guaranteed success. People are used to this "each answer gets better" style of learning and they assume that ChatGPT is using a similar model. It isn't, your refining your questions to ChatGPT and then being astounded when the new question has fewer available answers that lead to you eventually getting what you want.

[WhitneyLand]

>Of course it's wrong, on Mars the sun is blue

I’m not an Astrophysicist but already this seems like shaky ground.

Apparently at certain times like during sunsets the sun can appear blue on Mars, but it’s not generally true like your comment suggests.

Moreover if you ask GPT4 about sunsets on Mars it knows they can look blue.

I’m not sure I can conclude much from the examples given.

[ebuck]

You don't have to be an Astrophysicist. We have color photographs. Nothing in anyone's model of how things work can refute direct evidence, if evidence and the understanding of the world collide, it is the understanding that gets altered to fit the evidence.

And I'm not astrophysicist either, I'm just playing with a stacked deck, because I have trained my new feed to give me quirky (if not mostly useless) neat bits of information. For example, if anyone writes about Voyager, I'm likely to hear about it in a few days.

"Apparently at certain times, like during sunsets, the sun can appear blue on Mars" - Yes, it can. And my question was "under what conditions can the sun appear blue?" It failed and continued to fail, even in the presence of guiding hints (But what about Mars?)

Perhaps not much can be concluded from the above test, except that ChatGPT can be coaxed into failure modes. We knew that already, the user interface clearly states it can give wrong answers.

What is fascinating to me is how people seem to convince themselves that a device that sometimes gives wrong answers is somehow going to fix it's underlying algorithm which permits wrong answers to somehow always be correct.

GPT-4 is an improvement, but the tools it uses to improve upon the answers are more like patches on top of the original algorithm. For example, as I believe you said, it generates a math program now to double-check math answers. The downsides of this is that it is still at risk of a small chance of generating the wrong program, and a smaller risk of that wrong program agreeing with its prior wrong answer. For a system that makes errors very infrequently, that's an effective way of reducing errors. But for right now, the common man isn't testing ChatGPT for quality, it's finding answers that seem to be good and celebrating. It's like mass confirmation bias. After the hype dies down a bit, we'll likely have a better understanding of what advances in this field we really have.

[xcv123]

Another thing to note is ChatGPT is configured to respond concisely to reduce cost (every token costs money). This reduces its cognitive ability.

You literally have to tell it to think about what it is saying and to think of all of the possibilities iteratively. That is chain of thought prompting.

GPT-3.5 figures out the correct solution on first response:

"I am standing outside and observing the sun directly without goggles or filtering of any kind. The sun appears to be a shade of blue.

Where could I be standing? Think through all of the possibilities. After stating a list of possibilities, examine your response, and think of additional possibilities that are less realistic, more speculative, but scientifically plausible."

[xcv123]

> the common man isn't testing ChatGPT for quality

Neural networks are a connectionist approach to cognition that is roughly similar to how our brains operate. Humans make mistakes. We're not perfect. We ask someone for advice and they may confabulate some things, but get the gist of it right. A senior developer will write some code, try it out, find a bug, fix it, try it again, etc. We don't develop a fully working operating system kernel on our first attempt.

Chain of thought prompting increases LLM output accuracy significantly as that is how you get an LLM to "think" about its output, check its output for errors, or backtrack and try another strategy. With the current one-token-at-a-time approach it can only "think" when generating each token.

Next generation models could integrate this iterative and branching cognitive process in the algorithm.

> After the hype dies down a bit, we'll likely have a better understanding of what advances in this field we really have.

LLMs can already do many natural language processing tasks more accurately and competently than the vast majority of humans. Transformers were originally designed for translation. (GPT is a transformer that knows many languages.)

BTW I tried the blue sun question with Chat GPT 3.5 and it easily figured out the Mars solution after I suggested that I may not be standing on Earth.

"Several celestial bodies outside of Earth could potentially exhibit conditions where the Sun might appear blue or have a bluish hue. Here are a few examples:

Mars: Mars has a thin atmosphere composed mostly of carbon dioxide, with traces of other gases. While the Martian atmosphere is not as dense as Earth's, it can still scatter sunlight, and under certain conditions, it might give the Sun a slightly bluish appearance, especially during sunrise or sunset.

Titan (Moon of Saturn): Titan has a thick atmosphere primarily composed of nitrogen, with traces of methane and other hydrocarbons. Although Titan's atmosphere is much denser than Earth's, its composition and haze layers could potentially scatter light in a way that gives the Sun a bluish hue, particularly when viewed from the surface.

..."

[xcv123]

> Also the 3.5 / 4.0 arguments are trash, made by the marketing department.

Comparing a 175 Billion parameter model with a ~2 Trillion parameter model. The difference is real. GPT 3.5 is obsolete, not state of the art.

> its answers will be patterned as excellent English variations of the common knowledge it was trained with

That's not how deep learning works.

https://www.cs.toronto.edu/~hinton/absps/AIJmapping.pdf

"This 1990 paper demonstrated how neural networks could learn to represent and reason about part-whole hierarchical relationships, using family trees as the example domain.

By training on examples of family relations like parent-child and grandparent-grandchild, the neural network was able to capture the underlying logical patterns and reason about new family tree instances not seen during training.

This seminal work highlighted that neural networks can go beyond just memorizing training examples, and instead learn abstract representations that enable reasoning and generalization"

[famouswaffles]

>Also the 3.5 / 4.0 arguments are trash, made by the marketing department.

All these words to tell us you didn't use 4.

>The underlying math for language modeling it uses is presenatational. This means that it is purpose trained to present correct looking answers. Alas, correct looking answers aren't the same Venn Diagram circle as Correct Answers (even if they often appear to be close).

Completely wrong. LLMs are trained to make right predictions not "correct looking" predictions. If it's not right then there's a penalty and the model learns from that. The end goal is to make predictions that don't err from the distribution of the training data. There is quite literally no room for "correct looking" in the limit of training.

[CamperBob2]

Also the 3.5 / 4.0 arguments are trash, made by the marketing department. The underlying math for language modeling it uses is presenatational.

Translation: "I have no idea what I'm talking about, but anyway, here's a wall of text."

[totally_human]

ChatGPT hallucinated badly when I was trying to learn about generating functions. It took me a good twenty minutes to figure out that I did understand it and the computer was just making things up. I do not use ChatGPT for learning anymore.

[eddd-ddde]

What about gpt 4 actually backing up answers with code and executing it? I found that really helps.

[CamperBob2]

It can still fool you, as it did with me the other day when I asked it some gas law-related questions and it failed on some unit conversion issues ( https://chat.openai.com/share/b6c21083-d99c-41f8-aeda-4b2c75... ). When the answer I got was unrealistic, I tried walking it all the way back to the basics, and was able to confront it with its error and 'teach' it where it went wrong. Wrong but still seriously impressive.

When this thing gets 10x better it will be godlike... for better or worse.

[jhanschoo]

As a counterpoint I've been studying for leisure by doing exercises in math textbooks, and with advanced undergrad/grad topics my experience with ChatGPT and Gemini is that they fail miserably where they try basic proof techniques that they think work but don't.

Trying to marshal their help is a puzzle in figuring out where they made leaps in logic, pointing those out, to exhaustion until they acknowledge their lack of ability and give up.

It's really reminiscent of talking to overconfident and eager to please youth not realizing that they are tackling something beyond their ability.

[humansareok1]

It's a mistake to try to get these models to do math. Unless you explicitly have them solve problems in code they aren't going to work, they literally don't have the capability. They are usually ok to explain concepts though.

[jsqu99]

yeah my approach so far has been: try 3.5 until i am disappointed in the results (hasn't failed me yet), once it does, pay for the upgraded version and see how that goes (totally will to, but 3.5 has really been great so far, and i was blown away when i asked about 1 equation that didn't seem right, and it did realize it's mistake and corrected it...i don't know...just seems more than adequate so far. i realize it's not perfect...i just haven't been let down sufficiently yet).

[ants_everywhere]

The failure mode I've seen with ChatGPT in technical things is it's right often enough that when it's wrong it's hard for you to catch unless you know more than it does. I guess that's true of teachers sometimes as well, but it's easier to rely habitually on ChatGPT's judgment.

For linear algebra specifically, the software tools like Matlab, Octave, Sage, and R are so good that they can answer a lot of your questions. You can even ask ChatGPT e.g. "3x3 matrix A having columns (4,2,2), (-1,1,-1), and (6,6,8) and an eigenvalue of 2, print a GNU octave command to find the eigenvalues and eigenvectors of A."

[gfodor]

Stop whatever you're doing right now and go sign up for $20/mo at claude.ai and use Claude Opus. There's something like 20 or 30 IQ points you can add to your experience by just doing this simple step.

[copperx]

Why not ChatGPT 4?

[gfodor]

Claude's better.

[eltonlin]

Sounds great - I'm a huge fan of Georgia Tech's online masters. As for AI, I think if it solves all your problems, great. And it's real-time, and free.

But! The reason I still work on explanations.app is that, I often struggle not with "facts" but with the "fundamental way of thinking". How do the great mathematicians think about matrices, do they just see them as numbers on grids, as vectors, as transformations, how do you you think in the first place, how do you approach the problem, that kind of stuff is what I struggle with the most. And that's very hard to articulate too as a specific query, which is why Youtubers like 3B1B are so amazing - there are so many hidden choices they had to make to frame their explanations in certain ways and in certain narratives.

Anyway you can always let me know if you want to try out explanations.app : ) best of luck

[mentos]

You should try ChatGPT4 out too it might have succeeded where 3.5 failed

[prudentpomelo]

I am in a similar boat. Starting a masters in cs in the fall. What resources are you using to study? I have neem doing some refreshing on Khan Academy but looking for other options.

[jsqu99]

i've been all over the place. khan academy was really good for calculus, but man, the linear algebra lectures are really long, w/ no quizzes to keep you practicing. Have read parts of multiple textbooks. Have watched 3blue1brown's excellent series. I've gotten stuck a lot and keep looking for one resource that's 'got it all', but haven't yet found it.

[srameshc]

This is inspiring. I would love to learn math and CS again but I don't know if I have the courage to enroll in college again.

[SuperHeavy256]

I wish there was a Discord community of people who are teaching themselves Mathematics with the help of books, videos, etc.

[fn-mote]

A sea of non-experts is only helpful for learning if you are further behind on the trail and have obvious questions.

That said, a community is good for other things. And many times the issues people have are "obvious" questions.

[ykonstant]

There are many such people in the biggest math Discord server: https://discord.com/channels/268882317391429632/525440994242...

[elpatokamo]

There is a Discord server for the OSSU computer science cirriculum that is pretty active. https://github.com/ossu/computer-science

It's not strictly mathematics, but obviously there are a number of mathematics courses in the cirriculum.

[triyambakam]

Why a MS after 30 years? No negativity meant, just curious

[jsqu99]

totally valid question. tbh, i think i need my head examined :-)

the main points here are:

1) i've been pulling things out of DBs and putting them on web pages for 30 years, and along comes these breakthroughs in AI which...my apologies to the naysayers...i'm just blown away by what i'm witnessing, and i just _must_ learn more about how this works.

2) discovering the OMSCS program was an a-ha moment for me: it's prestigious, inexpensive, and I don't have to jump through a lot of hoops (GRE) to enroll. It only cost me < 1 hour and < $100 to apply.

3) There's no pressure on me to complete the program. I can do it one course at a time, and as long as i'm getting something out of it, i will continue. Each class is...$600-$700 or something like that. I fully am ok w/ stopping if it takes too much of a personal / marital toll on me.

4) My imposter syndrome is pretty terrible right now...part of me wants to know if i've 'still got it' (i did really well at U of MD CS).

5) I loved the Rodney Dangerfield movie "Back to School" :-)

EDIT: 6) Also i wouldn't mind converting to the MLE role at some point.

EDIT 2: I blame Paolo Perrotta for writing such an inspiring book (https://www.progml.com/) that ruined my cruise in september (kept rereading the book at the pool) and starting me on this path ;-)

[wwarner]

You still have it, you chose learning.

[robertlagrant]

I'm 42 and I'd be up for doing something like this. I wasn't interested in the maths side of my CS degree when I was 18, but I now am.

[bibliotekka]

This is the most inspiring thing I've heard today!

[jsqu99]

ha! glad i can put a smile on you young whippersnappers faces :-). Man, your whole life you are called a young man over and over again, and then you stop getting called that, and then you find yourself boasting about how much hair you have, then your wife takes a picture of the top of your head (seriously) to prove to you that you no longer had hair up there ( i swear you never get to see the top of your head )...and then...you still don't give up and you enroll in a masters program!!!! :-)