He never states he is bad at math. It sounds like to me he was much more interested in subjects like art, literature and film. Which, given his career makes a lot of sense.
Also, there is a difference between math and computation. If the course only required 9'th grade algebra level, I personally would rather drop it and take additional calculus courses.