Octave is an easy language for beginners and has excellent (less ceremony than numpy) support for linear algebra out of the box without having to learn any libraries. The point of the class isn't to teach you how to use libraries but to teach you at a high level how to use gradient descent to optimize parameterized models. Once you understand how it works, it is easy to translate what you know to run well on different systems or to use existing frameworks already implemented on different systems.
Numpy is kind of a funky library with some weird (but good!) syntactic sugar that doesn't translate to the rest of Python. Scipy is a different beast. And pandas. I could go on. Making, and using matrices, feels weird in python and interoperability/efficiency doesn't come for free.
Compare to matlab, where matrices are first-class, syntactic sugar is consistent and rather lovely. But then the rest of the language is detestable.