[abecedarius]

iirc connections with linear algebra come up in Conway's https://store.doverpublications.com/0486485838.html (which I only skimmed).

[Jaxan]

There is a whole field of “weighted automata” which combine linear algebra and automata theory.