Not exactly math, but physics, since that was my background. Perhaps my favourite textbooks are Landau and Lifschitz[1]. They pretty much exemplify what I mean. They are concise, lucid, entirely self-contained. All the relevant information is there and not one bit more. The material is reduced to its key ideas, therefore making their exposition as clear as it can possibly be. Many people I've spoken to, however, don't like it precisely for being "too terse", so there you have it as well: ymmv.
Another book that fits this is Cohen-Tannoudji's Quantum Mechanics[2]. The first two chapters explain quantum mechanics from absolute scratch. It starts as all physics does: with an experiment. Then everything else follows from looking carefully at the consequences of that observation, and the concepts are truly explained for what they are, because they are presented in the simplest way possible. "Idea" is more understandable than "Idea+Cruft". This is remarkable to me: much fuss is made about how quantum mechanics is strange and confusing and difficult to grasp, and it is indeed so. In that first chapter, however, I've found the most illuminating explanation of quantum mechanics I've ever read.
EDIT: I just opened Landau's Mechanics Vol I, and here's what his colleague had to say about his writing and teaching style: "[These are] all the features of his characteristic scientific style: clarity and lucidity of physical statement of problems, the shortest and most elegant path towards their solution, no superfluities.". Indeed.
I find this a bit funny. You are comparing books that are used by advanced undergraduates or graduate students to a book written essentially for the subset of programmers with almost no math background. There are beautiful mathematics books by Springer (Undergraduate Texts in Mathematics), Carus Mathematical Monographs and Dover that provide both intuition and rigor at an introductory level. The main problem for a beginner though is knowing where to start and getting a survey of the arena they are entering. They might think linear algebra is adding vectors together with a vector being just a list of numbers.
This book hopefully fills that gap. There are other introductory books like Mathematics: Its Content, Methods & Meaning by Aleksandrov and Kolmogorov but that requires a serious investment of time.
Lastly, I apologize but I am going to rant about this physicist phenomenon. I am a physicist myself (high-energy theory) that entered quantitative finance after my PhD (a heavily disliked field on HN). I found a lot of physicists outside academia who tend to be aggressive and condescending and often haven't actually gone through the rigors of a PhD program themselves. Maybe it makes one feel smarter to tell other people that they are reading easy books or can't jump straight to Landau or Bourbaki (those two couldn't be more different but you get the point) and there are no other physicists around to correct the notion that there are multiple ways to gain knowledge and build intuition.
It doesn't matter who says what about which book. Sample some books, find what you like as long as its not junk science/mathematics, do plenty of exercises, and try to derive things yourself and don't fool yourself that you understand something deeply by studying it for a year.
Landau-Lifschitz as a reference is also very polarizing. There are people that really like the style (myself included), but I have also met many people that really dislike it, think it is too hard, too brief, too dry, too little description of intuition... . Similarly, the Feynman lectures are loved by some and disliked by others.
Another book that fits this is Cohen-Tannoudji's Quantum Mechanics[2]. The first two chapters explain quantum mechanics from absolute scratch. It starts as all physics does: with an experiment. Then everything else follows from looking carefully at the consequences of that observation, and the concepts are truly explained for what they are, because they are presented in the simplest way possible. "Idea" is more understandable than "Idea+Cruft". This is remarkable to me: much fuss is made about how quantum mechanics is strange and confusing and difficult to grasp, and it is indeed so. In that first chapter, however, I've found the most illuminating explanation of quantum mechanics I've ever read.
[1] https://en.wikipedia.org/wiki/Course_of_Theoretical_Physics
[2] https://en.wikipedia.org/wiki/Claude_Cohen-Tannoudji#Selecte...
EDIT: I just opened Landau's Mechanics Vol I, and here's what his colleague had to say about his writing and teaching style: "[These are] all the features of his characteristic scientific style: clarity and lucidity of physical statement of problems, the shortest and most elegant path towards their solution, no superfluities.". Indeed.