As a self contained, foundational course, Georgia Tech's OMSCS offering [1] is solid. Charles Isbell and Michael Littman are great at building intuition into equations.
Isbell's course in person was great. And if the exams for the online version are anything like the in person ones, it really does test your understanding of foundational concepts.
Yup, just took the online RL class and the average grade for the final exam was 45 out of 100, high score of 76. The format was true/false with a short explanation for your answer. I never thought I'd be proud about getting a 53 on a true/false exam, but it was an extremely challenging and rewarding class.