I thought Finnish was categorised as "finno-ugric", which I understood to be a language group that included only Finnish, Magyar, and Basque. Perhaps my understanding is outdated.
Well, definitely not Basque. Basque is a language isolate, like Burushaski and Sumerian. It definitely is related to other languages at some time depth, but the depth is so great that this relation is impossible to discern now with any confidence. And I don't think anyone with any credibility currently argues that its closest extant relations are languages like Finnish and Hungarian whose homeland is north-central Asia.
Finno-Ugric is a subset of Uralic. Basque is not Uralic language and may be an isolate, although distant connections to various minor language families have been proposed.