The "core" library was designed for portability and it's not as old as the .NET used for Windows desktop development, which is very mature and very performant.
One can hardly call .NET Core "a library." It's an API surface area (.NET Standard), a cross-platform implementation of said surface area (.NET Core), a cross-platform runtime (CoreCLR), and a CLI tool (dotnet). It's already introduced numerous performance enhancements over .NET Framework (e.g., Span<T>, Memory<T>, and the runtime itself).
IMO, .NET Framework should be relegated to the dustbin of history. There is no longer a reason to use it unless you are writing legacy code (which, granted, is a legitimate reason in many cases). For greenfield projects, I hope you're using .NET Standard and .NET Core.