Mammals are tetrapods, which are indeed under Osteichthyes (unless you consider Ostreichtyes to be paraphyletic).
If you look at the cladogram on the wikipedia page I linked, you see a picture of a salamander near the top representing the category Tetrapodomorpha. We (and whales) are also on that branch. Sharks are outside of that whole cladogram, as they are cartilaginous fish. ( * )
In other words, any clade which includes all bony fish also includes mammals.
[*] which means that the most recent common ancestor of sharks and goldfish lived longer ago than the most recent common ancestor of whales and goldfish
If you look at the cladogram on the wikipedia page I linked, you see a picture of a salamander near the top representing the category Tetrapodomorpha. We (and whales) are also on that branch. Sharks are outside of that whole cladogram, as they are cartilaginous fish. ( * )
In other words, any clade which includes all bony fish also includes mammals.
[*] which means that the most recent common ancestor of sharks and goldfish lived longer ago than the most recent common ancestor of whales and goldfish