They are, but in that case you'd use raw pointers.
The reason this works is because of discipline. Generally, there is a FooManager class which owns Foos. The FooManager is responsible for both allocating and deallocating Foos, regardless of where they're used. And in a game, "When should something be deallocated?" usually has a clear answer: When the level loads, for example, or when you move from one part of the continuous world to another part.
Then there are Subsystems (singletons) for each division of the engine: GraphicsSubsystem, InputSubsystem, etc.
Between those two patterns, there aren't a lot of ways to lose track of a pointer.
I think another important factor is the pseudo-realtime update loop that synchronization is tied to.
Unless there is some garbage collection between game state changes (unloading unused resources during a level, etc...), its rare that references are invalidated unexpectedly, or accessed in parallel to the cleanup step between game state changes.
eg. a global state change from RUN to CLEANUP tells the subsystems to stop using a resource, so during the CLEANUP state the subsystems can safely delete any resources they have ownership of.
Idea of levels or areas isn't always applicable, and in vast open space games like Star Citizen or Witcher 3 with tons of random events you might need to load / free something pretty regularly I suppose, and it should happen seamlessly (i.e. if some event is triggered or passed).
So limiting lifetime of the object by some scope should work pretty well for it, and if you can pass raw pointers to transfer ownership, you can as well pass managed ones. At least it's safer.
Anyway, by using something like Rust a lot of such problems are solved by the language itself.
The idea of levels or areas is always applicable. A vast open game like Star Citizen is partitioned into area spaces, usually in a cubic array. When you move out of range of one of the cubes, past the point where you can see it, then all resources in that cube can be safely freed.
I don't know where Rust came from, since the discussion was about C++. But feel free to write an engine in Rust. It seems like a promising approach.