Sorry I don't have a reference, but as long as you are outside of an object, its gravitational force on you is the same as if it were a point-mass (at least for classical physics).
This is only true for objects that have a spherically symmetrical mass distribution and for a point test object. That is, the field of a spherically symmetric object is the same as the field of a point mass of its mass located at its center.
As soon as your test object is non-point you get tidal forces due to the field typically being nonuniform. As soon as your object is not spherically symmetric you get a field that looks nothing like that of a point mass.
As soon as your test object is non-point you get tidal forces due to the field typically being nonuniform. As soon as your object is not spherically symmetric you get a field that looks nothing like that of a point mass.