[elihu]

You could have something like that supplies heat to a building by nuclear decay, but generating useful amounts of electricity from relatively small temperature differences is hard. You could theoretically have a steam-generating nuclear reactor in a building, similar to what you'd find in a nuclear-powered submarine, but it wouldn't be small or safe or simple, and it would require large amounts of cold water.

Small research reactors exist, but they tend to generate in the neighborhood of tens of watts.