Correct. This is why SpaceX rejected using black coatings alone for Starlink. Mirrored surfaces that they tested instead were darker than VANTA black.[0]
On some surfaces they use a SpaceX-designed coating called Low Reflectivity Black, which is the least specular space-rated paint by a factor of 5.[1]