While you're right, I'll be happy with just empty data for now. Generating statistically-likely false data is only recently available generally and turns out to be rather expensive.
For the most sensitive fields (names, addresses, phone numbers) it's quite simple. For names, you get a list of the (say) 1,000 most common names, and pick randomly from the list. For phone numbers, you generate random numbers with valid formatting (not all area codes are valid, etc.). For addresses, you pick randomly from a database of real addresses. Etc. No LLM-style generation needed.