[polm23]

You can't do this because language models are not magic.

Let's say you write a sentence like: "[your name] picked up the book." Censor it to take out your name and let a language model fill it in. It might give you "John picked up the book" or "Mary picked up the book", which are grammatically correct, but it has no way of guessing your name reliably because it has no information about the situation of the real world. Language models work by predicting the most likely filler for a slot - if they can predict something it's not surprising.

Emily Bender wrote about this on Twitter.

https://twitter.com/emilymbender/status/1119081131234611201

If you want to use pixel data to fill in text that's a different approach that could work if they did a poor job with black bars, though it seems unlikely they'd do that.

[elicash]

Would they need to have done a poor job with the black text? For example, let's say in the sentence: First, ______________ Next, this happened.

If you know where the comma ends and the next sentence begins, you still know the exact number of pixels in between. Let's then assume that there are no proper nouns in the missing text. There probably aren't very many combinations of letters that fit the space perfectly while still making for valid words, given different letters have different pixel widths.

But the idea wouldn't even be to come up with the CORRECT answer. It would be to assign a score to different options of what it could possibly say.

I agree you couldn't do paragraphs.