[yonkshi]

AI can also be used to identify these fake videos, and they are probably better than human identification. There will be a rise of AI forensics.

The type of neural network they use (GAN) works by having two networks battling each other, one tries to generate fake, and another (discriminator) tries to identify fake, it's a constant arms race. As the generator gets better, the discriminator also gets better. Which means, if the fake video is this good, there must be a discriminator network that identifies fakes just as good.

We did a similar project using GAN, generating images from a text description. You can see the progression of generator and discriminator battling each other, and both get better with time.

https://github.com/yonkshi/text2imageNet

[nickbecker]

This is how their model was trained, but I think what you've said may not quite be the case.

Because the discriminator (D) and generator (G) usually compete in a minimax game, the equilibrium probability of D correctly classifying an image as fake tends to 1/2 (ignoring distributional factors). If the competing networks have enough capacity and can be stably trained, then in theory they will reach equilibrium as the data distribution from G converges to the actual data distribution. If this is the case, then the discriminator correctly identifies fake videos with a probability of 1/2.

They may not reach equilibrium (making D > 0.5), but it's not clear that the discriminator itself is a panacea for identifying fake videos/images.

[lambdadmitry]

The whole GAN thing is useless though if neither of the models have a concept of symmetry or that those pixels do represent actual solid objects in space. Modern neural nets can't represent abstract spatial and semantical concepts or reason, so that videos are full of glaring perspective inconsistencies and large-scale artifacts. So nope, neither AI can help to "identify fake videos", nor do we need AI for that.

[jcims]

The problem is that it's a game of superhuman cat and mouse.

We'll have systems arguing with each other and no way to tell which is correct. If, for example, someone is able to get a copy of the forensic AI model, they can train their decoder/generator to work against it until the results pass as legitimate. With no human ability to argue with the results of the forensics AI, we'll just trust it and pass it off as truth.

If you can clone the jury and conduct a quadrillion private trials, your chance of success in court is going to increase substantially.

Things gonna get creepy.