| A couple of years ago I started architecting transcoding/streaming video pipelines on AWS for my work. It sparked a general interest/curiosity into the internal workings of video compression algorithms, which then allowed me to revisit a subject I've always wanted to try out, datamoshing. Something about purposefully breaking codecs to generate artifacts that lets you peak at their inner workings really nerd sniped me. The most advanced video I was able to create is this datamoshed version of Carpenter Brut - Turbo Killer clip : https://streamable.com/7d9h4b. I'm particularly proud/pleased with the transition at the 2:00 minute mark.
It still a long way from what I would want to be able to accomplish (as the moshing is still pretty random and looks good only by "chance"), but I'm pretty happy with my progress overall. My long term goal for it would be able to create moshing of music video clips, and sync the glitching to the beat/structure of the music (using Spotify Audio Analysis feature [0][1]). As a sidenote, a tool I discovered and has been a godsend into tinkering with video codecs is FFglitch[2]. It's a fork of FFmpeg with the ability to directly access and modify internal MPEG2/4 values (such as the Motion Vectors, macroblocks, DCT coefficient, etc). Compared to other datamoshing techniques (such as corrupting AVI files with random data, manual key frames deletion, etc) it's order of magnitudes more advanced and precise, allowing you to precisely and purposefully influence the codec, instead of randomly breaking it and seeing what sticks. However its downside is that you have to have a solid grasp of the inner workings of those codecs to achieve anything. Finally, for anyone looking for a good resource to get an understanding of Video Encoding, I highly recommend this[3] github repo as a starting point. It contains a good description of the workings, as well as an amazing references list. [0]: https://developer.spotify.com/documentation/web-api/referenc... [1]: https://spotify-audio-analysis.glitch.me/ [2]: https://ffglitch.org/ [3]: https://github.com/leandromoreira/digital_video_introduction |