They are "just composites" in the sense that the NN learns to decompose an image into multidimensional concepts both local and global (the latent space) and then assemble new images from anywhere in that space even at points that weren't in the training set. Of course if the training set is small then it'll overfit to mostly reproduce the inputs.
it sounds like jargon to me as idk much about neural networks. i guess its kind of a highly complex and nuanced composition. but essentially its still just a sophisticated combination of existing things.
i guess this is the same as human creativity in the sense it depends on prior experience. its more the "does not exist" that bugs me a bit.
tx for the detailed response tho, wish i was smart enough to understand.
> i guess this is the same as human creativity in the sense it depends on prior experience.
That's roughly the idea. Although I didn't want to make that analogy because someone would try to overextend it and point out that an artist can do X and a neural network can't and therefore it's wrong.
> its more the "does not exist" that bugs me a bit.
Well, if someone learns what a sneaker generally is and then draws a fantasy-sneaker that is still recognizable as a sneaker because it stays in the general confines what is considered "a sneaker" then does that sneaker suddenly exist?
You’re somewhat right. The machine learning algorithm tries to replicate the general patterns on the images it’s trained on (so in this case, images of a lot of sneakers). If the algorithm is trained on enough images, it should, to an extent, learn to generalize and “understand” what a sneaker should look like and generate new ones rather than copying images. In my case, there is definitely some memorization going on, based on some shoes looking suspiciously similar to existing shoes, but there is also definitely some design going on not copied from other sneakers.
But it builds the images “pixel by pixel” rather than e.g. taking the sole of one shoe with the upper of another if that’s what you’re asking.
As a bit of a sneaker head, it's been quite fun to see how the algorithm changes the shapes when playing with the sliders on the Sneaker Editor.
Impressive that it isn't copying/pasting parts of shoes but actually building them pixel-by-pixel, I was editing a very AF1-looking sneaker and playing with the sliders made actual sensible changes to the shape and design.
Pretty entertaining :)
Quick edit: as a side question, the training dataset included all kinds of sneakers or was it biased towards more popular ones? I ask because from what I scrolled I missed seeing more avant-garde designs and wasn't sure if the model was trained on these or not.
The training set included pretty much all sneakers I could find online. However, the algorithm tends to bias towards patterns that are more common e.g. hundreds of colorways of Air Jordans.
BUT I can fine tune the model by training on a small subset of sneakers. So if you have any types of brands or sneakers that you would like to see generated, please feel free to mail me some examples at stan@thissneakerdoesnotexist.com and I will see what I can do.
It could also be interesting to define multiple subsets of sneakers - "2021 avant garde sneakers", "skate-inspired shoes", "2010s most popular shoes in the NBA" (based on https://ballershoesdb.com/ or a similar database). It could be cool to start with a model trained on a subset of sneakers, and tweak/combine things from there.
https://openai.com/blog/multimodal-neurons/