I've been working on NightCafe for a few years now and have submitted to Show HN in the past, but since VQGAN+CLIP is a new technology and, to my knowledge, no-one else has built an app around it yet, I thought I'd submit again!
Advantages over using Google Colab:
- It's >2x faster (runs on NVIDIA RTX A5000 GPUs). E.g. 800 iterations at 600x600px in 8 mins vs 19 mins on Colab.
- Can do higher resolution (894x894px > 700x700px).
- Run multiple jobs in parallel.
- Works from any device.
- Obviously it's much easier to use for anyone non-technical.
- It's easy to "evolve" creations. I.e. tweak the prompt, increase the resolution, and/or just run them for longer.
- You don't need to keep the tab open to see the result.
It's free for basic use, but does cost money for extended use. Unfortunately I don't have the same access to GPUs as Google does.
Try it out. Would love to get the thoughts of the HN community!
Looks like they're both new apps, so they've done well to get everything up and running so quickly! I already had a lot of the groundwork laid for NightCafe Creator, so it was much easier for me to get this done. There's a lot involved, especially if you're going to charge for use (which is a must if you expect to have any sort of volume in the long run).
These are cool. I wish it was possible to move beyond Colab for generative type ML as the UX is horrible and there is no practical way to use them on mobile. Unfortunately it seems to be a problem of who has the hardware and who doesn't. Hopefully as time passes it will be easier to host these kinds of services, and not have to feel hopeless after the next AI Dungeon-type corporate implosion.
Yep, NightCafe has struggled for years with the expense of running algorithms on GPUs. It started out free, but as soon as you start to get any kind of volume, costs balloon.
I almost gave up on NightCafe when I was trying to make money from selling prints to people. Then, one user generated 3,000 creations in a week, which cost him nothing and me $300. I decided to spend the weekend implementing a credit system to prevent abuse, and was surprised to find that more people were willing to pay for credits than I expected.
It's still not much and I've had to spend a lot of effort on reducing costs in various ways just to be able to break even. But I love working on this app and have high hopes that _one day_ it will become profitable enough for me to work on it full time.
Interesting. I suppose if it takes minutes on a GPU, it might take ages to generate one on a mobile ML/NN chip? If it were able to be optimized to somewhat-less-than-ages, it would be a different business model w/o the need to manage credits and worry about abuse.
Yeah it would, but aside from the cost there are other benefits to cloud computation. It can be done in the background (i.e. it's ok to close your tab) and you can run multiple at once.
Style transfer on NightCafe has a bulk-create mode that has been used in the past to create 500+ creations at once and they all finished in 10 mins.
Is the option to buy prints still there? Some services offer the possibility to buy prints on canvas tied on a wooden frame, possibly manually retouched, which look much like an actual picture you can hang in your living room
I've been meaning to experiment with GAN art for a while now, but never managed to actually study or run any notebooks. I saw NightCafe Creator mentioned in another thread earlier today, and it made it too easy not to try a few prompts. Very neat first experience!
It's not, but I can see why the "Graffiti sunset" would make you think that. It does seem to be two different images combined.
I won't claim to have a good knowledge of the inner workings, but basically CLIP will give a score for how well the image matches your prompt, and that's used as a loss function for the GAN.
On the Colab notebooks you can specify multiple prompts with different weights, in which case I assume it has multiple loss functions that it's trying to optimise for.
I'm more of a web app guy than a deep tech guy. Some of what I wrote can probably be corrected by someone with a better knowledge of the ML.
I've been playing around with AI art generators for a while now and they are always great fun. Though it weirdly motivates to do some painting or creating of my own as if I need to compete with it. I think AI generators as inspiration tools is a very interesting use-case that I'd like to see more of!
As a side note, have any people bought any of the prints?
Most of my regular users are artists who either use it for inspiration or use it to mix into their digital art. (NightCafe has been around for a while as a style transfer app).
It's rare that people buy prints. Most money comes from people buying credits.
Looking through the recent and top creations, it seems that including specific artistic styles or moods in the prompt produces more recognisable results. Do you have any other tips for crafting the prompt?
You're right, adding modifier keywords like "impasto" or "cubist" have a big effect. There are also some powerful community-discovered keywords that have a huge impact on your results, like "Trending on Artstation" and "Unreal Engine".
My best tips for good results - use a phrase describing objects you'd commonly see in pictures and art, like "House on a hill" or "Futuristic city", add some modifiers, then start with the smallest resolution and runtime. Try a few variations of phrase and modifiers, and see what looks promising. Then pick the best one(s) and use the "Evolve" button to run them for longer and increase the resolution.
How is your experience with re-seeding images to the system going from small resolution to a larger one? When I tried it, results were not as good as starting a high res image from scratch.
The text to image results look pretty awesome, but I feel like they all have a "neural net" quality to them. Whilst the style transer results come out more unique. Do you think it would become possible to combine the two techniques? Eg. Generate me a pic of this text, but in this images style?
> but I feel like they all have a "neural net" quality to them
it is getting significantly better though! The first AI art generators were just dreams now we're getting decent stuff as long as you stick to nature concepts.
Here's one I got for:
"a starry, dark night sky over vast plains of desert"
The Colab notebooks actually have a "target image" parameter. I haven't added it to NightCafe Creator yet, and haven't even experimented much with it on Colab, but it's definitely on my to-do list.
However, adding keywords like "watercolour painting", "van gogh painting" etc goes a long way to getting more precise results.
One big difference between style transfer and GAN art is that style transfer never changes the shape of things. E.g. you can't put in a woman's face and a cubist painting and get out a rearranged version of the face (as cubists do). It will have all the hallmarks of cubism, but it's still recognisable as the original photo.
With GAN art though, you can put that photo in as a start image, say "Cubist painting of a woman's face", and the GAN will actually rearrange the face like a cubist would.
Cool. I never used Shopify. Didn't know you can use them to sell digital goods.
Do you have to integrate your own payment or do they handle it all for you? Do they collect the money from users and then send it to you once per month? Do they handle the tricky parts like dealing with VAT per country?
I basically use the API to create a checkout specific to the user's order, then listen to the webhook to fulfill the credits. Gateways are Shopify and PayPal. Shopify payments are in my bank account in a couple of days.
Shopify lets you choose which payment options you'll let customers check out with. Examples are Stripe, PayPal, Shopify Payments, Google Pay, etc. A Shopify account is enough to get started, you won't need to dabble with any APIs.
However, if you're wanting to sell digital products, it may not be the best option for you. I've had to do a bit of extra work to get it to work for my situation.
Advantages over using Google Colab:
- It's >2x faster (runs on NVIDIA RTX A5000 GPUs). E.g. 800 iterations at 600x600px in 8 mins vs 19 mins on Colab.
- Can do higher resolution (894x894px > 700x700px).
- Run multiple jobs in parallel.
- Works from any device.
- Obviously it's much easier to use for anyone non-technical.
- It's easy to "evolve" creations. I.e. tweak the prompt, increase the resolution, and/or just run them for longer.
- You don't need to keep the tab open to see the result.
It's free for basic use, but does cost money for extended use. Unfortunately I don't have the same access to GPUs as Google does.
Try it out. Would love to get the thoughts of the HN community!