Show HN: AI magics meet Infinite draw board

[carefree0910]

I have been dreaming of creating things (arts, games, etc.), but that wouldn't be possible because I can hardly draw anything. But now with Stable Diffusion everything is hopeful again, so I further collected the AI magics that I considered necessary in creation activities, and integrated them into an Infinite draw board (like Figma).

Hope that I can hear from you, any advice or suggestions will be really appreciated!

[WhitneyLand]

“AI magics meet Infinite draw board”

No! Your project is much cooler than the title suggests.

It’s like, some kind of AI art studio app that goes beyond just text prompts. It’s a natural seeming evolution towards using generative tech in more of an iterative / workflow way.

Great work.

[carefree0910]

Wow, I'm so glad that you like it so much!

Just as you said, I'm trying to make it a studio-ish product which combines various AI magics, so they can work together and create something that a single AI model can hardly achieve.

Here's my future plan: I'm going to integrate natural language generation, music generation, video generation... Into this project, so you can make something really cool with and only with AI!

And the most important part: it will always be completely open source and free, and (maybe) a Google Colab is enough to serve all these stuffs XD.

[blagie]

I'll make a suggestion before you go too deep:

Integrate coding.

I have a hundred ideas of things I'd like to build with SD for cool drawing. There's a mile-long gap between doing things in Python and in GUIs. I'd like that to be an inch-long gap. For example:

- If I can use these tools in Python code (outside of your system, as a documented library), that's awesome

- If I can build plug-ins into your system, that's awesome

... and so on. Photoshop, and later gimp, sort-of did this with filters, and it was an amazing boost to both. What I'd really like is something like OpenProcessing meets carefree-creator, where I can easily do creative coding.

I have a neat model for how to use SD for rather creative, iterative artwork which I've wanted to hack, and I'd like a framework to hack it in.

[carefree0910]

That's a ton of great suggestions! I'll try to digest them and use them to guide my future works!

And for your requirement, actually carefree-creator is built to be a stand-alone backend server, so you can indeed customize your own frontend if you like, as I mentioned here (the first point): https://github.com/carefree0910/carefree-creator/issues/11#i...

And if you are interested in customizing carefree-creator itself (like integrating your fancy model into it), I haven't written any documentation yet, but here are some brief guides: https://github.com/carefree0910/carefree-creator/issues/8#is...

Feel free to contact me if you encountered any trouble!

[blagie]

I don't want a stand-along backend server with an AJAX API. I want a clean Python API on the back-end I can `pip install` (and, now that you mention it, a corresponding JavaScript on the front end). That should be wrapped in a thin layer to translate into a service-oriented API, which people might or might not use.

Performance issues aside (most of the integrations I'd want to build involve doing things between SD's iterative steps, and there would be a ton of data going back-and-forth):

- A service-oriented API only lets me use your code. I'd like to integrate with it to build new tools.

- Even if I want to use your code, a service-oriented API also doesn't let me do so practically. Adding your library to my code base involves launching a whole new server.

Flip it around: If you wanted to do a little bit of NLP in your tool, you'd probably be happy to `import spacy` or `import nltk`, add 5 commands, and use it. If you needed the devops of launch a server on a different tech stack, you'd probably make due without it.

And if you wanted to, in turn, extend `spacy` in some way, it's easy enough to toss in a few new things into it's pipeline, which, for example, annotate text. If you needed to launch a service to do so, it'd never be done.

Regarding documentation, my comments were architectural considerations. Your documentation is great for the stage you're at. I've basically never seen better documentation in a 6-week-old project since documentation make sense only once things have stabilized that they're not changing week-by-week. Otherwise, it's a lot of work for negative benefit. If people (like me) reuse or integrate with a 6-week-old project, you'll have pressure to support legacy APIs down-the-line. That only makes sense once those APIs stabilize. External contributions also make sense once the architecture has stabilized (which might or might not be the case 6-weeks-in).

[carefree0910]

Hi, thank you so much for the valuable advice and information!

If I didn't get you wrong, maybe another project (carefree-learn) meets your needs better: https://github.com/carefree0910/carefree-learn.

This is an old project, with a VERY outdated document, but all the features of carefree-creator can be achieved with one or few lines of code from carefree-learn.

Take the typical text to image feature as an example, here's a piece of minimal-working-codes:

```python

import torch, cflearn

api = cflearn.cv.DiffusionAPI.from_sd("cuda:0", use_half=True)

api.txt2img("a beautiful, fantasy landscape", "out.png") # this will save the generated image to `out.png`

```

And you can install it with `pip install carefree-learn[cv]` XD.

[la64710]

Totally absolutely amazing

[rajnathani]

This is a really wholesome top comment for a Show HN :)

[tfsh]

This looks really cool and I'd love to give it a go, however the installation instructions don't seem to be working for me.

I've tried both the Docker and non-docker instructions. Using Docker I get an error stating "Could not find a version that satisfies the requirement onnx-simplifier>=0.4.1" (I'm running Python 3.9.6 per the installation instructions).

If I manually install the dependencies I got a bunch of errors whilst executing pip install.

Has anyone else experienced these errors and know how to go about resolving them? I'm on an m1 macbook.

[carefree0910]

I haven't tested on m1 macbook yet, but the Docker solution should be OK...

Here's the onnx-simplifier project: https://pypi.org/project/onnx-simplifier/, looks like it already goes to 0.4.8.

I'll try to reproduce the error, before that maybe you can try the Google Colab solution:

https://colab.research.google.com/github/carefree0910/carefr...

[tfsh]

Thanks! I won't try the colab just yet, as I think it would be fun if I could locally run this.

I'm now getting different error messages by installing cmake (brew install cmake) before the manual install - it still looks like some odd error caused by Onnx, which is also why the manual install is failing - "fatal: not a git repository (or any of the parent directories): .git" whilst installing onnx.

The Docker solution failing is weird, containers are meant to be hermetic and deterministic, maybe an upstream change broke something..., but given the Docker install error messages and the fact Onnx is a large-scale project I fail to see how that's the case either.

I'll continue to debug and update the top comment if I resolve it :)

[carefree0910]

Thank you so much for your patience! You experience will definitely help me make this project compatible to the m1 chip systems XD!

And since ONNX is not a forced requirement, maybe the following guides can make things easier:

1. Install `carefree-learn` instead of `carefree-learn[cv]`

2. Install the following packages manually:

ftfy regex lmdb albumentations carefree-cv

> I'll consider removing the `onnx` requirements from the `cv` spec in the near future!

[tfsh]

Ah! That seems to have done it. Thank you for your help - ! I'm still downloading the models but I expect it'll work fine on the m1 chip ...

The installation steps are as follows:

$ git clone https://github.com/carefree0910/carefree-creator

$ cd carefree-creator

$ pip3 install carefree-client carefree-learn carefree-cv ftfy regex lmdb albumentations kafka-python redis[hiredis] cos-python-sdk-v5 uvicorn torchvision

$ python3 -m uvicorn apis.interface:app --host 0.0.0.0 --port 8123

[carefree0910]

Coooool! And yes, I'm not sure whether the models can run on the m1 chip, but I think this is more of a PyTorch issue instead of this project XD.

[permanent]

Yes please open source the webui codes!

> *: The WebUI codes are not open sourced yet, but we are happy to open source them if it is truely helpful .

[carefree0910]

That's exactly what I'm always looking for! Here's the related GitHub issue:

https://github.com/carefree0910/carefree-creator/issues/2

Apart from the technical concerns (huge code base, lack of documentation, etc.), I'm also pretty afraid of someone (or some company) just taking away the codes and make profits or even claim that they make it, because webui codes (especially as production ready as this project) are something that's easy to copy and HARD to protect...

But if more and more people get to know this project, this concern will no longer exist, and I'll definitely open source the webui codes XD!

After all, I do want more and more people to be able to enjoy the current AI magics, for free, and forever!

[zacmps]

It won't stop everyone, but if you license it under AGPL they'd be required to share any modifications they make under the same, open, license.

[blagie]

<== This is the answer.

The MIT license is wrong for this, as is proprietary. You want AGPL.

[bredren]

I don’t know how it is viewed here, but have a look at the licensing scheme adopted by Sentry.io.

They faced a similar concern when people were selling the exact same product using a new name.

> For example, this past year, we’ve had to deal with funded businesses plagiarizing or copying our work to directly compete with Sentry. This has included taking marketing content from our website, plagiarizing our documentation and framing it as their own, or straight-up copy/pasting our product visuals. Their defense? “Well, it’s free open-source, and we can do that.” These businesses are not using Sentry to improve how they develop software; they’re lifting its code and assets to build their closed-source products to compete directly with us.

https://blog.sentry.io/2019/11/06/relicensing-sentry/

[minraws]

Love the idea, hope you can continue improving on the execution, I am also fairly incapable of producing good manually drawn illustrations even though I used to be a junior designer early in school. Moved away from it as coding seemed like a more natural fit for me and helped me write games and do generative art.

But this really seems like something that would let me scratch my itch to produce nice illustrations, though ofc won't give the pleasure of learning that skill myself but surely stuff like this can't hurt. :3

Really cool project mate. :)

[carefree0910]

I'm so glad that you like it!

Currently the UX is not perfect, and I'll definitely try my best to improve it XD.

Looking forward to your advice and feedbacks!

[challenger-derp]

The UI/UX is great as many commenters have pointed out. Your write-up (readme) is also excellent. Though, at the moment, your cloud server is having trouble processing requests, I got error notifications about your cloud server posting.

[carefree0910]

Yes indeed, my poor cloud server burnt down just now... But I fixed it so it should be OK now!

[vongomben]

Wonderful project! How do I create an account for your cloud server? My phone is not accepted (maybe the hn hug of death?)

[carefree0910]

Oops, that's in fact a bug from my side, I'm afraid it's currently only available on PC...

I'll try to fix it ASAP!

(p.s. I personally recommend using this project on PC, because the features on the mobile site are very, very limited XD)

[carefree0910]

And BTW, you don't need an account to use this project if I fixed the bug / you visit it on PC!

[GaggiX]

This program should implement the runway inpainting model, it's so much better in in/outpainting than the usual "hack" as it supports them natively.

[carefree0910]

Guess what, I'm actually using the runway inpainting model as the backend of the `Erase & Replace` feature XD!

[cactusplant7374]

This could benefit from a Runpod template. Anything that makes access to high powered GPU’s easier is a win.

[carefree0910]

Yes indeed! In fact, I've already created a Google Colab based server:

https://colab.research.google.com/github/carefree0910/carefr...

Which means you can use the Google Colab as the product's backend, so you can enjoy all the AI magics for free forever XD.

[jjcm]

img2img seems to be broken for me, as it's just generating things entirely based on the prompt. Examples here: https://i.imgur.com/bEn94Zg.png

[carefree0910]

Oh, that looks like a bug... Could you please share the `fidelity` property of your `img2img` process, or share the `AI Parameters` on the right (after you select the generated image) (there should be a little `copy` icon) here? Thanks!

[dbieber]

Would love to see some of the things you've drawn using this tool!

[carefree0910]

Of course! But I'm new to HN so I actually don't see any way to post images XD.

I'm also concerned that posting links of my creations will be some sort of spam...

Any advice will be really appreciated!

[mhb]

What does "XD" mean?

[codefreakxff]

It is a text based laughing emoji

[jagged-chisel]

Oh the days when “emoticons” were cool

[shrimp_emoji]

uwu

[29athrowaway]

Does it generate pixel art?

[carefree0910]

It is not designed to generate pixel art, but you can try something like:

a lovely little cat, pixel art.

It will generate something weird, but (maybe) the quality of the generated image will be better with some tweaks!

[cube2222]

This looks very nice!

However, I'm worried that a big part of these AI tools built around SD are being posted on GitHub, marketed as open source, accept contributions, but don't have any license.

[sophrocyne]

Invoke, shared a few weeks ago, uses MIT and adheres to best practices in respecting model licenses/requirements (e.g., agreeing to not use it for hate speech, harming others, etc.)

Automatic on the other hand, seems actively opposed to responding to the calls (cries?) for him to clarify a license for the repo. Closed without being addressed - https://github.com/AUTOMATIC1111/stable-diffusion-webui/issu... 21 days open without response - https://github.com/AUTOMATIC1111/stable-diffusion-webui/issu...

We definitely need more projects who are completely open source (both WebUI and backend) to contribute to the ecosystem, and do so responsibly, adhering to lawful and ethical usage of the work of others.

[RobotToaster]

My understanding is the "ethical requirements" of the model licence preclude it from being used in any completely FOSS software, as it would violate freedom 0 by restricting the purposes someone can use the software for.

[carefree0910]

I noticed that they added a `LICENSE-ModelWeights.txt`! Maybe I should dive into their repo deeper and follow their good practices.

Thanks a lot for the advice! It really helps!

[Kiro]

If he doesn't want to add a license he shouldn't be bullied into adding one.

[tough]

But then how will big corps be able to steal his free code without any legal repercussions?

</s>

[carefree0910]

Oops, I thought I've used the MIT License, but it turns out that I have not!

I'm also curious whether I should somehow follow the SD license... But I'm not actually using their codes though...

Any advice will be really appreciated!

[cube2222]

IANAL but the SD license is for the model weights and using them. As long as you're not redistributing the weights, I think you're good to use the MIT license.

Thanks for adding the license, very appreciated! This makes me want to take it for a spin much more!

[carefree0910]

Thank you so much for pointing out the missing-LICENSE-issue as well, that's really a huge miss from me XD.

[kmeisthax]

Wait 'till you find out how they get their training data :P

The AI world does not seem to give a toss about licensing. I admire the spirit of "make stuff first, worry about copyright later", but you at least have to pay lip service to the system if you don't want to get roadkilled when "later" comes too quick.

If you're on GitHub there is a fallback license in the TOS that covers forking and PRs, but you have nothing to protect you if you go off-site for whatever reason. It's there mainly to keep their users from getting sued for just being on GitHub. I don't remember how it interacts with multiple collaborating users all under the same fallback license; you may be able to argue implied license in that case, but implied license is even narrower than the GitHub fallback license.

[thih9]

> If you find software that doesn’t have a license, that generally means you have no permission from the creators of the software to use, modify, or share the software. Although a code host such as GitHub may allow you to view and fork the code, this does not imply that you are permitted to use, modify, or share the software for any purpose.

Source: https://choosealicense.com/no-permission/

[cube2222]

Additionally, if you accept a contribution to your repo that doesn't have a license, you as the owner of the repo actually have no right to the contributed code.

This also means that technically speaking, you can't add the license after the fact without contacting (and getting the agreement of) all contributors that have ever contributed to your repo.

[googlryas]

The only reason you can't draw is because you haven't practiced(assuming you're not severely disabled). Why not work a little bit for your dreams?

[ImprobableTruth]

Where "work a little bit" of course means thousands of hours among several years.

[googlryas]

If you only consider being a commercial artist as success, then sure. I started drawing 5 years ago(in my 30s) and have been making steady progress. I consider that a success. And I could barely draw a stick figure when I started.

[Xelynega]

Or paying someone who put in the thousand of hours among several years instead of getting a mashup of their art out of a statistical model.

[carefree0910]

In fact, I've tried! I bought many books and just cannot get myself better at drawing...

I think you've mentioned something important: I haven't got enough time practicing. Maybe when I have more free time, I'll try to pick up these skills XD!

[_448]

> just cannot get myself better at drawing...

There is no such thing as "better" drawing. Drawing cannot be categorised as "better" or "worse". Just have a look at paintings of some of the well-known artists, example Picasso, and you will know what I mean.

[simondw]

I think a valid meaning might be "better at drawing what I want to draw." This is absolutely a skill that can be improved, even if the aesthetic quality of the output can't be evaluated objectively.

[carefree0910]

Lol maybe I got your meaning!

By saying 'better', I'm not actually 'willing' to become better. In fact, I enjoy drawing even if the outcomes are 'bad'.

The hard part is, when I want to draw something 'concrete', I can't... For example, a story between a boy and a girl - The only thing I can do is to draw two sticks in order to make them consistent frame by frame XD.

So yes, I have to admit that I still want to share my creations with others, and without being better I can hardly do so... I think artists like Picasso are at another level, because they might treat art as their whole life, and might have the ability to focus and only focus on their own arts.

And me, as a normal person, will still need some compliment XD.

[com2kid]

> The hard part is, when I want to draw something 'concrete', I can't... For example, a story between a boy and a girl - The only thing I can do is to draw two sticks in order to make them consistent frame by frame XD.

If you ever take an intro art class, you will spend a lot of time doing boring exercises like drawing a bowl of fruit, again and again.

This teaches you how to really "see" something, how to understand the shape, lighting, the true details. When we look at a thing we typically briefly glance at it long enough for our brain to categorize it, and then we move on. To get good at drawing, you need to train your brain to truly see things.

Then you learn how to hold something in your mind's eye and draw it. You start by briefly glancing away, drawing, then you look away for longer and longer, and keep drawing.

Drawing is not a mechanical skill of your hands, it is a mental skill.

[tchaffee]

Drawing is also a mechanical skill. Drawing a perfect circle for example takes practice, and does not involve the kind of mental / seeing skills you described.

[UncleEntity]

> Just have a look at paintings of some of the well-known artists, example Picasso, and you will know what I mean.

Picasso was a very good artist, look at some of his “better“ works where he was just doing a portrait or something and he’s up there with any other artist known to history.

You kind of have to be a really good technical artist to be a really good “bad” artist.

[Gigachad]

Yes it can or else there would be no such thing as practicing. Just because it’s somewhat subjective doesn’t make it all entirely subjective. This extends to many other things like food, music, beauty, etc. while people have different tastes, we can easily identify the traits and examples of “better”

[tchaffee]

Drawing absolutely can be categorized as better or worse if you are talking about its accuracy to reality, or your ability to draw exactly what you want. Which is what people are talking about when they say they are not good at drawing.

[moritonal]

Everything is built on the shoulders of others. A 'computer' was once a clever worker with a sliderule and most stop motion is now made in 3D software. We use tools, just as artists will use this to rapid prototype.

[smrtinsert]

The time to create assets is immense even for those who can draw.