Hacker News new | ask | show | jobs
by zsikelianos 1200 days ago
Hey, Scribble Diffusion author here.

Thanks for your kind words and feedback. I love seeing all these links to your scribbles.

I'm an engineer at Replicate, which is a place to run ML models in the cloud. [0] We built Scribble Diffusion as an open-source app [1] to demonstrate how to use Replicate.

This is all built on ControlNet [2], a brilliant technique by Lvmin Zhang and Maneesh Agrawala [3] for conditioning diffusion models on additional sources, in this case human scribbles. It also allows for controlling Stable Diffusion using other inputs like pose estimation, edge detection, and depth maps.

ControlNet has only existed for three weeks, but people are already doing all kinds of cool stuff with it, like an app [4] that lets you pose a stick figure and generate DreamBooth images that match the pose. There are already a bunch of models [5] on Replicate that build on it.

I see a few bits of feedback here about issues with the Scribble Diffusion UI, and I'm tracking them on the GitHub repo. If you want to help out, please feel free to open an issue or pull request.

[0] https://replicate.com

[1] https://github.com/replicate/scribble-diffusion

[2] https://github.com/lllyasviel/ControlNet

[3] https://arxiv.org/abs/2302.05543

[4] https://twitter.com/dannypostmaa/status/1630442372206133248

[5] https://replicate.com/explore?query=controlnet

3 comments

What a simple, but excellent concept! I kind of expected it to crash after being posted to HN, but surprisingly it is still going strong. What does it cost to host something like that? What does it cost to generate each scribble image?