Show HN: Riffusion with Lyrics

[haykmartiros]

Almost a year ago someone stumbled upon our hobby project and posted it to HN (edit: https://news.ycombinator.com/item?id=33999162). It was magical to see so many of you enjoy something we built for our love of music. We’re back today with a real product and are excited to see what you think!

riffusion.com lets you create original music via short, shareable audio clips called “riffs”. You describe the lyrics and a musical style to generate riffs complete with singing and custom artwork in a few seconds. From inspiring musicians, to wishing your mom a ‘good morning!’, we are having a lot of fun with riffs as a form of expression and communication.

We hope you have as much of a blast traveling the latent space of music as we did making it. You can generate music from text, remix other riffs, or record your own lyric timings.

A riff for HN: https://www.riffusion.com/riffs/ac7ff4c3-d86d-4dfa-884e-bb19...

Hayk + Seth

[dang]

Related ongoing thread:

AI-generating music app Riffusion turns viral success into $4M in funding - https://news.ycombinator.com/item?id=37915393 - Oct 2023 (49 comments)

[isoprophlex]

I can't believe how much you managed to improve this since that first HN post. You seem to have solved the phase distortion problem completely, and the lyrics are just insane.

A more fully featured UI to eg. align lyrics & append more (temporally consistent) generated music and you have a killer app.

Super well done :)

[cmaury]

Riffs need a downvote button. I'm not sure what the team's training/eval approach is, but you need a user signal for poor quality results, so you can improve over time.

ex: https://www.riffusion.com/riffs/2a42259b-1557-4913-bc04-5f0a...

Amazing progress since the original announcement.

[abledon]

I am blown away at how the lyrics sync up with the melodies... like some of the outputs are JINGLES. that a company would pay an agency like 300,000$ to come up with in 1995. The audio quality is so good, akin to an old VHS casette you'd find playing a commercial or something.

Just incredible.

feature requests:

- Need a Delete button on the song tiles so we can garbage the bad generations we don't want to keep.

[rhythmofrest]

I absolutely love this, and it nails genre really well. I think it’s optimised for English, right? The result of trying to get it to speak Spanish is pretty funny!

An attempt at norteño/gaucho/polka: https://www.riffusion.com/riffs/28aeb4a7-cf41-4efc-9519-3a10...

Reggaeton: https://www.riffusion.com/riffs/1afbbb7a-9987-4e17-89af-1805...

[seth_]

Nice! It is best at pronouncing in English, but we've had a bunch of fun trying to get other languages too. Sometimes you can make things happen phonetically.

Even for english words that it doesn't get right the first time haha

[rhythmofrest]

It produces this beautiful nonsense in Japanese also, and even correctly extracted the topic from the Japanese lyrics : https://www.riffusion.com/riffs/71fa41f3-1488-4e9e-8f99-5f81...

[seth_]

Very cool. The latent space is a wild place.

[ramoz]

country: https://www.riffusion.com/riffs/79a0fa02-5ee6-4fdf-bdba-9aac...

[moralestapia]

Really nice!

Made this one, https://www.riffusion.com/riffs/48311e5a-ec7e-4e37-9631-3c76...

Some hallucinations at the end, but pretty good overall, and fun!

[unshavedyak]

This is kinda crazy to say for me.. but this is one of the very few apps i actually want an iOS app for. Which to me personally, signals the value of the service. Great job

edit: Though it feels a bit weird to need my phone number from a desktop hah. Interesting mix of not-mobile-first-yes-mobile-first. GPU cost mitigation for bots i assume?

[seth_]

We would have an iOS app if we were better mobile engineers... hopefully someday we'll make one!

[Alifatisk]

I highly suggest Flutter if you are open for ideas!

[divyapatel12]

I am blown away by how well the country and opera genres work. Country: https://www.riffusion.com/riffs/746af7e4-0578-4688-9a5f-69e3... Opera: https://www.riffusion.com/riffs/7bce0bd1-4faf-4919-ac66-a5d0...

[seth_]

Those are awesome! Sorry if the app is down for you or anyone else right now btw. Our servers are a little overloaded but we'll be back up in no time

[speial]

Is it still open source?

The GitHub seems kinda stalled with the last changes from 7 months ago?

[vlugorilla]

Seems like it's no longer open source, as you say old repo seems abandoned, but I'd also like to know if it's somewhere else or why they decided to close it.

[speial]

Got an error when I tried to make a riff:

    TRPCClientError: JSON.parse: unexpected character at line 1 column 1 of the JSON data
    NextJS 3
    TRPCClientError
    from
    B
    Caused by: SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data

[realrogerchen]

Sorry guys i gotta go. I need to return some video tapes: https://www.riffusion.com/riffs/ee65d404-536c-4f54-a7f1-0a38...

[al_be_back]

>> Almost a year ago someone stumbled upon our hobby project and posted it to HN

which HN post was that?

[seth_]

This one! Was a wild day for us :)

https://news.ycombinator.com/item?id=33999162

[al_be_back]

nice

[earthnail]

Riffusion is great! Wishing you the best of luck, Hayk and Seth.

Question: what are role models of companies that generated a fun "toy" like yours (hope you don't mind the word; not meant negatively) and somehow figured out how to monetise it later?

[seth_]

We're happy to be building a toy!

Your comment reminds me of this post: https://cdixon.org/2010/01/03/the-next-big-thing-will-start-...

It's still really early innings for this technology, so we're happy to be learning and building fun technology helps people to do creative things. Main thing we're focused on is turning it from a toy you enjoy once, to something you come back to and dive deeper into (even if it's still just for fun).

[earthnail]

I agree, but my question is: what are specific companies that pulled this off well that you consider as role models?

Not meaning to put you in a corner, just really curious about this - I love Riffusion and hope you guys succeed.

[fishkingsin]

Love to see multiple languages support

[eknight15]

These are sweet: https://www.riffusion.com/riffs/48375c60-7b44-41d7-8aec-0ce5... https://www.riffusion.com/riffs/205bd650-4453-4239-a5ec-0e72...

[neom]

Fun.

How long till I can write a full song, "riffusionise" it and export an .als into Ableton? Then we're really cooking with gas. I was surprisingly impressed at how good of a job it did. Nice work.

https://www.riffusion.com/riffs/65768974-409b-4c93-943f-6f73...

[drev]

Just like this radio ad in GTA vice city

https://youtube.com/clip/UgkxdC87tLJ-QeYg-Qiue8PJpGdiiUFxOR1...

[c420]

Everytime I try and remix smth it starts processing and then tells me the lyrics don't work and to create smth original.

Edit: after playing with it for an hour, I agree with other posters that midi and/or .als export options would be very desirable. Worth paying for even.

[locusofself]

Are any of these music AI's trying to use VSTi instruments and the like? It seems like with the wealth of amazing quality instruments that just accept MIDI data, this would be a really good idea and require almost no compute power comparitively.

[mdrzn]

"These lyrics didn't work, try mixing it up with something original" doesn't give enough clues to figure out WHY those lyrics didn't work and what I have to change.

[tinytera]

This is great! For remixing, are you guys actually saving the latent assets/embeddings? Or regenerating the music from some "seed"?

[haykmartiros]

Remix is using latent space interpolation under the hood to generate three variations of a given riff with (light, medium and heavy) modifications

[Yenrabbit]

This is excellent! I've been turning my family's memorable sayings into smooth jazz. Congratulations

[sisomercy]

LFG..again!!!

[tsaiDavid]

Congrats! Love seeing this project take off in this way, it’s so exciting.

[gremlinsinc]

well.. f**k... I lost all hope of being productive today.

[milkweed]

What fun! Made some awesome birthday greetings.

[smusamashah]

Please add a way to delete riffs.

[haykmartiros]

as of a few minutes ago, you can mark individual riffs as public or unlisted, in addition to the global setting!

[abledon]

AMAZING

[sjkoelle]

awesome, good job

[m3kw9]

I don't see the appeal? It's pretty stty music, fine it has good novelty, but would I keep clicking it for 10 second riffs of mediocre songs? I do see a viral angle, and that is good because as an app dev, getting something viral is a home run hit. But I don't see it grabbing me to come back, hopefully there is going to be a lot more features to keep people coming back

[bemmu]

I just lost an hour to this site, it's a lot of fun.

Could have potential to generate a one-in-a-million hit that becomes a tiktok sensation just from the pure volume of these mediocre songs where one just happens to be a diamond.

Check out my soundcloud https://www.riffusion.com/riffs/3a91e0e4-a069-4282-b060-c389...

[m3kw9]

I can see people attaching songs to TikTok, not! TikTok would have their own version out before they allow Riff to easily attach outside music on their platform.

On a footnote, everyone of the songs seem to sound a bit derivative in a very “I’ve heard that somewhere before” vibe, you may get lyrics that vibes but it all seem “weak”. I’m just saying it as it is, not trying to be mean or putting the devs down, it’s just honest useful critique

[earthnail]

When the drum machine was released, tons of musicians said it sounded rubbish and not like real drums. Time will tell whether this falls in that same category, but generally it’s easy to underestimate new stuff.

[vunderba]

Mmm, this really isn't the same thing. With a drum machine you're still programming the riffs themselves and people's chief complaint was the sound itself, whereas I've listened to a number of samples from this and the quality is definitely there, but the lyrical melodies themselves are not particularly harmonically engaging or memorable. Like the voices are just all over the place on the scale.

That being said, this is definitely a "meme" kind of thing rather than a tool for actual orchestration.

[earthnail]

I agree with the control thing, but I can also see that this can be solved. Not sure how but that’s the whole point of their funding round - figuring out how to make it fun in a lasting way.

The non-memorable bit… I’m certain people complained that all drum sounds from drum machines sound the same and are thus not memorable. So I’m cautious here personally, even though I see your point.