[jzemeocala]

Awesome work, I've had the idea for a while of setting up a pipeline like this that could take input from all available sources of the person to clone their voice and image as well as dialogue.

The intent being to create digital avatars of lost loved ones to help people with the grieving process.

I know that there would be tremendous opportunity in such tech for malicious actors to do serious harm, but the stated goal is still a worthwhile endeavor.

[advaith08]

Yeah, this is definitely a dicey ethical question. Would be interested to know what guardrails you're considering for these digital avatars, and how you'll ensure that people use them in a healthy manner and dont get dependent on them.

[jzemeocala]

Right now I haven't gotten past the drawing board phase. But I have spent more than a few hours hashing out the morality of the idea with friends and family. In the end, I always come back to the fact that the tech itself would HAVE TO BE FOSS; Ipso facto, the extent of any guardrails would have to be limited to sternly worded recommendations for how it SHOULD be used. Much like the stance that the US NAVY has taken with the TOR project.

Edit:

However, I also know that the average grieving person doesn't necessarily have the skills or desire to compile source, fine-tune a llm, etc..., so there would most likely be an opportunity for people to create a paid turnkey system wherein a digital archivist helps the grieving person create the avatar. In such a system, I would probably recommend that the product be tied with:

1) A Grief Counselor

2) a nag system for when interaction time reaches unhealthy levels

3) some method of alerting the grief counselor directly when certain thresholds of toxic interaction are met