I like this a lot. I've been fascinated by Suzuki Toshitakas work on mapping bird sounds to syntax and want to experiment in that space too.
Ive been assisting at a wild bird rehab but not until I got pet birds (released pet birds that no owner could be found for) did I realize they make these extremely faint sounds to each other that I can sometimes just barely make out when I'm right next to them but are not the other quiet humming they make.
My mic can capture those sounds sometimes, but I don't know how to analyze for example 24h of recording in the cage to find slight variations to background noise. It doesn't have to be real-time and not bird specific (want to capture sounds they make that doesn't register as bird in the models).
If anyone has a suggestion please point me in any direction you know of. Audio is pretty new for me.
This is a bit off-the-wall but maybe you could use a variant of the approach used to see tiny movements in video? There you offset two clips by a frame and "subtract" them so that anything not moving just kind of vanishes, and purposeful movement shows up as a bright line.
Maybe doing something similar with a spectrogram would work? Two spectrograms, one delayed slightly with respect to the other, and subtract one from the other, and you might see bright spots that appear where the sound changes.
Building off that idea, if the offset between the two spectrograms was based on the relative distance between the two mics (ideally 3), you could eliminate noises not coming from a certain point in space, e.g., the birdcage.
I wanted to do something similar to this, then I started doing some research on birds in general, and those in my locality, then I started learning about Audio and spectograms and Nyquist Theorem and many other interesting audio stuff.
Then I started going through the Intro to Conservation Bioacoustics by Cornell course, and started watching Bioacoustic Talks by the K. Lisa Yang Center cornell center.
And now I am almost at the point where I cant start manually tagging audio sets, for target species so that I can train custom classifiers to identify birds in Rwanda which are poorly detected by birdnet.
TLDR: Being jobless can lead you into interesting ventures.
Thanks for sharing these resources and your story! I followed a very similar path, and ended up doing a biodiversity related MSc, with my dissertation being a custom classifier for poorly detected species in Príncipe. BirdNET and Perch are phenomenal achievements, but struggle in regions where, ironically, most of the world’s biodiversity is. What you’re doing for Rwandan species is so important!!
I have alas not published it yet, but I really should. How about you?
For me, on the research front, I’m very interested in methods that can be sustainably applied in remote, resource constrained locations. Heavy cloud dependent workflows to adapt a huge foundation model just aren’t practical on an island that doesn’t have 24/7 electricity and sporadic connectivity.
I know google has general sound classifiers like Yamnet, trained on youtube data but they are not very good for specific usecases. So you would have to create a custom model for you usecase.
just thinking, I've talked with a mechanic but he told me that now when they connect the car to a computer they almost always find anything wrong with a car, that and the experience they have they almost always know what's wrong.
I think sound + location could be really interesting, because you can filter parts of the car that could be making noises that are similar knowing where the mic is.
If you dig this idea, highly recommend the Cornell Merlin Bird ID app (likely built on the exact tech used here). Use it on my walks every week and it spots some wild stuff (it even correctly identifies birds who are being mocked by a Mockingbird). Helped me learn the Eastern Blue Bird call (personal favorite) so I know when they're nearby.
Excellent kachō-e prompt - working on something similar and found it hard to get the right balance between sharp outlines and watercolors, and especially plant morphology is dicey (eg plants like Cacao that fruit from the trunk instead of branch tips).
Did anyone come across projects that also nail that aspect well?
Ive been assisting at a wild bird rehab but not until I got pet birds (released pet birds that no owner could be found for) did I realize they make these extremely faint sounds to each other that I can sometimes just barely make out when I'm right next to them but are not the other quiet humming they make.
My mic can capture those sounds sometimes, but I don't know how to analyze for example 24h of recording in the cage to find slight variations to background noise. It doesn't have to be real-time and not bird specific (want to capture sounds they make that doesn't register as bird in the models).
If anyone has a suggestion please point me in any direction you know of. Audio is pretty new for me.