Is this same-language, or does it work for translated subtitles?
I have some Korean TV DVDs with Japanese subtitles I was trying to use as a guide for adding in US subtitles that came from a different source. I assumed that aside from commercial break timings (one break in the middle) and start/end gaps they should be the same, but my results automating based on timing clusters didn't work too well. Some subtitles were split into multiple segments, sometimes grunts or ambient noise had subtitles, etc.
The model was trained with features of human voice bound to a frequency range so it may work for "cross-language" sync. Why not give it a go and check the quality? It won't change the content of original segments but only shift them along the timeline if there are gaps.
Just realised another user reported that it did not work well for Russian movie and Polish subtitles. Nonetheless, it doesn't stop you from training your own subaligner with those media assets you possess.
That makes me wonder how much of the work is being done by aeneas vs. your own model. If parts of the audio are in a low voice or noisy (which tends to cause aeneas to slip) will subaligner be able to fix that?
Aeneas is used for stretching pre-synced subtitle cue blocks, which is still experimental in subaligner. If durations of cues are correct in you case, there is no need to use this feature and passing in flag -so will switch it off. So why not get rid of stretch and see the difference?
What I know is Aeneas is using DWT which does not guarantee triangle inequality and for low voice or noise, DNN can handle those better with good enough model capacity.
I tried it with a Russian movie [1] and Polish subtitles. With a single pass it was still off a bit, and dual pass didn't really work super well either. Nevertheless, interesting technology!
Oh good to know! Never tried that combination before. Maybe this was due to the model pre-trained with the speech in English. Nonetheless, have you tried switching off the stretch with "-so"?
This is magic, how is this done? This is amazing, desynced subtitles are the bane of my existence, if this tool can fix that this day will get even better.
The tool uses ffmpeg to load the video, and according to the "anatomy" section it's based on mel-frequency cepstral coefficients, so it's only using the audio to do the alignment.
Maybe it is my bubble speaking, but the only subtitle synchronization I ever needed were either a constant or a continuously increasing/decreasing offset, both easily solved without pretrained deep neural networks.
That's a good bubble to be in. I very often happen upon subtitles that need an offset, a constant multiplier because they run out of sync, and sometimes have a gap somewhere. It's very annoying, and I'm glad there's software to fix them without me having to faff about for ten minutes.
Unsynched subtitles are hell. I don't know how Kodi knows that a subtitle in open subtitle is "synched". Looks like magic to me.