Automatic lyric track alignment

[AAH]

Over on Falcon Christmas someone posted about having issues with a python script to do automatic alignment of lyrics to audio files. This is potentially well worth following as it is the first major step when synching lyrics to music. Between this and the papagayo stuff in xlights it's possible that you could get a lyric track done in minutes rather than hours or more.

Problem with python script to generate xtiming lyric track

Problem with python script to generate xtiming lyric track

falconchristmas.com

AutoLyrixAlign

<<<-- this is the site that does the cool stuff.

auto-lyrics

Xtiming generator for singing faces

pypi.org

<<<--- Ben's page to convert from the good stuff to the better stuff

GitHub - chitralekha18/AutoLyrixAlign: Pre-trained model and script to automatically align lyrics to polyphonic audio

Pre-trained model and script to automatically align lyrics to polyphonic audio - GitHub - chitralekha18/AutoLyrixAlign: Pre-trained model and script to automatically align lyrics to polyphonic audio

github.com

<<<--- github repository for the good stuff

It seems that there is some sort of worldwide competition to create programs to synch lyrics to songs. The AutoLyrix creators look to have won at MIREX 2 years in a row.

 

[OzAz]

Agree with @AAH (FB comments) the audacity file from AutoLryrixAlign looks like it would be easier to turn into a xtiming track

 

[AAH]

I have had to add some silence to the start of some tracks to get them to line up with premade lyric tracks ^_^

That will sometimes be due to the source of the audio files. If you're not using the exact source mp3/mp4a etc file that the person who created the track with there can be slight differences.

 

[Max Kabilafkas]

Went to try it myself but the auto-lyrics page does not seem to be working

 

[OzAz]

how odd it was there this arvo

 

[AAH]

I left out the link to the good bit. It's possible the github link is down as Ben may be working on an upgrade.
https://autolyrixalign.hltnus.org/ is the page that does the good stuff.

 

[AAH]

The lights community is awesome.
Phillip Littlewood created a notepadd++ macro to process the lyric file and also did a video.

View: https://www.youtube.com/watch?v=CGwlmtLDaTI

 

[OzAz]

He's quick that guy.

 

[Joshaaye]

Well this is bloody cool! Definitely going to have to test it out.

 

[OzAz]

Agree with @AAH (FB comments) the audacity file from AutoLryrixAlign looks like it would be easier to turn into a xtiming track

After watching Phil's video, all he's done is change the case of the words and replaced the spaces with tabs, so it was fairly easy. Just needed someone smarter than me (which wouldn't be difficult ) to work out that was all that was needed.

You can do it without a macro if you want (though Phil's macro makes it 1 click instead of a few) or record your own macro.

with the text in Notepad++
Ctrl+A (select all)
Ctrl+U (convert to lowercase) or Alt+U (convert to proper case - upper case first letter) - xlights doesn't seem to care which
Unselect all
Ctrl+H (to open Replace dialog box)
enter a space in Find what: text box
enter \t in Replace with: text box (converts space to tab)
click Replace All, close the replace dialog box
Save


Don't expect this to be perfect. Have tried it with 4 lyric tracks and there's problems with them all. The autolyricalign site does have difficulties with more than one voice singing at same time. In hey come look at my house it missed the word 'music' nearly every time, I think because the voice trails off as it's sung. It got Thunderstruck pretty good, the lyrics I submitted were missing the la las near the end so it just had one big long oh.

Certainly think using autolyricalign and then tidying the audacity file result would cut down the time to do a lyric track by heaps, especially if the song just has 1 main singer.

 

[Joshaaye]

Next project will be getting the phonyms to import to. That would be incredible.

 

[Joshaaye]

Trying this out for the first time. Does it normally take over 30 minutes for 3 a minute song? I'm at just over 30 minutes and still not finished processing it for me to download the file. (Tried twice now with long waits)

 

[LawrenceDriveLights]

I used it last week and took about 3 minutes to do one song

Maybe try another song and see how that goes

 

[Joshaaye]

Too easy, will try it on another song later.

 

[OzAz]

Trying this out for the first time. Does it normally take over 30 minutes for 3 a minute song? I'm at just over 30 minutes and still not finished processing it for me to download the file. (Tried twice now with long waits)

No, I've done 4 songs now, each one takes about as long as the song is. I've used mps, are you using youtube?

I've also attached a sequence (Thunderstruck. Sequence from the shared google drive), it has 2 lyric timing tracks, the raw and my corrected one. This is just so people who might be interested, can see a "before and after". My corrected one may not be 100% perfect to some people but good enough for this test.

• results_lyrix_thunderstruck which is the raw timing from https://autolyrixalign.hltnus.org/
• thunderstruck my modified/corrected timing track
• chorus additional track for the chorus lines, this was fairly easy to add as most of the words were in the original raw track

As you can see lyrix is not perfect, but it only took me about 2 hours to do the corrected one, usually would take me at least 10 hours, The la la las weren't in the lyrics txt I uploaded so had to add them and that took a while.

As I think I've mentioned earlier, for songs like Thunderstruck where there's basically only 1 singer with occasional chorus, lyrix works reasonably well. I think it would struggle with a song with lots of singers layering over each other.