Automated lights with Python and some AI

[tume]

A showcase of some Python code I've written to automate DMX lights with QLC+.

Using an mp3 file, the code runs the song through some models to predict bpm, find segments, and split instrument tracks. Some simple audio analysis with Librosa on volume helps with recognizing the more energetic parts. Then some hardcoded scripts are fitted to those segments and strobes based on drum onsets are added (really only works with metal).

The code writes a script for each element separately and synchronizes them, after which they are placed into a collection that contains the whole show. All of that, as well as some chasers and buttons (for the fun of it), are written to a new QLC file built on an existing setup.

This is a very experimental project and the code needs a lot of refactoring. I've built the program on WSL2, and getting all the necessary installations can also be annoying. I can try running the code for your lighting rig or give insights on the code if interested. Add me on discord username: _tume_

The models: https://github.com/mir-aidj/all-in-one? ... me-ov-file

Song is Sleepwalker by Parkway Drive

[prophy17]

Yes. It is a very interesting theme. I am not a professional in the soft industry. But I have a big interest for your great work. But I have not discord

[prophy17]

I would like to find for example automatically by AI places in the track, where strobo effect should be start and stop. Would you mind to help with this?

[prophy17]

And one more theme. I am interested in colormusic effects for lyric compositions. Trigger of QLC+ and even beat signal of OS2L plugin of Virtual DJ often can not help with such music. Do you have any works with AI in such direction?

[GGGss]

(really only works with metal)

Vladimir, the OP, already quoted the limitations of such a system. I've seen some pretty nifty programs that can analyse music, but the outcome is always too harsh to be used practically. If you like Chinese light plans, go ahead. Bright, randomly flashy colours without any ambience or sphere. It will look like the wedding DJ with his four pars behind him.

[prophy17]

I think there is much work for AI in the World of Colormusic.

[prophy17]

Dear Mr GGGss thanks for your reply. I know about it. And so I think we should change such situation. AI should be the future of colormusic.

[prophy17]

And one more question for the author. Would you mind to answer why you use only Linux (from wsl2 how it has been written by you) and NATTEN, which has no good friendship with Windows. Many users work in Windows with music and colormusic.

[tume]

I would like to find for example automatically by AI places in the track, where strobo effect should be start and stop. Would you mind to help with this?

I used librosa's onset feature to find peaks in the drums track. Here's the code responsible for that:

https://github.com/tumffa/aidmx/blob/ma ... /onsets.py

With some tweaks, you could probably get it to work for different instruments or perhaps even just the snare drum, for example.

[tume]

And one more question for the author. Would you mind to answer why you use only Linux (from wsl2 how it has been written by you) and NATTEN, which has no good friendship with Windows. Many users work in Windows with music and colormusic.

Can't remember exactly, but I think the models were harder to get working on Windows. You could probably get it to work though. Sorry for not having put the time into making installing this easier.

[prophy17]

Thanks a lot, Dear Mr tume, for your reply. But I am sorry you did not answer for my question about using AI for colormusic with lyrical melodies? And as I have written before, I am not a professional in the soft industry. I know only VBA a little . So excuse me for my question, but there is a string at the start of your onset.py file:
dev detect_onsets(filename, threshold)
What does it mean? What filename (music track mp3 or wav with the Path for it or another one) and what threshold? Would you mind to send a real example of this string of your soft.

[prophy17]

And thanks one more time for your great work with AI for QLC+, Dear Mr tume. I wonder of it. I think it will give a start for using of AI in QLC+ for another users, who know Python or another language. As I have said, I think that: AI is the future of colormusic

[tume]

Thanks a lot, Dear Mr tume, for your reply. But I am sorry you did not answer for my question about using AI for colormusic with lyrical melodies? And as I have written before, I am not a professional in the soft industry. I know only VBA a little . So excuse me for my question, but there is a string at the start of your onset.py file:
dev detect_onsets(filename, threshold)
What does it mean? What filename (music track mp3 or wav with the Path for it or another one) and what threshold? Would you mind to send a real example of this string of your soft.

Sorry for being unclear. The filename is just the path to the audio file (I use the drums track).

This is the function I use to find the time windows for strobes https://github.com/tumffa/aidmx/blob/3a ... ts.py#L128.

After normalizing the onsets, the threshold is used to filter quiet onsets. Then it finds closely grouped onsets and creates time windows for them. Lastly, it further merges time windows close to each other.

The segments parameter is just a list of section start and end times, for example:
{
"start": 45.33,
"end": 59.79,
"label": "verse",
}

If you hardcode some filepath and a list of segments like [{"start": 0.00, "end": 60:00}] to the get_onset_parts function, you should get it to work

Also, if you haven't already, I suggest copy-pasting the code to some LLM to get some further explanations.

[prophy17]

Greate thanks for your reply. I wonder yours work one more time. LLM it will be wonderful. But for today I can't test even yours file... After it start on my IDLE - there is nothing .. unfortunately.. as I don't know Python at all, maybe I do something wrong. For example: path and name of some music file should be only in the:
"def get_onset_parts"
as your drum.wav
or in the start of yours script in the:
"def detect_onsets"
too.
In the file under yours link there is only:
def detect_onsets (filename, threshold).
And in what place should be install start and end time? There is not one place in the file, where I can see "start" and "end" position ..And what does it mean Demix Path in "def get_onset_parts" ? Real path to the file from name of the disk (C:/....)? In your file there is only ("htdemux"). I am very very sorry for my weak knowledge in Python((..

[prophy17]

And I am very surprised that unfortunately for now I have been only one, who was interested in your work with AI in QLC+.

[tume]

Greate thanks for your reply. I wonder yours work one more time. LLM it will be wonderful. But for today I can't test even yours file... After it start on my IDLE - there is nothing .. unfortunately.. as I don't know Python at all, maybe I do something wrong. For example: path and name of some music file should be only in the:
"def get_onset_parts"
as your drum.wav
or in the start of yours script in the:
"def detect_onsets"
too.
In the file under yours link there is only:
def detect_onsets (filename, threshold).
And in what place should be install start and end time? There is not one place in the file, where I can see "start" and "end" position ..And what does it mean Demix Path in "def get_onset_parts" ? Real path to the file from name of the disk (C:/....)? In your file there is only ("htdemux"). I am very very sorry for my weak knowledge in Python((..

I modified the file to make it easier. Just replace your_filepath with the path to the audio file: https://github.com/tumffa/aidmx/blob/ma ... ts.py#L149

[prophy17]

Thanks for your reply. I understand - you have not many time for my personal questions. And thank you for your work one more time. Good luck