Nutcracker: Help understanding the asq/msq file for LSP

[smeighan]

Well, i am willing to give another try at making a native asq file for LSP.

I will work on making a msq file for lsp but first need someone to explain why this doesnt work?

msq or asq files are 7zip archives.

I made a simple 5 rgb (15 channel) sequence in LSP 2.0 that is 10 seconds long.

http://meighan.net/nutcracker/sequences/JUL6.asq

So i did a 7zip to unzip this.

I then saw the xml file called JUL6

Before doing anything else, i wanted to see that i could then 7zip this back

so i added JUL6 into a 7zip archive and got a JUL6.7z.

Didnt touch anything, what was strange is that this new file is smaller than the original asq file

JUL6.asq was 4,199 bytes
JUL6 (The xml file after unzipping) was 34,737 bytes
JUL6.7z was 2,970 bytes. It is smaller than the original file.

I renamed JUL6.7z as JUL6_after.asq

When i try to open this file it gives an error.

Unless i cam figure out how to get an xml file out and then get it compressed back as an asq file, i will not start to reverse engineer the xml file.

any ideas why this didnt work?

 

[AAH]

Sean I just had a look at the file and as I don't have 7zip and haven't had that proggy for years i opened it with winrar. Winrar reported to me that the file was a Gzip archive. Out of curiosity I then zipped up the contents again and opened it up with winrar again. It showed that it was a native zip file which suggests that there was a definite difference in the file format.

 

[lithgowlights]

I believe the 2.5 files are a different archive type. I think it is just ZIP - want me to send you one from 2.5 to see?

Yes 2.0 report as GZip here, and 2.5 are reported as Zip files in WinRar

 

[smeighan]

I believe the 2.5 files are a different archive type. I think it is just ZIP - want me to send you one from 2.5 to see?

Yes 2.0 report as GZip here, and 2.5 are reported as Zip files in WinRar

were you able to unzip it, then zip it and then open that zip file in LSP?


What i am looking for is how to take an xml file i have and get LSP to open it. If someone has the steps, that would be great.


Has anyone modified the xml before and got it to load into LSP?
thanks

 

[lithgowlights]

No it wont reopen in LSP ???

 

[kane]

I believe the 2.5 files are a different archive type. I think it is just ZIP - want me to send you one from 2.5 to see?

Yes 2.0 report as GZip here, and 2.5 are reported as Zip files in WinRar

Yes, just confirming - LSP 2.0 uses GZIP (as it is just one single XML file that is compressed). LSP 2.5 uses a standard ZIP, as it has multiple files - one main file, and then a subdirectory with a single XML file for each controller.

were you able to unzip it, then zip it and then open that zip file in LSP?

What i am looking for is how to take an xml file i have and get LSP to open it. If someone has the steps, that would be great.

Has anyone modified the xml before and got it to load into LSP?

Yes, I have been able to do this - by un-gzipping, making changes to it, and re-gzipping - and it is able to be opened back in LSP 2.0 okay. I did this a lot last year..

I used the inbuilt GZipStream classes in .NET to do this.

Can I just ask what the plan is - do you plan on creating an .asq (or .msq) file from scratch, or would you plan on people uploading their .asq file, and you applying your effects to it?

 

[smeighan]

I believe the 2.5 files are a different archive type. I think it is just ZIP - want me to send you one from 2.5 to see?

Yes 2.0 report as GZip here, and 2.5 are reported as Zip files in WinRar

Yes, just confirming - LSP 2.0 uses GZIP (as it is just one single XML file that is compressed). LSP 2.5 uses a standard ZIP, as it has multiple files - one main file, and then a subdirectory with a single XML file for each controller.

were you able to unzip it, then zip it and then open that zip file in LSP?

What i am looking for is how to take an xml file i have and get LSP to open it. If someone has the steps, that would be great.

Has anyone modified the xml before and got it to load into LSP?

Yes, I have been able to do this - by un-gzipping, making changes to it, and re-gzipping - and it is able to be opened back in LSP 2.0 okay. I did this a lot last year..

I used the inbuilt GZipStream classes in .NET to do this.

Can I just ask what the plan is - do you plan on creating an .asq (or .msq) file from scratch, or would you plan on people uploading their .asq file, and you applying your effects to it?

I want to generate the xml file and then compress it into an asq file. This way nutcracker effects could be loaded as native files.

 

[kane]

I want to generate the xml file and then compress it into an asq file. This way nutcracker effects could be loaded as native files.

Given that all the controller information is stored as part of the LSP file, I really don't think that would be a good idea, as you would have to generate the whole file upfront (keeping details of all the controllers for each user, etc) I have been suggesting to David at LSP that the controller files should be kept outside of the sequence files, but I don't know if he's keen on doing that.

My opinion is that you'd be best off allowing people to upload a blank (or perhaps not blank!) .asq file (with all their controllers setup, and song chosen etc). You would then allow the user to apply a nutcracker effect against certain controller(s) for a certain time period.

That's pretty much what I do - I load my .asq file into my tool, choose which controllers to apply my effect to, and at what point in the sequence, and then apply the effect. The tool then saves the file and I then load it back into LSP for previewing, and editing other channels. I might load it up 10-15 times between LSP and my tool.

 

[smeighan]

I want to generate the xml file and then compress it into an asq file. This way nutcracker effects could be loaded as native files.

Given that all the controller information is stored as part of the LSP file, I really don't think that would be a good idea, as you would have to generate the whole file upfront (keeping details of all the controllers for each user, etc) I have been suggesting to David at LSP that the controller files should be kept outside of the sequence files, but I don't know if he's keen on doing that.

My opinion is that you'd be best off allowing people to upload a blank (or perhaps not blank!) .asq file (with all their controllers setup, and song chosen etc). You would then allow the user to apply a nutcracker effect against certain controller(s) for a certain time period.

That's pretty much what I do - I load my .asq file into my tool, choose which controllers to apply my effect to, and at what point in the sequence, and then apply the effect. The tool then saves the file and I then load it back into LSP for previewing, and editing other channels. I might load it up 10-15 times between LSP and my tool.

I would love an easier file to produce than creating a full asq file.


Have you looked at the hls output?
try generating an effect and select hls as the output )*.hlsnc.
This file has one line for each channel. Eask token represents a frame. The values are 24bit RGB values.


Maybe your tool could open this file and process it into the asq?
that would be great , if it were possible.

 

[kane]

Have you looked at the hls output?
try generating an effect and select hls as the output )*.hlsnc.
This file has one line for each channel. Eask token represents a frame. The values are 24bit RGB values.


Maybe your tool could open this file and process it into the asq?
that would be great , if it were possible.

I'll have a play when I get a chance and see what I can do.

I'm yet to adapt my tool to support the LSP 2.5 file format, but I can't see it being too difficult - from my quick look, all that has been done is the XML for each controller has been spun off into it's own file

 

[kane]

I'll have a play when I get a chance and see what I can do.

I'm yet to adapt my tool to support the LSP 2.5 file format, but I can't see it being too difficult - from my quick look, all that has been done is the XML for each controller has been spun off into it's own file

I figured I'd take the plunge and switch my tool over to support the 2.5 file format, and all went well until I tried to save the file again. It seems that LSP cant read in my zip archive. I've emailed David to ask about it, so hopefully I'll get a reply when he gets back from holidays...

Doing a bit more testing tonight, if I unzip a .msq file, and rezip it (without modifying it), LSP fails to open up the file, which lines up with LL's test results. I've tried zipping up with my .NET libraries, with WinZip, WinRar and 7-Zip, and none of them are able to be loaded by LSP. (Interestingly, all 4 of these end up with different file sizes when zipping up the same files)

Once I get past this issue, I think it'll be relatively straightforward to apply the .hlsnc data to a sequence.

 

[kane]

Okay, worked out the zip file issue. After a lot of experimenting, and delving into the innerds of zip files, it seems as though LSP requires particular comments to be stored against each file in the zip archive as metadata ("LightShow Pro Light Sequence" against the whole zip archive, "Sequence" against the sequence file, and "Controller" against each controller file) So if I add these in, I'm able to get my exported .msq file to load up in LSP.


So I've written a quick and dirty import procedure, and tested it using a 10 second nutcracker animation - a matrix of 18x36 (648 channels). My tool took several minutes to load this data in and save the xml, but it worked eventually. Will have to look at more efficient ways of manipulating the xml.


I'm not 100% sure if the effects in LSP line up with what nutcracker produced, but will experiment more with that when I get a chance.


The resulting .msq file was around 2Mb, and uncompressed it was around 60Mb, so I can see issues popping up here - if you had 2000 rgb channels and 3 minutes of nutcracker effects, you'd be looking at around a 3-4GB file, and I have no idea if LSP could handle that..

 

[lithgowlights]

Absolutely brilliant news - I cant wait to give it a try

 

[smeighan]

Okay, worked out the zip file issue. After a lot of experimenting, and delving into the innerds of zip files, it seems as though LSP requires particular comments to be stored against each file in the zip archive as metadata ("LightShow Pro Light Sequence" against the whole zip archive, "Sequence" against the sequence file, and "Controller" against each controller file) So if I add these in, I'm able to get my exported .msq file to load up in LSP.


So I've written a quick and dirty import procedure, and tested it using a 10 second nutcracker animation - a matrix of 18x36 (648 channels). My tool took several minutes to load this data in and save the xml, but it worked eventually. Will have to look at more efficient ways of manipulating the xml.


I'm not 100% sure if the effects in LSP line up with what nutcracker produced, but will experiment more with that when I get a chance.


The resulting .msq file was around 2Mb, and uncompressed it was around 60Mb, so I can see issues popping up here - if you had 2000 rgb channels and 3 minutes of nutcracker effects, you'd be looking at around a 3-4GB file, and I have no idea if LSP could handle that..

Hi Kane; i brought up the issue of size last mar/apr. When i first looked at the LSP xml file, it is very chatty.
I wondered how big a full sequence of a RGB tree would take in LSP.


Forget Nutcracker, has anyone sequenced a 2000 channel tree for a 3 minute song using just LSP? How big was your asq file?


Maybe 2.5's plan to put each controller into a different file was meant to help the large file size?

 

[smeighan]

Okay, worked out the zip file issue. After a lot of experimenting, and delving into the innerds of zip files, it seems as though LSP requires particular comments to be stored against each file in the zip archive as metadata ("LightShow Pro Light Sequence" against the whole zip archive, "Sequence" against the sequence file, and "Controller" against each controller file) So if I add these in, I'm able to get my exported .msq file to load up in LSP.


So I've written a quick and dirty import procedure, and tested it using a 10 second nutcracker animation - a matrix of 18x36 (648 channels). My tool took several minutes to load this data in and save the xml, but it worked eventually. Will have to look at more efficient ways of manipulating the xml.


I'm not 100% sure if the effects in LSP line up with what nutcracker produced, but will experiment more with that when I get a chance.


The resulting .msq file was around 2Mb, and uncompressed it was around 60Mb, so I can see issues popping up here - if you had 2000 rgb channels and 3 minutes of nutcracker effects, you'd be looking at around a 3-4GB file, and I have no idea if LSP could handle that..

Hi Kane;


currently Nutcracker produces these files


Vixen: *.vir, *.vix
LOR: *.lms, *.lcb
HLS: *.hlsnc


I understand from your post that you had prototyped a tool that could take the hlsnc file and modify someones existing asq file?


any progress on this tool? I get pinged daily by lsp users who want a way to get nutcracker into lsp without going down the lms path.


i am so glad you are coming close to a solution.


thanks