Nutcracker: Help understanding the asq/msq file for LSP

Discussion in 'xLights (Nutcracker)' started by smeighan, Jul 9, 2012.

  1. smeighan

    smeighan Dedicated Elf

    Joined:
    Jan 19, 2012
    Messages:
    1,055
    Likes Received:
    4
    Location:
    4217 Greenfinch Dr CO 80126
    Well, i am willing to give another try at making a native asq file for LSP.

    I will work on making a msq file for lsp but first need someone to explain why this doesnt work?

    msq or asq files are 7zip archives.

    I made a simple 5 rgb (15 channel) sequence in LSP 2.0 that is 10 seconds long.

    http://meighan.net/nutcracker/sequences/JUL6.asq

    So i did a 7zip to unzip this.

    I then saw the xml file called JUL6

    Before doing anything else, i wanted to see that i could then 7zip this back

    so i added JUL6 into a 7zip archive and got a JUL6.7z.

    Didnt touch anything, what was strange is that this new file is smaller than the original asq file

    JUL6.asq was 4,199 bytes
    JUL6 (The xml file after unzipping) was 34,737 bytes
    JUL6.7z was 2,970 bytes. It is smaller than the original file.

    I renamed JUL6.7z as JUL6_after.asq

    When i try to open this file it gives an error.

    Unless i cam figure out how to get an xml file out and then get it compressed back as an asq file, i will not start to reverse engineer the xml file.

    any ideas why this didnt work?
     
  2. AAH

    AAH I love blinky lights :) Community Project Designer

    Joined:
    Dec 27, 2010
    Messages:
    2,507
    Likes Received:
    1
    Location:
    Eaglehawk
    Find Me On:
    Sean I just had a look at the file and as I don't have 7zip and haven't had that proggy for years i opened it with winrar. Winrar reported to me that the file was a Gzip archive. Out of curiosity I then zipped up the contents again and opened it up with winrar again. It showed that it was a native zip file which suggests that there was a definite difference in the file format.
     
  3. lithgowlights

    lithgowlights Senior Elf

    Joined:
    May 6, 2010
    Messages:
    742
    Likes Received:
    16
    Location:
    Lithgow, NSW
    Find Me On:
    I believe the 2.5 files are a different archive type. I think it is just ZIP - want me to send you one from 2.5 to see?

    Yes 2.0 report as GZip here, and 2.5 are reported as Zip files in WinRar
     
  4. OP
    OP
    smeighan

    smeighan Dedicated Elf

    Joined:
    Jan 19, 2012
    Messages:
    1,055
    Likes Received:
    4
    Location:
    4217 Greenfinch Dr CO 80126

    were you able to unzip it, then zip it and then open that zip file in LSP?


    What i am looking for is how to take an xml file i have and get LSP to open it. If someone has the steps, that would be great.


    Has anyone modified the xml before and got it to load into LSP?
    thanks
     
  5. lithgowlights

    lithgowlights Senior Elf

    Joined:
    May 6, 2010
    Messages:
    742
    Likes Received:
    16
    Location:
    Lithgow, NSW
    Find Me On:
    No it wont reopen in LSP ???
     
  6. kane

    kane Dedicated Elf

    Joined:
    Dec 23, 2010
    Messages:
    1,019
    Likes Received:
    0
    Location:
    Trigg (Northern suburbs of Perth)
    Find Me On:
    Yes, just confirming - LSP 2.0 uses GZIP (as it is just one single XML file that is compressed). LSP 2.5 uses a standard ZIP, as it has multiple files - one main file, and then a subdirectory with a single XML file for each controller.



    Yes, I have been able to do this - by un-gzipping, making changes to it, and re-gzipping - and it is able to be opened back in LSP 2.0 okay. I did this a lot last year..

    I used the inbuilt GZipStream classes in .NET to do this.

    Can I just ask what the plan is - do you plan on creating an .asq (or .msq) file from scratch, or would you plan on people uploading their .asq file, and you applying your effects to it?
     
  7. OP
    OP
    smeighan

    smeighan Dedicated Elf

    Joined:
    Jan 19, 2012
    Messages:
    1,055
    Likes Received:
    4
    Location:
    4217 Greenfinch Dr CO 80126

    I want to generate the xml file and then compress it into an asq file. This way nutcracker effects could be loaded as native files.
     
  8. kane

    kane Dedicated Elf

    Joined:
    Dec 23, 2010
    Messages:
    1,019
    Likes Received:
    0
    Location:
    Trigg (Northern suburbs of Perth)
    Find Me On:
    Given that all the controller information is stored as part of the LSP file, I really don't think that would be a good idea, as you would have to generate the whole file upfront (keeping details of all the controllers for each user, etc) I have been suggesting to David at LSP that the controller files should be kept outside of the sequence files, but I don't know if he's keen on doing that.

    My opinion is that you'd be best off allowing people to upload a blank (or perhaps not blank!) .asq file (with all their controllers setup, and song chosen etc). You would then allow the user to apply a nutcracker effect against certain controller(s) for a certain time period.

    That's pretty much what I do - I load my .asq file into my tool, choose which controllers to apply my effect to, and at what point in the sequence, and then apply the effect. The tool then saves the file and I then load it back into LSP for previewing, and editing other channels. I might load it up 10-15 times between LSP and my tool.
     
  9. OP
    OP
    smeighan

    smeighan Dedicated Elf

    Joined:
    Jan 19, 2012
    Messages:
    1,055
    Likes Received:
    4
    Location:
    4217 Greenfinch Dr CO 80126

    I would love an easier file to produce than creating a full asq file.


    Have you looked at the hls output?
    try generating an effect and select hls as the output )*.hlsnc.
    This file has one line for each channel. Eask token represents a frame. The values are 24bit RGB values.


    Maybe your tool could open this file and process it into the asq?
    that would be great , if it were possible.
     
  10. kane

    kane Dedicated Elf

    Joined:
    Dec 23, 2010
    Messages:
    1,019
    Likes Received:
    0
    Location:
    Trigg (Northern suburbs of Perth)
    Find Me On:
    I'll have a play when I get a chance and see what I can do.

    I'm yet to adapt my tool to support the LSP 2.5 file format, but I can't see it being too difficult - from my quick look, all that has been done is the XML for each controller has been spun off into it's own file
     
  11. kane

    kane Dedicated Elf

    Joined:
    Dec 23, 2010
    Messages:
    1,019
    Likes Received:
    0
    Location:
    Trigg (Northern suburbs of Perth)
    Find Me On:
    I figured I'd take the plunge and switch my tool over to support the 2.5 file format, and all went well until I tried to save the file again. It seems that LSP cant read in my zip archive. I've emailed David to ask about it, so hopefully I'll get a reply when he gets back from holidays...

    Doing a bit more testing tonight, if I unzip a .msq file, and rezip it (without modifying it), LSP fails to open up the file, which lines up with LL's test results. I've tried zipping up with my .NET libraries, with WinZip, WinRar and 7-Zip, and none of them are able to be loaded by LSP. (Interestingly, all 4 of these end up with different file sizes when zipping up the same files)

    Once I get past this issue, I think it'll be relatively straightforward to apply the .hlsnc data to a sequence.
     
  12. kane

    kane Dedicated Elf

    Joined:
    Dec 23, 2010
    Messages:
    1,019
    Likes Received:
    0
    Location:
    Trigg (Northern suburbs of Perth)
    Find Me On:
    Okay, worked out the zip file issue. After a lot of experimenting, and delving into the innerds of zip files, it seems as though LSP requires particular comments to be stored against each file in the zip archive as metadata ("LightShow Pro Light Sequence" against the whole zip archive, "Sequence" against the sequence file, and "Controller" against each controller file) So if I add these in, I'm able to get my exported .msq file to load up in LSP.


    So I've written a quick and dirty import procedure, and tested it using a 10 second nutcracker animation - a matrix of 18x36 (648 channels). My tool took several minutes to load this data in and save the xml, but it worked eventually. Will have to look at more efficient ways of manipulating the xml.


    I'm not 100% sure if the effects in LSP line up with what nutcracker produced, but will experiment more with that when I get a chance.


    The resulting .msq file was around 2Mb, and uncompressed it was around 60Mb, so I can see issues popping up here - if you had 2000 rgb channels and 3 minutes of nutcracker effects, you'd be looking at around a 3-4GB file, and I have no idea if LSP could handle that..
     
  13. lithgowlights

    lithgowlights Senior Elf

    Joined:
    May 6, 2010
    Messages:
    742
    Likes Received:
    16
    Location:
    Lithgow, NSW
    Find Me On:
    Absolutely brilliant news - I cant wait to give it a try
     
  14. OP
    OP
    smeighan

    smeighan Dedicated Elf

    Joined:
    Jan 19, 2012
    Messages:
    1,055
    Likes Received:
    4
    Location:
    4217 Greenfinch Dr CO 80126

    Hi Kane; i brought up the issue of size last mar/apr. When i first looked at the LSP xml file, it is very chatty.
    I wondered how big a full sequence of a RGB tree would take in LSP.


    Forget Nutcracker, has anyone sequenced a 2000 channel tree for a 3 minute song using just LSP? How big was your asq file?


    Maybe 2.5's plan to put each controller into a different file was meant to help the large file size?
     
  15. OP
    OP
    smeighan

    smeighan Dedicated Elf

    Joined:
    Jan 19, 2012
    Messages:
    1,055
    Likes Received:
    4
    Location:
    4217 Greenfinch Dr CO 80126

    Hi Kane;


    currently Nutcracker produces these files


    Vixen: *.vir, *.vix
    LOR: *.lms, *.lcb
    HLS: *.hlsnc


    I understand from your post that you had prototyped a tool that could take the hlsnc file and modify someones existing asq file?


    any progress on this tool? I get pinged daily by lsp users who want a way to get nutcracker into lsp without going down the lms path.


    i am so glad you are coming close to a solution.


    thanks
     

Share This Page