3.x Vixen 3 Lip Sync Overview - Updated

Discussion in 'Vixen' started by ebrady, Jun 25, 2014.

  1. ebrady

    ebrady New Elf

    Joined:
    Dec 26, 2010
    Messages:
    21
    Likes Received:
    0
    Location:
    Delaware, OH
    Hi all, I have put together a set of overview videos for the Lip sync functionality I am working on for Vixen 3. This is the first time I have put together summary videos, so bear with my "newbie-ness" in this area. Here are the links and their table of contents.

    Video #1
    0:00 - Introduction
    1:58 - setting up Display, new Template
    4:34 - Setting up the preview
    5:50 - New sequence items for Lipsync
    6:55 - Lipsync to String/Pixel Mapping Overview
    10:04 - Creating a Lipsync Map
    13:20 - Default Mappings
    14:55 - Setting Default Map

    Video #2
    0:00 - Introduction and recap of Video #1
    1:05 - Sequencing a Lipsync element manually
    4:54 - Overview of Papagayo
    8:32 - Importing Papagayo Files
    11:50 - Lyric to Phoneme Converter - Part #1

    Video #3
    0:00 Introduction and recap of Video #2
    1:33 - Sequencing of Alvin and the Chipmunks
    2:24 - Audio Processing to enhance Vocals
    5:17 - Converting Lyrics to Phonemes with Text converter
    7:01 - Text to Phoneme conversion using Marks
    12:30 - Chipmunks Sequence with audio
    16:55 - Other effect possibilities


    Video #4 - (Added 6/24/14)
    0:00 - Introduction
    0:50 - Review some possible types of Lip Sync element configurations
    3:35 - Review of setting up 8 "Dumb" String configuration
    4:21 - New program options to simplify setup for smart pixel strings of varying length
    14:38 - New Feature - Pixel Matrix / Mega Tree Mapping
    20:10 - Import/Export of Images into Map Matrix Editor

    (NOTE: In the section beginning at 0:50, I make several references to the "software" not supporting a Pixel Matrix while pointing at the Vixen3 Display Configuration.
    I realize after posting the video that this will likely cause some confusion, "software" in this case means the LipSync module code, not Vixen3 itself).


    Please note, the desktop recording software was unable to keep up when I tried recording the previews in videos #2 and #3 so the LipSync effect does not look as if it is working correctly even though it was.

    The latest builds of my code can be found at the following links -

    32-bit (6/24/14) - Updated
    64-bit (6/24/14) - Updated

    Note - These are not official development builds from the Vixen 3 team, they are from my own computer. To take a look at then, download and unzip to a known location and run the application exe in root of the folder directory. Back up any Vixen 3 files before hand and you mush already have Vixen3 installed for these to work properly. PM me if you have issues getting them to run.

    (6/24/14) - Note: The Vixen development team has merged this work into the latest development build for 3.11, which should be available soon. The development build has quite a few UI enhancements, performance improvements, and other bug fixes the versions above do not.</blockquote>Last edited by ebrady; Today at 02:52 PM.[/color][/size]
    Website:
    http://www.bradychristmas.com
    Facebook:
    https://www.facebook.com/BradyChristmasLights
     
  2. OP
    OP
    ebrady

    ebrady New Elf

    Joined:
    Dec 26, 2010
    Messages:
    21
    Likes Received:
    0
    Location:
    Delaware, OH
    --- Repost of message from DIY Christmas ----


    OK, so for all you fine folks out there who have taken time to try out this functionality and provide feedback THANK YOU! You "bravery" has helped find a number of issues with the code and provided valuable feedback. If you haven't look at it yet, I encourage you to come on over to the "Light Side"

    My time is becoming limited as I have two other major Christmas projects going on right now, so I wanted to hear from you on what you feel are the most important features are missing. I figure with a list in hand, I can organize my efforts and spend time where it yield the biggest bang for the buck this year.

    So far, here is a list based upon your feedback in no particular order.

    #1 - Add the ability for the Lip Sync Map to be used as a mask for other effects. For each element in the map, instead of picking a static color, allow the user to specify another Sequencer Row which contains the effect(s) to use. This would allow for more sophisticated effects to be specified for Lip sync elements. (i.e. Color Gradients, Random Eye blinking, alternating colors, etc).

    #2 - Modify the text converter to insert phonemes based upon marks made at word syllable divisions. - This will make using the Tapper function much more easier to use. The user would then be able to tap out marks based upon the Lyric's "beat" vs having to consciously think about word start/end locations.

    #3 - Allow phonemes to be grouped into a single logic unit representing a word. This "Word" effect would act as a master to its associated phonemes and would allow them to be more easily managed as a group.

    #4 - Add new mapping functionality to control Servos for Animatronics.

    #5 -????????

    Ed
     
  3. gnarmstr

    gnarmstr Full Time Elf

    Joined:
    Feb 15, 2014
    Messages:
    326
    Likes Received:
    27
    Location:
    Karalee
    Great work on the lip sync effect.


    Can the matrix function be used on a matrix that has been built as a zigzag matrix? My matrix is zigzag starting at top right and working down.


    Would be great if you could include this unless already possible.
     
  4. OP
    OP
    ebrady

    ebrady New Elf

    Joined:
    Dec 26, 2010
    Messages:
    21
    Likes Received:
    0
    Location:
    Delaware, OH
    I have had multiple requests for this feature, and have am thinking about it.


    I am coming to the conclusion that this feature is really not specific to the lip sync module and would likely require changes to the system in multiple places. Specifically, to the templates used to setup the matrix and possibly mega tree display items, and possibly to the matrix preview element. Nutcracker code may require changes also.


    I will see what I can get in this year, I want to get the dev team to weigh in on it also.


    Ed


     
  5. OP
    OP
    ebrady

    ebrady New Elf

    Joined:
    Dec 26, 2010
    Messages:
    21
    Likes Received:
    0
    Location:
    Delaware, OH
    Forgot to mention - It is possible to use zig-zag matricies with Vixen3 today, it just requires a lot more work to patch channels, setup preview elements etc. So the "feature" would be more of one that would make it easier to use, which is what I am talking about in my previous post.



     

Share This Page