E1.31 next gen

Discussion in 'DMX, E1.31 & Networking' started by multicast, Mar 12, 2017.

  1. multicast

    multicast Senior Elf Generous Elf

    Joined:
    Jul 13, 2013
    Messages:
    711
    Likes Received:
    5
    Work continues on making e1.31 more useful and more flexible.. I'm currently working some extras and would like your feedback on what you think..






    EXPANDED UNIVERSE COUNT

    Firstly I know some people have a strong view point that the current 16 bits of universes is sufficient ( and I accept that for many situations it is ), there are situations where it isn’t. I also accept that there are ‘work-arounds’ for more universes and space, but they are just that, workarounds.

    I also understand that sending lots of uncompressed data in e1.31 frames to single devices may to some seem like an odd thing to do. There are ways of sending more information compressed, and potentially more more efficiently. If you are building large video walls, then this is very appropriate.
    If you are building very large scale installations with tens of thousands of end devices that consume in the range of 1 to 100 universes each, and you have a different set of parameters to that of the video wall situation. You want to be able to use very low cost receiver devices, where having to run decompression is not an option. Of course this has an impact on network bandwidth.. But networks are cheap in comparion to the compute resource required to do a near real time decode of a compressed image.

    Why this needs to be standardised, is no different from any of the standards. It means that multiple vendors can work together using a common template.

    Fortuantly the way the protocol work ( with the layers in the headers ) supporting more universes while maintaining backwards compatility is easy to achieve.

    I would like to propose that we can make a simple addition to the protocol, which a Sender and Receiver MAY implement.

    We create a new Vector type ( VECTOR_ROOT_131_EXPANDED_DATA) ß- I used expanded as a place holder. I don’t’ care what its actually called.



    [table][tr] [td] Octet
    [/td] [td] Field Size
    [/td] [td] Field Name
    [/td] [td] Field Description
    [/td] [td] Field Contents
    [/td] [/tr] [tr] [td][/td][td][/td][td][/td][td][/td][td] Root Layer (See Section 5)
    [/td] [/tr] [tr] [td] 0-1
    [/td] [td] 2
    [/td] [td] Preamble Size
    [/td] [td] Define RLP Preamble Size.
    [/td] [td] 0x0010
    [/td] [/tr] [tr] [td] 2-3
    [/td] [td] 2
    [/td] [td] Post-amble Size
    [/td] [td] RLP Post-amble Size.
    [/td] [td] 0x0000
    [/td] [/tr] [tr] [td] 4-15
    [/td] [td] 12
    [/td] [td] ACN Packet Identifier
    [/td] [td] Identifies this packet as E1.17
    [/td] [td] 0x41 0x53 0x43 0x2d 0x45 0x31 0x2e 0x31 0x37 0x00 0x00 0x00
    [/td] [/tr] [tr] [td] 16-17
    [/td] [td] 2
    [/td] [td] Flags and Length
    [/td] [td] Protocol flags and length
    [/td] [td] Low 12 bits = PDU length
    High 4 bits = 0x7
    [/td] [/tr] [tr] [td] 18-21
    [/td] [td] 4
    [/td] [td] Vector
    [/td] [td] Identifies RLP Data as 1.31 Protocol PDU
    [/td] [td] VECTOR_ROOT_E131_EXPANDED
    [/td] [/tr] [tr] [td] 22-37
    [/td] [td] 16
    [/td] [td] CID
    [/td] [td] Sender's CID
    [/td] [td] Sender's unique ID
    [/td] [/tr][/table]

    We then have a modified Framing layer. The only changes that is required here is that the Universe field is made to 4 bytes
    And you have a different VECTOR.


    [table][tr] [td][/td][td][/td][td][/td][td][/td][td] E1.31 Framing Layer (See Section 6)
    [/td] [/tr] [tr] [td] 38-39
    [/td] [td] 2
    [/td] [td] Flags and Length
    [/td] [td] Protocol flags and length
    [/td] [td] Low 12 bits = PDU length
    High 4 bits = 0x7
    [/td] [/tr] [tr] [td] 40-43
    [/td] [td] 4
    [/td] [td] Vector
    [/td] [td] Identifies 1.31 data as DMP Protocol PDU
    [/td] [td] VECTOR_E131_DATA_PACKET (DMX512-A [DMX] data)_EXPANDED
    [/td] [/tr] [tr] [td] 44-107
    [/td] [td] 64
    [/td] [td] Source Name
    [/td] [td] User Assigned Name of Source
    [/td] [td] UTF-8 [UTF-8] encoded string, null-terminated
    [/td] [/tr] [tr] [td] 108
    [/td] [td] 1
    [/td] [td] Priority
    [/td] [td] Data priority if multiple sources
    [/td] [td] 0-200, default of 100
    [/td] [/tr] [tr] [td] 109-110
    [/td] [td] 2
    [/td] [td] Synchronization Address
    [/td] [td] Universe address on which sync packets will be sent
    [/td] [td] Universe on which synchronization packets are transmitted
    [/td] [/tr] [tr] [td] 111
    [/td] [td] 1
    [/td] [td] Sequence Number
    [/td] [td] Sequence Number
    [/td] [td] To detect duplicate or out of order packets
    [/td] [/tr] [tr] [td] 112
    [/td] [td] 1
    [/td] [td] Options
    [/td] [td] Options Flags
    [/td] [td] Bit 7 = Preview_Data
    Bit 6 = Stream_Terminated
    Bit 5 = Force_Synchronization
    [/td] [/tr] [tr] [td] 113-116
    [/td] [td] 4
    [/td] [td] Universe
    [/td] [td] Universe Number
    [/td] [td] Identifier for a distinct stream of DMX512-A [DMX] Data
    [/td] [/tr][/table]
    The Universe Numbers in this instance SHALL be restricted to be greater than 65,635, so they do not conflict.
    The DMP layer remains unchanged. ( other than the offsets changing )

    These expanded universes can be sent unicast or multicast. There would be no implied mapping of universe numbers to multicast address’s. For these universes it would be up to the implementation to decide where to send them. This is essentially the same as you have now with unicast. You have to somehow tell your transmitter where the receivers are. This could be a manual thing, or an automatic process. But it doesn’t have to happen within the context of e1.31

    This saves the entire massive effort of needing a discovery process, and or needing to deal to multicast group issues ( in the standard )

    I believe this is a *VERY* simple thing to do, that only requires a small hand full of changes, and has no impact on backwards compatibility. The standard could go as far as to say that any device that does implement EXPANDED_DATA frames, SHALL also support ‘DATA’ frames and it has to be done the same way it is done today.

    You get to eat your cake, keep your cake and have a new cake ( and ice-cream ).. I’d be very keen to know what negative impact this would have on anyone. I

    If you don’t’ need it, that’s fine, you are not impacted by it.. But for those that do desire to do things this way then this really helps and it also helps spread the use-case of e1.31 into the massively growing ‘architainment’ space where large scale deployments are growing.

    16 bits of universes sounds a lot until you have lots of buildings / sites that are all co-ordinated together. Overlapping the universe numbers ( by vlan or separated routing spaces ) is troublesome. ( you end up with the possibility that you have multiple universe ‘10’s or ‘207’ etc. )


    Manufacturer specific Frames.

    I have had reason to want to send some streaming data that does not fit well in a dmx frame to my own devices.. While I don’t’ want to go into too much detail, because its commercially interesting, it broadly falls into what I’d call a ‘dynamic configuration’ where things are interactive. Its not so much ‘configuration’ but more a data set that is used to manipulate other data.


    Fortunately thanks to the structure of the packets, and its layers, its easy enough.


    At the root layer, we create a new Vector. I’ve called this VECTOR_ROOT_E131_MANUFACTURER



    [table][tr] [td] Octet
    [/td] [td] Field Size
    [/td] [td] Field Name
    [/td] [td] Field Description
    [/td] [td] Field Contents
    [/td] [/tr] [tr] [td][/td][td][/td][td][/td][td][/td][td] Root Layer (See Section 5)
    [/td] [/tr] [tr] [td] 0-1
    [/td] [td] 2
    [/td] [td] Preamble Size
    [/td] [td] Define RLP Preamble Size.
    [/td] [td] 0x0010
    [/td] [/tr] [tr] [td] 2-3
    [/td] [td] 2
    [/td] [td] Post-amble Size
    [/td] [td] RLP Post-amble Size.
    [/td] [td] 0x0000
    [/td] [/tr] [tr] [td] 4-15
    [/td] [td] 12
    [/td] [td] ACN Packet Identifier
    [/td] [td] Identifies this packet as E1.17
    [/td] [td] 0x41 0x53 0x43 0x2d 0x45 0x31 0x2e 0x31 0x37 0x00 0x00 0x00
    [/td] [/tr] [tr] [td] 16-17
    [/td] [td] 2
    [/td] [td] Flags and Length
    [/td] [td] Protocol flags and length
    [/td] [td] Low 12 bits = PDU length
    High 4 bits = 0x7
    [/td] [/tr] [tr] [td] 18-21
    [/td] [td] 4
    [/td] [td] Vector
    [/td] [td] Identifies RLP Data as 1.31 Protocol PDU
    [/td] [td] VECTOR_ROOT_E131_MANUFACTURER
    [/td] [/tr] [tr] [td] 22-37
    [/td] [td] 16
    [/td] [td] CID
    [/td] [td] Sender's CID
    [/td] [td] Sender's unique ID
    [/td] [/tr][/table]

    If your device doesn’t know about this because its following earlier standards, it would simply drop the packet. ( assuming that your device actually checks packets.. groan ) and at least in theory, you shouldn’t be sending these packets to devices that don’t’ need them anyway.

    At the Framing layer, another VECTOR to identify this is a M



    [table][tr] [td][/td][td][/td][td][/td][td][/td][td] E1.31 Framing Layer (See Section 6)
    [/td] [/tr] [tr] [td] 38-39
    [/td] [td] 2
    [/td] [td] Flags and Length
    [/td] [td] Protocol flags and length
    [/td] [td] Low 12 bits = PDU length
    High 4 bits = 0x7
    [/td] [/tr] [tr] [td] 40-43
    [/td] [td] 4
    [/td] [td] Vector
    [/td] [td] Identifies 1.31 data as DMP Protocol PDU
    [/td] [td] VECTOR_E131_MANUFACTUER
    [/td] [/tr] [tr] [td] 44-107
    [/td] [td] 64
    [/td] [td] Source Name
    [/td] [td] User Assigned Name of Source
    [/td] [td] UTF-8 [UTF-8] encoded string, null-terminated
    [/td] [/tr] [tr] [td] 108
    [/td] [td] 1
    [/td] [td] Priority
    [/td] [td] Data priority if multiple sources
    [/td] [td] 0-200, default of 100
    [/td] [/tr] [tr] [td] 109-110
    [/td] [td] 2
    [/td] [td] Synchronization Address
    [/td] [td] Universe address on which sync packets will be sent
    [/td] [td] Universe on which synchronization packets are transmitted
    [/td] [/tr] [tr] [td] 111
    [/td] [td] 1
    [/td] [td] Sequence Number
    [/td] [td] Sequence Number
    [/td] [td] To detect duplicate or out of order packets
    [/td] [/tr] [tr] [td] 112
    [/td] [td] 1
    [/td] [td] Options
    [/td] [td] Options Flags
    [/td] [td] Bit 7 =
    Bit 6 =
    Bit 5 =
    [/td] [/tr] [tr] [td] 113-116
    [/td] [td] 4
    [/td] [td] Universe
    [/td] [td] Universe Number
    [/td] [td] Identifier for a distinct stream of DMX512-A [DMX] Data
    [/td] [/tr] [tr] [td] 117-118
    [/td] [td] 2
    [/td] [td] Manufacturer ID
    [/td] [td] Identifies which manufacturer
    [/td] [td] Manufacturer ID
    [/td] [/tr] [tr] [td] 119-120
    [/td] [td] 2
    [/td] [td] Manufacturer_Vector
    [/td] [td]
    [/td] [td]
    [/td] [/tr][/table]

    I think its helpful to leave the Synch/priority/sequence/options there so you can use them if you want but don’t’ have to. The Manufacturer ID can be the same ID’s that are already issued for RDM.

    The manufacturers Vector code allows the the manufacturer to have multiple frame types.. From a helicopter view the frame ‘type’ is unique when you combine the ‘Vector’ and ‘Manufacturer_Vector’ together.

    Rather than ‘hack’ in things that are not in standard, ( and have other vendors devices do ‘odd’ things ), this gives you a way to send out your ‘interesting proprietary’ data in a “non hacky” way.

    This encourages a lot of room for innovation without impacting on core feature interoperability.


    ( Note: Its easy and free to get a manufacturers ID code, so if Xligths / Falcon / smartalec lights ) want codes then its just a form filling exercise.
     
  2. damona

    damona Full Time Elf

    Joined:
    Oct 23, 2013
    Messages:
    253
    Likes Received:
    5
    I still think its worth removing the DMX512-A limit of 512 Channels Per Universe.

    E.131 Could go to a device which could have the OLD DMX connectors which daisy chain and allocate the first output with channel 1 to 512, the next output 513 to 1024 and so on. Allow devices which do not support greater than 512 to be connect to a single Universe that uses over 512 channels.

    Like to see as many channels which will fit into a Ethernet frame/packet (Standard or Jumbo), and the packet size determines the number of channels in the data the device needs to read. I would be surprised if and existing E.131 device could not read the whole standard Ethernet frame, however they may not be able to process the data if they don't have the memory.

    These days most enterprise switches support 8k frames which would allow future devices to receive a large number of channels in one packet. This could be implemented with the current number of Universe with multi-casting. Or with Universe greater than the existing standard.



    I don't have an issue increasing the number of Universes or allowing device dependent data within E.131
     
  3. Barnabybear

    Barnabybear New Elf

    Joined:
    Mar 30, 2012
    Messages:
    14
    Likes Received:
    0
    Location:
    UK
    Hi, I’m not sure I’m following this correctly (but that’s probably because I’m not clever enough). My understanding is at the packets can be sent via unicast directly to the IP of the receiving device which knows the data contained within is relevant to it or it wouldn’t have been sent, or via multicast globally to all devices, who can pre-filter this, as the last two octets of the IP address contain the universe number. If you moved to a system that used four octets for universes it wouldn’t be possible to pre-filter the packets in this way and controllers would have to parse the packet to find if contained data relevant for it as opposed to just leaving it to be pushed out of the buffer on overflow. The danger is that there would be 65,536 different universes that all appear to be valid to current controllers and could swamp it. To current controllers the following packets when sent by multicast would appear the same until parsed:
    0x0001
    0x0000 0001
    0x0001 0001
    0xFFFF 0001
    Does that make sense?
     

Share This Page