Video
Low-latency video is essential for high-quality interactive events. In general, total-latency <60ms is great and 100ms is the upper-limit for good real-time immersion. Total-latency is from (camera) photon to (screen) photon. This keeps the dancers in step with the beat and lip-sync looking natural. In general, musicians often talk about 5-7ms to stay in sync with their group and VR studies have shown that humans can visually perceive a single millisecond!
Sync your Cameras! For the lowest feasible latency you need to have all of your video inputs (genlock) synchronized. This avoids delays caused by Time Base Correction. Traditional (and the most robust method) is to have a house sync generator that locks the timing of ALL video devices: cameras, computers, switchers, DDR's, etc. Most HD equipment will accept both bi-level (NTSC/PAL) and tri-level (analog HD) sync. If your camera's do not have genlock, then sync your (GPU) FX system to your primary camera.
Contact us for PROFESSIONAL VIDEO support at: [email protected]
- YUV - 10bit, 12bit (4:4:4 or 4:2:2)
- RGB - 8bit, 10bit, 12bit
- XYZ - 12bit, (custom 14bit, 15bit, 16bit)
- HD - 1280 x 720p @ 59.94
- HD - 1920 x 1080p @ 59.94
- UHD - 3840 x 2160p @ 59.94
- UHD2 - 7680 x 4320p @ 59.94
- 2K - 2048 x 1080p @ 23.98, 24, 47.95, 48
- 4K - 4096 x 2160p @ 23.98, 24, 47.95, 48
- 8K - 8192 x 4320p @ 23.98, 24, 47.95, 48
- 32768 x 8640 @ 59.94 (control rooms)
- 16384 x 16348 @ 59.94 (dome projection)
We ONLY recommend using PROGRESSIVE video formats (480P, 720P, 1080P).
- We do NOT support interlaced textures and you will get artifacts!
- In general, other formats (525i, 576i, 1080i) and frequencies (25, 29.97Hz) tend to work, but with artifacts intrinsic to interlacing, scaling or conversion (up/down/cross, frame rate, 3:2 pulldown, etc). Interlaced signals also double the latency due to the (GPU & Video IO) frame buffers moving both fields (odd & even) as a complete frame.
Minimum video latency with a GPU is ONLY obtained using NVIDIA GPUdirect or AMD DirectGMA with compatible video IO hardware.
*Currently we do NOT support AMD DirectGMA, (contact us if needed, API differences are minimal).
We support GPUdirect video capture using either AJA (IoXT, Kona & Corevid series) or Blackmagic Design (DeckLink, Intensity Shuttle, UltraStudio Mini Recorder...) hardware. Note that both AJA and BMD can theoretically be used in the same system (and some do), but this is NOT recommended by the vendors, (potential driver conflicts).
In theory, the latest HW is directly compatible, and we would be happy to test any system you provide!
AJA support includes: IoXT, Kona 5, 4, 3G, LHi, LHe Plus, IP and Corvid series.
Blackmagic Design support for: Intensity Shuttle USB and Thunderbolt, UltraStudio Mini Recorder and various DeckLink cards.
nVidia Quadro (4000 series and above) is required to use GPUdirect and/or house sync. Tested using nVidia SDI Output Option II, SYNC or SYNC II.
- Quadro SDI Output Option II - Quadro: 4000, 5000, 6000, K4000, K4200, K5000, K5200, K6000
- Quadro SYNC - Quadro: 4000, 5000, 6000, K4000, K4200, K5000, K5200, K6000, M4000, M5000, M6000
- Quadro SYNC II - Quadro: P4000, P5000, P6000, GP100, GV100, RTX 4000, RTX 5000, RTX 6000, RTX 8000
You can connect the reference input to either house sync or an NTSC output from the video capture hardware.
Cross-Industry Confusion - There is much confusion (outside of broadcast) about 60.0Hz vs 59.94Hz video. Many devices auto-select and misrepresent the actual format. Monitors will round the number up. Windows display settings will show 59 or 60 Hz while actual video timing can be +/-1 Hz which means that either can be 59.94Hz (usually 59 is and 60 is NOT, but you have to check!). GoPro (Hero3) will display 720P60 (but is actually true 720p59.94) and so & so on.... KEEP IN MIND that you have to check your actual frequencies, auto-selected modes often get confused and computers especially are very good at producing video signals that are not broadcast (SMPTE) compliant! There are 3rd party utilities (discussed below) that can help you force video output to 59.94Hz from OSX and MSW machines. For pro-video, CentOS/RHEL is your friend.
-
Quadro Sync - be patient, this card is mode dependent. Monitors (connected to the GPU) need to be capable of matching frequency with the genlock/sync source (59.94Hz). It is possible to use a standard NTSC composite video source for the sync signal, such as the analog output of the AJA Kona cards, (or you can also find cheap HDMI to CVBS converters online). It is quick to detect a sync signal, but takes a moment to lock an (output) monitor to sync. As noted in the nVidia Quadro Sync Users Guide Framelock can take a few minutes!!!!
-
Quadro SDI Output Option II - Avoid frame tearing when using broadcast 59.94Hz standards (1080i, 720p, 525i, etc.) Check that nVidia Control Panel 'View system topology' shows 59.94Hz for all display outputs accept the one labelled 'NVIDIA-SDI' will show 60.00Hz (weird, but do not believe it, it's actually 59.94Hz if the rest of the GPU outputs are 59.94Hz). The 'SDI Output' should also show 59.94Hz for the 'Sync signal (External sync)'. You may need to connect ALL of your GPU display ports to a broadcast monitor, TV or HDMI -> SDI converter in order to expose the 59Hz option under 'Change Resolution'.
-
Quadro SDI Output Option II - VSYNC should be OFF (unless using DVI or DP output). Can control V-Sync only if the V-Sync in the nVidia Conrol Panel is set to 'controlled by application'.
-
BMD Intensity Shuttle - Best with 720p59.94, note that it does NOT detect input format, which must be set manually.
-
BMD UltraStudio Mini Recorder - Best with 720p59.94, does NOT auto-switch between inputs. HDMI or SDI source must be set using the utility (it remembers its last setting when powered off).
-
BMD MicroConverter HDMI to SDI - May NEED to configure for 1080p59.94b using the BMD converter utility (factory default was 1080p59.94a)... verify firmware ver X.X+ (upgrade if older).
-
CAUTION - Generic (non-pro) SDI <-> HDMI -> CVBS converters - Tend to produce troublesome video signals. We have tried several generic converters, most have failed one way or another, some worked for a while before burning out, others seemed to work but caused timing problems (non-compliant video signals). For example, we were able to use a generic HDMI to CVBS (composite NTSC/PAL) into the Quadro Sync, which provided a functional (59.94Hz) sync signal from the HDMI camera. However, the converters HDMI output was detected by the AJA Kona 3G as 720p60 and NOT 59.94! Though the Kona could capture the signal at both 60Hz 59.94, the BMD UltraStudio Mini Recorder auto-detected it at 720p59.94 and would not accept 720p60, while the BMD Intensity Shuttle (USB) could NOT see the signal at all!
-
Use professional cross converters (scaling, frame rate, signal format) from reputable video manufactures such as: AJA, BlackMagic Designs (BMD), DECIMATOR, Gefen and BlueFish444 - Synapse
-
Gefen GTV-HDMI-2-COMPSVIDSN - HDMI to composite scaler could be useful for extracting sync from a camera.
NVIDIA X Server Settings
- Edit your monitor(s) frequency 'RATE' to 59.94 (vim ~/.config/monitors.xml). Hint, you will need to have set at least one resolution in the GUI (Settings->Devices->Displays->Resolution) to generate the 'monitors.xml' file with default resolution settings. After editing the monitors file, save and reboot, then go back to the 'Displays' dialog and set the 'Resolution' (1280 x 720 OR 1920 x 1080) and 'Refresh Rate' to 59.94Hz.
- Enable Frame Lock:
- Open NVIDIA X Server Settings and click 'Frame Lock' (left menu).
- Click 'Add Devices' (hit 'OK' for default 'localhost.localdomain:0.0').
- Check the 'Server' box next to your primary output monitor (frequency needs to be close to the House Sync.)
- Click 'Enable Frame Lock', if you get a green blinking SYNC LED on the board, then your good! You should also see a green indicator next to 'Receiving' with 'Rate: 59.9400Hz' and green for 'House' and 'Timing' as well. If you do not and you have a solid lit green LED for SYNC (meaning you have a valid House Sync signal), then it is likely you need to configure the 'RATE' in your ~/.config/monitor.xml file to 59.94 for all monitors.
- Optionally, set Color Space & Range to YCbCr444 (or default RGB) by selecting the monitor and selecting the 'Controls' tab.
Hint: widen the utility window to see all settings: HOUSE(genlock) indicator, Server, Client, etc.
We use the app 'SwitchResX4' to set a custom monitor (EDID) resolution to 59.94Hz. The app currently (Jan, 2020) has a free 10day trial and is $16/seat to purchase:
- DOWNLOAD 'SwitchResX' at: https://www.madrau.com/
You need to manually create the custom video timings. This is non-trivial, however we found that these settings worked for 1920x1080 @ 59.94Hz (connected to a BMD MicroConverter).
- Install the app, allow it under: System Preferences... -> Security & Privacy -> General
- Shutdown, start and hold 'Command+R' to get into the recovery tools.
- Open a 'Terminal' under the 'Utilities' pulldown.
- Type 'csrutil disable' and hit Enter.
- Reboot.
- Launch 'SwitchResX' (directly or from System Preferences).
- Select the monitor (to set to 59.94Hz).
- Select 'Custom Resolutions' tab.
- Click '+' to add a new resolution.
- Double-Click the new item to edit.
- Change the Vertical 'Hz' to 59.94 (lower right setting, see image below.)
- Hit 'OK'.
- Click 'Save settings' from the 'File' pulldown.
- Reboot.
- Launch SwitchResX and select the monitor.
- Select 'Custom Resolutions' tab and click 'Activate Immediately'.
- Your monitor should now be 59.94Hz
- Verify custom settings by selecting the 'Current Resolution' tab and click the 'eye' (lower left icon) and you should see:
- Repeat steps 7-18 for all displays you wish to conform to 59.94Hz.
*Note that it shows '59.939Hz' which is within nVidia's specification of +/-0.002Hz (the difference may do to valid timing increments, more homework is needed here!)
Steps are similar for 720P_59.9 settings:
Venue size makes a difference, since sounds travels 343m/s, audio is delayed an entire frame every 5.7 meters from the stage. Someone at 17m from stage has 3 frames equivalent of audio delay, quite convenient! A good video editor knows that you want the audio video sync within +/- 1 frame, and it's better to have the audio get behind the video, rather then vice versa. Otherwise you get an unnatural feel, likely because in the real world we tend to see something before the sound arrives, like thunder & lighting.