How To use the E2E with Unreal
Table of Contents
This How To will guide you through the Unreal Integration - E2E / "Engine to Engine" and send some very basic data from Ventuz to Unreal.
Introduction
Please note that the current version of the Ventuz <-> Unreal Engine communication may have some synchronization issues which are caused by the Unreal Engine API. Epic is aware of this problem and will improve its API within the next months. See limitations for further details.
With this How To you will learn the basics of how to use the "Engine to Engine" integration with Ventuz. We will learn about the available nodes from within Ventuz, see how to create and use E2E Blueprint Nodes on the Unreal side and create a basic Data In/Out scenario. There are several ways to use and run the Unreal Engine. For creating in Unreal and Ventuz in parallel use the Live Link. But Ventuz is also capable of running and managing a compiled version of your Unreal project in the background or you can import your .uproject file easily to your Ventuz Scene. Even several instances of Unreal can work simultaneously with the Ventuz Designer or the Ventuz Runtime. Please be aware of the limitations in the cooperation of Ventuz 6.9.0 and Unreal 4.26.2.
Requirements
The VentuzE2E integration is only available with Ventuz 6.9 and upwards. Make sure you installed the latest Version of Ventuz. You will need to install the Unreal Engine by using the Epic Games Launcher. The E2EIntegration works with Unreal Engine 4.26.1 and upwards. The latest Unreal Plugin can be found here: http://www.ventuz.com/downloads
Preparations
Download and install at least Ventuz 6.9. Download and install the latest Unreal Engine. In the next step we will create a new Project and copy the Ventuz Plugin for Unreal.
Basic Setup
We will start to create a basic setup to get started with the E2E Integration using a Live Link. To do so, we need to have a existing project first where we can put in the plugin. If you dont have a existing project start the Unreal Engine Editor and create one. Otherwise you can continue from here
Create a new Project and use the Film, Television, and Live Events Project Category and click Next.
We start with a Blank Template.
Make sure the following Project Settings are set (default) - Here you can change the project location and the project name - we used UnrealVentuz as project name.
Click the Create Project button and let the Unreal Editor create and start the new Project. This may take a while since it will also load and include the "Starter Content".
This will create a new project in the selected location. When Unreal started, there might be several messages popping up. The one might be related to "New plugins detected" and the other will claim that your "Project file is out of date". click Dismiss for the new plugins-.. and Update for the Project file is out of date.... message.
Inserting the Plugin
Now it is time to copy the Ventuz Plugin into your project. To do so, we extract the contents of the Unreal Plugin which can be found on our Homepage for Download http://www.ventuz.com/downloads .
Copy and Paste the content, which should be a VentuzPlugin folder into your recently created project location under Plugins. It might be possible that you need to create the Plugins folder manually first. The structure would look like following:
Restart your Unreal Editor by closing it. This will throw you back to the Epic Games Launcher. Start the Unreal Engine and this time the Project Browser will contain the recent projects category. You can doubleclick the recent project or click the project to select it and click Open Project. We will check if the VentuzPlugin has been loaded successfully by using the Plugin Browser from within the Unreal Editor. As well you might have noticed that a small message might have pop up in the lower right corner of the Unreal Editor that a new Plugin has been installed.
click Edit->Plugins to open the Plugins Browser and use the Searchbar to locate the VentuzPlugin easily. The following Screenshot is just a representation of the installed Plugin and the version number might differ from you result.
If the VentuzPlugin is not listed there, check for the correct Project and Location - remember to create a Plugins directory inside the Project and copy the VentuzPlugins Folder into it.
Now since the Plugin has been installed successfully we need to have some Editor- and Project Settings adjusted.
Unreal Editor- and Project Settings
To ensure Ventuz and Unreal behave as expected, we need to adjust the Project settings. These are mandatory steps and would apply for any Project you work with if you plan to use the Unreal and Ventuz E2E Integration.
Project Settings
Anti-Aliasing:
click Edit-> Project Settings and use the Searchbar to search for anti-aliasing or look under Engine - Rendering - Default Settings - Anti-Aliasing Method and switch from TemporalAA to FXAA
Optional Settings
So far the Ventuz plugin will apply some changes to the project automatically as soon as it is enabled/loaded. If you still run into trouble like your Camera is not moving synchronously, or Unreal does stall whenever the Editor is out of focus, check for the following settings:
Editor Settings
click Edit-> Editor Preferences and use the Searchbar to locate the CPU related settings - by typing CPU into it. Disable Use Less CPU in Background in the Editor Preferences to avoid the situation where the frame rate in Unreal drops whenever the Unreal Editor loses application focus.
Close the Editor Preferences.
Project Settings
GameMode:
Use the Searchbar again and search for default which will bring up the default settings for the project or look under Project - Maps & Modes - Default Modes - Selected Game Mode - Default Pawn Class and set SpectatorPawn. It might be possible that the settings are greyed out there and we would need to create our own Default GameMode. To do so you click the + next to the Default GameMode and use UnrealVentuz as new GameMode name. click ok to create a new GameMode.
Now you should be able to select the SpectatorPawn as Default Pawn Class to avoid problems with controlling the Unreal Camera from within Ventuz.
Local Player Class:
In order to have full control over the camera settings (in particular to be able to set the off-center cameras required for some projection scenarios) you'll need to set the Local Player Class to the one provided by the Ventuz plugin called VentuzLocalPlayer. If this is not set the camera control will still partially work but on a limited feature set. After changing the Local Player Class you might have to restart the Unreal Editor for changes to take effect. Use the Searchbar and search for local player class or look into Engine - General Settings - Default Classes - Local Player Class and select VentuzLocalPlayer from the drop down list.
Close the Project Settings and your Project is now ready to be linked with Ventuz.
Live Link Unreal and Ventuz
Now it is time to check if the Live Link between Unreal and Ventuz is established. Open or create a new Ventuz Project and create a new Scene. In the Layer Editor add a new E2E Live Link Layer.
To bring everything to "live" - click the Play button in the Unreal Editor and you should see immediately the Unreal Editor rendering in Ventuz!
You should have an instant connection to your running Unreal Instance and see the Unreal Rendering inside the Ventuz Render Window. Also note that if you have configured Unreal correctly, the Unreal Camera is now controlled by Ventuz. To manipulate the Ventuz Camera either create a Ventuz Camera Node inside the E2E Live Link Layer or use the Ventuz Free Flight Camera.
The E2E Live Link Layer behaves like a normal Ventuz 3D Layer and can be used the same way as a traditional Ventuz 3D Layer. Then populating the Unreal Layers contents with Ventuz Nodes, the Unreal Rendering is in the background and depth composited with Ventuz Elements.
Activate the Free Flight mode by pressing TAB and navigate the camera to have nice perspective view of the checkerboard surface - while in the free flight mode press ctrl + insert to create a new camera from the actual position/view. This will create a new camera inside the E2E Live Link Layer and automaticaly activate it if you disable the Free Flight mode with TAB.
From here we can start building our Ventuz and Unreal content side by side and use all the Ventuz specific E2E Nodes and Unreal E2E Blueprint Nodes to communicate bidirectionally. click Stop to stop the running Unreal Engine for now.
Working with the E2E with Live Link
The very first example of the Engine2Engine usage will be to send Data from Ventuz to Unreal. You can make use of the E2E Data Out node in this case. We will send data, in this case the position values of a mover, to Unreal to move an Actor in Unreal.
On the Ventuz side, if not done already, create a E2E Live Link Layer. Select the E2E Live Link Layer and create a mover and a E2E Data Out Node in the Content Editor. Add a float Property for the E2E Data Out Node by click the icon in front of Fields and name it PosX. Connect the PosX to the mover as shown below.
We want to animate an Actor inside of Unreal. For this create a simple Empty Actor by drag&drop the Empty Actor into our 3D View.
The Actor is already selected and we click Add a Blueprint.
The default name for the Actor will work fine, just create the blueprint by click the green Select button. This will bring up the Blue Print Editor. click the Event Graph to switch to the Event Graph view and create a Get Ventuz Value Blueprint Node. Scroll down until you see the Event Tick Blueprint Node and drag&drop a connector from the Exec pin to create a new Blueprint Node. Type in get ventuz value and select it to create the node.
Type in the Ventuz Node Name to the Blueprint Node field - we used the default E2E_Out_1. The Field name itself has to match the Property Name we used on the Ventuz Side. Type in PosX in this case.
click into the Blueprint Graph and use the searchfield, search for SetActorRelativeLocation blueprint node and create it. If you connect the Result of the Get Ventuz Value to the SetActorRelativeLocation we notice that the PosX value on the Ventuz side would not transform the unreal Actor along the X-Axis. This is simply because both Engines use different coordinate systems. We can use the Ventuz UE4 Position Blueprint node to do so. This works in both directions, but would need to use the respective node for it depending from where to where you would like to convert the coordinate system. Below is an example list of available Blueprint Nodes for the Blueprint editor to be used.
click into the Blueprint node editor and search for the Ventuz to UE4 Position blueprint node and create it. click the Position and select Split Struct Pin. By doing so the Position will be split into seperate X/Y/Z Pin inputs. The layout of the created blueprint nodes should look like the picture below:
Now you can create bindings for the exec and from Get Ventuz Value - Result to the Ventuz to UE4 Position - Positon X Input. Bind the Ventuz to UE4 Position - Return Value to the SetActorRelativeLocation - New Relative Location input pin. The com plete bindings should look like below:
Actually we have just created a link between Ventuz and Unreal to send the mover position or PosX values to just an Actor. As soon as we would press Play on the Unreal side you will notice this Sphere like Actor will disappear. We will now simply use this Actor and add a Geometric component to it, like a cube. To do so, you select the Actor_Blueprint in the World Outliner or click the Actor inside the Unreal Editor Renderer. click Add Component from the Details window and choose the Cube.
We can use the Transformation Manipulator to move the Cube above the surface and as soon as we hit Play now from inside the Unreal Editor we will see the realtime movement of the Cube. As soon as you change the Mover property values from inside Ventuz, you will see the changes reflected at the cubes movement.
Sending an Alpha Channel to Ventuz
By default Unreal is not able to render an alpha channel and instead only supports a Color Buffer without transparency. To get rid of that behavior and use the Unreal rendering in your Ventuz Layer Composition you can do one of two things: use the Depth Keyer or enable experimental alpha usage in Unreal. To do the latter you need to go to the project settings and change the Enable alpha channel support in post processing (experimental). setting to Allow through tonemapper.
Note that this setting is experimental and you do not have much control over the alpha channel in Unreal. Sometimes you might get better results using the Depth Keyer approach.
Conclusion
This is just an example of using the Unreal Live link from Ventuz to send Position Data by the usage of the E2E Data Output node. As you might have spotted already, ventuz offers a bunch of E2E Nodes which can be found in the E2E Blueprint Nodes Overview - or by have a look into the Tool Box from Ventuz.
Keep in mind that we have bidirectional communication - therefore you can send data from Unreal to Ventuz too. The possibilities are endless and it would be possible to even have Camera Tracking working in Unreal.