NAV Navbar
C#
  • Introduction
  • What's New
  • Requirements
  • Getting Started
  • 8th Wall XR - Unity API
  • Advanced Topics
  • Troubleshooting
  • Changelog
  • Introduction

    8th Wall XR enables any developer to quickly and easily build AR apps that run on any Android or iOS device.

    8th Wall XR works by bringing the best possible AR experience to each user's device. It integrates seamlessly with native APIs like ARKit, ARCore and Tango, while allowing the very same apps to be run on any Android or iPhone by using the phone's camera and inertial sensors. It provides easy-to use APIs for lighting, surfaces, textures, and 3D tracking and installs natively into a standard Unity workflow.

    This document describes how to integrate 8th Wall XR into Unity applications.

    What's New

    8th Wall XR Release 3 is now available! Release 3 provides a number of updates and enhancements, including:

    Click Here for to see a full list of changes.

    Requirements

    Mobile OS:

    Unity:

    XCode:

    Features supported by platform

    Platform Lighting AR Background AR Textures Camera Motion Surface Detection
    ARKit (iOS 11) Yes Yes Yes 6 DoF Yes
    ARCore Yes Yes Yes 6 DoF Yes
    Tango Yes Yes Yes 6 DoF Fixed Surfaces**
    iOS 7/8/9/10 Yes Yes Yes 3 DoF* Fixed Surfaces**
    Android Yes Yes Yes 3 DoF* Fixed Surfaces**

    *8th Wall's computer vision team is actively working on adding support for 6DoF for these platforms.1

    **8th Wall's computer vision team is actively working on adding support native surface detection for these platforms.1To maintain compatibility with older Android and iOS phones, 8th Wall XR currently provides fixed surfaces in four directions around the camera. These will appear parallel to flat surfaces around you (tables and floors) and can be used to give the appearance that an object is resting on a surface even without native surface detection.

    Note: Unity projects using 8th Wall XR will automatically gain support for newly released features as they are made available.

    Getting Started

    This guide provides all of the steps required to get you up and running quickly and easily.

    Download Unity

    If you don't already have Unity installed, please download it from www.unity3d.com

    Note: During installation, make sure you install BOTH Android & iOS build support packages, even if you only plan to develop for one:

    Unity Component Selection

    Download XR

    The 8th Wall XR Unity package is available here:

    https://releases.8thwall.com/xr/unity/download/release

    Optional: Download a sample 8th Wall XR enabled Unity project:

    https://github.com/8thwall/xr-unity

    Start a New Unity Project

    Open Unity and on the welcome screen, click "New" to create a new project. Give it a name and click "Create Project"

    Create Unity Project

    Install XR

    First, add 8th Wall XR to your Unity project. Locate the .unityplugin file you just downloaded and simply double-click on it. A progress bar will appear as it's loaded.

    Once finished, a window will display the contents of the XR package. Leave all of the boxes checked and click "Import".

    import-xr-unity-package

    Create an XRController

    Create an XRController object in your scene. You can do this in the Hierarchy panel via Create -> XRController or GameObject menu -> XRController:

    create-xrcontroller

    created-xrcontroller

    At this point you are ready to create AR-enabled objects in your scene. Here are some features to take advantage of while building your application:

    8th Wall XR - Unity API

    XRCameraController

    Description

    XRCameraController attaches to your Main Camera. It's primary function is to control the position and rotation of the camera in your scene as you move your device in the real world.

    Note: 6DoF tracking is currently supported for ARKit, ARCore and Tango enabled devices. For all other phones, as of today 8th Wall XR falls back to 3DoF tracking.*

    *8th Wall's computer vision team is actively working on adding support for 6DoF for these older platforms.1

    Variables

    Parameter Type Default Description
    scale float 1.0 scale the camera's position vector by this value to affect movement speed

    Example

    Select the Camera in your scene and perform one of the following actions:

    To scale the movement speed of the camera, adjust the "scale" parameter.

    XRCameraController Setup

    Videos

    6DoF Experience (ARKit, ARCore & Tango enabled phones):

    3DoF Experience (iOS 10 or lower & Non-ARCore Android devices):

    XRLightController

    Description

    XRLightController adjusts the intensity of your scene light based on the lighting conditions in the world around you.

    Example

    Select the light in your scene and perform one of the following actions:

    XRLightController Setup

    XRSurfaceController

    Description

    Attach XRSurfaceController to a game object in your scene. When XR detects a surface, the game object will be placed on the surface. Typically this will be a Plane and other game objects will be children of the plane so everything moves together.

    AR Shadows: If you have attached an XRSurfaceController to a plane (i.e. a "ground" object) in your scene, and want it to both be transparent AND receive shadows, click here for more info.

    Note: To maintain compatibility with older Android and iOS phones, 8th Wall XR currently provides fixed surfaces in four directions around the camera. These will appear parallel to flat surfaces around you (tables and floors) and can be used to give the appearance that an object is resting on a surface even without native surface detection.

    Variables

    Parameter Type Default Description
    deformToSurface bool false If true, modify the rendered mesh and the collider mesh of the surface so that it matches the detected surface. This allows for interactions like shadows that clip to surface boundaries, and objects that can fall off surfaces.
    displayImmediately bool false If true, game object will appear as placed in the scene prior to surface detection. If false, game object will be moved out of view until surface is detected. In either case, when a valid surface is detected, object will be moved to surface position.
    groundOnly bool false If true, only place game object on "ground" surfaces. If false, use any deteced surface.
    lockToFirstSurface bool false If true, place on first detected surface and don't move. If false, move game object to currently active surface.

    Example

    First, create a plane and call it "GameSurface":

    XRSurfaceController Create Plane

    Select the object in your scene and perform one of the following actions:

    XRSurfaceController Setup

    Videos

    deformToSurface = TRUE deformToSurface = FALSE
    displayImmediately = TRUE displayImmediately = FALSE
    groundOnly = TRUE groundOnly = FALSE
    lockToFirstSurface = TRUE lockToFirstSurface = FALSE

    XRVideoController

    Description

    XRVideoController script captures camera input and sets it as the background of your scene.

    Example

    Select the Camera in your scene and perform one of the following actions:

    Unity Editor:

    XRVideoController Setup

    Result:

    XRVideoController Result

    XRVideoTextureController

    Description

    XRVideoTextureController script captures camera input and sets it as the main texture on a game object.

    Example

    Select the object in your scene and perform one of the following actions:

    It's recommended that the material on your object is "unlit" (i.e Unlit/Texture) to avoid appearing washed out. XR takes care this automatically for you!

    Unity Editor:

    XRVideoTextureController Setup

    Result:

    XRVideoTextureController Result

    XRController

    Description

    XRController provides low-level access to 8th Wall XR and can be used to interface directly with the engine instead of using the controllers provided by 8th Wall.

    Public Variables

    Parameter Type Default Description
    enableLighting bool If true, adjust the intensity of your scene light based on the lighting conditions in the world around you
    enableCamera bool If true, control the position and rotation of the camera in your scene as you move your device in the real world
    enableSurfaces bool If true, enable surface detection and ability to place objects on surfaces

    Public Functions

    Function Description
    GetActiveSurfaceId Returns the ID of the currently active surface.
    GetActiveSurfaceMesh Returns the Mesh of the active surface
    GetCameraIntrinsics Get the instrinsic matrix of the camera
    GetCameraPosition Get the position of the Camera
    GetCameraRotation Get the rotation of the Camera
    GetLightExposure Returns the exposure of the environment
    GetRealityTexture Returns what the phone's camera is capturing as texture of type Texture2D
    GetRealityTextureAspectRatio Returns aspect ratio (width/height) of captured image
    GetVideoShader Returns the appropriate Video shader (for AR scene background)
    GetVideoTextureShader Returns the appropriate Video texture shader (for AR video textures on objects)
    UpdateCameraProjectionMatrix Set the instrinsic matrix of the camera
    public class XRSurfaceController : MonoBehaviour {
      private XRController xr;
      private long surfaceId = Int64.MinValue;
    
      void Start() {
        xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
      }
    
      void Update() {
        // If there are no meshes, reset the id to default and don't change anything
        Mesh mesh = xr.GetActiveSurfaceMesh();
        if (mesh == null) {
          surfaceId = Int64.MinValue;
          return;
        }
    
        surfaceId = xr.GetActiveSurfaceId();
      }
    }
    

    XRController.GetActiveSurfaceId()

    public long GetActiveSurfaceId()

    Parameters

    None

    Description

    Returns the ID of the currently active surface.











    public class XRSurfaceController : MonoBehaviour {
      private XRController xr;
      private long surfaceId = Int64.MinValue;
    
      void Start() {
        xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
      }
    
      void Update() {
        // If there are no meshes, reset the id to default and don't change anything
        Mesh mesh = xr.GetActiveSurfaceMesh();
        if (mesh == null) {
          surfaceId = Int64.MinValue;
          return;
        }
      }
    }
    

    XRController.GetActiveSurfaceMesh()

    public Mesh GetActiveSurfaceMesh()

    Parameters

    None

    Description

    Returns the Mesh of the active surface










    public class XRCameraController : MonoBehaviour {
      public const float METERS_SCALE = 1.0f;
      public const float FEET_SCALE = 3.28084f;
    
      private XRController xr;
      private Camera sceneCamera;
    
      // XRCameraController.scale allows for scaling the effective units of a scene. For example, if
      // feet is a more natural unit for a scene than meters, set scale to 3.28084f.
      public float scale = METERS_SCALE;
    
      void Start() {
        xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
        sceneCamera = GetComponent<Camera>();
        xr.UpdateCameraProjectionMatrix(sceneCamera, transform.position, scale);
      }
    
      void Update () {
        transform.position = xr.GetCameraPosition();
        transform.rotation = xr.GetCameraRotation();
        sceneCamera.projectionMatrix = xr.GetCameraIntrinsics();
      }
    }
    

    XRController.GetCameraIntrinsics()

    public Matrix4x4 GetCameraIntrinsics()

    Parameters

    None

    Description

    Get the instrinsic matrix of the camera


















    public class XRCameraController : MonoBehaviour {
      public const float METERS_SCALE = 1.0f;
      public const float FEET_SCALE = 3.28084f;
    
      private XRController xr;
      private Camera sceneCamera;
    
      // XRCameraController.scale allows for scaling the effective units of a scene. For example, if
      // feet is a more natural unit for a scene than meters, set scale to 3.28084f.
      public float scale = METERS_SCALE;
    
      void Start() {
        xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
        sceneCamera = GetComponent<Camera>();
        xr.UpdateCameraProjectionMatrix(sceneCamera, transform.position, scale);
      }
    
      void Update () {
        transform.position = xr.GetCameraPosition();
        transform.rotation = xr.GetCameraRotation();
        sceneCamera.projectionMatrix = xr.GetCameraIntrinsics();
      }
    }
    

    XRController.GetCameraPosition()

    public Vector3 GetCameraPosition()

    Parameters

    None

    Description

    Get the position of the Camera in world space stored as a Vector3













    public class XRCameraController : MonoBehaviour {
      public const float METERS_SCALE = 1.0f;
      public const float FEET_SCALE = 3.28084f;
    
      private XRController xr;
      private Camera sceneCamera;
    
      // XRCameraController.scale allows for scaling the effective units of a scene. For example, if
      // feet is a more natural unit for a scene than meters, set scale to 3.28084f.
      public float scale = METERS_SCALE;
    
      void Start() {
        xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
        sceneCamera = GetComponent<Camera>();
        xr.UpdateCameraProjectionMatrix(sceneCamera, transform.position, scale);
      }
    
      void Update () {
        transform.position = xr.GetCameraPosition();
        transform.rotation = xr.GetCameraRotation();
        sceneCamera.projectionMatrix = xr.GetCameraIntrinsics();
      }
    }
    

    XRController.GetCameraRotation()

    public Quaternion GetCameraRotation();

    Parameters

    None

    Description

    Get the rotation of the Camera in world space stored as a Quaternion















    public class XRLightController : MonoBehaviour {
      private XRController xr;
      private Light sceneLight;
    
      void Start() {
        sceneLight = GetComponent<Light>();
        xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
      }
    
      void Update () {
        float exposure = xr.GetLightExposure();
        // Exposure ranges from -1 to 1 in XR, adjust to 0-2 for Unity.
        sceneLight.intensity = exposure + 1.0f;
      }
    }
    

    XRController.GetLightExposure()

    public float GetLightExposure()

    Parameters

    None

    Description

    Returns the exposure of the environment as a value in the range -1 to 1.









    public class XRVideoTextureController : MonoBehaviour {
      private XRController xr;
    
      void Start() {
        xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
    
        // Set reality texture onto our material. Make sure it's unlit to avoid appearing washed out.
        // Note that this requires Unlit/Texture to be included in the unity project.
        Renderer r = GetComponent<Renderer>();
        r.material.shader = Shader.Find("Unlit/Texture");
        r.material.mainTexture = xr.GetRealityTexture();
      }
    }
    

    XRController.GetRealityTexture()

    public Texture2D GetRealityTexture()

    Parameters

    None

    Description

    Returns what the phone's camera is capturing as texture of type Texture2D







    public class XRVideoController : MonoBehaviour {
    
      private XRController xr;
      Resolution currentRes = Screen.currentResolution;
    
      public void Start() {
        xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
        float scaleFactor =
          ((float)currentRes.width / (float)currentRes.height) / xr.GetRealityTextureAspectRatio();
      }
    }
    

    XRController.GetRealityTextureAspectRatio()

    public float GetRealityTextureAspectRatio()

    Parameters

    None

    Description

    Returns aspect ratio (width/height) of captured image









    public class XRVideoController : MonoBehaviour {
    
      private XRController xr;
    
      private Material xrMat;
      private Camera cam;
    
      public void Start() {
        xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
        cam = GetComponent<Camera>();
        cam.clearFlags = CameraClearFlags.Depth;
        xrMat = new Material(xr.GetVideoShader());
        xrMat.SetInt("_ScreenOrientation", (int) Screen.orientation);
      }
    

    XRController.GetVideoShader()

    public Shader GetVideoShader()

    Parameters

    None

    Description

    Returns the appropriate Video shader (for AR scene background)











    public class XRVideoTextureController : MonoBehaviour {
      private XRController xr;
    
      #if !UNITY_EDITOR
      void Start() {
        xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
    
        Renderer r = GetComponent<Renderer>();
        r.material.shader = xr.GetVideoTextureShader();
        r.material.mainTexture = xr.GetRealityTexture();
        r.material.SetInt("_ScreenOrientation", (int) Screen.orientation);
      }
      #endif
    }
    

    XRController.GetVideoTextureShader()

    public Shader GetVideoTextureShader()

    Parameters

    None

    Description

    Returns the appropriate Video texture shader (for AR video textures on objects)











    public class XRCameraController : MonoBehaviour {
      public const float METERS_SCALE = 1.0f;
      public const float FEET_SCALE = 3.28084f;
    
      private XRController xr;
      private Camera sceneCamera;
    
      // XRCameraController.scale allows for scaling the effective units of a scene. For example, if
      // feet is a more natural unit for a scene than meters, set scale to 3.28084f.
      public float scale = METERS_SCALE;
    
      void Start() {
        xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
        sceneCamera = GetComponent<Camera>();
        xr.UpdateCameraProjectionMatrix(sceneCamera, transform.position, scale);
      }
    }
    

    XRController.UpdateCameraProjectionMatrix()

    public void UpdateCameraProjectionMatrix(Camera cam, Vector3 origin, float scale)

    Parameters

    Parameter Type Default Description
    cam Camera The camera that should have it's projection matrix updated
    origin Vector3 (0,0,0) Initial position of the camera (camera's tranform.position)
    scale float 1.0 scaling the effective units of the camera

    Description

    Set the instrinsic matrix of the camera

    Advanced Topics

    ARCore Support

    In order to fully support ARCore functionality within your 8th Wall XR enabled Unity application, you will need to install the ARCore Service onto your ARCore supported mobile device.

    ARCore Supported Devices: https://developers.google.com/ar/discover/#supported_devices

    Steps:

    1. Visit https://developers.google.com/ar/develop/downloads#arcore-service-app
    2. Download the ARCore Service from the URL in Step 1.
    3. Install the ARCore Service by running the following adb command:

    adb install -r -d arcore-preview.apk

    NOTE: Unsupported devices or devices without the ARCore Service app installed will fallback to standard Android functionality.

    AR Shadows

    AR is more realistic when virtual objects in your scene cast shadows onto the physical world. 8th Wall XR now provides a shader that can be used on transparent surfaces.

    Create a material (for example, call it MyXRTransparentSurface), and select 8thWall -> XRShadow from the list of shaders, then apply the material to the surface object you want to make transparent (i.e. a ground plane):

    XR Shadow Shader

    To adjust the opacity of the shadows, select your Directional Light, and adjust the Strength of the shadow. To improve shadow quality, set resolution to "Very High Resolution"

    XR Shadow Shader Opacity

    For improved shadow quality, go to Edit -> Project Settings -> Quality. Set Shadow Resolution to "Very High Quality" and Shadow Projection to "Close Fit"

    XR Shadow Quality

    Result:

    XR Shadow Shader Result

    Tango Support

    In order to fully support Tango functionality within your 8th Wall XR enabled Unity application, you will need to install:

    Steps:

    1. Visit https://developers.google.com/tango/downloads
    2. Under C APIs, download both: Tango SDK for C and Support Library
    3. Tango SDK for C: Copy the lib/armeabi-v7a/libtango_client_api.so into your Unity project’s Assets/Plugins/Android/libs/armeabi-v7a directory.
    4. Support Library: Copy the lib/armeabi-v7a/libtango_support_api.so into your Unity project’s Assets/Plugins/Android/libs/armeabi-v7a directory.

    NOTE: If these libraries are not added to your Unity project, the application will still run on Tango-enabled devices, but it will not support Tango functionality.

    Troubleshooting

    Camera Position Not Changing

    Issue: Camera position isn't changing as I move my phone. XRController.GetCameraPosition() always returns (0,0,0)

    Explanation: 6DoF tracking is currently supported for Tango and ARKit enabled devices. Supported features by platform is documented here:

    https://docs.8thwall.com/xr/unity/#requirements

    **8th Wall is actively working to add 6DoF support for ARCore Android phones. For all other phones, 8th Wall XR falls back to 3DoF tracking, which explains why the position variables are zero. 6DoF tracking for non-ARKit/ARCore/Tango phones will be delivered in a future release. Once available, Unity projects using 8th Wall XR will automatically gain this functionality1

    Disable Mulththreaded Rendering

    Issue: My mobile application asks for camera access, but doesn't render the camera view.

    Resolution: Disable Multithreaded Rendering

    1. Go to Edit -> Project Settings -> Player
    2. Click on the Android Settings tab
    3. Uncheck Multithreaded Rendering

    MTRendering

    Objects Not Placed on Surface

    Issue: Surface detection isn’t working properly. My object appears to be floating instead of being placed on a surface!

    Explanation: Native surface detection is currently supported on ARKit-enabled phones. For older Android and iOS phones, 8th Wall XR currently provides fixed surfaces in four directions around the camera. These will appear parallel to flat surfaces around you (tables and floors) and can be used to give the appearance that an object is resting on a surface even without native surface detection.

    Supported features by platform is documented here:

    https://docs.8thwall.com/xr/unity/#requirements

    **8th Wall's is actively working on adding support native surface detection for these platforms. Unity projects using 8th Wall XR will automatically gain support for newly released features as they are made available1

    Missing XR Controller

    Issue: None of the AR features are working (camera position/rotation, Video background, etc)

    Looking through logs, you'll see NullReferenceExceptions such as:

    "NullReferenceException: A null value was found where an object instance was required. at XRController.GetCameraPosition () [0x00000] in :0"

    Resolution: Add an XRController (with "XRController" tag) to your scene. You can do this in the Hierarchy panel via Create -> XRController or GameObject menu -> XRController:

    create-xrcontroller

    created-xrcontroller

    Changelog

    Release 3:

    Release 2:

    Release 1:

    [1] Intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for 8th Wall’s products remain at the sole discretion of 8th Wall, Inc.

    C#