Introduction

8th Wall XR enables any developer to quickly and easily build AR apps that run on any Android or iOS device.

8th Wall XR works by bringing the best possible AR experience to each user's device. It integrates seamlessly with native APIs like ARKit, and ARCore, while allowing the very same apps to be run on any Android or iPhone by using the phone's camera and inertial sensors. It provides easy-to use APIs for lighting, surfaces, textures, and 3D tracking and installs natively into a standard Unity workflow.

This document describes how to integrate 8th Wall XR into Unity applications.

Tutorial API Reference Need Help?

What's New

8th Wall XR Release 5 is now available! Release 5 provides a number of updates and enhancements, including:

  • New 8th Wall Console - provides developer access to:

    • Generate and manage App Keys.
    • View application analytics.
    • Access downloads and documentation.
  • New Features:

    • ARCore Preview 2 support.
    • XRCapabilities API.
    • OpenGL support on iOS.
    • Added support for Android phones without gyroscopes.
  • Enhancements and Fixes:

    • Performance improvements to rendering.
    • Removed Android Support Library dependency.
    • Camera preview rendering performance improvements.
    • Fix rendering on non-16x9 phones.
    • Fix tracking orientation on Android tablets.
    • Fix ARCore aspect ratio.
    • Updated default camera usage descriptions.
    • Fix tracking for x86 architectures.
    • Improved compatibility with developer ProGuard configurations.

Click Here for to see a full list of changes.

Requirements

Mobile OS:

  • iOS:

    • Minimum: iOS 7.0 or later
    • Recommended: iOS 11 (ARKit)
    • OpenGL ES3 or higher or Metal
  • Android:

    • Minimum: Android Kitkat (4.4) or higher
    • Recommended: Android Nougat (7.0) or higher on an ARCore compatible phone.

      • Click Here for instructions on installing ARCore on supported devices.
    • OpenGL ES3 or higher

Unity:

  • Unity version 5.5 or higher

XCode:

  • XCode 9 or higher

Features supported by platform

Platform Lighting AR Background AR Textures Camera Motion Surfaces
ARKit (iOS 11) Yes Yes Yes 6 DoF Yes, Deformable
ARCore Yes Yes Yes 6 DoF Yes, Deformable
iOS 7/8/9/10 Yes Yes Yes 6 DoF (Scale Free) Yes, Instant Planar
Android Yes Yes Yes 6 DoF (Scale Free) Yes, Instant Planar

Tutorial

This guide provides all of the steps required to get you up and running quickly and easily.

Download Unity

If you don't already have Unity installed, please download it from www.unity3d.com

Note: During installation, make sure you install BOTH Android & iOS build support packages, even if you only plan to develop for one:

  • Android Build Support
  • iOS Build Support

Unity Component Selection

Download XR

If you don't already have an 8th Wall account, sign up for one. It's free. Once logged in, the 8th Wall XR Unity package is available here:

https://console.8thwall.com/downloads

Optional: Download a sample 8th Wall XR enabled Unity project:

https://github.com/8thwall/xr-unity

Start a New Unity Project

Open Unity and on the welcome screen, click "New" to create a new project. Give it a name and click "Create Project"

Create Unity Project

Install XR

First, add 8th Wall XR to your Unity project. Locate the .unityplugin file you just downloaded and simply double-click on it. A progress bar will appear as it's loaded.

Once finished, a window will display the contents of the XR package. Leave all of the boxes checked and click "Import".

import-xr-unity-package

Generate App Key

  1. To create an app key, go to https://console.8thwall.com and login. Select "Applications" from the left navigation and then click the "+ Create a new App Key" button:

CreateAppKey1

  1. Enter the bundle identifier of the application you’ll be creating, then click "+ Create"

IMPORTANT: The bundle identifier entered will here needs to be identical to the bundle identifier in your Unity project.

CreateAppKey2

Note: A bundle identifier is typically written in reverse domain name notation and identifies your application in the app store. It should be unique.

Install App Key

  1. Select "Applications" from the left navigation.
  2. In the "Your App Keys" table, locate the row containing your App and its associated App Key.
  3. Click the "Copy" button.

AddAppKey1

  1. In Unity, select Assets / XR / XRAppSettings. Paste your key into the App Key field.
  2. IMPORTANT: Go to File -> Save Project to save these settings.

IMPORTANT: Make sure that the Bundle Identifier matches the bundle identifier you entered in Step #2.

AddAppKey2

Create an XRController

Create an XRController object in your scene. You can do this in the Hierarchy panel via Create -> XRController or GameObject menu -> XRController:

create-xrcontroller

Note: You'll notice that the XRController object is automatically configured with a Tag of "XRController". This is for your convenience so that other XR scripts can automatically find this one. Don't change it!

created-xrcontroller

Add Controllers to Scene

At this point you are ready to create AR-enabled objects in your scene. Here are some features to take advantage of while building your application:

  • Camera Movement: To control the position and rotation of the camera in your scene as you move your device in the real world, add an XRCameraController to your camera.
  • AR Scene Background: To capture camera input and use it as the background of your scene, add an XRVideoController to your camera.
  • AR Textures on Objects: To capture camera input and use it as a "live" texture on an object in your scene, add an XRVideoTextureController to your object.
  • Lighting Control: To adjust the intensity of your scene light based on the lighting conditions in the world around you, add an XRLightController to your light.
  • Surface Detection: To anchor objects to detected surfaces, add an XRSurfaceController to your object.

Console Overview

The 8th Wall Console is a web based portal that allows developers to:

  • View application analytics
  • Generate and manage App Keys
  • Manage accounts and teams
  • Access downloads and documentation

Creating an account is free and doesn't require you to provide billing information.

To access the 8th Wall Console, go to https://console.8thwall.com

Console

App Keys

An App Key is required to build and run mobile apps using 8th Wall XR. Each mobile app you create requires a unique App Key that is tied to the bundle identifier of your app.

It is recommended that you use the same bundle identifier, (i.e., package), on Android and iOS so one App Key can be used for both the Android and iOS versions of the same app.

Create App Key

  1. To create an app key, select "Applications" from the left navigation and then click the "+ Create a new App Key" button:

CreateAppKey1

  1. Enter the bundle identifier of the application you’ll be creating, then click "+ Create"

IMPORTANT: The bundle identifier entered will here needs to be identical to the bundle identifier in your Unity project.

CreateAppKey2

Note: A bundle identifier is typically written in reverse domain name notation and identifies your application in the app store. It should be unique.

Add App Key to Unity Project

  1. Select "Applications" from the left navigation.
  2. In the "Your App Keys" table, locate the row containing your App and its associated App Key.
  3. Click the "Copy" button.

AddAppKey1

  1. In Unity, select Assets / XR / XRAppSettings. Paste your key into the App Key field.
  2. IMPORTANT: Go to File -> Save Project to save these settings.

IMPORTANT: Make sure that the Bundle Identifier matches the bundle identifier you entered in Step #2.

AddAppKey2

Controller Overview

8th Wall XR provides a number of controller scripts that enable you to easily add AR functionality to your application without having to write any code. Simply attach a controller to a relevant game object.

The following controllers are provided:

Controller What Does It Do? Attached To Description
XRCameraController Camera Movement Main Camera XRCameraController modifies the position and rotation of the camera in your scene as you move your device in the real world.
XRLightController Light Estimation Directional Light XRLightController adjusts the intensity of your scene light based on the lighting conditions in the world around you.
XRSurfaceController Surface Estimation Game Object XRSurfaceController places objects onto detected surfaces.
XRVideoController AR Scene Background Main Camera XRVideoController captures camera input and sets it as the background of your scene.
XRVideoController AR Textures Game Object XRVideoTextureController captures camera input and sets it as the main texture on a game object.

XRCameraController

Description

XRCameraController attaches to your Main Camera. It's primary function is to control the position and rotation of the camera in your scene as you move your device in the real world.

Release 4 adds full 6DoF tracking for all phones. This includes phones without ARKit or ARCore!

Variables

Parameter Type Default Description
scale float 1.0 scale the camera's position vector by this value to affect movement speed

Example

Select the Camera in your scene and perform one of the following actions:

  • Drag the XRCameraController.cs script (default location: Assets/XR/Scripts/XRCameraController.cs) onto to the inspector panel for your camera
  • Click "Add Component" in the inspector panel and search for "XR Camera Controller"

To scale the movement speed of the camera, adjust the "scale" parameter.

XRCameraController Setup

Videos

6DoF Experience (Now on all phones):

XRLightController

Description

XRLightController adjusts the intensity of your scene light based on the lighting conditions in the world around you.

Example

Select the light in your scene and perform one of the following actions:

  • Drag the XRLightController.cs script (default location: Assets/XR/Scripts/XRLightController.cs) onto to the inspector panel for your light
  • Click the "Add Component" button in the inspector panel and search for "XR Light Controller"

XRLightController Setup

XRSurfaceController

Description

Attach XRSurfaceController to a game object in your scene. When XR detects a surface, the game object will be placed on the surface. Typically this will be a Plane and other game objects will be children of the plane so everything moves together.

AR Shadows: If you have attached an XRSurfaceController to a plane (i.e. a "ground" object) in your scene, and want it to both be transparent AND receive shadows, click here for more info.

Variables

Parameter Type Default Description
deformToSurface bool false If true, modify the rendered mesh and the collider mesh of the surface so that it matches the detected surface. This allows for interactions like shadows that clip to surface boundaries, and objects that can fall off surfaces. Only for ARKit/ARCore based phones.
displayImmediately bool false If true, game object will appear as placed in the scene prior to surface detection. If false, game object will be moved out of view until surface is detected. In either case, when a valid surface is detected, object will be moved to surface position.
groundOnly bool false If true, only place game object on "ground" surfaces. If false, use any deteced surface.
lockToFirstSurface bool false If true, place on first detected surface and don't move. If false, move game object to currently active surface.

Example

First, create a plane and call it "GameSurface":

XRSurfaceController Create Plane

Select the object in your scene and perform one of the following actions:

  • Drag the XRSurfaceController.cs script (default location: Assets/XR/Scripts/XRSurfaceController.cs) onto to the inspector panel for your camera
  • Click "Add Component" in the inspector panel and search for "XR Surface Controller"

XRSurfaceController Setup

Videos

deformToSurface = TRUE deformToSurface = FALSE
|
displayImmediately = TRUE displayImmediately = FALSE
|
groundOnly = TRUE groundOnly = FALSE
|
lockToFirstSurface = TRUE | lockToFirstSurface = FALSE --- | ---
|

XRVideoController

Description

XRVideoController script captures camera input and sets it as the background of your scene.

Example

Select the Camera in your scene and perform one of the following actions:

  • Drag the XRVideoController.cs script (default location: Assets/XR/Scripts/XRVideoController.cs) onto to the inspector panel for your camera
  • Click the "Add Component" button in the inspector panel and search for "XR Video Controller"

Unity Editor:

XRVideoController Setup

Result:

XRVideoController Result

XRVideoTextureController

Description

XRVideoTextureController script captures camera input and sets it as the main texture on a game object.

Example

Select the object in your scene and perform one of the following actions:

  • Drag the XRVideoTextureController.cs script (default location: Assets/XR/Scripts/XRVideoTextureController.cs) onto to the inspector panel for your camera
  • Click the "Add Component" button in the inspector panel and search for "XR Video Texture Controller"

It's recommended that the material on your object is "unlit" (i.e Unlit/Texture) to avoid appearing washed out. XR takes care this automatically for you!

Unity Editor:

XRVideoTextureController Setup

Result:

XRVideoTextureController Result

API Overview

This section of the documentation contains details of 8th Wall XR's scripting API.

Use this information to interface directly with the 8th Wall XR engine instead of using the controllers provided by 8th Wall.

XRCapabilities

Struct

Description

AR capabilities available to a given device. (i.e. is it using ARKit vs ARCore vs. 8th Wall's Computer Vision technology)

Properties

Property Type Description
positionTracking PositionTracking The PositionTracking engine to use.
surfaceEstimation SurfaceEstimation The SurfaceEstimation engine to use.

Constructors

Constructor Description
XRCapabilities Creates an new XRCapabilities struct with a specified positionTracking and surfaceEstimation.

Public Functions

Function Description
IsPositionTrackingRotationAndPosition If true, the device is using ARKit/ARCore for camera tracking. This is a convience method. Equivalent to calling: XRCapabilities.positionTracking == XRCapabilities.PositionTracking.ROTATION_AND_POSITION;
IsPositionTrackingRotationAndPositionNoScale If true, the device is using 8th Wall's 6DoF camera tracking. This is a convience method. Equivalent to calling: XRCapabilities.positionTracking == XRCapabilities.PositionTracking.ROTATION_AND_POSITION_NO_SCALE;
IsSurfaceEstimationFixedSurfaces If true, the device is using 8th Wall instant surface placement (Non-ARKit/ARCore). This is a convience method. Equivalent to calling: XRCapabilities.surfaceEstimation == SurfaceEstimation.FIXED_SURFACES;
IsSurfaceEstimationHorizontalOnly If true, the device is using ARKit/ARCore for surface estimation. This is a convience method. Equivalent to calling: XRCapabilities.surfaceEstimation == SurfaceEstimation.HORIZONTAL_ONLY;

XRCapabilities.PositionTracking

Enumeration

Description

Position tracking engine used by the device.

Properties

Property Description
UNSPECIFIED Unable to determine the tracking engine used by the device.
ROTATION_AND_POSITION The device is using ARKit/ARCore for camera tracking.
ROTATION_AND_POSITION_NO_SCALE The device is using 8th Wall's 6DoF camera tracking.

XRCapabilities.SurfaceEstimation

Enumeration

Description

Surface estimation engine used by the device.

Properties

Property Description
UNSPECIFIED Unable to determine the surface estimataion engine used by the device.
FIXED_SURFACES The device is using 8th Wall instant surface placement (Non-ARKit/ARCore).
HORIZONTAL_ONLY The device is using ARKit/ARCore for surface estimation.

XRCapabilities Constructor

public XRCapabilities(PositionTracking positionTracking, SurfaceEstimation surfaceEstimation)

Parameters

Property Type Description
positionTracking PositionTracking
surfaceEstimation SurfaceEstimation

Description

Creates an new XRCapabilities struct with specified positionTracking and surfaceEstimation.

Example


XRCapabilities.IsPositionTrackingRotationAndPosition()

public bool IsPositionTrackingRotationAndPosition()

Parameters

None

Description

Returns true if running on an ARKit/ARCore device.

This is a convience method. Equivalent to calling:

XRCapabilities.positionTracking == XRCapabilities.PositionTracking.ROTATION_AND_POSITION;

Example


XRCapabilities.IsPositionTrackingRotationAndPositionNoScale()

public bool IsPositionTrackingRotationAndPositionNoScale()

Parameters

None

Description

Returns true if running on a device without ARKit or ARCORE. On these devices, 8th Wall's computer vision tracker is being used.

This is a convience method. Equivalent to calling:

XRCapabilities.positionTracking == XRCapabilities.PositionTracking.ROTATION_AND_POSITION_NO_SCALE;

Example


XRCapabilities.IsSurfaceEstimationFixedSurfaces()

public bool IsSurfaceEstimationFixedSurfaces()

Parameters

None

Description

Returns true if running on a device without ARKit or ARCORE. On these devices, 8th Wall's computer vision engine is being used.

This is a convience method. Equivalent to calling:

XRCapabilities.surfaceEstimation == SurfaceEstimation.FIXED_SURFACES;

Example


XRCapabilities.IsSurfaceEstimationHorizontalOnly()

public bool IsSurfaceEstimationHorizontalOnly()

Parameters

None

Description

Returns true if running on an ARKit/ARCore device.

This is a convience method. Equivalent to calling:

XRCapabilities.surfaceEstimation == SurfaceEstimation.HORIZONTAL_ONLY;

Example


XRController

Description

XRController provides low-level access to 8th Wall XR and can be used to interface directly with the engine instead of using the controllers provided by 8th Wall.

Public Variables

Parameter Type Default Description
enableLighting bool If true, adjust the intensity of your scene light based on the lighting conditions in the world around you
enableCamera bool If true, control the position and rotation of the camera in your scene as you move your device in the real world
enableSurfaces bool If true, enable surface detection and ability to place objects on surfaces

Public Functions

Function Description
GetActiveSurfaceId Returns the ID of the currently active surface.
GetActiveSurfaceMesh Returns the Mesh of the active surface
GetCameraIntrinsics Get the instrinsic matrix of the camera
GetCameraPosition Get the position of the Camera
GetCameraRotation Get the rotation of the Camera
GetCapabilities Returns the position tracking and surface estimation capabilities available to the device.
GetLightExposure Returns the exposure of the environment
GetRealityRGBATexture Returns what the phone's camera is capturing as an RGBA texture.
GetRealityYTexture Returns what the phone's camera is capturing as a Y texture stored in the R channel.
GetRealityUVTexture Returns what the phone's camera is capturing as a UV texture stored in the RG channels.
GetRealityTextureAspectRatio Returns aspect ratio (width/height) of captured image
GetSurfaceWithId Returns the Mesh of a specified surface
GetVideoShader Returns the appropriate Video shader (for AR scene background)
GetVideoTextureShader Returns the appropriate Video texture shader (for AR video textures on objects)
Recenter For Non-ARKit/ARCore phones, reset surface to original position relative to camera
SetAppKey Set App Key. Needs to be set prior to Start() in the Unity lifecycle (i.e. in Awake or OnEnable).
ShouldUseRealityRGBATexture Returns if device requires RGBA textures. (e.g. ARCore)
UpdateCameraProjectionMatrix Set the instrinsic matrix of the camera

XRController.GetActiveSurfaceId()

public long GetActiveSurfaceId()

Parameters

None

Description

Returns the ID of the currently active surface.

Example

public class XRSurfaceController : MonoBehaviour {
  private XRController xr;
  private long surfaceId = Int64.MinValue;

  void Start() {
    xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
  }

  void Update() {
    // If there are no meshes, reset the id to default and don't change anything
    Mesh mesh = xr.GetActiveSurfaceMesh();
    if (mesh == null) {
      surfaceId = Int64.MinValue;
      return;
    }

    surfaceId = xr.GetActiveSurfaceId();
  }
}

XRController.GetActiveSurfaceMesh()

public Mesh GetActiveSurfaceMesh()

Parameters

None

Description

Returns the Mesh of the active surface

Example

public class XRSurfaceController : MonoBehaviour {
  private XRController xr;
  private long surfaceId = Int64.MinValue;

  void Start() {
    xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
  }

  void Update() {
    // If there are no meshes, reset the id to default and don't change anything
    Mesh mesh = xr.GetActiveSurfaceMesh();
    if (mesh == null) {
      surfaceId = Int64.MinValue;
      return;
    }
  }
}

XRController.GetCameraIntrinsics()

public Matrix4x4 GetCameraIntrinsics()

Parameters

None

Description

Get the instrinsic matrix of the camera

Example

public class XRCameraController : MonoBehaviour {
  public const float METERS_SCALE = 1.0f;
  public const float FEET_SCALE = 3.28084f;

  private XRController xr;
  private Camera sceneCamera;

  // XRCameraController.scale allows for scaling the effective units of a scene. For example, if
  // feet is a more natural unit for a scene than meters, set scale to 3.28084f.
  public float scale = METERS_SCALE;

  void Start() {
    xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
    sceneCamera = GetComponent<Camera>();
    xr.UpdateCameraProjectionMatrix(sceneCamera, transform.position, scale);
  }

  void Update () {
    transform.position = xr.GetCameraPosition();
    transform.rotation = xr.GetCameraRotation();
    sceneCamera.projectionMatrix = xr.GetCameraIntrinsics();
  }
}

XRController.GetCameraPosition()

public Vector3 GetCameraPosition()

Parameters

None

Description

Get the position of the Camera in world space as a Vector3

Example

public class XRCameraController : MonoBehaviour {
  public const float METERS_SCALE = 1.0f;
  public const float FEET_SCALE = 3.28084f;

  private XRController xr;
  private Camera sceneCamera;

  // XRCameraController.scale allows for scaling the effective units of a scene. For example, if
  // feet is a more natural unit for a scene than meters, set scale to 3.28084f.
  public float scale = METERS_SCALE;

  void Start() {
    xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
    sceneCamera = GetComponent<Camera>();
    xr.UpdateCameraProjectionMatrix(sceneCamera, transform.position, scale);
  }

  void Update () {
    transform.position = xr.GetCameraPosition();
    transform.rotation = xr.GetCameraRotation();
    sceneCamera.projectionMatrix = xr.GetCameraIntrinsics();
  }
}

XRController.GetCameraRotation()

public Quaternion GetCameraRotation();

Parameters

None

Description

Get the rotation of the Camera in world space as a Quaternion

Example

public class XRCameraController : MonoBehaviour {
  public const float METERS_SCALE = 1.0f;
  public const float FEET_SCALE = 3.28084f;

  private XRController xr;
  private Camera sceneCamera;

  // XRCameraController.scale allows for scaling the effective units of a scene. For example, if
  // feet is a more natural unit for a scene than meters, set scale to 3.28084f.
  public float scale = METERS_SCALE;

  void Start() {
    xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
    sceneCamera = GetComponent<Camera>();
    xr.UpdateCameraProjectionMatrix(sceneCamera, transform.position, scale);
  }

  void Update () {
    transform.position = xr.GetCameraPosition();
    transform.rotation = xr.GetCameraRotation();
    sceneCamera.projectionMatrix = xr.GetCameraIntrinsics();
  }
}

XRController.GetCapabilities()

public XRCapabilities GetCapabilities()

Parameters

None

Description

Returns the position tracking and surface estimation capabilities available to the device.

Example

public class MyClass : MonoBehaviour {
  private XRController xr;
  
  void Start() {
    xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
    if (xr.GetCapabilities().IsSurfaceEstimationHorizontalOnly()) {
      // Device is using ARKit/ARCore for surface estimation.
    } else {
      // Device is using 8th Wall instant surface placement (Non-ARKit/ARCore). 
    }
  }
}

XRController.GetLightExposure()

public float GetLightExposure()

Parameters

None

Description

Returns the exposure of the environment as a value in the range -1 to 1.

Example

public class XRLightController : MonoBehaviour {
  private XRController xr;
  private Light sceneLight;

  void Start() {
    sceneLight = GetComponent<Light>();
    xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
  }

  void Update () {
    float exposure = xr.GetLightExposure();
    // Exposure ranges from -1 to 1 in XR, adjust to 0-2 for Unity.
    sceneLight.intensity = exposure + 1.0f;
  }
}

XRController.GetRealityRGBATexture()

public Texture2D GetRealityRGBATexture

Parameters

None

Description

Returns what the phone's camera is capturing as an RGBA texture.

Example

public class XRVideoTextureController : MonoBehaviour {
  private XRController xr;

  void Start() {
    xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
    Renderer r = GetComponent<Renderer>();
    r.material.shader = xr.GetVideoTextureShader();
    if (xr.ShouldUseRealityRGBATexture()) {
      r.material.mainTexture = xr.GetRealityRGBATexture();
    } else {
      r.material.SetTexture("_YTex", xr.GetRealityYTexture());
      r.material.SetTexture("_UVTex", xr.GetRealityUVTexture());
    }
    r.material.SetInt("_ScreenOrientation", (int) Screen.orientation);
  }
}

XRController.GetRealityYTexture()

public Texture2D GetRealityYTexture()

Parameters

None

Description

Returns what the phone's camera is capturing as a Y texture stored in the R channel.

Example

public class XRVideoTextureController : MonoBehaviour {
  private XRController xr;

  void Start() {
    xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
    Renderer r = GetComponent<Renderer>();
    r.material.shader = xr.GetVideoTextureShader();
    if (xr.ShouldUseRealityRGBATexture()) {
      r.material.mainTexture = xr.GetRealityRGBATexture();
    } else {
      r.material.SetTexture("_YTex", xr.GetRealityYTexture());
      r.material.SetTexture("_UVTex", xr.GetRealityUVTexture());
    }
    r.material.SetInt("_ScreenOrientation", (int) Screen.orientation);
  }
}

XRController.GetRealityUVTexture()

public Texture2D GetRealityUVTexture()

Parameters

None

Description

Returns what the phone's camera is capturing as a UV texture stored in the RG channels.

Example

public class XRVideoTextureController : MonoBehaviour {
  private XRController xr;

  void Start() {
    xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
    Renderer r = GetComponent<Renderer>();
    r.material.shader = xr.GetVideoTextureShader();
    if (xr.ShouldUseRealityRGBATexture()) {
      r.material.mainTexture = xr.GetRealityRGBATexture();
    } else {
      r.material.SetTexture("_YTex", xr.GetRealityYTexture());
      r.material.SetTexture("_UVTex", xr.GetRealityUVTexture());
    }
    r.material.SetInt("_ScreenOrientation", (int) Screen.orientation);
  }
}

XRController.GetRealityTextureAspectRatio()

public float GetRealityTextureAspectRatio()

Parameters

None

Description

Returns aspect ratio (width/height) of captured image

Example

public class XRVideoController : MonoBehaviour {

  private XRController xr;
  Resolution currentRes = Screen.currentResolution;

  public void Start() {
    xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
    float scaleFactor =
      ((float)currentRes.width / (float)currentRes.height) / xr.GetRealityTextureAspectRatio();
  }
}

XRController.GetSurfaceWithId()

public Mesh GetSurfaceWithId(long id)

Parameters

None

Description

Returns the Mesh of a specified surface

Example

public class XRSurfaceController : MonoBehaviour {
  private XRController xr;
  private long surfaceId = Int64.MinValue;

  void Start() {
    xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
  }

  void Update() {

    Mesh mesh = surfaceId == Int64.MinValue ? xr.GetActiveSurfaceMesh() : xr.GetSurfaceWithId(surfaceId);
    if (mesh == null) {
      surfaceId = Int64.MinValue;
      return;
    }

    surfaceId = xr.GetActiveSurfaceId();
  }
}

XRController.GetVideoShader()

public Shader GetVideoShader()

Parameters

None

Description

Returns the appropriate Video shader (for AR scene background)

Example

public class XRVideoController : MonoBehaviour {

  private XRController xr;

  private Material xrMat;
  private Camera cam;

  public void Start() {
    xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
    cam = GetComponent<Camera>();
    cam.clearFlags = CameraClearFlags.Depth;
    xrMat = new Material(xr.GetVideoShader());
    xrMat.SetInt("_ScreenOrientation", (int) Screen.orientation);
  }

XRController.GetVideoTextureShader()

public Shader GetVideoTextureShader()

Parameters

None

Description

Returns the appropriate Video texture shader (for AR video textures on objects)

Example

public class XRVideoTextureController : MonoBehaviour {
  private XRController xr;

  #if !UNITY_EDITOR
  void Start() {
    xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();

    Renderer r = GetComponent<Renderer>();
    r.material.shader = xr.GetVideoTextureShader();
    r.material.mainTexture = xr.GetRealityTexture();
    r.material.SetInt("_ScreenOrientation", (int) Screen.orientation);
  }
  #endif
}

XRController.Recenter()

public void Recenter()

Parameters

None

Description

For Non-ARKit/ARCore phones, reset surface to original position relative to camera

Example

public class XRViewController : MonoBehaviour {
  private XRController xr;
  private long surfaceId = Int64.MinValue;

  void Start() {
    xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
    xr.Recenter();
  }

XRController.SetAppKey()

public void SetAppKey(string key)

Parameters

None

Description

Set App Key. Needs to be set prior to Start() in the Unity lifecycle (i.e. in Awake or OnEnable).

You will need to create a unique license key for each 8th Wall app that you develop.

Example

public class MyXRClass : MonoBehaviour {
  private XRController xr;
  private string myAppKey = "Example123";

  void OnEnable() {
    xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
    xr.SetAppKey(myAppKey);
  }
}

XRController.ShouldUseRealityRGBATexture()

public bool ShouldUseRealityRGBATexture()

Parameters

None

Description

Returns if device requires RGBA textures. (e.g. ARCore)

Example

public class XRVideoTextureController : MonoBehaviour {
  private XRController xr;

  void Start() {
    xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
    Renderer r = GetComponent<Renderer>();
    r.material.shader = xr.GetVideoTextureShader();
    if (xr.ShouldUseRealityRGBATexture()) {
      r.material.mainTexture = xr.GetRealityRGBATexture();
    } else {
      r.material.SetTexture("_YTex", xr.GetRealityYTexture());
      r.material.SetTexture("_UVTex", xr.GetRealityUVTexture());
    }
    r.material.SetInt("_ScreenOrientation", (int) Screen.orientation);
  }
}

XRController.UpdateCameraProjectionMatrix()

public void UpdateCameraProjectionMatrix(Camera cam, Vector3 origin, float scale)

Parameters

Parameter Type Default Description
cam Camera The camera that should have it's projection matrix updated
origin Vector3 (0,0,0) Initial position of the camera (camera's tranform.position)
scale float 1.0 scaling the effective units of the camera

Description

Set the instrinsic matrix of the camera

Example

public class XRCameraController : MonoBehaviour {
  public const float METERS_SCALE = 1.0f;
  public const float FEET_SCALE = 3.28084f;

  private XRController xr;
  private Camera sceneCamera;

  // XRCameraController.scale allows for scaling the effective units of a scene. For example, if
  // feet is a more natural unit for a scene than meters, set scale to 3.28084f.
  public float scale = METERS_SCALE;

  void Start() {
    xr = GameObject.FindWithTag("XRController").GetComponent<XRController>();
    sceneCamera = GetComponent<Camera>();
    xr.UpdateCameraProjectionMatrix(sceneCamera, transform.position, scale);
  }
}

ARCore Support

In order to fully support ARCore functionality within your 8th Wall XR enabled Unity application, you will need to install ARCore onto your ARCore supported mobile device.

ARCore Supported Devices: https://developers.google.com/ar/discover/#supported_devices

Prepare your device:

  1. Enable developer options
  2. Enable USB deebugging

Install ARCore on the device:

  1. Visit https://developers.google.com/ar/develop/downloads
  2. Download the ARCore app from the URL in the previous step.
  3. Install ARCore by running the following adb command:

adb install -r -d arcore-preview2.apk

NOTE: Unsupported devices or devices without the ARCore Service app installed will use 8th Wall's 6DoF tracking instead of ARCore.

AR Shadows

AR is more realistic when virtual objects in your scene cast shadows onto the physical world. 8th Wall XR now provides a shader that can be used on transparent surfaces.

Create a material (for example, call it MyXRTransparentSurface), and select 8thWall -> XRShadow from the list of shaders, then apply the material to the surface object you want to make transparent (i.e. a ground plane):

XR Shadow Shader

To adjust the opacity of the shadows, select your Directional Light, and adjust the Strength of the shadow. To improve shadow quality, set resolution to "Very High Resolution"

XR Shadow Shader Opacity

For improved shadow quality, go to Edit -> Project Settings -> Quality. Set Shadow Resolution to "Very High Quality" and Shadow Projection to "Close Fit"

XR Shadow Quality

Result:

XR Shadow Shader Result

Switching Channels

You can test the latest features of 8th Wall XR by switching to the Beta channel.

To Switch Channels:

  1. Select Assets / XR / 8thWallXR
  2. Select the desired channel from the resulting inspector pane:

Switching Channels

Changelog

Release 5:

  • New 8th Wall Console - provides developer access to:

    • Generate and manage App Keys.
    • View application analytics.
    • Access downloads and documentation.
  • New Features:

    • ARCore Preview 2 support.
    • XRCapabilities API.
    • OpenGL support on iOS.
    • Added support for Android phones without gyroscopes.
  • Enhancements and Fixes:

    • Performance improvements to rendering.
    • Removed Android Support Library dependency.
    • Camera preview rendering performance improvements.
    • Fix rendering on non-16x9 phones.
    • Fix tracking orientation on Android tablets.
    • Fix ARCore aspect ratio.
    • Updated default camera usage descriptions.
    • Fix tracking for x86 architectures.
    • Improved compatibility with developer ProGuard configurations.

Release 4:

  • 6DoF camera motion for phones without ARKit or ARCore!
  • Enhancements and Fixes:

    • XRSurfaceController:

      • New: GetSurfaceWithId() - Ability to select surface with specific ID.
      • lockToFirstSurface: Fixed issue on phones without ARKit or ARCore.
      • deformToSurface: Fixed issue on phones without ARKit or ARCore.
    • XRController:

      • New: Recenter() - Reset camera camera to original position on phones without ARKit or ARCore.
      • UpdateCameraProjectionMatrix() - Fixed issue on phones without ARKit or ARCore.
    • XREngine:

      • Up to 3x improved performance on phones without ARKit or ARCore.

Release 3:

  • XRSurfaceController enhancements:

    • Specify if object should be displayed in scene immediately, or only after surface is detected
    • Specify if any surface should be used, or only "ground" surfaces.
    • Specify if object should be locked to first detected surface, or move to currently active surface
  • XRLightController performance improvements
  • Fixed frame timing for ARCore
  • Fixed issues with mismatched surfaces in ARCore
  • New Sunlight Controller available on community GitHub repo

Release 2:

  • ARCore support
  • Deformable surface meshes
  • Improved surface mesh stability
  • AR Shadow support
  • ARKit native resolution support
  • ARKit frame delay fixes
  • Unity Editor play mode fixes
  • Support for latest iOS 11 release

Release 1:

  • Initial release of 8th Wall XR

iOS Crash on Scene Load

Issue: Scene crashes on iOS devices that support Metal, if Metal API is removed from Unity project settings.

XCode reports an "EXC_BAD_ACCESS" error message within the setManagedCameraTexture() function:

XCodeExcBadAccess

This happens on certain phones if "Auto Graphics API" is unchecked and Metal is removed from the list of Graphics APIs as seen here:

MTRendering

See Apple Documentation for the list of phones that support Metal

Resolution: Add Metal to list of Graphics APIs

  1. Go to Edit -> Project Settings -> Player
  2. Click on the iOS Settings tab
  3. Under Graphics APIs, click "+" and add Metal back into the list:

MTRendering

Mulththreaded Rendering

Issue: My mobile application asks for camera access, but doesn't render the camera view.

Resolution: Disable Multithreaded Rendering

  1. Go to Edit -> Project Settings -> Player
  2. Click on the Android Settings tab
  3. Uncheck Multithreaded Rendering

MTRendering

Missing XR Controller

Issue: None of the AR features are working (camera position/rotation, Video background, etc)

Looking through logs, you'll see NullReferenceExceptions such as:

"NullReferenceException: A null value was found where an object instance was required. at XRController.GetCameraPosition () [0x00000] in :0"

Resolution: Add an XRController (with "XRController" tag) to your scene. You can do this in the Hierarchy panel via Create -> XRController or GameObject menu -> XRController:

create-xrcontroller

created-xrcontroller

Help & Support

Need some help? 8th Wall is here to help you succeed. Contact us directly, or reach out to the community to get answers.

Ways to get help:

Email Support Stack Overflow GitHub
Email support@8thwall.com to get help directly from our support team Get help and discuss solutions with other 8th Wall XR users by using the 8thwall-xr tag Download sample applications and view step-by-step setup guides on our xr-unity GitHub repo


























[1] Intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for 8th Wall’s products remain at the sole discretion of 8th Wall, Inc.