holokit / holokitsdk

HoloKit Unity SDK for HoloKit 1 (Cardboard version). This repo is deprecated. Please refer to https://github.com/holokit/holokit-unity-sdk for latest SDK for HoloKit X.

HoloKitSDK v1.5 (Branch)

HoloKitSDK is the Unity plugins to build AR/MR apps for HoloKit.

Features

• Stereosopic rendering
• Calibration profile for different holokits and phones
  • It's the key to have a correct Holokit experience. Wrong calibration may lead to eye discomfort.
• AR/MR Mode Switch
  • You can easily switch betwen AR and MR mode.
• Screen Brightness Control.
  • Make sure that the brightness of phone screen are the best for the current environment during the MR experience.
• Universal wrapper for ARCore and ARKit.
  • HoloKit SDK integrates both ARCore and ARKit. It will seamlessly compile to Android and iOS devices. You don't need worry about the switch between platforms.
• [ARKit v1.5 or above] Horizontal and vertical plane detection.
• [ARKit v1.5 or above] Image marker detection.

News

• Feb 23, 2018. Added ARcore 1.0 support.
• Feb 23, 2018. Added ARKit 1.5 beta support.

Supported Devices

• iOS (ARKit): iPhone 6S, 7, 8, 6S+, 7+, 8+, X.
• Android (ARCore): Google Pixel, Pixel XL, Pixel 2, Pixel 2 XL, Asus Zenfone AR, LG V30, OnePlus 5, Samsung Galaxy S7, S7 Edge, S8, S8+, Note8. See more: the full list of supported Android devices)

Version Index

HoloKit SDK v1.5 (Current version):

• For ARKit 1.5 beta and ARCore 1.0, please checkout "master" branch.

HoloKit SDK v1.0 (Legacy version):

• For Google Tango, please checkout "tango" branch.
• For Apple iOS (ARKit), please checkout "ios" branch.
• For Android (ARCore), please checkout "android" branch.

How to get started?

Prerequists(iOS)

• You must be an Apple Developer.
• An iOS device that supports ARKit v1.5 and running iOS 11.3.
• XCode 9.3. You need to log in to download it with your Apple Developer account.
• Unity 2017.3.1f3 or later. Make sure you installed iOS components.
  1. Change the target platform to iOS and click Switch Platform.
  2. Open "File" -> "Build Settings" and click "Build". Make sure that the paramaters below is correct. Player Settings -> Other Settings
     • Artectecture: ARM64
     • Camera Usage Description: "Blahblah...".
     • Target minimum iOS version: 11.3

Prerequists(Android)

• An Android device that supports ARCore with Android SDK version 7.0 (API Level 24) or higher.
  • Prepare your device
  • Enable developer options
  • Enable USB debugging
• Android Studio with Android SDK installed.
• Unity 2017.3.1f3 or later. Make sure you installed Android components.
  1. Change the target platform to Android and click Switch Platform.
  2. Click Player Settings to open the Android Player Settings. Then change the following settings:
     • Other Settings > Multithreaded Rendering: Off
     • Other Settings > Package Name: a unique app ID that looks like a Java package name, such as com.example.helloAR
     • Other Settings > Minimum API Level: Android 7.0 or higher
     • Other Settings > Target API Level: Android 7.0 or 7.1
     • XR Settings > ARCore Supported: On

Quick Start - Universal (iOS and Android)

1. Import "HoloKitSDK" folder under "Assets" folder into a new Unity project.
2. Open the example scene "HoloKitSDK/Examples/HoloKitSample_Universal".
3. Build your scene and run! This scene can run on both iOS and Android.
4. After the app runs, you should see a cube floating in the air somewhere. You may gaze at the sphere.
5. The app detects planes, and you may click on the screen to place the cube on the plane.
6. You may touch the small "AR/MR" button to switch to HoloKit mode.

Quick Start - ARKit v1.5 Only (iOS 11.3)

1. Import "HoloKitSDK" folder under "Assets" folder into a new Unity project.
2. Open the example scene "HoloKitSDK/Scenes/HoloKitSample_ARKit1.5".
3. You can detect the vertical plane.
4. You can detect the image marker under "Assets/UnityARKitPlugin/Examples/ARKit1.5/UnityARImageAnchor/ReferenceImages/unitylogowhiteonblack.png".

• 

Create your own experience

1. Create a new scene in Unity.
2. Drag and drop everything in "HoloKitSDK/Prefabs/Universal" to the scene.
3. Put anything you like under "HoloKitPlacementRoot", and your model should have a comparable size as "DebugCube". Then feel free to turn off or delete "DebugCube".
4. Build your scene and run!
5. If you don't like the ambient light, please disable HoloKitAmbientLight in your scene.

Manual

Prefabs

HoloKitCamera

• The most basic component to have your game responds to ARKit tracking.
• It has a child VideoSeeThroughCamera that renders video seethrough VR experience. There are two extra cameras "Left Eye", "Right Eye" as its children, to render stereo for HoloKit. In "HoloKitCameraRig" object, you can set default see through mode and the key for toggling modes.

HoloKitObjectGenerator

DisplayPlanes

• To generate collidable planes when ARKit detects a plane. Optionally, you can set a Plane Prefab as its property to visualize the generated planes.

PointCloud

• To visualize point clouds that ARKit detects, for debugging purpose.

AmbientLight

• A directional light, which intensity is controlled by ARKit. The intensity will adjust automatically based on the current environment's lighting in reality.

HoloKitPlacementRoot

• A convenient prefab to help you place your objects on the ground. See "How to place your object on the ground" for details.

HoloKitGazeManager

• A convenient prefab for you to emit gaze events. See "How to respond to gaze events" for details.

How to place your object on the ground

1. Drag "HoloKitPlaneGenerator" and "HoloKitPlacementRoot" to your scene.
2. Put your object under "HoloKitPlacementRoot", and your object should have a comparable size as "DebugCube". Then feel free to turn off or delete "DebugCube".
   • 
3. Set the input key for placement.
   • 

Troubleshooting

How to upgrade to iOS 11.3 Beta?

1. Backup your device. See "Prepare your device before you update to beta software" section in About iOS beta software
2. Follow iOS beta Software Installation Guide to install iOS 11.3 Beta.

I don't have an Apple Developer account.

You need one.

I cannot see any "Team" in my XCode project settings.

Make sure "Automatically manage signing" is checked, and you've logged in with your developer account in "XCode" -> "Preferences" -> "Accounts".

Attribution

You shall read the How to Attribute section.

App developer shall mark with the words, "Works with HoloKit", or display either of the following two Holokit Logos in your app.

or

For academic work, please cite Monocular Visual-Inertial State Estimation for Mobile Augmented Reality, P.Li et al (ISMAR 2017, accepted)

holokitsdk's Issues

change VR camera with ARkit camera

it is possible to change VR camera with ARkit Camera?
I tried that but camera view turn narrow and tracking not in accordance

does holokitsdk support ipad(A9+)?

Supported Devices
iOS (ARKit): iPhone 6S, 7, 8, 6S+, 7+, 8+, X.(support ipad?)
Android (ARCore): Google Pixel, Pixel XL, Pixel 2, Pixel 2 XL, Asus Zenfone AR, LG V30, OnePlus 5, Samsung Galaxy S7, S7 Edge, S8, S8+, Note8. See more: the full list of supported Android devices)

How to caluclate viewport size with different real device screen any guide or advice?

Hi,
Thanks for the great work!
As the newest version SDK, the PhenoType didn't contains the newest iPhone. e.g. iPhone XS Max or iPhone XR.

So the viewport is wired in iPhone XS Max or iPhone XR.
I'm trying add those device myself. But I can't get the correct parameters to render the viewport correctly.

Does there any advice or guide to help me get the correct parameters?

Thanks so much!

ios 多次打开场景会造成内存泄漏 停止服务的时候内存没有完全清理

Distortion Correction?

Because the maths is difficult to deal with. So i decided to buy the official holokit cardboard. And i am using iphone 8 for test.

So far, i feel the depth is correct ,which is good (i bought a third party headset and depth is incorrectly closer than the real thing). However, it has a strong distortion problem in the edge area. So only the position close to the center of view is marching the real world. I see that in the code, holokit v1 model set the distortion correction to zero. And if i manually adjust the distortion, it will break the perspective.

So what should i do?

Placement of smartphone on headset + calibration

First of all, great work with the Holokit!

I was now taking the first steps with it, and was wondering, do you have any recommended lateral placement for the smartphone in the headset, for example, for the Iphone? was wondering how many millimeters from the edge (for example from the camera edge) should it be placed so that your transforms screen-to-display would be according to the pre-calibration you have done (reducing distortion and have a good alignment between the real world and displayed content)?

The other question was if you had any calibration sw available that a user could run, to reduce distortion and have a good alignment between the real world and displayed content?

Thanks!

ARKit 2.0

Hi guys,

Was just wondering if you can advise what has to be changed to get the Holokit to be able to be integrated with the new ARKit 2.0 (iOS 12)?

Thanks!

Need XCode 9 beta 5 to deploy

The iOS app need a XCode 9 beta 5 or above to deploy, otherwise, will fail.

any plan update to ar foundation?

https://github.com/Unity-Technologies/experimental-ARInterface arinterface seems deprecated. any plan to update to https://docs.unity3d.com/Packages/[email protected]/manual/index.html ?

HolokitBug?

Hi guys,I found that Android SKD packaged in the back of the mobile phone will cause flicker problem when the screen changes horizontally and vertically. Anyone can answer this question, thank you.

No support for ARCore

Why is that it's not having any support for ARCore? It doesn't seem to work with it though it says that it has support for ARCore

HoloKitScreenBrightnessManager throwing an exception on Android

There's an issue when getting and setting the screen brightness on Android:

First: android.provider.Settings.System should be android.provider.Settings$System (System is a nested class of Settings) and system.CallStatic("putInt", contentResolver, system.CallStatic<string>("SCREEN_BRIGHTNESS_MODE"), 0); should be system.CallStatic<bool>("putInt", contentResolver, system.GetStatic<string>("SCREEN_BRIGHTNESS_MODE"), 0);

Second: You cannot get the write settings permission by only adding it to the manifest.
Here's a quick workaround :

// We are using ARCore so Build.VERSION.SDK_INT will always be greater than VERSION_CODES.M
private bool HasWriteSettingsPermission()
{
    var unityPlayer = new AndroidJavaClass("com.unity3d.player.UnityPlayer");
    var activity = unityPlayer.GetStatic<AndroidJavaObject>("currentActivity");

    var system = new AndroidJavaClass("android.provider.Settings$System");
    if (!system.CallStatic<bool>("canWrite", activity))
    {
        var settings = new AndroidJavaClass("android.provider.Settings");
        var uri = new AndroidJavaClass("android.net.Uri");

         var intent = new AndroidJavaObject("android.content.Intent", settings.GetStatic<string>("ACTION_MANAGE_WRITE_SETTINGS"));
        intent.Call<AndroidJavaObject>("setData", uri.CallStatic<AndroidJavaObject>("parse", "package:" + activity.Call<string>("getPackageName")));

        activity.Call("startActivityForResult", intent, RESULT_CODE);

        return false;
    }
    return true;
}

It will open the settings dialog of the app and it's up to you how you want to handle onActivityResult. I might be wrong but at one point you'll need to have a custom activity in native android code and handle the result from there.

PS: The AndroidManifest.xml under HoloKitSDK/Plugins/HoloKitScreenBrightnessManager/Android doesn't seem to be merged. I had to copy the default Unity Android manifest and add the WRITE_SETTINGS permission to Plugins/Android/AndroidManifest.xml

How to use Google ARcore instant preview?

I wanted to test out the app without building it using unity. AR core supports instant preview to connected supported devices to use their camera inside editor. Please try including that as its really helpful to speed up the development process.

Detecting vertical and horizontal planes

Hello,

I've ordered the Holkit a month ago and now I started developing simple tests with the SDK.
I am trying now to detect a generate a plane. However, I could not succeed. Could you give me some hints how can I achieve this functionality?

Thanks in advance!

Best regards,
Ivan