HoloLens Programming Tutorial: AirTap & Spatial Mapping

AirTap & Spatial Mapping
HoloLens Programming Tutorial:

Development Tools
Development environment：Unity
 Game engine and development environment for multi-platform
 You can visually place CGs and add functions by using GUI
 Interactive behavior can be described in C# or javascript
 HoloLens is supported from Unity 5.5
Development support tool:：HoloToolkit
 Toolkit to support the development of HoloLens compatible systems
 You can use AirTap and cursor behavior, spatial mapping easily
 Various sample programs are provided
 Also provides functions such as moving viewpoint on Unity editor
It is possible to develop without HoloLens or emulator
2

Goal of this tutorial
YouTube → https://youtu.be/_BmbL4yw8R0
3

First of all, the basic operation of Unity
4

Create a project
(1) Extract HoloToolkit-Unity-master.zip to any directory
(2) After starting Unity, click OPEN at the upper right of the screen
OPEN
5

Create a project
Open HoloToolkit-Unity-masterfolder and click [folder selection]
HoloToolkit-Unity-master
Select folder
HoloToolkit-Unity-master
6

Unity operation screen (overview)
Game space design area
Name list of objects
placed in space
List of Asset (CG, program, etc.) added to the project 7

Let's add CG to the game space
Right click
3D Object→Cube
8

Let's run
Click and run
Click to Exit
Space seen from
a camera
9

Adjust the position, orientation and size of the object
Click and select
move rotate zoom in / out
10

Let's change the perspective of Scene
[←][→] -Move left and right
[↑] [↓] -Zoom in/out
[Alt] + drag -Rotation
Other thingshttp://goo.gl/Lq1ILT
→ http://goo.gl/Lq1ILT
11

Designation of CG position/ size using numerical values
Click
Change Position to 0 0 1.2 Change all scale to 0.15
※ The unit of position and size is meter
Inspector(≒Detailed Information
12

Editing CG detailed information :color setting (1/3)
②Right click
③ Create
Make detailed material to set colour and texture
④ Material
①Assets
13

Editing CG detailed information: color setting (2/3)
Edit colour (Albedo) of the created material
(2) Click on the white area next
to Albedo and select a color
(1) Select created material
14

Editing CG detailed information: colour setting (3/3)
Open the material of the Cube to assign the created material
(2) Top of Materials▼
(1)Cube
(3) Drag and drop to
Element 0 review
15

Check
It is OK if the colour is reflected. You can change the colour later.
16

Setting the influence of gravity
Add Component
選択
RigidBody
17

Operation check and fine adjustment
Let's run it
When Cube looks smaller, change Position of MainCame to 0, 0, 0
18

Creation of ground
Click right
3D Object→Plane
19

Adjust the ground position
Plane
Change Position to 0 -0.8 0
20

Save Scene
Scene Name
 Open the dialog with [Ctrl] + [s] and save with your favourite name
 Frequently save during content creation
21

Development using HoloToolKit
22

Setting HoloLens Camera
Delete Main Camera
23

Adding HoloLens Camera
(1) Assets → HoloToolkit → Input → Prefabs
(2)Drag & Drop HoloLens Camera
24

Check
Light & Left by[←] [→]
Forward & Backword by[↑] [↓]
Up & down by[Q] [E]
Rotation by right click + mouse move
25

Supplement: Adjust movement / rotation sensitivity
(1) Select operation content
KeyboardXZTranslation
(2) Sensitivity Scale to 2
(3) Axis Type to Keyboard Arrows
Amount of movement when pushing keys :↑↓← → , will become less than the default
26

Spatial recognition using SpatialMapping
Remove Plane
27

Spatial recognition using SpatialMapping
Assets → HoloToolkit → SpatialMapping → Prefabs
Drag & drop SpatialMapping
28

Operation
Space is recognized by actual HoloLens, but not by PC
29

Importing a space model (official sample)
② Assets → HoloToolkit → SpatialMapping → Tests → Meshes
③ Drag & drop SRMesh into Room Model
① Spatial Mapping
30

Importing a space model (official sample)
(1)Run
The model of the room is displayed
More simplified one provided instead of more complexed one 31

Import of space model (for study group)
Assets
→ Import New Asset
→ isit.obj
Import spatial model (isit.obj) prepared as sample
33

Import of space model
(2)SpatialMapping
(3) Drag & drop into Room Model
(1)Assets
34

Change appearance of mesh
Click button next to
Surface Material SpatialMappingWireframe
Before After
35

Create and drop Cube at the same time as AirTap
What to do next
36

Add function to manage input operations such as AirTap
(1)HoloToolkit → Input → Prefabs
(2)Drag & drop Input Manager
37

Check motion
Run
(1) Press [Shift] to use hand
(2) Left-click emulates AirTap
38

Create a script to receive AirTap
Right click on
blank of herarchy
Create Empty
Create an empty object to attach script
39

(1)GameObject
(2)Add Component
40

New Script
Scriot name
e.g.RcvAirTap
Create and Add
Added
41

Open script file by VisualStudio
Assets
Double click on
script (RcvAirTap)
42

Writing a script
using HoloToolkit.Unity.InputModule;
public class RcvAirTap : MonoBehaviour
{
// Start function is executed only once for initialization
void Start () {"R
obot Kyle");
}
// Update function is executed every frame
void Update () {
}
}
First, load InputModule and implement IInputClickHandler
, IInputClickHandler
43

Writing a script
public class RcvAirTap : MonoBehaviour , IInputClickHandler
{
// Start function is executed only once for initialization
void Start () {
}
// Update function to be executed every frame
void Update () {
}
}
•Move the cursor to llnputClickHandler,
Select lmplement interface from hint
44

Check the status quo
using HoloToolkit.Unity.InputModule;
{
//Called up function when AirTapped
public void OnInputClicked(InputEventData eventData)
{
throw new NotImplementedException();
}
void Start () {
}
void Update () {
}
}
throw new NotImplementedException();
Delete this line as unnecessary
45

What to do next
InputManager knows AirTap is done or not
Previous script is written here
Notify GameObject when InputManager recognizes AirTap
46

Receive AirTap
{
//Called up function when AirTapped
{
Debug.Log("AirTap!");
}
void Start () {
InputManager.Instance.
PushFallbackInputHandler(gameObject);
}
void Update () {
}
}
Register so that AirTap will
be notified to gameObject
with this script attached
47

Check motion
Run
AirTap! To be displayed
Console tab
48

Prefabrication of Edited Cube
(1) Assets
(2) Drag & drop Cube
49

What to do next
Cube prefab
Create a Cube (Prefab) in space for each AirTap
50

Generation of cube using script
{
//Variables to handle Cube prefab
public GameObject original;
{
// Instantiate cube using Cube prefab
GameObject cube = GameObject.Instantiate(original);
//Convert the position 1.2 m ahead from your
//viewpoint into the position in real space
cube.transform.position =
Camera.main.transform.TransformPoint(0, 0, 1.2f);
Debug.Log("AirTap!");
}
/*Omitted below*/
51

Assign Cube Prefab with variable of script
(1) Assets
(2) GameObject
Associate variable(=original) of script with Cube Prefab
Drag & drop Cub into Original
52

Check
Run
Generate cubes as many times as AirTapped
53

Limit the number of cubes
//container holding several cubes
List<GameObject> list = new List<GameObject>();
{
GameObject cube = GameObject.Instantiate(original);
cube.transform.position =
Camera.main.transform.TransformPoint(0, 0, 1.2f);
if (list.Count == 10)
{
Destroy(list[0]); //Delete the oldest cubes from space
list[0] = null;
list.RemoveAt(0); // Remove element in list 0
}
list.Add(cube);
}
54

Let's run on HoloLens: Confirm Quality
Edit
Project Settings
Quality
Fastest
55

Run on HoloLens: Build Setting
(1)Edit
(2)Project Settings
(3)Player
(4)Name ProductName
for yours
(5)Windows Store App
56

Name Short
namefor yours
icon Other Settings
Virtual Reality
Supported
Publishing Settings
Name Package
Namefor yours
57

File
Build Settings
58

Run on HoloLens: Build
Add OpenScenes
Windows Store
Switch Platform
・ SDK
Universal 10
・ Target device
HoloLens
・ UWP Build Type
D3D
Finally Build
59

Run on HoloLens: Create a folder for Build
Name App
New folder
Select folder
60

Project
yourname.sln
Appfolder
61

Run on a real machine: Deploy to HoloLens
x86
Click ▼
Remote
(In case of
Wifiapplication)
62

In put HoloLens IP
Click Select
63

Debug
Start Without
Debugging
64

Hide the mesh
Spatial Mapping Smatial Mapping Manager
Click 〇
Select Spatial Mapping Occlusion for material
66

Increase the display range
HoloLens Camera
Change Nearto 0.3
Decreasing the value of the camera's Near, will work out
even in case of approaching to some extent
67

Import images pasted on Cube
[Assets]→
[Import New Asset]
→ Select the image you brought up
68

Material setting change
Click on the material (New Material) assigned to Cube
69

Material setting change
(1) Shader
(2) Unlit
(3) Texture
70

Paste texture
Drag and drop images
71

Any questions?
Please Contact Me!
Name：
Takashi Yoshinaga
Affiliation：
Institute of Systems, Information
Technologies and Nanotechnologies
Title：
(1) Researcher
(2) Microsoft MVP for Windows Development
taka.yoshinaga @tks_yoshinaga
YoshinagaTakashi

74
I’m also waiting for your suggestions on English expressions :)

HoloLens Programming Tutorial: AirTap & Spatial Mapping

HoloLens Programming Tutorial: AirTap & Spatial Mapping

More Related Content

Similar to HoloLens Programming Tutorial: AirTap & Spatial Mapping

More from Takashi Yoshinaga

Recently uploaded

In this document

HoloLens Programming Tutorial: AirTap & Spatial Mapping