HoloLens is an augmented reality (AR) introduced by Microsoft, also known as MR. Its technology is very revolutionary. With it, it can superimpose the virtual world in the real world and bring people the immersive experience of combination of virtual and reality. Such interaction is more natural and convenient, and is likely to become a common form of human-machine interaction in a few years.
The development version that has been sold now can be viewed in https://www.microsoft.com/microsoft-hololens/en-us/development-edition. Briefly, it has a 64GB Flash storage space, 2GB RAM, 2GB CPU, and 1GB HPU (I do not know what HPU is, and the official web is written), which can be used about two or three. For an hour, the weight is 579g.
More official descriptions can also be taken to HoloLens’s official website to view https://www.microsoft.com/microsoft-hololens/en-us.
In the Hackathon link at Microsoft students’ summer camp, we have the chance to reach HoloLens. We want to develop a real game similar to the Ingress and tribal conflict, and can be interconnected to allow players to experience the non offensive thinking. But without the Unity development experience and time limit (36h), we set the goal as a Demo for picking up energy and replenish energy.
Here is a summary of the general development process of this HoloLens. If you don’t have time to look at it, look directly at the bottom of the reference. Many articles of dry goods and Daniel are worth reading.
The official words are over, tell me how I feel.
When I got the HoloLens, I found many limitations, which was not as good as I imagined. The first is that its vision is limited (much smaller than expected, just like a monitor in front of us, which is not enough at all! I hope that the later version can solve this problem), second is not enough computing power (but the disadvantage is the advantage, it is a complete PC, do not have to be tow the line like other VR devices), the third is the hand gesture is very difficult, now only two gestures, one is only the point of the index finger, the other is five fingers. Open Bloom, it’s not very convenient to interact. There are still only support for English, the picture depth control is unreasonable, will cause eye pain, too heavy machine and so on.
But with the experience of several days, I found the essentials of using HoloLens – stand up and walk! When you stand up, you will find that the whole world has changed. It is no longer a sense of sitting without HoloLens. When you stand up, you can see all of you nailed to the wall, pin in the window, even floating in the air window, of course, you can also drag the window by hand to a suitable place. The real world and the virtual world are so unclash together, and even after a while, you will forget what the real world is like, and when you take the HoloLens off, you will not be able to adapt to the feeling that you can’t control the window. Interactive operation is not so unbearable, Bloom gesture is very beautiful, the moment the screen starts to open, it is pretty beautiful! The sensitivity of Cortana is beyond my imagination. I can speak with her at the level I only have six level English. What is most worth mentioning is that when you put an object, such as an aircraft model or an earth, or even the whole galaxy on the table, you can carry it around it for 360° watch, see the details of each direction, click on a star ball into that planet or enlarge a plane. Make it full of the whole house. The feeling is quite shocking! Never a VR product can give me such a real and unreal sense of shock. HoloLens really did it, perfect!
Pre – development preparation
Preparation: install PC with Windws10. If you need HoloLens simulator, you need more than 4GB of memory and support for virtualization of more than 4 core CPU.
Install Visual Studio 2015: any version is available, recommend free community version, download link https://www.visualstudio.com/
Install Unity For HoloLens: the common Unity version is not supported by HoloLens related settings, and you need to use the Unity version specially designed for HoloLens and download the address http://aka.ms/HoloLensUnity
Install HoloLens Emulator: if no HoloLens can be debugged for the time being, you can use the simulator for debugging, download the address http://go.microsoft.com/fwlink/? LinkID=823018
The goal of this development is a 3D game, so we mainly use Unity to develop the game, use VS to debug the code and transmit it to HoloLens.
In addition to using Unity development, you can also use DirectX to develop, or you can directly run the developed UWP program on HoloLens.
Game development on Unity
First of all, the HoloLens customized version of Unity is newly built, and then need to change the settings to adapt to HoloLens. Specific reference: https://developer.microsoft.com/zh-cn/windows/holographic/unity_development_overview
Because of the stressful events, all the teammates are selling at present, the level is the level of one day, so the game is not so good that it doesn’t go to the code. The general effect is to pick up the energy ball from the air and pick up the energy to supply its energy crystal by clicking.
Dealing with the interaction of games and HoloLens
Now, pretending to emulate (or generate exe files) on Unity will be able to run. However, because HoloLens interaction is very different from ordinary mouse and keyboard interaction, we need to deal with HoloLens interaction.
Let’s first talk about the main types of HoloLens interaction. HoloLens provides the following six sets of API:
World coordinates World coordinates
Gaze into the input Gaze input
Gesture input Gesture input
Sound input Voice input
Space sound Spatial sound
Space matching Spatial mapping
We need to use gaze input, gesture input and space matching this time. Gaze at the energy ball or energy crystal, then launch the staring ray, then detect the hand clicks, and pick up the energy or add energy if the crystal is clicked and clicked. The whole space needs to be scanned when placing the energy crystal, and then the crystal is placed on a surface, and when the energy ball falls down, it will disappear automatically if it falls to the ground.
When looking at the API introduction in the official document, it’s very dizzy, and there’s no example of API, so it’s a direct reference to how the Demo101 learned how to use the API. After reading the source code and understanding the relevant meanings, we can understand the idea and add it to our own code. Then we found a better way to do it directly using the SDK given on Github, and the steps were simplified, but because the code version did not match, many libraries could not be compiled and passed, only to be deleted.
Debug on simulators and real machines
The Unity project is exported, using Visua Studio (set to D3D and UAP), using USB cable to connect HoloLens. (you can also use the WiFi connection, when the host and HoloLens are needed to connect to the same WiFi, and select the Remote in the debug device in the compilation options, according to the prompt).
Adjust the target device: right click the Package.appxmanifest file in the solution manager, select the view code, find the TargetDeviceFamily label, change the Name= “Windows.Universal” to the Name= “Windows.Holographic”, and change the MaxVersionTested= “10.0.10240.0” to MaxVersionTested=. “10.0.10586.0”.
Adjust compilation options: change the target to x86 (now support x86 only), and switch the generated configuration from Debug to Release, and debug device select Device.
Click start, compile and remote deploy to open in HoloLens.
Specific steps can be used as reference: https://developer.microsoft.com/zh-cn/windows/holographic/holograms_100
In addition to debugging code, you can also use Device Portal to view the running status of HoloLens and view real-time images in HoloLens.
First, open the Device Portal option in the For developers option of Update in HoloLens’s Settings.
Use USB or WiFi connection, if it is a USB connection, enter the http://127.0.0.1:10080 in the browser, if it is a WiFil connection, input the https://< of the HoloLens; HoloLens’s IP address > (can be viewed in a reset network setting). If you need to install a safety certificate, you need to install it.
Then enter the PIN code displayed in HoloLens according to the prompt, then set the login password of Portal, then you can see the settings for managing HoloLens.
In fact, Portal is also written by the Restful interface provided by HoloLens, so you can use the Restful interface provided by HoloLens to extract data automatically.
The method of setting up Portal can refer to: https://developer.microsoft.com/en-us/windows/holographic/using_the_windows_device_portal
For Restful API, you can refer to: https://developer.microsoft.com/en-us/windows/holographic/device_portal_api_reference
The final effect of development
Well, not to mention it, many of the functions are BUG.
Well, if you find this blog stop, you must have been killed by your teammate.
Finally Tip: because of the limited computing power of HoloLens, so do not put too complex model, otherwise, the consequences of self – negative (well, in fact, there is no consequence, it is the black screen just).
By the way, I would like to publicize the answer above: what kind of experience is it to attend Microsoft student summer camp? Reply to the author: Zheng Zi http://zhihu.com/question/31743851/answer/59362202
Here is also a HoloLens real machine that you recorded, using video: http://v.youku.com/v_show/id_XMTcwMjU0NDEwOA==.html? Beta& #paction
Reference materials with dry goods
HoloLens official website: https://www.microsoft.com/microsoft-hololens/en-us (a lot of practical things!)
Microsoft student club official website WeChat Development Guide: first bullet, second shells, third bombs (dry goods very much)
MSP’s Chang Wei brother’s blog: http://www.cnblogs.com/mantgh/
Microsoft’s official example: https://developer.microsoft.com/zh-cn/windows/holographic/academy (be sure to look carefully, follow the one or two examples to get it done basically)
Microsoft official document: https://developer.microsoft.com/zh-cn/windows/holographic/documentation (document is not very good, many places are baffling).
Development of SDK:https://github.com/Microsoft/HoloToolkit-Unity and https://github.com/Microsoft/HoloToolkit on HoloLens on Unity
HoloLens document translation: https://github.com/HoloDocs/HoloLensDocs (this is written by Chang Wei’s brother of MSP, high in quality, many on GitHub, and other for reference)