|
|
<h1>Documentation for WMW</h1>
|
|
 |
|
|
|
|
\ No newline at end of file |
|
|
<h2>What is this user story about</h2>
|
|
|
|
|
In our user story, we looked at the integration of the Airsim simulation with the Hololens.
|
|
|
|
|
|
|
|
|
|
<h2>What stage has the project reached?</h2>
|
|
|
|
|
We have come so far that we can display the simulation, set room anchors and interact with objects. We also run the simulation on a computer and stream it to the Hololens.
|
|
|
|
|
|
|
|
|
|
<h2>Description of the work in the semester</h2>
|
|
|
|
|
The work during the semester was mainly focused on the Unreal engine and the creation of blueprints.
|
|
|
|
|
|
|
|
|
|
<h2>Short introduction to the project</h2>
|
|
|
|
|
Within the project you will mainly deal with mixed reality. This work includes how you can display and manipulate objects that are rendered in the simulation on the computer in space. However, it is important to note that you will be working almost exclusively on the Unreal engine and will therefore mainly be dealing with “game development” and its logic.
|
|
|
|
|
|
|
|
|
|
<h2>General documentation</h2>
|
|
|
|
|
General setup:
|
|
|
|
|
|
|
|
|
|
- For a general introduction to the capabilities of the Hololens, we recommend the following link:
|
|
|
|
|
https://learn.microsoft.com/en-us/windows/mixed-reality/develop/unreal/unreal-hand-tracking?tabs=426
|
|
|
|
|
|
|
|
|
|
- Unreal engine or Airsim is started on the laptop
|
|
|
|
|
|
|
|
|
|
- The simulation is streamed to the Hololens using the Microsoft Open XR plugin
|
|
|
|
|
|
|
|
|
|
- The app: holographic remoting is started on the Hololens
|
|
|
|
|
|
|
|
|
|
- If you now press Play in the unreal editor, content should be transferred to the Hololens
|
|
|
|
|
|
|
|
|
|
- To create a new project, we recommend the following quick start:
|
|
|
|
|
https://learn.microsoft.com/en-us/windows/mixed-reality/develop/unreal/unreal-quickstart
|
|
|
|
|
|
|
|
|
|
Tutorial for holographic remoting:
|
|
|
|
|
https://dev.epicgames.com/documentation/en-us/unreal-engine/hololens-streaming-quick-start?application_version=4.27
|
|
|
|
|
|
|
|
|
|
Instead of installing the Windows Mixed Reality plugin as in the tutorial, we recommend using Microsoft OpenXR. You can find it under the following link:
|
|
|
|
|
https://github.com/microsoft/Microsoft-OpenXR-Unreal/tree/main/MsftOpenXRGame
|
|
|
|
|
|
|
|
|
|
If the objects are to be scaled down in the Unreal engine, for example to fit into the existing space, this can lead to incorrectly displayed collision meshes. In our project, this was the case when the car was scaled down. This can be fixed by scaling the linked meshes separately. After scaling, the car floated above the ground because the height and width of the tires had to be changed.
|
|
|
|
|
|
|
|
|
|
Local room anchors can be set in our project using “Right select” and are automatically reloaded after a restart. All set room anchors can be removed again with “Left select”.
|
|
|
|
|
|
|
|
|
|
Spatial perception is also activated in our project. This means that the Hololens permanently scans the environment and builds an Unreal engine mesh from it. How this is used can be set in the ARSessionConfig. A bug or future feature that results from this is that this mesh ensures that it is not possible to interact with objects that are “outside” the room.
|
|
|
|
|
|
|
|
|
|
The spawned car can be manipulated with handinteractions in our project.
|
|
|
|
|
|
|
|
|
|
The logic for the room anchors is located within the blueprint: MRPawn.
|
|
|
|
|
|
|
|
|
|
Tutorial for room anchors: https://learn.microsoft.com/en-us/windows/mixed-reality/develop/unreal/unreal-spatial-anchors?tabs=426
|
|
|
|
|
|
|
|
|
|
The logic for the hand visualization is located in the blueprint: MRGameMode.
|
|
|
|
|
|
|
|
|
|
Tutorial for hand visualization: https://learn.microsoft.com/en-us/windows/mixed-reality/develop/unreal/unreal-hand-tracking?tabs=426
|
|
|
|
|
|
|
|
|
|
<h2>Open tasks</h2>
|
|
|
|
|
|
|
|
|
|
- So far, only placeholders have been set for the anchors; in the future, it could be checked how the map and the car can be connected to these anchors.
|
|
|
|
|
|
|
|
|
|
- The mesh generated by Spatial Perception could be created as a collision mesh so that the car does not drive on the map but on the ground in space |
|
|
|
\ No newline at end of file |
|
|
