Whodunnit? - Microsoft Hololens (Augmented Reality)

Production and Game Design

Building Virtual Worlds, Round 2

Development Duration: 2 Weeks

Developed in Unity for Microsoft Hololens

Programmers: Jett Lee, Sally Xuejun Wang

Artists: Ketul Majmudar, Euna Park

Sound: Ricardo Tucker

Co-Producers: Ketul Majmudar, Euna Park

Communication tools: WhatsApp, Google Sheets
Daily update meetings
- Longer meetings scheduled separately on an as-needed basis

When we got the Hololens, there was some debate over how we could best put augmented reality to the test. The general consensus among the team was that we wanted to do something difficult and ambitious for the project.

RESEARCH AND BRAINSTORMING

We played around with the idea of building out a puzzle world, but the biggest challenge was taking into account that the scheduled guests were not going to be familiar with the Hololens platform at all. Instead of deciding on a concept right away, we analyzed the list of Hololens features we wanted to use in our experience and push what could be done with the platform.

We had another meeting in which we discussed what each team member was interested in, and what they were looking for from this particular round. This was doable since we were a team of five, and we wanted everyone to be invested in the idea. After evaluating each team member's interests and throwing some more ideas around based off of that discussion, we decided to commit to a murder mystery. We would build a world in which the guest would be invited to play the part of a detective and save the day.

GAMEPLAY AND WORLD BUILDING

We quickly realized that this was going to be extremely challenging from a world-building and story perspective. While the programmers got to work understanding the nuances of working with the Hololens and the limitations of how we could present this world, the rest of the team got to work mapping out the story.

The goal was to go for a humorous vibe, a Pink Panther-style experience. We struggled to find a story that would walk that line, and we ended up with a placeholder story to start mapping out the different kinds of interactions that our guest would be experiencing. In this version, the guest was charged with the task of finding out who murdered the Building Virtual Worlds Head TA, Cody Fraser.

One of the biggest goals of our experience was to take advantage of the wireless Hololens and use the entire room, not just the stage in the front. In order to craft a more interactive experience for the guest, we decided to plant a few team members into the audience with one assistant accompanying the guest through the experience. The guest would become a supercop Sherlock, and the accompanying team member would play the role of Watson.

In this version, Building Virtual Worlds instructor, Jesse Schell, was the culprit who killed our head TA.

The next challenge came in the form of mapping out the details of the content, now that we had an idea of what the presentation and structure would be. The sense of accomplishment upon discovering the solution is an essential part of a good murder mystery experience, and as such, the clues need to point the guest towards the solution without feeling too obvious. We collectively brainstormed what information our audience plants would have for our guest, and wrote a theatrical script based off of the chart below. As the guest interacted with clues and suspects, we initially planned for a 'mind palace' map which would collect and display the information that the guest had discovered.

Map of interactions and clues for the guest's experience.

During our interim presentation, we struggled with Hololens drift and our presentation went on for almost ten minutes. Much of the feedback we received was not unexpected.

While the idea of turning the guest into Sherlock Holmes was intriguing, the execution of the world did not deliver because of a confusing plot.
- The "inside joke" nature of the script catered to the audience, but was unappealing.
The world felt inconsistent as the rules of the virtual and real space were mixed (some clues were virtual, others were not).
The guest was given a surprising amount of autonomy, too much. It would result in confusion and frustration.

After the presentation, our team threw out the whole story we had built and struggled to find a new one. We started with the underlying mechanics and interactions we had built, and worked to repackage it into a believable and engaging story. The major things we needed to reconsider were:

The guest's role in the story
The rules of the physical and virtual world
Setting, story, and motive of the crime
The use of the Hololens at all in our experience
- How do we make the Hololens a core part of the experience, rather than a quirky add-on?

After a lot of discussion, we cut the story down and simplified it in order to focus on delivering a more complete experience. A panacea was stolen from a top secret lab at the CDC, and the guest would investigate the scene to find out who had done the deed. Mechanically, these were the largest changes we made to the experience following the story shift:

We reversed the roles of the guest (junior security guard) and the accompanying team member (senior security guard), to provide the guest with more guidance through the world.
We changed the use of the Hololens to be an actual piece of crime-solving technology for the guest, with the UI designed in the manner of an Iron Man-like HUD
- We added a "training scene" to introduce a naive guest to the use of the Hololens in this way.
- The senior officer was not using the technology himself because we wrote him as a Luddite
Rather than using the entire presentation area, we cut down our use to the entire front of the room.

Mapping out the reworked story and presentation of the experience.

We took exact measurements of the area in order to arrange the clues and the world with the physical space.

When we invited playtesters to test our modified version, we modified aspects of the UI and the arrangement of certain clues. There were also modifications made to the overall experience and some story problem points were identified. Overall, we focused on whether guests felt like they felt free, were having fun, and were satisfied with the experience.

FINAL PRESENTATION

In our final presentation with an actual naive guest, many of the changes we implemented proved to be effective and the feedback was also not unexpected.

Pros:

Our naive guest understood his role and what he was expected to do while feeling free to investigate and explore as he liked.
The modified story helped streamline and clarify the experience.
The music, sound effects, and interfaces set up the aesthetic of the world very well.

Cons:

The tutorial was slightly heavy-handed with teaching, and the ending needed to be pushed to fit within the time constraints.
Some of the interface elements drifted or were poorly placed, and caused frustration for the guest and the audience.
The interest curve was flat, with no twists or surprises.

If we had more time after this feedback, we would have created animations or direct visual connections that showed the connections between the clues and their corresponding popups. We would have also worked on fleshing out story details to be more engaging, and have refined the use of the physical staging and prop use.