AR_Hack: Inspiration for an AR-Based Gameplay

For the past few months, I’ve been working on a networked game between Augmented Reality (AR) and PC (titled AR_Hack), and I want to spend a few minutes to talk about the inspiration behind the project.

AR_Hack is a stealth-based, asymmetric game where one player acts as a thief trying to steal a relic, while another player sets up traps and tries to track down the thief. The game utilizes ARKit to scan the room around the player and builds a shared world where both players can interact with one another. The ‘PC Player’ acts as this thief, while the ‘AR Player’ sets up traps around the room and physically moves around to search for the other player. A demo of the project can be viewed here.

During the summer, I worked on a networked AR-VR experience which got me started to thinking about ways to gamify an AR experience. With the current technology, AR is very good at building a world on top of what it sees, but it’s not very good at providing ways to interact with them. With the HoloLens, for example, you can easily map where the floor and walls are within a room, but interaction with those objects are limited to gestures and voice commands. Since gameplay is built on interactions, that limited the form of play that could be introduced to AR.

To talk about why that was so frustrating to me, I want to talk a little bit about game literacy. The world of games had come to rely on established rules to help players grasp the control of a game. In any console-based FPS game, for example, one can probably assume that the left control stick is for movement and the right control stick is for camera controls. However, nothing about this control scheme can be deduced from the set up of the controller or the mechanics of the game. Sure it feels right as a gamer, but we have to first learn it before we can enjoy the game.

Virtual Reality (VR) has done a great job at bridging this gap in game literacy by building interactions from actions that people do everyday. If you see something on the floor you want to grab in a VR game, you can just reach down and grab it (much like how you would do it in real life). A VR game can then build a fantasy on top of all of those interactions, by introducing ‘super powers’ — such as the ability to point at objects and move them with telekinesis.

Instead of trying to build a different ‘controller’ for an AR game, I tried to focus on one of the strongest interactions with AR — movement. If a player wants to get closer to an object on a table or turn the camera towards a different object, they can simply move or turn the AR device towards that object. One of the first ideas that came to mind was a Hide-and-Seek game of sorts — one where the player can use an iPad to search around the room. The fantasy of holding an iPad as a monitoring device to search for this ‘virtual thief’ then slowly took the game towards a more sci-fi theme. The gameplay then slowly developed to involve a heist aspect and a “tower-defense-esque” aspect.

As for the multiplayer portion of the game, this developed from the common complaint that VR is an experience that is too isolating. I really like social games and I wanted this AR experience to have a strong social aspect where the play would be enhanced by these social interactions. I also originally wanted the project to be a networked game between AR and VR, but working on the project alone restricted me from touching any form of VR (PSA: networked games are not easy to make).

Watch out for the next blog post where I’ll try to talk about more post-mortem-type stuff involving play in AR.