Do we even really need augmented reality?
( THE ANSWER IS MAYBE DINGUS)
Disclaimer: As a designer I try to analyze flaws and opportunities in our world. While I'm not an expert in augmented reality or even my own reality, let's still discuss it from a design standpoint.
Augmented reality is pretty much a staple for so many social apps that we don't even realize it that often. It kind of just gets buried into the word "filter" on social photo apps and other clever gimmicks Though not as highly touted as VR because you still have to interact with the real world, AR as a concept might be considered...boring?...to the average consumer. Or maybe just considered novel because once you look past your phone, you can see that there's obviously nothing there.
Which more or less is kind of sad because the potential applications of augmented reality are actually insanely usefully. You can literally augment (here's some synonyms: amplify, boost, enhance, intensify) the world around you. Can't augment your virtual reality, can ya, punk?! Wait, I guess you can but that's just better graphics and coding, which is still unrelated to your real world (unless...simulation theory?).
Either way, augmented reality and virtual reality, while using similar capture technologies, generally are completely different applications for the most part. VR puts you into a completely different world, while AR (yes, I'm going to abbreviate from now on) enhances the world you actually live your everyday life in.
Since AR isn't as immersive as VR, doesn't that mean it's less interesting?
Well first of all, you aren't supposed to be immersed with AR, you are simply enhancing the world around you. Second of all, interest is relative to each person. So as of right now, I'd be inclined to say yes, that's somewhat correct. VR goggles completely cover your eyes and give you a realistic depth of field that transport you to a completely different world where anything is possible (visually). So naturally it's a more suitable for imaginary creations because it has the luxury of not having to mesh with the real world.
That is unless you want to buy the Microsoft HoloLens for $3,500 bucks. And even then, I don't think Halo is on there yet. That is until technology catches up. AR isn't even meant to immersive and if it is, it will probably not happen in my lifetime. But I'm guessing that technology will get to the point where AR/VR will be indistinguishable from each other - its simply because our sensory measurement technology isn't there yet. VR has to only sense your direction and motion in space and then provide you with whatever it feels like throwing at your eyeballs.
So if you want realistic immersion in augmented reality, then something technically needs to capture every freakin' single aspect that surrounds you and apply an overlay that matches exactly that. Light, geometry, distance, optical, and a potentially never-ending list of sensors would need to be optimized to achieve that same kind of VR "immersion". Ridiculously far-fetched and difficult? Yes. But not impossible (I think).
What's currently available in terms of augmented reality gaming?
Sometimes people forget that augmented reality gaming doesn't just have to be a cute imaginary creature that you can tap on when it shows up on a screen. Technically projecting something onto an object or listening to world-altering audio is also augmenting reality, not to mention the hardware format matters as well.
But let's stick to the one we're all probably the most familiar with - smartphone AR gaming. It's fun, it's accessible, and it's not extremely technologically demanding. In most cases, any higher-end smart phone that you've bought in the last 5 years is AR-capable. You may not have played any AR games but you've probably used an Instagram filter and that's fun in it's own way. So let's take a look at what differentiates mobile AR games from conventional screen-based video games:
• You must interact with your real physical world around you in some way
Dang - that's a shorter list than I thought. But that's the truth, really having only having to interact with the environment can technically constitute as an AR game.
Is a real world + overlaid object on a screen enough to make AR games fun?
Frankly, probably not, solely based on the answers I've gotten from people who have played a decent amount of AR video games. It's probably fun at first to see mythical creatures or recognizable characters show up in your living room or dance by you while you take a poo. Yet eventually that loses its appeal and is just a waste of batteries and eventually you turn the AR mode off. However, if the game still stays fresh and interesting because it's probably a well-designed game that holds it's own as a game.
An artistically-designed chess board overlaid on top of a surface, is still just chess at its core. At the end of the day, AR games suffer the same dilemmas and obstructions that other video games have to deal with - making appealing gameplay, visuals, entertainment, challenge, etc. The only extra factor is that AR developers have to deal with is the extra layer of complexity which is the world (which is layers upon layers) around us all. As with all games, developers should strive to make sure that a feature is intrinsically enjoyable first rather than using it as a gimmick.
What else can elevate AR games to be better?
I think the major improvement that can open up a world of possibilities is technology getting better and more suited for AR. Right now, we are technically bound by the apparatus it's being developed for. There certainly are devices that are made purposely for augmented reality but that limits it's real-world adoption and gaming would seem like it's one of the last applications that enthusiasts would be developing for. Computers don't really have AR hardware (regularly) in them. Therefore, for gaming and the average consumer, they are relegated to a smartphone, which has the basic sensors needed for AR to function.
This does limit what is possible for AR gaming on a smartphone but as average consumer technology gets updated with tech such as LiDAR - better sensors, processing speed, image recognition, and who knows what else will be huge for consumer devices. This paves the way to new avenues to approach gaming with these advancements. Imagine if phones were able to capture the landscape of your surrounding areas with extreme accuracy without you have to move your phone around carefully and recognize surrounding objects around you. This alone could make interesting features such as applying attributes/triggers/systems for each player that are universal despite being in different locations because the recognition capabilities are that advanced. This definitely seems like a long ways away though.
So as designers what can we do to make AR gaming more interesting NOW?
Well, let's take a look at what makes conventional games so appealing. Actually we already listed them above but how do we make those more appealing with AR technology on mobile phones? Currently (based on Wikipedia) AR is fueled by "digital cameras and/or other optical sensors, accelerometers, GPS, gyroscopes, solid state compasses, Radio-frequency identification". These are all super useful technologies, but the one thing that none of those mechanics can enhance is to have an engaging story-driven game.
This has yet to be achieved at the level of the great storylines such as Final Fantasy series, The Last of Us (Part 2 included - come at me haters), or Witcher 3 to randomly name some. However, why is this not replicable in AR? It's not that it not achievable, it just hasn't happened yet. In fact I think that a story-driven AR game can be highly enjoyable, both as an individual player or as a community. Both cases you can set the pacing of storytelling while having mini-games or a core gameplay loop be secondary choice.
"Augmented reality is the 'boy who cried wolf' of the post-Internet world - it's long been promised but has rarely been delivered in a satisfying way."
Om Malik, Tech Writer, Founder of GigaOm, and Partner at True Ventures via The New Yorker
Here are some random story-driven AR game concepts to ponder:
•Tying the storyline to be very correlated to areas such as blank walls/mirrors, this would give a lot more creative freedom in making sure the user experience is uniform overall
•An evolving storyline that has GPS/touch screen/camera requirements by a certain amount of users/actions in a certain area to advance (A character is stuck and has to be visited and fed by 'x' amount of individuals before it can be freed and have the storyline progress)
•Create random events that alert users through the use notifications and if they open the app within a certain time period, they receive some sort of reward (personal visit from a character, additional lore, points, whatever really)
•Community-driven storylines (it could be that a number of users have to be logged on at a certain time and perform a certain action, vote, solve a puzzle to drive the story in a certain way) and have it so that users would be able to swap information
•Use the accelerometer to function as a tool and you have to shake your phone or move quickly to emulate spinning action (digging someone out of a hole, shaking a character that is "stuck" in your phone, anything really)
•Have very accurate contextual voice lines that change based on what is needed to be seen on camera (for example someone being dizzy or confused sounding until your camera points at a desired object/space
Quick AR screens made with wiarframe.com and assets from free3d.com
Hold on a second my fellow readers, I hear some distant naysayers...
"But Zack! There's no way you can make an cohesive story with AR because it'll look different for everyone!"
Well, possibly, who knows? It depends so many things, ESPECIALLY the story. I think the challenge in creating a story using augmented reality, isn't that it needs to look the same for everyone but needs to FEEL the same for everyone. Either everyone gets the same story or everyone gets a different story based on the actions you take. You could have a story about germs that talk to you and since they're so small, you'd have to get your phone really close to the floor or a surface and it would practically look the same for everyone. Story should be first. Logistics can come second.
Wow, that's almost like every single movie, book, tv show and every single other kind of media. Of course, some individuals might be unhappy but there will also be an inverse of people who will be quite happy because they got what they wanted. Can we also not forget that it would be a story and stories can consistently change and some people might end up liking it? I think that if there was a community storyline idea, it would drive user engagement and investment in the story. People that would want to see their outcome would make the extra effort (of whatever the developer required) to ensure that the story would flow in their favor.
What other innovations can we do for AR gaming?
There are a few ideas that come to mind but one concept that I feel might be overlooked is level design. Since everything is technically based in the real world, the level design aspect of gaming kind of takes a back seat in general for AR. How can we make it be one of the highlights of an AR game? Well the technology really isn't in a place where we can do something drastically different but we can rely on what does constantly change and that is what is in physical in the world.
I'm not sure what the logistics/requirements/laws are for being able to receive user-recorded data but if users were able to use their cameras to record video or take pictures of areas for developers in combination with GPS there could be a huge change in what is possible for augmented reality, especially in level design.
Here are some random ideas based being able manipulate physical objects or the world around you or level design:
•Have the game recognize features of previously uploaded user data and then create content from said data to share with other users
•Create more realistic interactive environments, scenes, and gameplay options from uploaded user data
•Allow players to create checkpoints or pins relating to gameplay elements that other users to access or find (however, there is a very dangerous aspect of this feature as it could be used predatorily and should have safeguards in place to prevent any harmful actions)
•Use a commonplace object as an anchor to be the decisive point of your level design (this might allow you to be able to make your scale smaller while this object always being in sight)
•Being able to place objects/glyphs/whatever into another user's gameplay to aid or sabotage them
I think AR gaming as a whole still has a long way to go if it wants to reach new heights in what is considered a significant breakthrough that isn't novelty or what feels like an add-on. The experience should be something shouldn't feel like it was forced - it should just work. I think once it is grounded deeply within our reality while still providing an escape from what we know, there will be a breakthrough in augmented reality experience. Something that is deeply relatable to users. Something that is obviously fictional or unrealistic but realistic in the sense that it's plausible to a certain extent because we're invested.
Despite the hurdles that an ideal augmented reality game would need to overcome, I am still excited about what it might be. I say will be because I don't think this is some sort of impossible goal or monumental achievement - it's completely in the realm of possibilities and when it does finally exist - people will know and it will set the standard for AR gaming and lead to future improvements and innovations. I'd say I'd even love to be part of the discovery of process of a new form of AR gaming because it really is a puzzle that has to draw from what works in existing video games and while having some designing ingenuity for something that doesn't technically exist. If anyone is interested in collaborating or just brainstorming some fun ideas, feel free to reach out to me!
Either way, thank you for reading my blog post and would love to hear your comments, concerns, or thoughts via my Twitter account.
Happy and stressful designing!
Zack
