Hot on the heels of reports that reveal AR and VR will become mainstream sooner than we expect, media companies are lining up to test the waters and push the boundaries. While they recognize that implementation of immersive technologies can bring audiences deeper into a story than text, 2D images, and video ever could, many have yet to grasp that they have to create stories that move people, not just “wow” them.

One company that is further along on learning curve is USA TODAY NETWORK, part of Gannett Co, Inc. and the largest local-to-national media organization in the U.S. It’s cleverly harnessed talent from a variety of disciplines, including video games development, to bring new life and excitement to environments (life on the USS Eisenhower), events (launching a Falcon 9 rocket from Kennedy Space Center), and formats (an investigative podcast revealing the hidden power structures behind our cities really work).

Peggy Anne Salz—mobile analyst and Content Marketing Strategist at MobileGroove—catches up with Ray Soto, Director of Emerging Technology for USA TODAY NETWORK, to discuss recent AR projects and delve into how media companies can leverage growing interest in AR and VR to elevate storytelling and engage audiences.

---

PAS:  Please bring us up to date on the storytelling experiences you have created and what you have learned.

RS: We recognize that both virtual and augmented reality are fairly new within our industry. But that’s what really motivates and inspires us to take that extra step to experiment and try something new. An example is the 321 LAUNCH app, our first AR app. It gives users a close-up live view of real launches and landings, allowing them to build and launch their own Falcon 9 rocket within the app—an experience enhanced with access to live video and real-time updates.

The aim was to combine immersive storytelling with a live launch AR broadcast, paving the way for users to engage with content in new ways. Metrics show strong retention rates and long session times, even on non-launch days. Overall, we have seen engagement times of around 3.5 minutes, and we observed that user engagement for our live launch broadcast spiked up to around 8 minutes.  Right now, we hovering at around 3.5 minutes on non-launch days and about 5 minutes on launch days.

We learned a lot from the app about how to reach and engage audiences. We also learned a lot about the development cycle and the UX. The experience of creating the app—and seeing how users wanted to be in control of their experience—helped us to focus on building the foundation that would support all our augmented reality experiences moving forward. We saw that there was deep engagement with AR through interactivity, and what we didn’t want to do was leverage existing APIs that would limit us in the types of stories we could tell.

PAS: So, it sounds like you have “augmented” augmented reality platforms to suit your requirements and ensure you can support how audiences have shown they want to interact with your app…

RS: You’re right. So, we ended up leveraging the GUI game engine, and that is essentially the foundation of our AR platform. Unity provided us that foundation, where we can leverage AR kit and AR core. But we also built on top of that to incorporate the functionality we envisioned we would need for our storytelling experiences. So what we have created is a series of editorial templates, that include design templates and animations, allowing us to quickly support interactive storytelling through AR.

A good example is the augmented reality Hurricane Florence tracker. At first, it was a challenge that I presented to my team, which is made up of former video game developers with experience working with companies including Electronic Arts and NC Soft, to turn this AR experience around quickly. And we did this. By drawing from our game design experience and working closely with our product development, mobile dev team, and the editorial team, we leveraged our template to create the tracker—which we were able to ramp up, turn around and publish within 24 hours.

PAS:  In the case of the hurricane tracker, you harnessed AR to bring breaking news to life. But to go mainstream AR will need to enhance content and experiences that are a little more downtempo. Do you have an example where AR adds a layer of storytelling, not just effect?

RS: A great example is our AR experience for The City, an investigative storytelling podcast where we developed the augmented reality trailer. The podcast is already a powerful story paired with an immersive audio experience—but it was missing an equally powerful visual. We recognized that, through our AR platform, we could create an experience that would allow you to explore the community that is the focus of the podcast and “tease” users to engage with the powerful story that was being told within the podcast.

In this scenario, AR gives the audience a sense of place. Clearly, it was a different type of story to tell and a different goal from launching a Falcon 9 rocket. However, we found we could just leverage what we’d learned from earlier projects and that was a plus. It was exciting to think no one had ever done this before—and we did. And through this, we recognized there is a real opportunity to fill in some of the gaps in content, such as audio, with AR and adapt what we create based on the story we’re trying to tell.

PAS: It’s exciting to push the envelope, but you also need to know where to draw the line. How do you decide if AR is a match with the storytelling in the first place?

It can be a tough one to call. So, we start by drawing from our experience and our understanding of the unique opportunities that emerging tech can provide and support when it comes to creating a very strong interactive story. The story comes first. Before we commit resources or time to develop an augmented reality experience, we work very closely with the editorial team to define what the story should be and how augmented reality can drive value to the story we’re trying to tell.  It’s very much a collaborative process. There have been times when we’ve gone down a path of exploration and determined that the particular story might actually be best suited to being told with the support of an interactive graphic on the web or a video experience.

We’re at the point now where we’re able to produce an offline prototype quickly to really understand how we can tell the story in a new way that the audience can engage with again and again. The goal for an AR project can never be to create a one-time gimmick that folks will see and walk away from saying, “oh hey, that was neat.” The goal—and this is our goal—has to be to give audiences the feeling they have gained something from the experience or learned something. You don’t want the technology to overshadow the story because it comes down to the story first. We understand this because our team has had years of experience, not just with game design but in VR and more recently AR.

PAS: Your team is made up of talent from video games companies, which you’ve pointed out is a plus. What talent mix should publishers assemble and orchestrate if they are serious about creating AR experiences?

RS: We’re fortunate to have a team that is made up of former game developers, but I don’t think it’s a team that has to made up of game developers exclusively. Yes, it definitely makes it much easier to create AR experiences and ramp up a team to support AR and VR. But it’s not the only way. Publishers also need to bring together people who have a passion for the work and a desire to explore what’s possible.  It’s through passion that companies can build a team able to go outside of their comfort zone. Passion guides you and helps you find ways to tell a story that hasn’t been done before. So, game development is helpful, but it’s not required for interactive storytelling.

PAS:  So, what is required? What is the checklist publishers should have top of mind?

RS: Start by thinking through how you can create a narrative arc. As I said before, it’s not about gimmick. It’s about trying to do things differently and avoid getting too comfortable with how you have told stories in the past. Even if you have a template to build a story, you don’t want it to be the same every time. The aim is to encourage people to engage and come back. We want audiences to recognize that USA Today is telling interactive stories in a new way with AR, and that means pushing the boundaries through seeing the opportunities and exploring approaches.

It’s also important to identify the gaps within the story. This is where AR platforms can enhance that story. Again, a great example is the City podcast where we could improve an immersive audio experience with AR. It still just boggles my mind that we even considered it in the first place and I’m proud of the results. We recognized that it was a strong story that we could do differently.  That’s the mindset you have to have for every project: try to do something that’s different but do it by creating a narrative arc that engages your audience and helps them coming back.

Everyone has a unique opportunity to define their path, but don’t only focus on the visual experience. You have to think about scalability and accessibility.  You don’t want to exclude users who are impaired in any way –and this is why we have incorporated subtitles within our USA Today app. We want everyone to be engaged with the experience and able to make it what they want.

PAS: Brands are also focused on engaging users with brand storytelling and experiential content. Is there a place for brands in your future plans?

RS: We’re looking for opportunities to work on branded content. That conversation started because we observed such high engagement and retention rates. These metrics tell us there’s an opportunity—through AR—to create a personal connection. We’re poised to tell our stories in new ways, and we see ways brands can build on the growing excitement around augmented reality to engage with their existing audiences and acquire new ones.