Defining audience engagement can feel daunting at times. In our work at Harvard Business Review, we’re looking at engagement – and experimenting to understand that engagement better – through several different lenses: What is most valuable to current subscribers or potential subscribers or social audiences? And what formats hold the most growth potential across social, text, audio, video, bots, and so on.

Here are a few things we’ve learned from some of our recent projects and experiments:

1. When doing something brand new, get (at least a few) true believers in the mix.

Like others, we barreled into doing Facebook Live videos in early 2016. We weren’t doing much live video on the editorial side at the time, but we had a few ideas about what would work and what wouldn’t. We were especially eager to try videos that allowed the experts who write for HBR to explain their ideas to the audience directly. These videos, called whiteboard sessions, put a professor or business leader or other expert in front of a camera (or, early on, in front of a phone) to talk about their idea or concept for up to 40 minutes.

We knew this format would be fairly natural to the presenters because they know how to lecture or give presentations. That gave us a constant point of quality and allowed us to focus on the engagement side: questions, comments, visuals, takeaways.

However, while we had confidence in the format, we needed to work on getting the audience to engage. We discovered the importance of enthusiastic presenters. We had a few early on on who really believed in the opportunities that live, social video offered. One of those early whiteboard experts was entrepreneur and author Evan Baehr. He immediately understood the challenges: a live audience, a social platform, and a lot of competition for your attention from other content. But his enthusiasm got the audience’s attention right away and gave us a glimpse of what was possible with this format.

That confidence in the whiteboard format has been part of what has led us produce and experiment with more than 100 live whiteboards with experts from all over the world. We’re now working on creating a larger video strategy for them.

But without the enthusiasm of people like Evan and others early in the process, we might have assumed whiteboards were a nice-to-have and not invested our time as purposefully. So, keep in mind that, when you’re trying something new, you need to make sure you have some enthusiasts in the mix. The audience can feel it when you’re not enthusiastic about a medium (especially when it comes to video and audio), which will be reflected in popularity and engagement.

2. Text interactions can be tricky.

While we often seen hundreds of comments and shares on live videos, our results have been somewhat mixed when it’s come to text-based chats.

We write about a lot of work advice that’s very personal and a lot of business trends that are important to understand. We know that connecting our audience – and specifically our subscribers – to the experts who write for us helps people go even deeper in applying and sharing these ideas.

We’ve tried text-based engagement experiments around this premise in a couple of ways. In one open-to-all experiment around an important feature piece (an article series we call the Big Idea) on inequality between firms, we invited readers to a Slack channel where the author of the main piece was going to be available for questions and conversation about his research and work. We thought this had the right combination of platform plus deep, important topic and timely relevance.

The results were mixed. While we had more than 150 readers join the Slack channel, they mostly lurked. It’s tough to diagnose why. Was the text format intimidating? Too much work? Do people really want to listen to an expert first and foremost?

We’ve also tried this type of experiment in subscriber-only settings. We’ve invited authors of HBR pieces to answer questions in a closed Facebook group for subscribers, for example. The results have often been similar. Most people lurk and don’t contribute much. We’ll keep experimenting with this type of engagement. But we may need to pair text chat with something else – like video or audio – to really stir interaction. And we need to keep considering the right platform, as well as how much people want to interact with each other instead of an editor or author from HBR.

3. Constantly test your assumptions.

A central tenet of HBR’s experimentation and innovation practice is to write down our assumptions, goals, and hypotheses. This document is shared between product, editorial, and other teams involved in the project. We then check back on those assumptions at planned intervals to understand what we should do differently, what we’ve learned, and to help gauge whether the experiment should continue.

We’ve done two bot projects, one on Slack and one on Facebook Messenger, that help illustrate how this works. When we started the HBR bot on Slack, we assumed that people would love to see our big beautiful images in their conversations with our bot. However, our UX research proved that was wrong. People just wanted to get to work advice we were offering.

We also hypothesized that we wouldn’t drive any trackable subscriptions through the bot. That was right. But we also believed in driving frequent interactions through the bot should lead them to engage more deeply with us, even if we can’t draw a straight line back to their bot experience. That engagement could take a lot of shapes: Signing up for a newsletter, following a social channel, sharing an article with a friend.

It helps to write down even your most basic assumptions. For our recent Women at Work pop-up podcast, we documented the assumption that people wanted to engage with HBR about this topic in an audio experience. For the bots, we had to test our assumptions that people wanted to consume our content in bot form (and we’re still testing whether that’s true). It seems basic, but doing these things helps your build up a store of observations that can inform your future experiment plans and pitches.

4. Be clear on asks to your audience.

Don’t take this one for granted. When you ask your audience to do something, have a purpose and follow through. HBR has had a subscriber-only Facebook group for about a year now. The editor who runs it, my colleague Laura Amico, is clear that when editors post to that group, there’s a purpose and a plan. We’re not just throwing out a “What do you think?” to that community and walking away. Instead, we’re asking that community to join us in testing advice or sharing ideas on how they’ve solved tough work problems.

We’re also opening up new channels to listen to our audience and incorporate their feedback. Our new Dear HBR: podcast answers questions from the audience about tough workplace dilemmas. It’s our newest foray into the advice space, and we’re already seeing steady growth in the number of questions being posed.

And we’re taking a fresh look at the role of article comments in this space. Laura is working on this too, and I thought I’d share part of an internal report she’s developing to help push our thinking in this area:

Comments are about so much more than what is typed into a text box. Commenters need to know how to succeed in what we’re asking them to do and they need to be rewarded for doing it. That’s true on the other side, too; editors and others working with audience development also need to know how they and the business benefit from engagement. Developing use cases for comments, promoting good comments, and sharing engagement results internally and externally are all key to maintaining the relationships.

As we move into new areas of experimentation around engagement – involving everything from email to comments to visual storytelling to machine learning – we’ll keep testing our assumptions. We’ll keep pushing ourselves to think dynamically about our audience and how we ask for their time. And hopefully we’ll make their day at work a little better, too.

---

Maureen Hoch is Editor of HBR.org. Hoch leads Harvard Business Review’s digital newsroom and oversees the team working on HBR’s Facebook Live videos. She played an instrumental role in the redesign of HBR.org, which has since garnered three Webbys, and she helped lead the team that created HBR’s bots on Slack and Facebook Messenger. Hoch joined HBR as Senior Editor in 2013. She has previously worked for the PBS NewsHour, the World Bank, and The Washington Post Express.