Login
close
Login is restricted to DCN Publisher Members. If you are a DCN Member and don't have an account, register here.
Search

Digital Content Next logo

Menu

Tags containing: "comments"

Perspectives | Supporter Spotlight

Publishers: Read the comments to find misinformation you need to debunk

Uncertainty has been prevalent over the past couple of years thanks largely to a deadly and contagious virus. That’s only been exacerbated by right-wing extremism triggering dangerous protests in two North American capital cities and an armed conflict in Ukraine.

In times of crisis, uncertainty propels questions that come more quickly than answers, which creates the perfect breeding ground for misinformation.

“In these last couple of years, there’s been too much information, and not enough time to vet it,” says our director of moderation, Leigh Adams whose team manages comments sections for media brands in North America, Europe, and Latin America. And, increasingly, “audiences turn to online news instead of watching TV because they don’t have the time to wait.”

Comments reveal valuable insights

A media organization’s comments section is the perfect place to identify issues that need to be fleshed out and contextualized, because that’s where your eager audience will post its questions first. The added bonus is the comments also reflect what readers actually want to know about.

Adams’ exposure to the media sites of multiple major media organizations provides insights into what audiences are thinking, relative to different events. Her experience can also help publishers understand how to mine the comments for information that will help them better serve their audiences. So, let’s dig in.

Russian invasion of Ukraine

During the ongoing armed conflict in Ukraine, Adams’ team found that many people want to know what’s real and what’s not. This tracks, because many of them are aware that Russia ran a tireless disinformation campaign to interfere with the U.S.’s 2016 presidential election. In light of allegedly fake videos and photos that have circulated since the beginning of the conflict, commenters also want confirm whether what they’ve seen on social media is true.

“If the news responds specifically to misinformation, it’ll be productive and perform well,” Adams says. “Stories that fact-check will help users filter out the noise.”

Trucker convoy in Ottawa

The beginning of 2022 in Canada was marked by the organized protest against vaccine mandates on both sides of the Canada-U.S. border, which protesters contested would impact the livelihood of many truckers. Despite the fact that the majority of Canadian truck drivers are vaccinated, a “convoy” of truckers and supporters of unvaccinated truckers marched or drove to Ottawa, the capital of Canada, where they staged a disruptive blockade for a few weeks. The movement inspired other similar protests elsewhere in Canada and the U.S. It was even compared to the insurrection of Jan. 6, 2021.

In the early days of this protest, Adams says that commenters wondered how protesters could take that much time off — many protesters were not from the Ottawa region and spent several full days participating in events — and whether or not they had jobs. This line of questioning led journalists to dig into the protest’s financing, and it eventually emerged that the convoy was largely funded by American donors.

This means commenters’ questions at least partially influenced important reporting about the protest and who was behind it.

From the very beginning of this specific crisis, the situation has been in constant flux. Learning about the virus, reacting to it, adapting to it, treating it, vaccines, resurgences, plateaus: every ebb and flow of Covid-19 has been, and is still, happening in real-time.

Two years later, Adams finds that readers still tend to use the comments section to fact-check the latest developments in the pandemic. What is the science behind the lock-downs? What mitigation or preventive measures are they employing elsewhere in the world? What is Ivermectin? How do we measure what’s working and what isn’t? What does “living with the virus” really mean?

The pandemic

The other thing Adams observed is that commenters consistently turned to local media as a source of truth. Perhaps because pandemic strategies are decentralized and differ from one country to the next, when audiences turn to local media for pandemic information, they know they’ll be able to wade through the clamor and find content that’s both relevant and applicable to their region.

Cut through the confusion

“Comments are the most valuable data point if you want your audience to come back,” Adams says. “They’re like eavesdropping on water cooler conversations.”

By listening carefully to what your audience is saying, you can generate stories they’ll care about and want to return for. But during crises, Adams has seen that readers especially rely on media to answer their most pressing questions and give them the facts they need to stop worrying about the fiction.

Perspectives | Supporter Spotlight

Engaging audiences: How publishers can shift heated arguments into meaningful discussions

The Internet is renowned for bringing out our worst instincts. When anonymous strangers with differing views converge in online community forums, the result is often heated arguments that can quickly descend into insults, threats, and abuse.

Once upon a time, that tendency to “pile on” seemed like a good thing for publishers. If strong emotions drive engagement, and engagement means more page views and ultimately more revenue, then stirring up angry debate must be good for business, right? 

The risks of allowing online toxicity to flourish

Is it actually effective to sow controversy in an article, then sit back and let commenters fight it out below the fold? Based upon what we’ve seen over the past few years, not very. 

For one thing, social platforms and publishers have had to contend with advertiser boycotts, as brands rush to distance themselves from hate speech and misinformation. Publishers like The Atlantic and Vice, meanwhile, have felt they had no choice but to shutter comments sections completely, cutting themselves off from their readership in the process. 

There’s another, much more appealing option: building a safe and healthy online community of engaged commenters. Our research shows that quality conversations and respectful debate actively attract engaged users–and engaged users typically view 4.6x more pages, spend 3.6x longer on-site, and drive 3.2x more revenue than non-engaged users. 

What drives incivility in online community forums?

So how can publishers turn heated arguments into quality discussions? First, it’s useful to understand what drives online incivility. Then these underlying drivers can be taken into account in audience engagement strategies. Here are three for starters:

  • People behave in a less inhibited way when they interact online with people they don’t know and are never likely to meet, in an environment where they can be anonymous and where there are few or no repercussions for behaving aggressively.
  • People get addicted to being right. Winning an argument can produce a flood of feel-good hormones, similar to winning a bet. As with gambling, that feeling can become all-consuming, leading to a greater focus on “winning” than on debating respectfully.
  • Comments without context get misinterpreted. While it may not be 100% true that 93% of communication is non-verbal, a comment can seem hostile when elements like body language and tone of voice are missing. And that, in turn, can promote an aggressive response.

How to shift heated arguments to meaningful discussions

There are several options open to publishers who want to elevate the quality of debate in the communities they host. Here are four approaches that we believe in at OpenWeb:

Encourage visitors to become registered users

When people feel they belong to a community, and get to know their fellow community members, the online disinhibition effect starts to fade. Interactions between community members become more civil as a result. But that’s not the only benefit. Registered users are also more likely to return more often, spend more time on-site, and deliver more revenue than non-registered users.

Have editorial teams join the conversation

Editorial engagement is a powerful way to increase the quality of online debate. Editorial teams set the conversational tone and guide discussion by responding to the highest-quality contributions. We’ve found that editorial involvement of this kind typically leads to a 17% decrease in toxic comments. 

Define and reward civility

Most people don’t come to an online community forum looking for a fight: often, it’s the atmosphere they find on the site that tips them into incivility. Sophisticated moderation technology can help publishers cultivate a civil atmosphere. For example, you can:

  • Make it clear what kinds of language and behavior are encouraged, and what won’t be tolerated. For example, our OpenWeb Clarity Mode puts community guidelines front and center, ensuring everyone knows what’s expected of them.
  • Invite users to rethink their comment before posting, if it looks like it may breach the community guidelines. This draws on the “nudge” theory of behavioral economics, which holds that positive reinforcement at the right moment can spur more considered decisions. When we experimented with such prompts across some of our top publishers, we saw a 12.5% lift in civil and thoughtful comments being posted.  
  • Use multi-layered AI and ML-based moderation to analyze, sort and highlight comments based on their quality rather than their propensity to incite hostility. That way, community members who post high-quality, thoughtful and expert content will be rewarded by seeing their comments highlighted and featured on the site. 

Analyze and filter incivility

If rewarding quality conversations is the carrot, then filtering out incivility is the stick. For the most efficient moderation results, a hybrid approach using category-leading AI and ML moderation (combined with a healthy dose of intuitive, manual moderation) can reliably filter out a large proportion of toxic language–and analyze users to understand who are most likely to post toxic comments in the future.

Today, publishers no longer have to make a Hobson’s choice between enabling comments (and seeing them degenerate into petty arguments) and turning them off completely. With positive reinforcement, editorial engagement and multi-layered moderation, overheated arguments can become meaningful discussions–bringing healthier online communities, and higher reader loyalty, to publishers.

About the author

Andrew Sullivan is the Chief Product Officer at OpenWeb.

Featured Post | Innovation

Comment sections aren’t dead (yet)

Comments for news organizations’ digital publications took a blow when NPR closed its online comments section in 2016. Even before NPR’s monumental decision, other organizations including The Verge, Reuters, and Recode had also bid adieu to online comments. The feature was increasingly viewed as an incubator for name-calling, insults, and sometimes harassment. And the trend continues. In just the past year, USA Today and OregonLive added their names to the growing list of news outlets that have had enough of supervising online trolls and bots.

Comments definitely don’t always bring out the best in people. Some commenters use the shield of anonymity to share derogatory and abusive messages, communicate hate toward certain racial groups, or turn a constructive discussion into a name-calling match. And negative comments can even influence readers to have a more unfavorable view of the story itself.

Some national and regional newsrooms have employed comment moderators, but monitoring thousands of comments requires considerable labor. The task of deleting uncivil comments oy leads to emotional exhaustion for the moderators. However it turns out that deleting the litany of racist, sexist, and hompophobic comments also decreases trust in the news organization, according to a 2019 Center for Media Engagement study. While comments sections once held the promise of democratizing news discussions, that optimism seems to have backfired.

The death of comments may be exaggerated 

Is this the beginning of a slow dive into obsolescence for comment sections?

Coral helps moderators identify disruptive comments and surface the best submissions.

Not necessarily. Despite the woeful forecast from some commentators about the death of comments sections, experts who study reader engagement warn critics against hasty obituaries. The number of news organizations that have done away with comments sections do not represent the industry as a whole, says Andrew Losowsky, head of Coral by Vox Media, a system that aims to bring publishers and journalists closer to their communities. 

In fact, comments sections can bring great value that often goes overlooked. Done right, comments sections can increase reader loyalty and engagement, and even boost subscriptions. Simply deleting comments sections, Losowsky says, makes sense when those sections turn into vitriolic spaces. But he says this move can also be  “incredibly short-sighted.”

The key for news organizations is to develop a strategy. News outlets should think of their comments sections more deliberately. The need to ask questions like: What does it mean to involve people in the conversation? And, what sort of conversations do news organizations want to foster? From there, news organizations can ask pointed questions for readers to answer in the comments sections.

Managing the unmanageable

One way to make handling the volume of comments more manageable is by implementing artificial intelligence to help with the task of comment moderation. The New York Times is one outlet that actually expanded its comments section in 2017, with a machine learning technology called Moderator, which helps to moderate up to 12,000 comments a day. The technology scores each comment based on whether a human staffer might reject it based on inflammatory or inappropriate behavior. It then decides whether to keep or delete the comment based on that score.

Local news organizations, however, often lack the resources of larger sites. With revenues falling for smaller outlets, securing funding to implement a comments moderation system — much less hiring an employee specifically to moderate comments — might land quite far down the priority list.

Innovating with engagement

The State newspaper in Columbia, South Carolina has found that its Coral commenters are more engaged.

That doesn’t mean some local news sites aren’t experimenting with their own initiatives, though. The State newspaper in Columbia, South Carolina, which has a daily readership of around 270,000, implemented Coral about nine months ago. The platform uses AI to detect and remove toxic comments. Commenters at The State also need to sign into and register their email addresses to Coral separately from their subscriptions. This means that Coral users tend to be more engaged.

“We still see some of the bluster and posturing that can be negative in comments. But by and large, the comments that we do get are a lot more focused on the actual story,” said Cal Lundmark, southeast audience growth editor for McClatchy.

In addition to more constructive comments, general engagement on The State’s website flourished for Coral users. Active Coral commenters read twice as many stories on The State’s website per visit compared to other subscribers.These visitors spend almost 16 minutes longer on the site per visit as well.

Interaction breeds civility

When journalists interact with commenters, they can actually encourage civility in the comments sections, according to a report from the Democracy Fund and the Engaging News Project, now known as the Center for Media Engagement. The study found that journalists who answer reader questions and participate in discussions make the tone of comments sections more constructive.

It is effective even when local reporters only have enough time to dip into the comments sections rather than engage regularly. It “creates a sort of bond between the reader and the news organization, where people feel less okay to say nasty things,” says Gina M. Masullo, associate director of the Center for Media Engagement at the University of Texas at Austin.

Reporters at The State have been doing exactly this as part of the recent initiative to re-strategize the comments section. “The time in their day when they can really just stop and say ‘I can respond to some comments’ is pretty limited,” Lundmark said. “So it does take teamwork… between the audience team and the reporters themselves.”

In addition, news organizations can benefit from highly “person-centered” messages, according to Masullo’s research at the Center. These are comments from journalists that acknowledge a commenter’s anger, but also ask the commenter to keep their language civil.

“It worked across three different experiments that we did,” Masullo said. “I think it’s because even people who are angry like when somebody validates that they have a right to be angry.”

Social media, mediated

For the news organizations that have deleted comments sections over the past few years, engaging with their audiences is far from dead. In its one year update after doing away with its comments sections, NPR’s public editor Elizabeth Jensen declared the move a success.  She cited other ways of communicating with readers such as newsletters as garnering more attention. Journalists respond to readers and listeners through social media channels like Twitter and Facebook, Jensen wrote, and discussions also remain civil in closed Facebook groups.

Plus, it makes sense to take the conversation to social media if the community already regularly engages there, says Lynn Walsh, assistant director of Trusting News.

“We have to be realistic: Are people going to log into the website to comment when they may not already be engaging there actively?” Wash said. So, maybe “we should reach people by going to where they already are instead of asking them to come to us.”

But Losowsky warns against relying too much on social media to foster discussion.

“What you’re saying is that your engagement with your most loyal and engaged users belongs to a third party that you cannot control. And that data and that relationship belongs to them,” Losowsky said. “So if you say instead we will let Facebook manage this, then Facebook owns that relationship with your readers. You do not own it, they do. And they control it.”

Positive comments

So what does the future hold for comments and will news organizations abandon comments sections while reaching out to readers on other channels? Or will they attempt to take on the challenge of molding comments sections into places for constructive feedback?

Research continues to explore how to make comments sections into constructive spaces. Maite Taboada at Simon Fraser University, for instance, is working on a system to identify constructive comments that news organizations can highlight to create civil dialogue. And innovative organizations like Hearken develop different ways to tune into to readers and commenters, in addition to making listening to communities a fundamental directive in the reporting process.

“I’m cautiously optimistic that we will make the internet a better space,” Masullo said. “It’s not going to happen overnight, and it’s gonna be a combination of human and technological solutions. But I don’t think we should just say, ‘oh you know, this is a dismal mess and there’s no hope.’ Because I really think there is.”

Innovation | Perspectives

4 things HBR has learned by experimenting with audience engagement

Defining audience engagement can feel daunting at times. In our work at Harvard Business Review, we’re looking at engagement – and experimenting to understand that engagement better – through several different lenses: What is most valuable to current subscribers or potential subscribers or social audiences? And what formats hold the most growth potential across social, text, audio, video, bots, and so on.

Here are a few things we’ve learned from some of our recent projects and experiments:

1. When doing something brand new, get (at least a few) true believers in the mix.

Like others, we barreled into doing Facebook Live videos in early 2016. We weren’t doing much live video on the editorial side at the time, but we had a few ideas about what would work and what wouldn’t. We were especially eager to try videos that allowed the experts who write for HBR to explain their ideas to the audience directly. These videos, called whiteboard sessions, put a professor or business leader or other expert in front of a camera (or, early on, in front of a phone) to talk about their idea or concept for up to 40 minutes.

We knew this format would be fairly natural to the presenters because they know how to lecture or give presentations. That gave us a constant point of quality and allowed us to focus on the engagement side: questions, comments, visuals, takeaways.

However, while we had confidence in the format, we needed to work on getting the audience to engage. We discovered the importance of enthusiastic presenters. We had a few early on on who really believed in the opportunities that live, social video offered. One of those early whiteboard experts was entrepreneur and author Evan Baehr. He immediately understood the challenges: a live audience, a social platform, and a lot of competition for your attention from other content. But his enthusiasm got the audience’s attention right away and gave us a glimpse of what was possible with this format.

That confidence in the whiteboard format has been part of what has led us produce and experiment with more than 100 live whiteboards with experts from all over the world. We’re now working on creating a larger video strategy for them.

But without the enthusiasm of people like Evan and others early in the process, we might have assumed whiteboards were a nice-to-have and not invested our time as purposefully. So, keep in mind that, when you’re trying something new, you need to make sure you have some enthusiasts in the mix. The audience can feel it when you’re not enthusiastic about a medium (especially when it comes to video and audio), which will be reflected in popularity and engagement.

2. Text interactions can be tricky.

While we often seen hundreds of comments and shares on live videos, our results have been somewhat mixed when it’s come to text-based chats.

We write about a lot of work advice that’s very personal and a lot of business trends that are important to understand. We know that connecting our audience – and specifically our subscribers – to the experts who write for us helps people go even deeper in applying and sharing these ideas.

We’ve tried text-based engagement experiments around this premise in a couple of ways. In one open-to-all experiment around an important feature piece (an article series we call the Big Idea) on inequality between firms, we invited readers to a Slack channel where the author of the main piece was going to be available for questions and conversation about his research and work. We thought this had the right combination of platform plus deep, important topic and timely relevance.

The results were mixed. While we had more than 150 readers join the Slack channel, they mostly lurked. It’s tough to diagnose why. Was the text format intimidating? Too much work? Do people really want to listen to an expert first and foremost?

We’ve also tried this type of experiment in subscriber-only settings. We’ve invited authors of HBR pieces to answer questions in a closed Facebook group for subscribers, for example. The results have often been similar. Most people lurk and don’t contribute much. We’ll keep experimenting with this type of engagement. But we may need to pair text chat with something else – like video or audio – to really stir interaction. And we need to keep considering the right platform, as well as how much people want to interact with each other instead of an editor or author from HBR.

3. Constantly test your assumptions.

A central tenet of HBR’s experimentation and innovation practice is to write down our assumptions, goals, and hypotheses. This document is shared between product, editorial, and other teams involved in the project. We then check back on those assumptions at planned intervals to understand what we should do differently, what we’ve learned, and to help gauge whether the experiment should continue.

We’ve done two bot projects, one on Slack and one on Facebook Messenger, that help illustrate how this works. When we started the HBR bot on Slack, we assumed that people would love to see our big beautiful images in their conversations with our bot. However, our UX research proved that was wrong. People just wanted to get to work advice we were offering.

We also hypothesized that we wouldn’t drive any trackable subscriptions through the bot. That was right. But we also believed in driving frequent interactions through the bot should lead them to engage more deeply with us, even if we can’t draw a straight line back to their bot experience. That engagement could take a lot of shapes: Signing up for a newsletter, following a social channel, sharing an article with a friend.

It helps to write down even your most basic assumptions. For our recent Women at Work pop-up podcast, we documented the assumption that people wanted to engage with HBR about this topic in an audio experience. For the bots, we had to test our assumptions that people wanted to consume our content in bot form (and we’re still testing whether that’s true). It seems basic, but doing these things helps your build up a store of observations that can inform your future experiment plans and pitches.

4. Be clear on asks to your audience.

Don’t take this one for granted. When you ask your audience to do something, have a purpose and follow through. HBR has had a subscriber-only Facebook group for about a year now. The editor who runs it, my colleague Laura Amico, is clear that when editors post to that group, there’s a purpose and a plan. We’re not just throwing out a “What do you think?” to that community and walking away. Instead, we’re asking that community to join us in testing advice or sharing ideas on how they’ve solved tough work problems.

We’re also opening up new channels to listen to our audience and incorporate their feedback. Our new Dear HBR: podcast answers questions from the audience about tough workplace dilemmas. It’s our newest foray into the advice space, and we’re already seeing steady growth in the number of questions being posed.

And we’re taking a fresh look at the role of article comments in this space. Laura is working on this too, and I thought I’d share part of an internal report she’s developing to help push our thinking in this area:

Comments are about so much more than what is typed into a text box. Commenters need to know how to succeed in what we’re asking them to do and they need to be rewarded for doing it. That’s true on the other side, too; editors and others working with audience development also need to know how they and the business benefit from engagement. Developing use cases for comments, promoting good comments, and sharing engagement results internally and externally are all key to maintaining the relationships.

As we move into new areas of experimentation around engagement – involving everything from email to comments to visual storytelling to machine learning – we’ll keep testing our assumptions. We’ll keep pushing ourselves to think dynamically about our audience and how we ask for their time. And hopefully we’ll make their day at work a little better, too.

Maureen Hoch is Editor of HBR.org. Hoch leads Harvard Business Review’s digital newsroom and oversees the team working on HBR’s Facebook Live videos. She played an instrumental role in the redesign of HBR.org, which has since garnered three Webbys, and she helped lead the team that created HBR’s bots on Slack and Facebook Messenger. Hoch joined HBR as Senior Editor in 2013. She has previously worked for the PBS NewsHour, the World Bank, and The Washington Post Express.

Perspectives

Building a Business Case for Keeping Comments Sections

There was a time when online comments below stories enhanced and strengthened the stories with added commentary, smart insights and timely corrections from readers. Unfortunately, that only lasted until the first ad hominem attacks came. Then spammers. Then outright harassment.

So publishers are torn between wanting to keep readers engaged (and on the site) or streamlining operations and content so they don’t get pulled down into the swamp of trolls. Lately, publishers such as Re/code, Chicago Sun-Times, Bloomberg and Reuters have eliminated some or all comments. But others, like the New York Times, value reader comments but limit them to certain stories and close them after specified time periods.

The bottom line is that each community is different, with a site like Popular Science wanting to ward off climate-change deniers and a place like Reddit trying to keep things as open as possible. And what fits one business doesn’t fit another. Local news sites typically want more engagement with readers who live in their community, while a wire service like Reuters doesn’t really “live” in any community.

No Comment
There are many reasons to cut comments entirely. A recent study led by University of Wisconsin-Madison professor Dominique Brossard found that comments can change readers’ perception of the article. “Uncivil comments not only polarized readers, but they often changed a participant’s interpretation of the news story itself,” she and her co-author Dietram Scheufele wrote in a New York Times Op-Ed.

Another important reason to kill them is that comments and conversation have moved on to social networks such as Facebook and Twitter, where articles are being shared. Thus, publishers begin to question the resources required to moderate comments (The New York Times has 13 moderators) if they don’t pay enough of a dividend.

Dan Colarusso, executive editor for digital at Reuters told me in our recent PBS MediaShift Mediatwits podcast that they hadn’t seen a drop-off in engagement on Reuters after removing comments from news stories. “The normal organic users to our site weren’t engaging,” he said. “It was a fraction of 1% increase in engagement on our site from comments… So we decided to cull them out of there. And our wire service reporters don’t have the time to interact.”

It makes more sense for Reuters.com to keep comments on blog posts and opinion pieces, Colarusso said, because those stories, and their writers, are already associated with providing opinions and personal views. Reuters, as a brand, is not.

Moderate Comments
There are middle grounds, too. The Huffington Post, for example, banned anonymous comments in 2013 and has since instituted a system where readers must log in using their Facebook accounts. Though this has earned mixed reviews from its readers, some of whom don’t like giving personal information to Facebook, it’s proved useful for HuffPost in terms of accountability. “We’ve seen a marked [decrease] in the number of fake accounts in our system. It’s also helped with getting more quality comments and positive conversation as opposed to criticisms or insults,” Huffington Post community director Tim McDonald told Digiday last year.

The Engaging News Project released a study recently that showed that there are ways to increase civility in online comments. If journalists interact with commenters, the tone usually improves. Plus, some publishers such as Gawker do a better job of designing comments to highlight the best ones. And algorithmic systems that automatically flag cursing or spam in comments can help.

In the end, news outlets have to analyze the purpose of their comments to make the business decision that’s right for them. Is the purpose of commenting to encourage a community that fits in with the news organization’s brand? If so, then perhaps keeping a commenting system in-house is a must, so the organization can maintain control of that brand and identity. But if the purpose of comments is to drive engagement back to a news website, then referrals from social media — and therefore, commentary that takes place on social media — might be the best fit.

Or if the point of commenting is to encourage high-level discussions, then perhaps you have to make the process of commenting attractive only to those who really care — like Tablet Magazine’s announcement that it plans to charge would-be commenters.

We’ve seen social media take on so much heat and discussion, but publishers will have to weigh whether they want to improve commenting and discussion on-site or cede another territory to the social giants.

Listen to the whole Mediatwits podcast discussion on online comments here:

©2024 Digital Content Next. All Rights Reserved.