Tags containing: "chatbots"

How Outside, FT and Dow Jones use AI for search and solutions

June 6, 2024 Jessica Patterson – Independent Media Reporter

These days, digital media companies are all trying to figure out how to best incorporate AI into their products, services and capabilities, via partnerships or by building their own. The goal is to gain a competitive edge as they tailor AI capabilities to their audiences, subscribers and clients’ specific needs.

By leveraging proprietary Large Language Models (LLMs) digital media companies have a new tool in their toolboxes. These offerings offer differentiation and added value, enhanced audience engagement and user experience. These proprietary LLMs also set them apart from companies that are opting for licensing partnerships with other LLMs, which offer more generalized knowledge bases and draw from a wide range of sources in terms of subject matter and quality.

A growing number of digital media companies are rolling out their own LLM-based generative AI features for search and data-based purposes to enhance user experience and create fine-tuned solutions. In addition to looking at several of the offerings media companies are bringing to market, we spoke to Dow Jones, Financial Times and Outside Inc. about the generative AI tools they’ve built and explore the strategies behind them.

Media companies fuel generative AI for better solutions

Digital media companies are harnessing the power of generative AI to unlock the full potential of their own – sometimes vast amounts – of proprietary information. These new products allow them to offer valuable, personalized, and accessible content to their audiences, subscribers, customers and clients.

Take for example, Bloomberg, which released a research paper in March detailing the development of its new large-scale generative AI model called BloombergGPT. The LLM was trained on a wide range of financial data to assist Bloomberg in improving existing financial natural language processing (NLP) tasks, such as sentiment analysis, named entity recognition, news classification, and question answering, among others. In addition, the tool will help Bloomberg customers organize the vast quantities of data available on the Bloomberg Terminal in ways that suit their specific needs.

Launched in beta June 4, Fortune partnered with Accenture to create a generative AI product called Fortune Analytics. The tool delivers ChatGPT-style responses based on 20 years of financial data from the Fortune 500 and Global 500 lists, as well as related articles, and helps customers build graphic visualizations.

Generative AI helps customers speed up processes

A deeper discussion of how digital media companies are using AI provides insights to help others understand the potential to leverage the technology for their own needs. Dow Jones, for example uses Generative AI for a platform that helps customers meet compliance requirements.

Dow Jones Risk Compliance is a global provider of risk and compliance solutions across banks and corporations which helps organizations perform checks on their counterparties. They do that from the perspective of complying with anti-money laundering regulation, anti-corruption regulation, looking to also mitigate supply chain risk and reputational issues. Dow Jones Risk Compliance provides tools that allow customers to search data sets and help manage regulatory and reputational risk.

In April, Dow Jones Risk & Compliance launched an AI-powered research platform for clients that enables organizations to build an investigative due diligence report covering multiple sources in as little as five minutes. Called Dow Jones Integrity Check, the research platform is a fully automated solution that goes beyond screening to identify risks and red flags from thousands of data sources.

The planning for Dow Jones Integrity Check goes back a few years, as the company sought to provide its customers with a quicker way to do due diligence on their counterparties, Joel Lange, executive Vice President and General Manager, Risk and Research at Dow Jones explained.

Lange said that Dow Jones effectively built a platform which automatically creates a report for customers on a person or company, using technology from AI firm Xapien. It brings together Dow Jones’ data that is plugged into other data sets, corporate registrar information, and wider web content. It then leverages the platform’s Generative AI capability to produce a piece of analysis or a report.

Dow Jones Risk & Compliance customers use their technology to make critical, often complex, business decisions. Often the data collection process can be incredibly time consuming, taking days if not weeks.

The new tool “provides investigations, teams, banks and corporations with initial due diligence. Essentially it’s a starting point for them to conduct their due diligence, effectively automating a lot of that data collection process,” according to Lange.

Lange points out that the compliance field is always in need of increased efficiency. However, it carries with it great risk to reputation. Dow Jones Integrity Check was designed to reshape compliance workflows, creating an additional layer of investigation that can be deployed at scale. “What we’re doing here is enabling them to more rapidly and efficiently aggregate, consolidate, and bring information to the fore, which they can then analyze and then take that investigation further to finalize an outcome,” Lange said.

Regardless of the quality of the generated results, most experts believe that it is important to have a human in the loop in order to maintain content accuracy, mitigate bias, and enhance the credibility of the content. Lange also said that it’s critical to have “that human in the loop to evaluate the information and then to make a decision in relation to the action that the customer wants to take.”

Media-powered generative AI offers smarter search

When Google first launched Bard in February 2023, it provided incorrect information about many subjects, including the James Webb Space Telescope. This raised concerns about the accuracy and reliability of generative AI, and caused a drop in Alphabet’s stock value to the tune of $100b. Digital media companies’ generative AI products may avoid similar issues because they use their own content archives and data, rather than content scraped willy-nilly from the open web.

In recent months, digital media companies have been launching their own generative AI tools that allow users to ask questions in natural language and receive accurate and relevant results.

To name a few: The Washington Post is partnering with Virginia Tech’s Sanghani Center for Artificial and Data Analytics to develop a new generative AI project where readers can get answers to questions, using data taken from The Post’s previous coverage.

The Associated Press created Merlin, an AI-generated search tool that makes searching the AP archive more accurate. “Merlin pinpoints key moments in our videos to exact second and can be used for older archive material that lacks modern keywords or metadata,” explained AP Editor in Chief Julie Pace at The International Journalism Festival in Perugia in April.

Outside’s Scout: AI search with useful results

Chatbots have become a popular form of search. Originally pre-programmed and only able to answer select questions included in their programming, chatbots have evolved and increased engagement by providing a conversational interface. Used for everything from organizing schedules and news updates to customer service inquiries, Generative AI-based chatbots assist users in finding information more efficiently across a wide range of industries.

In January, Outside Inc. launched a custom chatbot OpenAI’s LLM and trained on Outside’s library of content as a knowledge base, to provide summaries and answers to audience questions. Called Scout, the chatbot is integrated into the top navigation bar of Outside’s 20+ websites and serves as the primary search engine across the Outside network.

Much like The Guardian, The Washington Post, The New York Times and other digital media organizations that blocked OpenAI from using their content to power artificial intelligence, Outside CEO Robin Thurston explained that Outside Inc. wasn’t going to let third parties scrape their platforms to train LLM models.

Instead, they looked at leveraging their own content and data. “We had a lot of proprietary content that we felt was not easily accessible. It’s almost what I’d call the front page problem, which is you put something on the front page and then it kind of disappears into the ether,” Thurston said.

“We asked ourselves: How do we create something leveraging all this proprietary data? How do we leverage that in a way that really brings value to our user?” Thurston said. The answer was Scout, Outside Inc.’s AI search assistant. Scout is a custom-developed chatbot.

The company could see that generative AI offered a way to make that content accessible and even more useful to its readers. Outside had a lot of evergreen content that wasn’t adding value once it left the front page. Their brands inspire and inform audiences about outdoor adventures, new destinations and gear – a lot of which is evergreen and proprietary content that still had value if it could easily be surfaced by its audience. The chat interface allows their content to continue to be accessible to readers after it is no longer front and center on the website.

Scout gives users a summary answer to their question, leveraging Outside Inc’s proprietary data, and surfaces articles that it references. “It’s just a much more advanced search mechanism than our old tool was. Not only does it summarize, but it then returns the things that are most relevant,” he explained.

Additionally, Outside Inc’s old search function worked by each individual brand. Scout searches across the 20+ properties owned by the parent company which include Backpacker, Climbing, SKI Magazine, and Yoga Journal, among others. Scout brings all of the results together, from the 20+ different Outside brands, from the best camping destinations, to the best trails, outdoor activities for the family, gear, equipment and food all in one result.

One aspect that sets Outside Inc.’s model apart is their customer base, which differs from general news media customers. Outside’s customers engage in a different type of interaction, not just a quick transactional skim of a news story. “We have a bit of a different relationship in that they’re not only getting inspiration from us, which trip should I take? What gear should I buy? But then because of our portfolio, they’re kind of looking at what’s next,” Thurston said.

It was important to Thurston to use the LLM in a number of different ways, so Outside Inc launched a local newsletter initiative with the help of AI. “On Monday mornings we do a local running, cycling and outdoor newsletter that goes to people that sign up for it, and it uses that same LLM to pick what types of routes and content for that local newsletter that we’re now delivering in 64,000 ZIP codes in the U.S.”

Thurston said they had a team working on Scout and it took about six months. “Luckily, we had already built a lot of infrastructure in preparation for this in terms of how we were going to leverage our data. Even for something like traditional search, we were building a backend so that we could do that across the board. But this is obviously a much more complicated model that allows us to do it in a completely new way,” he said.

Connecting AI search to a real subscriber need

In late March, The Financial Times released its first generative AI feature for subscribers called Ask FT. Like Scout, the chat-based search tool allows users to ask any question and receive a response using FT content published over the last two decades. The feature is currently available to approximately 500 FT Professional subscribers. It is powered by the FT’s own internal search capabilities, combined with a third-party LLM.

The tool is designed to help users understand complicated issues or topics, like Ireland’s offshore energy policy, rather than just searching for specific information. Ask FT searches through Financial Times (FT) content, generates a summary and cites the sources.

“It works particularly well for people who are trying to understand quite complex issues that might have been going on over time or have lots of different elements,” explained Lindsey Jayne, the chief product officer of the Financial Times.

Jayne explained that they spend a lot of time understanding why people choose the FT and how they use it. People read the FT to understand the world around them, to have a deep background knowledge of emerging events and affairs. “With any kind of technology, it’s always important to look at how technology is evolving to see what it can do. But I think it’s really important to connect that back to a real need that your customers have, something they’re trying to get done. Otherwise it’s just tech for the sake of tech and people might play with it, but not stick with it,” she said.

Trusted sources and GenAI attribution

Solutions like those from Dow Jones, FT and Outside Inc. highlight the power of a brand with a trusted audience relationship to create deep, authentic relationships built on reliability and credibility. Trusted media brands are considered authoritative because their content is based on credible sources and facts, which ensures accuracy.

Currently, generative AI has demonstrated low accuracy and poses challenges to sourcing and attribution. Attribution is a central feature for digital media companies who roll out their own generative AI solutions. For Dow Jones compliance customers, attribution is critical to customers, to know if they’re going to make a decision based on information that is available in the media, according to Lange.

“They need to have that attributed to within the solution so that if it’s flowing into their audit trails or they have to present that in a court of law, or if they would need to present it to our internal audit, the attribution is really key. (Attribution) is going to be critical for a lot of the solutions that will come to market,” he said. “The attribution has to be there in order to rely on it for a compliance use case or really any other use case. You really need to know where that fact or that piece of information or data actually came from and be able to source it back to the underlying article.”

The Financial Times’ generative AI tool also offers attribution to FT articles in all of its answers. Ask FT pulls together lots of different source material, generates an answer, and attributes it to various FT articles. “What we ask the large language model to do is to read those segments of the articles and to turn them into a summary that explains the things you need to know and then to also cite them so that you have the opportunity to check it,” Jayne said.

They also ask the FT model to infer from people’s questions when it should be searching from. “Maybe you’re really interested in what’s happened in the last year or so, and we also get the model to reread the answer, reread all of the segments and check that, as kind of a guard against hallucination. You can never get rid of hallucination totally, but you can do lots to mitigate it.”

The Financial Times is also asking for feedback from the subscribers using the tool. “We’re literally reading all of the feedback to help understand what kinds of questions work, where it falls down, where it doesn’t, and who’s using it, why and when.”

Leaning into media strengths and adding a superpower

Generative AI seems to have created unlimited opportunities and also considerable challenges, questions and concerns. However it is clear that an asset many media companies possess is a deep reservoir of quality content and it is good for business to extract the most value from the investment in its creation. Leveraging their own content to train and program generative AI tools that serve readers seems like a very promising application.

In fact, generative AI can give trustworthy sources a bit of a super power. Jayne from the FT offered the example of scientists using the technology to read through hundreds of thousands of research papers and find patterns in a process that would otherwise take years to read in an effort to make important connections.

While scraped-content LLMs pose risks to authenticity, accuracy and attribution, proprietary learning models offer a promising alternative.

As Jayne put it, “The media has “an opportunity to harness what AI could mean for the user experience, what it could mean for journalism, in a way that’s very thoughtful, very clear and in line with our values and principles.” At the same time, she cautions that we shouldn’t be “getting overly excited because it’s not the answer to everything – even though we can’t escape the buzz at the moment.”

We are seeing many efforts bump up against the limits of what generative AI is able to do right now. However, media companies can avoid some of generative AI’s current pitfalls by employing the technology’s powerful language prediction, data processing and summarization capabilities while leaning into their own strengths of authenticity and accuracy.

Are we entering the age of artificial journalistic intelligence?

March 9, 2023 Jessica Patterson – Independent Media Reporter

From simple chatbots to generative content creation tools, artificial intelligence has become a force to be reckoned with for digital media companies. But while interest has heated up lately, artificial intelligence (AI) has already been employed by media companies for about a decade. Among other things, AI has long enabled journalists to analyze large amounts of data, extract insights, and identify patterns and trends. It also offers ways to engage audiences through personalized experiences and recommendations.

However, while it is not a new phenomenon, AI has come a long way since 2014 when some of the first media companies started using it for journalistic purposes. In this two-part series, we explore the evolution of AI use in the media, what’s different now, the opportunities that it offers publishers, as well as challenges including the misuse and abuse of AI, errors and accuracy, and concerns about job losses for human journalists.

AI + journalism

The news industry has witnessed three phases of AI innovation over the past decade: automation, augmentation, and generation, according to Francesco Marconi, computational journalist and the co-founder of real-time information company AppliedXL. “During the first wave, the focus was on automating data-driven news stories, such as financial reports, sports results, and economic indicators, using natural language generation techniques,” Marconi said.

In that first wave, AI offered a way to free journalists from menial or repetitive tasks so they could engage in more complex, high-impact, and investigative journalism. It was touted as a way to save organizations time and money by automating the production of certain types of news stories based on data.

For example, The Washington Post used its in-house automated technology Heliograf to cover weekly high school football games in Washington, D.C. and The Associated Press used AI to automate quarterly earnings reports – generating more than 3,000 stories per quarter compared to just 300 previously.

During the second wave, the emphasis shifted to augmenting reporting through machine learning and natural language processing to analyze large datasets and uncover trends, Marconi explained. AI was a tool for journalists to use to analyze large amounts of data, text, video and images, to help them identify trends, patterns or outliers.

During this period, digital media companies also began experimenting with chatbots to help journalists tell stories differently. Chatbots simulated human conversations by responding to specific questions with pre-programmed answers. Early examples included Harvard Business Review’s Slackbot, and AccuWeather’s chatbot for Facebook Messenger that took questions from users and provided weather-related answers.

AI today

The third and current wave is driven by the AI generative movement, powered by large language models, which are capable of generating narrative text at scale.

OpenAI’s ChatGPT-3 was trained on massive amounts of text (large language models), which it scraped from the internet. This allowed it to learn the patterns and structures of language to “make educated guesses based on the words you’ve typed previously.” ChatGPT reached 100 million users in January, two months after its launch. Not wanting to be left behind, Google released its own AI chatbot called Bard Facebook announced its AI large language model called LlaMA, and search engine DuckDuckGo has announced DuckAssist.

In recent days, digital media companies have been experimenting with and examining how they can use ChatGPT and generative AI. Last month, Jim Mullen, chief executive officer at Reach PLC, told the Financial Times his company had set up a working group to examine how the tool might be used to assist journalists to write short news stories or compile coverage of local weather and traffic.

For media companies that have used AI for years, this new crop of AI has been interesting to watch. While ethical and quality uses are well-established, some of the current applications are still a work in progress.

One early adopter, Bloomberg Media, has used AI for more than a decade. Their deployment uses natural language processing (NLP) and machine learning to extract information from documents, including detecting people, companies and organizations found in the text of their stories. In addition, they also extract insights to help reporters find news about companies or sectors they cover.

“Like most consumers, I’ve been enchanted by the latest crop of AI tools. Tools like ChatGPT and Midjourney are showing us how much creativity these tools can unlock,” said Julia Beizer, Chief Digital Officer at Bloomberg Media.

“But alongside the enchantment, I’ve felt a fair bit of unease. These are early, very public experiments and we are seeing in real time how easy it is for unproven tech to get the facts wrong. This should be a reminder to all of us – particularly those of us in the journalism business – how important the work we do is,” she said.

The difference in today’s AI

For digital media companies, AI is different now for a number of reasons. Newer generative AI models have broader capabilities because of increased computational power, which means models can process larger amounts of data faster. There have been improvements in NLP, which has allowed AI to generate more nuanced language.

And, in recent years, AI tools have become more accessible for digital content companies. The current crop requires far less tech savvy and investment because they’re open-source.

“Initially, AI in the newsroom was limited to big newsrooms with significant resources,” explained Marconi. “However, with the widespread availability of AI infrastructure, such as open-source models and APIs, the technology is now accessible to every newsroom. This broad accessibility will also bring major challenges as the news industry rushes to adopt new standards and workflows.”

Bloomberg’s Beizer believes that the big difference right now is creativity. “The best technology feels like magic. It fades to the background so all you’re left with is the experience. ChatGPT is a simple interface over one of the most powerful pieces of technology we’ve seen,” she said.

So what’s new?

Given the leap forward in ease of use, AI may create new opportunities for those willing to adapt and innovate in this rapidly changing landscape.

“The economics of the media business are changing dramatically. AI has substantially reduced the cost of production” such as including editing, story writing, and creating multiple versions. “But newsgathering and fact-checking still necessitate specialized knowledge and infrastructure that technology companies lack,” Marconi said.

He added that media companies have an opportunity to become a major player in the AI space. That’s because they possess valuable assets for AI development: the text and image data for training models and ethical principles for creating trustworthy systems.

Advances in technology have disrupted the media industry for many years. According to Beizer, generative AI is only the latest chapter. “What I believe has changed this time around is that we are now mature digital companies with digital businesses to protect and grow. This time around, I have no doubt that our industry will rise to the occasion of thoughtfully exploring this disruption from all its angles – and setting the right strategies for our businesses and quality content to flourish,” she said.

Opportunities ahead

There are valid concerns about whether generative AI will be used to flood the market with content in the future. However, reputable media outlets will find that there are responsible applications of AI that will actually allow them to double down on the quality of their journalism and provide better content recommendations and experiences.

For Beizer, the biggest opportunity at the moment is “focusing our companies on what matters most: our customers… In this moment, we should lean even harder into building direct relationships with those users, getting to know what they want and delivering that value. We have the biggest opportunities in the kind of differentiated value that a chatbot cannot provide – analysis, novel points of view, human connection, nuanced reporting and writing. We should deprioritize anything that isn’t that. The future of our business depends on it,” she said.

However, as AI becomes more sophisticated, there are other promising applications for journalists. Imagine an AI-based tool that could help with large projects that newsrooms could never have enough staff to do, or one that checked facts, style, and grammar.

Marconi sees the evolution of AI in recent years as the foundation for the next phase of AI in journalism, which he believes will come to fruition over the next decade. He calls it artificial journalistic intelligence, where machines are able to automate news gathering and production, resulting in improved accuracy, efficiency, and impartiality, he said.

“Nonetheless, humans will continue to play a significant role. Creating editorial algorithms requires human supervision. As a result, in this new era, transparency and auditing of algorithms will become critical functions of newsrooms in the future,” he said.

PART 2: Media execs weigh risks, challenges of generative AI

In part 2, we look at challenges and concerns around generative AI, including the misuse and abuse of AI, errors and accuracy, legal and ethical issues, and concerns about job losses for human journalists.

Why the time is right to shine a light on ‘dark social’

November 1, 2018 Mark Glaser, Founder and Publisher – MediaShift

One of the least understood portions of the web is becoming one of the most important: dark social. And as the calendar turns from Halloween to the Day of the Dead, it’s time to shine a light into the darkest corner of the web to understand how people are sharing content via messaging services, email, texting, and other platforms that don’t show up easily on analytics programs.

For a long time, referrals that showed up on Google Analytics as “direct” were a mystery. Could that many people really be typing in long URLs into their browsers? No. Instead, they were cutting and pasting URLs into an email to friends, or WhatsApp or Facebook Messenger. Those referrals became known as “dark social” because of our ignorance about their true source. And with public sharing on Facebook and other platforms down, dark social is only growing more important, with an estimated 84% of all sharing, according to RadiumOne.

But there are ways to better gauge where those dark social referrals are coming from. There are also ways to reach people on messaging apps, through chatbots and targeted promotions. Even more important, reaching people on dark social requires a new way of thinking of them in their “tribes” or personal space where gauging sentiment might be more important than marketing a product directly.

The Rise of Dark Social

They are like two ships passing in the night: the rise of messaging apps and the decline of public social sharing. Just looking at Facebook user statistics gives us a stark picture:

From November 2014 to September 2017, Facebook Messenger users went from 500 million to 1.3 billion.
From December 2013 to December 2017, WhatsApp users went from 400 million to 1.5 billion.
From Q1 2017 to Q2 2018, Facebook interactions went from 29.1 billion to 12.8 billion.

Yes, 12.8 billion is still a lot of interactions, but you the trend is clear. People are losing trust in social networks and flocking to private messenging services.

As Arik Hanson at Business2Community noted, even Facebook’s Instagram is thinking beyond the public feed. “Let’s look at the direction they’re going: Investing heavily in Stories and direct messaging (both “private” for the most part, when it comes to engagement). Many of the new features added recently revolve around these two areas: poll sticker, slider stickers, polling via DM. These are the areas Instagram is focused on – not the public-facing feed as much.”

How to Get a Handle on Dark Social

There are ways to bring dark social into the light. While vanilla Google Analytics shows dark social traffic, there are ways to segment pages to see which ones are getting the most action from dark social. (99Signals has a good primer on that here.)

And there are always advanced tools like GetSocial and AddThis that can help identify which pages are getting hit the most from dark social. They also show where some of those sources are.

However, tracking down dark social the same as regular social referrals might not be the point. As Alpha Group’s David Cohn points out on MediaPost, “part of dark social’s appeal is its ‘off the grid’ nature, [so] we wouldn’t want to throw the baby out with the bathwater by looking for too much data and meaning among the noise.”

Taking Action to Reach Users

So how does a marketer or publisher get a return on investment by reaching folks on messaging apps, texting or emails? Cohn says the key is to figure out basic demographics on the platforms and the groups who congregate there. He uses the example of a foodie group in New York that could start to send more traffic to food publications and restaurants. “A marketer or publisher wouldn’t know exactly whom to target, but they would have a place,” he wrote. “And they would recognize this community as a gathering place for worthy targets.”

One example from India shows how ROI can grow when targeting dark social. When streaming service iflix wanted to increase its users, it found that its most loyal customers had one important thing in common: They liked sharing viral content and celebrity gossip on messaging apps. According to Business Today’s Sonal Khetarpal, iflx targeted messaging apps users. They converted 1 million more paid subscribers in six months. And the cost of acquisition dropped from $25 to $3 per person.

While it’s true marketers are tiring of social media planning that moves from one platform to the next each season, they’re going to have to get serious about messaging apps and dark social. More folks will have to take chatbots seriously, or just better targeting to users of messaging apps. Otherwise, they will miss a massive, growing opportunity as more people become disaffected with public social media spaces.

Innovation | Perspectives

4 things HBR has learned by experimenting with audience engagement

May 15, 2018 Maureen Hoch, Editor—HBR.org

Defining audience engagement can feel daunting at times. In our work at Harvard Business Review, we’re looking at engagement – and experimenting to understand that engagement better – through several different lenses: What is most valuable to current subscribers or potential subscribers or social audiences? And what formats hold the most growth potential across social, text, audio, video, bots, and so on.

Here are a few things we’ve learned from some of our recent projects and experiments:

1. When doing something brand new, get (at least a few) true believers in the mix.

Like others, we barreled into doing Facebook Live videos in early 2016. We weren’t doing much live video on the editorial side at the time, but we had a few ideas about what would work and what wouldn’t. We were especially eager to try videos that allowed the experts who write for HBR to explain their ideas to the audience directly. These videos, called whiteboard sessions, put a professor or business leader or other expert in front of a camera (or, early on, in front of a phone) to talk about their idea or concept for up to 40 minutes.

We knew this format would be fairly natural to the presenters because they know how to lecture or give presentations. That gave us a constant point of quality and allowed us to focus on the engagement side: questions, comments, visuals, takeaways.

However, while we had confidence in the format, we needed to work on getting the audience to engage. We discovered the importance of enthusiastic presenters. We had a few early on on who really believed in the opportunities that live, social video offered. One of those early whiteboard experts was entrepreneur and author Evan Baehr. He immediately understood the challenges: a live audience, a social platform, and a lot of competition for your attention from other content. But his enthusiasm got the audience’s attention right away and gave us a glimpse of what was possible with this format.

That confidence in the whiteboard format has been part of what has led us produce and experiment with more than 100 live whiteboards with experts from all over the world. We’re now working on creating a larger video strategy for them.

But without the enthusiasm of people like Evan and others early in the process, we might have assumed whiteboards were a nice-to-have and not invested our time as purposefully. So, keep in mind that, when you’re trying something new, you need to make sure you have some enthusiasts in the mix. The audience can feel it when you’re not enthusiastic about a medium (especially when it comes to video and audio), which will be reflected in popularity and engagement.

2. Text interactions can be tricky.

While we often seen hundreds of comments and shares on live videos, our results have been somewhat mixed when it’s come to text-based chats.

We write about a lot of work advice that’s very personal and a lot of business trends that are important to understand. We know that connecting our audience – and specifically our subscribers – to the experts who write for us helps people go even deeper in applying and sharing these ideas.

We’ve tried text-based engagement experiments around this premise in a couple of ways. In one open-to-all experiment around an important feature piece (an article series we call the Big Idea) on inequality between firms, we invited readers to a Slack channel where the author of the main piece was going to be available for questions and conversation about his research and work. We thought this had the right combination of platform plus deep, important topic and timely relevance.

The results were mixed. While we had more than 150 readers join the Slack channel, they mostly lurked. It’s tough to diagnose why. Was the text format intimidating? Too much work? Do people really want to listen to an expert first and foremost?

We’ve also tried this type of experiment in subscriber-only settings. We’ve invited authors of HBR pieces to answer questions in a closed Facebook group for subscribers, for example. The results have often been similar. Most people lurk and don’t contribute much. We’ll keep experimenting with this type of engagement. But we may need to pair text chat with something else – like video or audio – to really stir interaction. And we need to keep considering the right platform, as well as how much people want to interact with each other instead of an editor or author from HBR.

3. Constantly test your assumptions.

A central tenet of HBR’s experimentation and innovation practice is to write down our assumptions, goals, and hypotheses. This document is shared between product, editorial, and other teams involved in the project. We then check back on those assumptions at planned intervals to understand what we should do differently, what we’ve learned, and to help gauge whether the experiment should continue.

We’ve done two bot projects, one on Slack and one on Facebook Messenger, that help illustrate how this works. When we started the HBR bot on Slack, we assumed that people would love to see our big beautiful images in their conversations with our bot. However, our UX research proved that was wrong. People just wanted to get to work advice we were offering.

We also hypothesized that we wouldn’t drive any trackable subscriptions through the bot. That was right. But we also believed in driving frequent interactions through the bot should lead them to engage more deeply with us, even if we can’t draw a straight line back to their bot experience. That engagement could take a lot of shapes: Signing up for a newsletter, following a social channel, sharing an article with a friend.

It helps to write down even your most basic assumptions. For our recent Women at Work pop-up podcast, we documented the assumption that people wanted to engage with HBR about this topic in an audio experience. For the bots, we had to test our assumptions that people wanted to consume our content in bot form (and we’re still testing whether that’s true). It seems basic, but doing these things helps your build up a store of observations that can inform your future experiment plans and pitches.

4. Be clear on asks to your audience.

Don’t take this one for granted. When you ask your audience to do something, have a purpose and follow through. HBR has had a subscriber-only Facebook group for about a year now. The editor who runs it, my colleague Laura Amico, is clear that when editors post to that group, there’s a purpose and a plan. We’re not just throwing out a “What do you think?” to that community and walking away. Instead, we’re asking that community to join us in testing advice or sharing ideas on how they’ve solved tough work problems.

We’re also opening up new channels to listen to our audience and incorporate their feedback. Our new Dear HBR: podcast answers questions from the audience about tough workplace dilemmas. It’s our newest foray into the advice space, and we’re already seeing steady growth in the number of questions being posed.

And we’re taking a fresh look at the role of article comments in this space. Laura is working on this too, and I thought I’d share part of an internal report she’s developing to help push our thinking in this area:

Comments are about so much more than what is typed into a text box. Commenters need to know how to succeed in what we’re asking them to do and they need to be rewarded for doing it. That’s true on the other side, too; editors and others working with audience development also need to know how they and the business benefit from engagement. Developing use cases for comments, promoting good comments, and sharing engagement results internally and externally are all key to maintaining the relationships.

As we move into new areas of experimentation around engagement – involving everything from email to comments to visual storytelling to machine learning – we’ll keep testing our assumptions. We’ll keep pushing ourselves to think dynamically about our audience and how we ask for their time. And hopefully we’ll make their day at work a little better, too.

Maureen Hoch is Editor of HBR.org. Hoch leads Harvard Business Review’s digital newsroom and oversees the team working on HBR’s Facebook Live videos. She played an instrumental role in the redesign of HBR.org, which has since garnered three Webbys, and she helped lead the team that created HBR’s bots on Slack and Facebook Messenger. Hoch joined HBR as Senior Editor in 2013. She has previously worked for the PBS NewsHour, the World Bank, and The Washington Post Express.

Innovation

What AccuWeather’s Facebook chatbot says about AI

July 21, 2017 Michelle Manafy, Editorial Director – DCN

Artificial Intelligence is a term used to describe everything from Apple’s Siri to Google’s Deep Mind and is being leveraged for a wide range of applications from shopping to quantum computing. At its core, AI is the capacity of computing to perform tasks that correspond to decision making and learning by humans. True AI doesn’t just infer or make deductions, it understands natural language, and can develop based upon experience.

Today, AI powers everyday tools used by millions of people. With the rising popularity of voice-based interfaces such as Google Home and Amazon Alexa – and the increasingly-accurate recommendation tools offered by the likes of Netflix and Pandora – AI tools are becoming embedded in people’s everyday activities and expectations.

Meet the Bots

Chatbots are another booming implementation of AI. That said, not all of them are powered by AI. As Trish Mikita, AccuWeather’s VP of Digital Media Strategy points out, “There are plenty of dumb bots out there. Many bots are based upon simple decision trees that use a scripted format to (eventually) deliver answers to common questions. Mikita says AccuWeather experimented with this approach in the past, however its latest chatbot leverages true AI.

The company has just launched AccuWeather for Facebook Messenger. The AI-powered weather assistant handles plain-language questions from users and provides easy weather-related answers. According to Mikita, Facebook Messenger was the natural choice for this launch given its large user base and the fact that AccuWeather fans were already communicating with the company on Messenger. “Messenger is a great platform that extends our conversations with customers in a natural way.”

Natural Language, Real Answers

AccuWeather’s chatbot—which was dubbed Abbi in beta — strives to be natural in its conversation. “The goal with us is for users to be able to ask a question, for our chatbot to come with an intent, and then answer the question in a meaningful way,” says Mikita.

So, while Abbi can answer questions about the weather forecast, she can also advise on appropriate attire for the day. The idea is for the chatbot to provide an accurate response given the context of a run in Cape Cod or a business meeting in Manhattan.

According to Mikita, being able to respond, “no it’s going to be 75 and sunny, no need for a jacket today” is a great first step. However, as the chatbot evolves, she looks forward to answers like “not today, but you’ll need an umbrella if you are going to be out later tonight.” To get there, AccuWeather is logging all of its audience interactions with Abbi. “This helps us with all of our products, so we can better understand users’ intent.”

Evolution of AccuWeather AI

Another step in AccuWeather’s chatbot evolution will be developing a personality. “A bot absolutely needs to have a personality, though it is a big challenge.” First, says Mikita, you must understand intent and get the outputs to be accurate. “The next phase is that personality piece. Natural interactions should have a personality. But having a personality doesn’t mean it has to be irreverent, or jokey. It can be a science nerd. Getting that personality to match our brand voice is a very important aspect.”

Mikita says that in addition to the continued development of Abbi (which may well have a name change in her future as the bot’s personality and functionality evolve), we can expect more AI-based launches coming within the next few months. The long-term roadmap also includes wrapping the intelligence piece into the AccuWeather API and incorporating these functions into its subscription products. AccuWeather’s presence on Alexa – as well as its evolving chatbots and plans for improved and innovative implementations of AI in its premium products – offers what Mikita describes as an “innovation opportunity that helps our other ad supported platforms.”

The company is already seeing efficiency savings with customer service. And they look forward to emerging opportunities to monetize these innovations directly. In the meantime, however, the company’s investment in AI is paying off in its ability to better interact with consumers, to get to know them better and to better serve their needs.

Industry-Research | Innovation

Artificial Intelligence gains momentum with news media

June 13, 2017 Rande Price, Research VP – DCN

Artificial Intelligence (AI), a computer’s ability to replicate the human thought process and solve problems, is hard at work in today’s news media. And for those not already leveraging AI, the time is now. In the new report, Artificial Intelligence: News Media’s Next Urgent Investment, Martha L. Stone, CEO of the World Newsmedia Network, in association with the INMA, explores how AI is being applied in today’s news industry.

Stone explains that AI has three main forms of behavior: natural language processing; predictive analytics; and machine learning/neural networks. Publishers can apply all three forms to address a wide range of news challenges.

Natural Language Processing

The first form describes the way in which computers understand the natural language process (NLP). It allows for automatic creation of articles (“robo-journalism”). Both the Associated Press and the BBC use Wordsmith, an NLP automated database, to create huge volumes of stories within seconds.

Natural language processing enables speech recognition and is used in devices like Apple’s Siri, Amazon’s Alexa, or Google’s Home. This type of AI process allows CNN, The New York Times, The Washington Post, the Chicago Tribune, Quartz, the Huffington Post, and others to offer “flash” new briefings. Users signal these audio devices to inform them of the day’s news.

Media companies use sentiment analysis, a subset of NLP, to identify opinions in social media and blogs. The sentiment analysis sorts through comments about people, brands, etc. by analyzing both positive and negative words used in online discussions.

NLP also powers the recommendation engines used by many news publishers. Story recommendations help increase both traffic and user engagement. Chat apps and bots can also be used to drive traffic. However, they are especially good at repetitive tasks such as answering specific questions and offering data alerts. A successful example is The Washington Post’s Facebook Messenger feed bot. Users ask the bot questions and responds to overall news inquiries by suggesting links to other relatable news stories.

Predictive Analysis

The second form of AI, predictive analytics, allows analysts to predict trends and behaviors based on a subset of data. Predictive models are often used to target advertising, subscription, or membership offerings. The analytics identify consumer patterns and project the potential the outcome. The Financial Times uses predictive analytics to correlate revenue to content usage and conversion rate to engagement.

Schibsted, one of the biggest news media publishers in Europe, uses predicted analytics to identity the gender of their users. Using predictive analytics, Schibsted’s accuracy of gender prediction grew from 15% – 20% to 100%. Demographic assignments are extremely important in serving personalize content and advertising. Likewise, The Weather Channel uses weather trends to help predict the optimal time for advertising. A cold front or snow storm approaching is a perfect time to advertise hot breakfast foods or batteries.

Machine Learning

The third form of AI is machine learning, which is essentially computers learning to make decisions. Computers identify patterns and apply new logic based on the results. Algorithms allow publishers to make predictions on data, including consumer usage patterns and personal preferences. The New York Times uses machine learning to help identify content for readers. Pinterest uses machine learning to identify relevant user-generated content for their users.

Personalization is a great way to utilize machine learning. These practices include recommendations of text and video, location-specific content, segment-based personalization (identifying users or specific products or a demographic, etc.) and newsletter recommendations. Of course, there are concerns that personalization bubbles leave little room for new content discovery.

Artificial Intelligence offers support and efficiency in real-time using sentiment and machine based audience analytics. It presents the news media with a way to connect with consumers and provide relevancy. Importantly, the use AI technologies has direct and positive impact on revenue and customer engagement.

Innovation | Perspectives

Less chat, more info: Facebook tries to revive messenger bots

April 20, 2017 Mark Glaser, Founder and Publisher – MediaShift

At Facebook’s annual F8 developer conference last year, executives touted the rise of chatbots. These automated personalities within the Messenger app promised to answer your queries as seamlessly as a human — or so was the expectation. But in fact, mismatched expectations regarding the purpose of the service, the lack of strong AI to back it up, and lack of awareness by the public left many industry analysts doubtful of chatbots’ ability to fulfill the hype.

With this year’s F8 conference upon us,Facebook is trying to shift the gears on this with the announcement of “group bots” within Messenger, a new Discovery tab for bots and more commands related to bots in Facebook’s virtual assistant M. But it looks like the social giant must also do some inward scrutiny — and involve marketers — to get users on board in much bigger numbers.

The Promise of M

When Facebook released M to a select group of some 10,000 California power users in late 2015, the promise of the virtual assistant was usurped by insight into how it could maintain such intelligent operations. “M is so smart because it cheats,” as MIT Technology Review’s Tom Simonite put it succinctly. Unlike other virtual assistants, which admit defeat when a question stumps them, a human being “invisibly” stepped into the role of the bot to cure the disease of not knowing — without ever letting the user know otherwise.

Of course, this is an expensive design to scale for Messenger’s 1.2 billion users. But M — along with competition from Siri and Amazon’s Alexa — inevitably laid a foundation for misrepresenting how users might realistically benefit from such a chatbot.

Siri and Alexa are trained to be generalists. However, Facebook has largely pushed developers to create more specialist bots. Of the more than 30,000 bots on Messenger, most of them serve a single purpose. That, in itself, makes creating a habit of use difficult. After all, unless you truly have an affinity for pizza, how many times will you use Domino’s Messenger bot in a single week? But Facebook also let the promise of chatbots spiral downward by not doing enough to inform and educate users on how to best use them.

Discoverability and Bugginess

That brings us to the twin disorders plaguing chatbots: discoverability and bugginess.

There was no native way on Messenger to find a chatbot, so unless a user already knew the name of a bot to type into Messenger’s search box, there was no way to actually use one. While third-party bot browsing options like Botlist exist, they are little-known and thus, little-used.

In part, that’s what Facebook hoped to address when it announced a new class of group bots to work inside Messenger at this year’s F8 conference. Once one person introduces the bot into a group conversation, everyone will know about its existence (although how to get that first user to know of the bot remains a looming question). And whereas users might have previously been misguided into thinking chatbots could be true, humanlike discussion partners, these group bots are meant to be more “information bots” (think scores and or stock prices) rather than “chat bots.” Feeding chatbots enough data to maintain conversation flow, and ensuring the AI was up to task, were hopeless hurdles that have made “bugginess” a default bot personality type.

That people have spoken little about chatbots since their initial hype may actually work in Facebook’s favor if it wants to reset people’s expectations. With little regard and much ignorance about the service (in fact, a November 2016 online survey by Publicis Groupe’s Digitas LBi found 78% of U.S. adults hadn’t even heard of chatbots), Facebook could help bring the popularity of chatbots outside the music fandom crowd into the mainstream population.

Can Payments, Marketing Pave the Way?

Kik CEO Ted Livingston, for one, argues the real benefits of chatbots come from their enabling the offline world around us, rather than functioning solely as app replacements. Paying for items is one way to do that And, if U.S. chatbots can make that territory theirs, they’d follow in the footsteps of China, where it’s common for QR codes to trigger interactions, and where more than 600 million WeChat users use its peer-to-peer payment service. While Kik is in talks with its Chinese counterparts to introduce payments, Facebook has already launched this option. Still, addressing Facebook’s buggy bot issues is necessary for users to trust making financial transactions.

Yet aside from all the hype and wonder, there is huge potential for publishers and marketers to seize control of the potential for bots — if they can seize control of the AI and have a common goal for users. Just like voice assistants, bots can get smarter over time with the right technology enabled. Another issue is that a simple question to a bot can yield multiple responses. However, if the aim is to get the user to do something specific — whether it is go to a website, make a purchase, or look up a news item — the answers need to actually be both human and robotic enough to do just that.

It’s not only Facebook that has to do some inward reflection on bots — marketers and publishers must too. They must also demand that Messenger (or any other messaging platform) be the right kind of environment for their goals to thrive. This involves much more nuanced AI programming to make brand interaction a clear-cut goal. It also requires much more education by both Facebook and marketers to let their audiences know bots exist, without setting them up for disappointment.

Innovation | Perspectives

3 ways we’ve been looking at working with algorithms at The Guardian

April 10, 2017 Martin Belam, Social & New Formats Editor—The Guardian

Publishing into algorithms

There are two sets of people you’ll find constantly moaning about the implementation of algorithms on services like Facebook, Instagram and Twitter—which focus on showing users the best stuff. Power-users, who visit so frequently that the algorithmic approach really doesn’t work for them, and social media managers, frustrated that they can’t get their content to all of their followers instantly.

I’m more relaxed about it. If we had our way, every single piece of content pumped out by our news organizations would be hogging the user’s news feed on Facebook. But think about it for a second: If they want to see what our news organization is doing, they’d come to us. They’ve come to Facebook or Instagram or Twitter to see what their friends (or possibly with Twitter, their enemies) are doing.

And we have to work out how to fit into their lives at that moment. So it has got to be the earth-shatteringly good pieces. It must be things that will surprise and delight and invoke emotion, not just the mundane routine of pushing out articles because we have a quota of space on the internet we feel we need to fill up every day.

I look at the Facebook algorithm as being a whole series of A/B tests for your content. Facebook shows it to a few people, if they interact with it, more people will see it, and so the best things you do will snowball and travel far. In many ways in an algoritmic internet, pieces get the audience they deserve.

Now let’s flip it in its head for a second. It always slightly reassures me that, thanks to algorithmic social feeds, if we publish stuff that is boring and un-engaging, fewer people see it. So people get an overall better impression of your brand, because they only see the good stuff. Now do you hate algorithmic feeds?

Working with chatbots

Over the course of a now pretty lengthy career looking at how the web and news publishing interact, I’ve worked on some things that became very huge. This includes Facebook, Twitter, getting content indexed and ranked by search engines. I’ve also worked on some things that very much fell by the wayside — Google Wave, or desktop widgets anybody?

At the moment, I suspect that chatbots are much more likely to fall into the latter category. I’m unconvinced people are going to tap out messages asking what’s happening rather than go to a news site or a news aggregating app.

However, with the rise of devices like Alexa or Google Home, we are really beginning to see many more examples of ‘conversational UI’, and that interests me. I don’t think it is too far-fetched to think that in a couple of years time my kids will just be yelling at the telly ‘Put on Scooby Doo’ and it will play it out on demand. And that I’ll be able to walk in the room, call out ‘What’s the news?’ at some (or maybe several) devices, and they’ll know the last time I checked, and update me accordingly.

In that context, what does the news published by an organization like the Guardian even look like? I suspect it isn’t 800 word articles, and our Facebook Messenger chatbot and our Alexa Skill have given us an opportunity to explore that possibility.

Three things I have learned:

It is very hard to get people to discover that you have a chatbot.
It is very hard to quickly convey to people what they are expected to do with a bot, and what commands are available to them.
People really do chat to bots as if they are living sentient beings, and the technology at the moment is really nowhere near ready to cope with that.

Working with robot colleagues

The blurb description for a panel I participated in on collaborating with algorithms at the International Journalism Festival session stated that “Some journalists approach algorithmic assistance the way one might consider hopping into a self-driving car: some are delighted, some are wary.” I’ve certainly seen the looks in the news room when we are publishing stories about how robots are going to take all our jobs that have a definite expression of “and mine too?” about them.

Unsurprisingly, given the future-facing aspect of my role at the Guardian, I am perhaps more enthusiastic about the prospect than some others in the building. I’m not, at the moment, concerned that an AI is going to be adept enough to start writing 800 word opinion pieces with my authentic voice. My colleague Alex Hern has experimented in that area for one of our hack days, and the results didn’t look that promising. And while I would for sure read some of the articles that Clickbait Robot suggests, at the moment it is only generating headlines, not the actual stories.

But I think computers absolutely have a role to play now in helping us to write the first drafts in several areas. Imagine a bot that automatically ingests stock market movements, or sports data, or the latest YouGov opinion polls. A computer can instantly see “This is the biggest shift away from supporting Labour among the over-65s for a decade” or “Sunderland are on their longest run of not scoring since a similar spell in 1956” or “While the FTSE on the whole remained high, stocks in the mining sector collectively fell by 12% yesterday”.

Those may not be stories in themselves, but they might be the inspiration required jog to a journalist to phone up a source in the mining sector to get the lowdown on what is happening, or to try and see if anybody from that 1956 squad is still alive and available to interview, or give them a unique angle on a story that everybody else will be reporting in a broadly similar way. And sure, someone who knows their beat very well might be able to just look at those figures and instantly have all the relevant comparative data at her fingertips. But for most journalists, I reckon a robot saving them time and chucking out potential story ideas and angles would soon become a very useful little friend.

So what I’m interested in, is not whether algorithms can write my articles, but whether a combination of 70% algorithm grunt-work and 30% human inspiration, can produce stories that people are more engaged with and more likely to want to read.

Want to learn more? Watch Martin discuss this topic further at the April 2017 International Journalism Festival

Martin Belam is a designer, journalist, and Social & New Formats Editor at the Guardian. Martin has spent nearly twenty years building digital products for the likes of the BBC, Daily Mirror and Sony.

Perspectives