The Associated Press (AP) has launched an innovative artificial intelligence (AI)-powered search experience, which it says will enhance the efficiency and accuracy of content discovery for AP Newsroom users.

The global news agency believes that the new search functionality represents a significant shift in the discoverability of photos and videos that align with users’ search criteria. Unlike metadata-based searches, the AI-powered tool is able to understand descriptive language and generates search results based on the user’s description. It can also identify specific moments within an entire video clip, regardless of its length. 

This opens up new possibilities for users, allowing them to easily pinpoint precise moments of interest in videos, even if they haven’t been tagged or captioned. 

Dramatically improved results

Traditional searches for visual content depend entirely on descriptive information, which means many visual assets are difficult to find. 

“AP’s visual archives stretch back to the beginnings of photography, comprising tens of millions of photos and videos” explains Paul Caluori, the AP vice president of global products. “Across that history and volume, the amount and kind of descriptive information varies widely, and sometimes information is not as complete as we’d like it to be.”  

However, the AP’s new AI-enabled search is able to recognize elements within photos and videos and “understand” more specific search terms and concepts to find those elements. 

“This yields some dramatically improved search results and we are very excited to make this available to our customers,” says Caluori. “We believe it will speed their searching time, get them more useful results and ultimately help them be more successful with their projects.” 

The search engine driving the new technology is NOMAD™, developed by MerlinOne, a software company specializing in AI applications for visual objects. NOMAD™ is the result of four years of in-house advanced AI effort, and the developers say it is the first such tool that understands natural language and concepts.

AI pioneer

This is far from the first time AP has used AI. It was one of the pioneering news organizations to leverage this technology back in 2014 and issued a guide to using AI in the newsroom in 2017. At the AP, the Business News desk took the lead in the use of AI to automate articles covering corporate earnings and sports. This undertaking not only enabled the brand to to experiment with new projects, it positioned AP as a thought leader in the AI space, inspiring other news organizations to embrace the technology.

So, while generative AI has captured much of the limelight since ChatGPT’s 2022 introduction, NOMAD™ is a purely visual AI search technology that builds upon the APs long standing use of AI. 

Addressing common concerns

With AI moving so quickly, it is easy to anticipate that this technology will become the new normal for other agencies in the future. But what of the negative impact of AI-powered search engines, in terms of job losses? As media mogul Barry Diller recently told Time magazine, artificial intelligence (AI) could be as “destructive” to news publishers as free online news was in the early aughts. 

However, unlike the threat of Generative AI, Caluori states that when it comes to their new search engine, “nobody is losing a job over this.”

“Our journalists search our archives every day and this tool will make it easier for them; it improves the usefulness of our archives to people both inside and outside the AP.”

Another concern around the use of AI is algorithmic bias. Consumer Reports has just released a series of videos, called BAD INPUT, in partnership with the Kapor Foundation which explore the biases present in algorithms and data sets, and the harm they can cause, particularly within communities of color.

David Tenenbaum, CEO of MerlinOne, has led their machine learning efforts for the last seven years. He explains that issues of bias center around the content sets used to train the model. If they are small and exclude crucial data (in this case faces representing all genders and races in equal proportions) bias is certain. On the other hand, if you use a huge training set, reflective of our world, the probability of bias is greatly diminished.      

“Recognizing the danger that bias can creep into any system that uses a small, unbalanced training set, NOMAD was trained on over a billion images in a collection that is highly reflective of the world we all live in,” says Tenebaum. “No AI system will ever be perfect, but with millions of searches done, we have not had a single complaint about biased results.”

New normal

AP initially  launched the service in a beta mode on its self-serve site, for customers who have access to its ecommerce service for photos and video. The AI search facility is now being rolled out to all customers over the next couple of weeks.

With an ever-expanding archive that adds new content every minute from around the globe, AP’s visual collection consists of over 60 million photos, catering to the needs of professional image buyers and commercial customers alike. The enhanced search tool is designed to make much more of this content discoverable. With NOMAD™ for Video, editors can find a short clip in just a few seconds, freeing up time to focus on high-impact journalism.

Additionally, new ecommerce capabilities improve the AP Newsroom experience for non-subscription customers. With streamlined self-serve licensing and pricing models, users can easily license images and video, including for ad hoc usage.

Caluori described the AI-powered search as a “real sea change” in content exploration and discovery within the AP’s vast archives. 

“Want to see a fragment of Winston Churchill in a garden feeding birds? You can find something that specific using natural language, rather than typical one or two keyword searches that yield many unuseful results. The ability for customers to find very specific moments within a video or just the right photo is a powerful key to our archives.”