Digital media offers publishers new opportunities to better understand audience engagement. Using post-click measures within an article page of the client-side logging system provides insights into audience preferences and retention. Nir Grinberg, a research fellow at the Harvard Institute for Quantitative Social Science together with the Northeastern’s Lazer Lab, reports on new post-click user engagement measures. His report, Identifying Modes of User Engagement with Online News and Their Relationship to Information Gain in Text, identifies metrics that will help publishers anticipate how long a reader will stay with an article and improve the recommendation process for new content.

Grinberg’s analyzes 7.7 million-page views from Chartbeat data on seven different news publications with articles across finance, how to, magazine (longer-form), science, sports, technology, and women.

Grinberg enlists three core metrics in his study:

1. dwell time (an estimate of the total time a user spends on a page)
2. scroll depth (the furthest position the user reaches on the page)
3. page interaction (the amount of interaction with the page through any form of input such as touch, mouse move, etc.).

His first step is to project the modes of behavior present in the individual page views. Then, he uses this information to predict the common behaviors of a larger audience. Identifying the modes can enable publishers and recommendation system to distinguish between different forms of reading, scanning and other lighter forms of engagement.

More Metrics

While Grinberg valued the original three metrics, he finds the information they provide limiting. His next step combines and correlates the original metrics with new measures. For example, he analyzes the proportion of an article that is visible on a user’s screen (relative depth) with how quickly a user scrolls through the visible part of the page (average scrolling speed).

Combining these two measures provides information about an overall navigation experience throughout an article. This information also allows recommendations systems to distinguish between audiences reading, scanning and other lighter forms of engagement. Based on these findings, Grinberg’s classifies reading behavior into five types: “Scan,” “Read,” “Read (long),” “Idle,” and “Shallow” (plus bounce backs for users who get to a page and almost immediately leave). This offers insight into the variety of reading behaviors and the level of engagement present when a user is consuming content.

As an example, the chart above reflects the relationship between different reading behaviors and their levels of engagement. The top right panel shows that the median article on a sports site has about 28% of page views that are scanning the article. The percentage is considerably higher than the rate of scanning in the other content categories. This suggests that people consume sports articles with a different intent, perhaps simply looking for quick games scores and/or result of a sporting event. This data helps characterize the likely user behaviors and intent of the page view and factors into content recommendations.

Information Gain

In addition to user behaviors, Grinberg identifies a measure that he calls “Semantic Information Gain” (SIG), the flow of information on a page. SIG captures how quickly an article moves toward its final point, It can be very helpful for publishers to use to see if or where a reader gets lost in an article. SIG can be highly predictive of audience engagement.

Practical and informative measures of post-click user engagement can improve recommendations of news content and enable more informed editorial decisions. Distinguishing between different modes of engagement with an article, such as scan, skim, or in-depth reading, can enable recommendation systems to better match articles with potential readers based on their engagement profile.