The Influence of Search Result Listings (Captions) on Clickthroughs

You type in a query in a search box at Google or Yahoo or live.com or Ask, and hit the search button.

In the search results, you see lists of links to pages that should have something to do with the keyword phrase that you typed into the search engine. Which do you click upon? How might the words in the caption - the title, snippet and URL - influence what people will click upon?

That's the question raised in Microsoft's The Influence of Caption Features on Clickthrough Patterns in Web Search.

I really like it when the search engines share some of their findings on usability issues around the way people search. Here's the quick answer, from the paper's abstract:

The findings of our study suggest that relatively simple caption features such as the presence of all terms query terms, the readability of the snippet, and the length of the URL shown in the caption, can significantly influence users' Web search behavior.

Skipping to the quick answer misses of the other questions and observations raised in the paper, such as potential differences based upon whether the search is informational or navigational in nature. There's also some nice discussion of how captions where compared.

Choices of snippets displayed by the search engine may also play a role in which link is chosen by a searcher. When a link is redirected, the search engine may choose to display the original URL, or the redirected URL. A title or description (as snippet) from a Web directory such as the DMOZ might be shown instead of information found on the page listed.

The paper provides a nice rundown on some of the previous research involving document summarization used in search engines, including a review of captions that depend upon the query searched for, and those provided where the summary doesn't depend upon queries.

The study doesn't involve a small pool of users or a comparison of results selected through methods like eye-tracking, but instead explores user logs from the search engines, and compares clickthroughs of pairs of results that are adjacent to each other in search result lists.

This finding is also interesting:

The results support claims that missing snippets, short snippets, missing query terms and complex URLs negatively impact clickthroughs. While this outcome may not be surprising, we are aware of no other work that can provide support for claims of this type in the context of a commercial Web search engine.

Search engineers do make a number of decisions when they present title and snippet and URL for a page in search result sets, such as the length of title that they will show (long titles will get truncated if they are too long), which snippet they choose when keyword phrases appear more than once on a page, and which URL to show when there is a redirect in place. It's good to see that they are performing user testing on what they do show.

Originally published here