Is Google better for research than Google Scholar?

Sometimes an academic article delivers results that are so much at odds with your common sense that you immediately try to see holes in the argument.

Take, for example, an article in the Journal of Academic Librarianship, published this month, by two academic librarians, reaching the surprising conclusion that Google “outperformed Google Scholar in search effectiveness and content coverage”. Such a result is astonishing, given that the largest survey of researchers found that researchers typically started their search with Google Scholar, not with Google or other discovery tool.

For this article, the authors selected nine searches and ran them across four search tools, measuring results for precision and retrieval. However, I have some doubts about their methodology. Although their methodology seems sound, the results they obtain seem at variance with what you can see on the screen when you try to replicate their results.

The first of their searches was to enter the three terms “academic resource searching” (not a search I would have chosen for a research project, but never mind) and to compare the results. Above and below are the first results on Google and Google Scholar (they only looked at the first ten, and I’m only showing the first few hits).

It is immediately apparent that Google Scholar provides links to scholarly articles – which is what researchers want to find. Google, on the other hand, provides a wide range of results about the subject, but not, primarily, scholarly research articles. If you try hard, by ignoring the first result of “JSTOR”, irrelevant to scholars for this search, it might perhaps be possible to look at only the scholarly article hits from Google – but you would have to wade through many results that are not articles, and that takes time. A detailed examination of the results between the two search engines might suggest that Google delivers more relevant documents, but the time taken to find those documents on Google will considerably greater than the time taken to find them using Google Scholar.

Aaron Tay points out in a post, and I’m sure he is correct, that Google Scholar has not taken on board some of the improvements in search implemented on the main Google platform over the years, particularly some of the semantic expansion capabilities now seen on Google. But that does not appear to have led to a move away from Google Scholar to Google.

So, despite the criteria used by Pulikowski and Matysek, the difference in appearance of the results makes it obvious which resource will be chosen – and why, despite this article, Google Scholar will continue to be chosen as the first search tool for academics. There are some scholarly practices where, even if you show scholars the evidence, they won’t change their mind – and in this case, with good reason.