Reading Time: 3 minutes

This year’s annual European Conference on Information Retrieval (Cologne, April 2019) was remarkable for me in that it showed a dramatic renaissance of the world of search. Conferences labelled “enterprise search” have been running down or closing in the UK and the US for several years, indicating a lack of interest in the subject of search. To the best of my knowledge, not one person at this event mentioned the word “Boolean”; if it was mentioned, it was pretty incidental. .

For several years, the problem of search has been not so much a lack of interest as the inability to do anything about search. Information professionals might complain, but today the vast majority of searching is done via Google. The Google search engine has developed dramatically over several years, and includes some impressive AI tools behind the scenes, but given its monopoly situation, and the fact that the vast majority of Google search users are either very accepting or very uninterested in questioning or improving their search experience, it has almost stopped being a subject for study; why study something you can’t do anything about? For many years, the indefatigable Karen Blakeman has been running courses about the latest developments in Google search; I attended one of her events and I was full of admiration for the things she had discovered. Unfortunately, most users of Google remain unaware that search experience is changing regularly. But the fundamental challenge of Google search is that it seems to have developed little from the earliest search engines. When I was publishing online encyclopedias, we found that users, typically US high-school and college students, keyed in one word. If they didn’t find what they wanted, they started again, keying in a different word. Searching doesn’t seem to have progressed much since then; it is still the case that presented with a search box, most users go for the quickest possible result.

This conference, however, told a very different, and encouraging, story. Despite that minimal Google interface, the world of information retrieval is expanding dramatically towards looking at specific use cases and domains, and in some areas, notably ecommerce, a whole subject has grown up that is specific to this area, and with fundamental different characterstics to the standard Google search. A couple of simple examples will suffice to show how distinctive ecommerce searching is. Or Levi of eBay pointed out that expanding searches to include related terms might not be right for ecommerce at all: try searching for “monarch bed” when you want to buy a “king bed”.  

Two speakers from the German online department store Otto gave another very good example. Otto sells clothes, furniture, and a wide range of goods. If you go to the Otto site and search for “TV” you get 4,000 hits, even though they only sell around 400 different TVs. The remaining 3,600 hits are mentions of “TV” that are mainly irrelevant, such as “TV-advertised” clothing. If only users entered the site via the facets and the subject-based hierarchy, then Otto could immediately lead them to a page with all the relevant options for types of TV set, such as wide-screen, or Smart TV, but of course users almost always opt for the simplest option on screen. Trying to reduce the number of clicks before the user gets to the items they want is the classic problem of ecommerce; if you don’t know the word “king” means a type of bed, your search will involve many clicks. To a large extent, Amazon is doomed to providing a poor search experience because it has to guess what users are looking for.

Along with ecommerce talks, there were excellent talks by Diego Ceccarelli of Bloomberg mentioning the new (for me) technique of learning to rank, or LTR, using some AI tools to improve the relevance of search results – also mentioned by Or Levi; I enjoyed as well an excellent case study of a radiology app, which was revealing for two things. First, it took this startup company over a year to understand what the users wanted. It sounds incredible, but I can believe it. Secondly, the presenter, Allen Hanbury, stated something fundamental about the information retrieval business: “Radiologists are professional searchers, although they don’t realise it”. While a handful of information specialists continue to use highly refined Boolean tools to carry out literature searches, the rest of the world have become professional searchers, but hardly reflect on what they do. That is the challenge of present-day information retrieval.