The JISC Explore AI website

Here was a webinar with a crucial difference: in addition to a presentation of a topic, delegates had the chance to try some hands-on interactive tools for themselves, in addition to being given a guided journey through the building of an actual AI project.

Led by Andrew Cox and Suvodeep Mazumdar of the University of Sheffield, this UKeiG course, Artificial Intelligence for Information Professionals,  comprised an overview of AI, a review of factors involved in building a chatbot application, plus a demo of building a simple tool using AI. Taking all the reviews from a well-known book site (Goodreads), Suvodeep showed how the data from the site could be held in a spreadsheet and then, by use of a training set, used to predict what genre a submission should be.

Inevitably, much of the half-day session was introductory. Before getting to the nitty gritty, Andrew Cox presented a fascinating set of no fewer than five definitions of AI. How AI should be understood depends on your role and background: it can range from tools for information professionals in a library (managing low-level tasks such as a chatbot for interlibrary loans) to an agent for global domination by a handful of (mainly US-based) corporations (the theme of an article and book by Kate Crawford). The remarkable thing is that all the definitions can be valid. Suvodeep, not to be outdone, added a page full of further definitions of AI later in the session.

The course continued with a presentation of what to consider when building a simple chatbot app. Here were listed so many criteria and issues about chatbots that it made me think it might be better not to start at all.  Andrew Cox listed 8 “prompts”, but each of those had several questions, so there were around 25 things to take into consideration. But I appreciate that effective AI requires an awareness of context, so the considerations are relevant.

Suvodeer’s  wide-ranging overview of AI approached the subject from a more technical viewpoint. Some of his slides were very good at summarising visually the main flavours of AI, for example, comparing three of the most common methodologies: classification, regression and clustering.

How classification, regression and clustering can be depicted

The next part of the course was, for me, where things really came to life: some hands-on tools. This was a tour of some live examples of AI functionality available at the excellent newly launched JISC site, JISC Explore AI, and open to all.  That many of these tools were prototypes and proofs of concepts seemed to be borne out by his uploading an image of a building with trees in front of it, which the system confidently stated was a house with a bench. Looking at another image recognition tool, the system responded it was 92.5% certain that the image was not of an insect:

An AI-based check of an image for the presence of insects. No insects here!

All participants were encouraged to try out the tools for themselves, and I couldn’t help noticing that the first site I looked at, about film reviews, seemed to have some rather unexpected inferences. One of the sample reviews included the words “it absolutely hooks you … and has you glued to the screen”. That sounds pretty positive to me, but the tool indicated the phrase was just “neutral”.

AI check for positive or negative sentiment in reviews

These examples are not included here to suggest that AI tools simply don’t work.  They reveal, instead, the importance of context – the tool that stated a building was not an insect was doing exactly what it had been set up to do: this was a test to identify insects in an image, and clearly the image is not of an insect. It may seem obvious, but one of the lessons from this course is that AI tools have to be used for the purpose for which they were intended. JISC Explore AI is a fascinating site, and strongly recommended for anyone keen to see what tools are available.

The highlight of this course, however, was a walkthrough of actually building an AI tool from scratch. In less than 40 minutes, Suvodeep Mazumdar showed how a genre prediction model could be built from a website. Using a well-known collection of book reviews, all of which had been manually tagged by genre, he demonstrated how to prepare the text, then how the training set was identified, and finally the system, once trained, predicted the genre of new reviews. Existing, manually-tagged reviews were used for the test set, which means the success of the tool could be measured by comparing the machine results with human tagging: the score achieved was around 0.52 (not a very good result, incidentally, but unimportant for the purpose of this demo). Of course, there was insufficient time to go through all the details of the process, but a couple of things stood out:

  • It took more time, and more stages, to prepare the data than to run the AI tools.
  • The corpus used had some major bias, simply because the set of reviews comprised mainly fiction and fantasy titles, while other categories made up only a few percent of the whole. A better dataset would have had a more balanced corpus.

It was in the discussion at the end of this fascinating demonstration that for me the real issues emerged. As Andrew Cox mentioned, libraries are known for their skills at curating and managing metadata, so how could machine-created metadata be used alongside human-generated metadata of a higher quality? Would it compromise the results? In addition, information professionals in libraries have good skills at managing metadata and in data manipulation, so potentially they could play an important role in building these AI tools.

The session ended with a key question: where do you position yourself in relation to AI? Should all information professionals become coders? Should they create and manage the training data? Or is their role more in developing a vision for AI, and managing responsible AI projects? Whatever answer is chosen, it looks clear there is a vital role for information professional in the use and development of these AI tools.