What researchers and students actually do

The UKSG Forum (Birmingham, 5 December 2024) provided some remarkably informative sessions about AI, from the perspective from the library, as well as from researchers. As often happens at library conferences, the most insightful sessions were from smaller institutions providing a case study, revealing what their students actually do. In the same event, we heard from a much larger, more rigorous study of academics and their use of AI – and I was more inspired by the “from the chalk face” experience of inner-city institutions than to hear about what researchers are doing.

The real eye-opener on generative AI was a presentation by Dr Janice Fernandes of University of West London. With a PhD in information science, and over 25 papers to her credit, she clearly combines library work with her research, yet the focus here was clearly practical. She shared insights she has gathered from using genAI, and from comparing it with standard search engines. As she states, the wheel is coming full circle – initially there were only search engines. Next came genAI interfaces, and now search engines are are adding genAI elements, while at the same time genAI is busy adding search interfaces. Nonetheless, there are some fundamental differences in their approach.

As with all the best presentations, this one had me going back to the two systems to compare them. One thing I noticed was that a standard search engine like Google Search provides multiple hits, each with attributions, for example, a search for a recipe will result in several recipes by named chefs. In contrast, gen AI, asked for a recipe, will by default give a generic recipe and even if asked, the system will be unwilling to attribute this recipe to any individual (most likely for fear of infringing intellectual property). This isn’t necessarily better or worse, but as a user, you now have two choices.

Dr Fernandes also pointed to some fascinating research from the Higher Education Policy Institute (HEPI). This was a 2024 survey by Josh Freeman of students’ attitudes to AI. 1250 UK undergraduates was asked about their use of AI, and attitudes towards it. For me, the notable statistic was how students have rapidly found a reliable use of gen AI, and have steered clear of more controversial (and less controlled use of the tool: 66% of undergraduates thought it acceptable to use gen AI to explain concepts, while 53% have used gen AI to help them with assessments. However, only 5% put AI-generated text into assessments without editing.

Another interesting snippet is that 73% expect to use AI after they finish their studies, mostly for translation and for summarising. Equally remarkable was Dr Fernandes’ summary of prompt engineering strategies, including:

• zero-shot vs one/multiple-shot
• “chain-of thought” – break down the query step by step
• iterative prompting – asking the same question multiple times, each time fine-tuning the query. 
• role playing (“Imagine you are a film star…”)

This presentation provided excellent guidance for users. It was reassuring to hear how at one institution, the students are engaging with AI and new learning skills. That wasn’t the message I gathered from a large-scale study by Research Consulting investigating current academic practice. A survey of no fewer than 11,000 researchers found that less than half had ever posted a preprint as part of their publishing workflow, and less than half supported open peer review. In other words, the widely discussed OA workflow “publish, review, curate” model is a long way from common practice. It’s worth remembering the conservativism of many researchers, especially in humanities and social sciences.

Another great case study was from Trinity St Davids University, Birmingham Campus. The two presenters, Olivia Edmonds and Tarandeep Johal, described their students: all over 30, 95% from an ethnic minority background, and 50% of them travelling to the college to study – in other words, not the easiest of backgrounds for study. Here again, the two librarians provide tools and digital skills for making use of gen AI, including certificates in using Microsoft Office, and digital skills badges. In an environment where they work, they reason the key thing is to help the students make effective use of whatever tools are available. And when it comes to helping the students get a job, let’s try every possible means!