The current issue of Against the Grain (July 5 2024) contains two contrasting pieces about the use of generative AI for scholarly discovery. They are interesting not just for their conclusions, positive and negative, but also for their approach.
Traditionalists see the library at the centre of the discovery process – after all, the library is where you once started your literature review. This tradition dies hard. Gary Price (An Interview with Gary Price) is clear from the outset: he doesn’t trust generative AI to do research, because the content is so unreliable. He refers fondly to the Guide to Library Research Methods, now the Oxford Guide to Library Research, first published in 1987, now in its fourth edition, written by a Library of Congress librarian, Thomas Mann. This guide gives detailed lists of recommended subject encyclopedias, to which, in an ideal world, you would turn for information on most subjects. There are all kinds of problems with this model, and perhaps the biggest problem is availability. Except for the hypothetical perfect public library with an infinite budget, all these encyclopedias are only available to subscribers, and it is very unlikely they will have found their way into any major LLM available today; the one exception is, of course, Wikipedia. The wording in the Guide to Library Research Methods is very quaint: “None of these sources is freely available on the open Internet (sites that are available to anyone, anywhere, without requiring a password or credit card).”
Lucy Lu Wang, in contrast, gives a clear overview of generative AI tools and, fascinatingly, suggests a possible way forward that involves the library. Briefly, her argument is thus: many of the problems with generative AI arise from the unthinking use of the corpus the system uses to provide its answers. Some of this can be mitigated by using a human-in-the-loop, for example by curating and providing additional prompts, the RAG, retrieval-augmented generation principle. In this article, Wang describes it as RLHF, “reinforcement learning with human feedback”, to ensure the answers provided by the system align with human values and expectations.
But the real innovation in her article suggests a potential role for the library. She describes libraries as “community organizations with deep knowledge of their patrons and their needs”. She notes the existence of subscription-restricted content, such as subject encyclopedias, and suggests that perhaps the library could create a local collection of relevant content: “local instantiations of LLMs could be hooked up to what is institutionally accessible as an additional and unique-to-that-community source of data for retrieval augmentation.”
It’s a great idea, perhaps a little akin to the way that some reference titles, such as the Dictionary of National Biography, are available in many public libraries in digital form via a licensing agreement. But providing the DNB keeps the reader as a passive consumer. Lucy Wang’s essay suggests one step further: using locally generated content, perhaps, to create a resource. Why not a mini-Wikipedia relating to one’s home town? As Wang points out, building smaller, subject-specific corpora can provide equal or better results than a general-purpose generative AI LLM.
Of course, the truth is that both arguments have some justice on their side. Both parties agree that providing provenance would dramatically improve the authority of the results, because humans use provenance as one factor in evaluation. This brings us back to all those specialized encyclopedias described by Thomas Mann above – he states that Library of Congress has catalogued over 8,700 print-based subject encyclopedias since 2000, presumably to the date his book was last updated in 2015 – that’s more than one reference work published every day! Could the library assist in creating a corpus for its users? Mann’s suggested solution to paid access is to provide free access only within the library, but libraries don’t have an infinite budget, and I think the idea of restricting access to a specific location is already superseded by the principle of universal distant access: any member of the library, in fact any member of any library, should have access, physically or online. That would be quite some idea, but, alas, the world does not appear to be moving closer towards the universal digital library (as Google tried and failed to achieve), and as for public lending right, the world seems to be questioning whether such a right should exist for digital content (see the recent victory of publishers over the Internet Archive).
In the meantime, all that scholarly, peer-reviewed resource remains locked away, behind the paywall, while the generative AI companies continue to build larger and larger corpora of content scraped from the Web.
Leave a Reply