One of the most impressive capabilities of genAI is the ability to summarise and to define terms. Increasingly, I, like many users, ask natural-language questions of AI and I often find the answers valuable. One reason why the results are good is that I am asking the AI to do something for which it has ample evidence: all I’m looking for is to summarise what the literature says about a concept. This is a long way from hallucinations and creating spurious references.
To check this further, I compared genAI (in the form of the Google AI Overview) with some other reference sources, specifically, Wikipedia and one or two subject-specialist dictionaries.
This exercise came about because several commentators noted Elon Musk’s launch of an alternative to Wikipedia, complaining that Wikipedia has a liberal bias. For me, much more important than such a bias, which would be difficult to quantify, is a rather different, and I think more fundamental problem: many entries, and, specifically, many definitions, are becoming too complicated for the average reader.
In this post I will concentrate on the definitions, which, after all, are the main use of an encyclopedia. In another post I will look at some entries overall. Some years ago I suggested a number of rules for writing reference content, and I was intrigued to see to what extent genAI complied with those principles.
What is the problem?
The initial definition is essentially the first sentence of the article.
A good example of a Wikipedia article is stroke. The article begins with three sentences, all of them immediately intelligible:
But many Wikipedia articles contain technical terms without explanation. This may be explained, at least in part, by the practice of higher education institutes getting their students to edit Wikipedia content. But the result is sometimes a more technical presentation than the average reader requires. It might be correct, but it’s not very helpful.
One technique that many reference works uses is to make wide use of links. Wikipedia will no doubt suggest that technical terms, if included as hyperlinks, can be explained by the user hovering the cursor over the term and seeing an explanation of that term, but I think this practice introduces unnecessary complexity. A term should be defined in words that are no more technical than the term itself.
The thinking behind lots of links is that the reader can always go to the link if they wish. But one of the main developments of UX design ever since the Internet began is to try to reduce the number of clicks required by the user (even if UX design no longer insists on fewer clicks as the absolute goal). In this case, at least, more clicks are certainly not required.
Here is an example of lazy linking: there is no way the reader is likely to understand the term used, so they are forced to look at something else. This is the Wikipedia entry for Buddhism:
… It [Buddhism] arose in the Eastern Gangetic Plain as a sramana movement in the 5th century BCE.
What is a sramana movement? If you hover the cursor, “sramana” is defined as a type of person, not a movement.
Definitions too technical and complicated
Terms should be defined using words no more technical than the term itself (see the principles of reference definition). This is of course a rough-and-ready rule, but it means that you don’t overload the initial description with words that the user will probably have to look up. I could define a horse as a quadruped, but it would be better for most users if Ii defined it as having four legs. The person looking up the defintion of Buddhism is very unlikely to know what a “sramana movement” is
This problem is not restricted to Wikipedia. The Oxford Dictionary of Geology (Allaby), for example, defines basalt as follows:
basalt A dark-coloured, fine-grained, *extrusive, *igneous rock composed of *plagioclase feldspar, *pyroxene, and *magnetite, with or without *olivine, and containing not more than 53 wt. % SiO2. Many basalts contain *phenocrysts of olivine, plagioclase feldspar and pyroxene.
The problem is when you look up a related term such as “extrusive”, it is defined in the same dictionary as follows:
extrusive Applied to all ejected material of volcanic origin. The sense applies to *lavas and flows, rather than to *pyroclastic rocks.
Wikipedia’s entry for basalt has similar problems:
basalt an aphanitic (fine-grained) extrusive igneous rock formed from the rapid cooling of low-viscosity lava rich in magnesium and iron (mafic lava) exposed at or very near the surface of a rocky planet or moon. More than 90% of all volcanic rock on Earth is basalt.
- Poor: the definition is too complex and requires too much looking up to be understood.
- Poor: too many links: That’s 13 links in 45 words!
- Good: the term aphanitic is both linked but also glossed, so intelligible in context (fine-grained).
The same problem applies when you click on links in Wikipedia, for example, extrusive, which actually links to an entry for “extrusive rock”:
Extrusive rock refers to the mode of igneous volcanic rock formation in which hot magma from inside the Earth flows out (extrudes) onto the surface as lava or explodes violently into the atmosphere to fall back as pyroclastics or tuff.[1] In contrast, intrusive rock refers to rocks formed by magma which cools below the surface.[2]
In this case, the cross reference doesn’t quite apply. Better would be to use the term extrusive with the gloss “ejected into the atmosphere”.
Let’s compare this with the Google AI Overview version:
Basalt is a dark, fine-grained, extrusive igneous rock formed from rapidly cooled, magnesium- and iron-rich lava, making it the most common volcanic rock on Earth, forming the ocean floor and shield volcanoes.
This is quite remarkable. Although this is a survey of just one term, Google’s AI overview provides the best of the three definitions of this word – it’s certainly more readable. It could be further improved, but it’s an improvement on what humans have written. Here is my suggestion for an initial definition of basalt, which I fully acknowledge draws on the genAI version (in other words, AI-informed human editing):
basalt a dark-coloured and fine-grained, rock, the most common type of rock that is extruded (released into the air) from volcanoes. More than 90% of all volcanic rock on Earth is basalt.
I’m not saying that Wikipedia should not contain all the technical terms, but that those terms should be placed further down the entry, following the “onion-skin” technique: define the term simply, for those who know least, then define it again in more technical terms, then again in more technical terms, expanding if necessary as appropriate for the concept. I can’t claim any originality for this technique; it was in use by World Book Encyclopedia over 50 years ago, and was probably in use before then. But if something works well, why not continue to use it? And if genAI defines terms so well, perhaps we should use it when drafting any new reference content?
Leave a Reply