How do you communicate, or encourage, technical innovation in academic publishing? This is something the STM Association has been grappling with for years. Innovation is a good thing, but can you inculcate it? Each year, the STM Innovation Day tries a slightly different format, so it’s not entirely surprising that this year’s event was unlike the previous year, but not surprising either that it will probably change again next year.
The format for this year was simple. There was a keynote presentation on generative AI (because, let’s face it, you can’t have a conference about innovation without mentioning ChatGPT today), followed by presentations by many startups – no fewer than 16 presentations, to be precise, and that was just the morning. In the afternoon, there was a panel session, followed by a further set of (unrelated) presentations, this time five startups that had been shortlisted for the Karger-sponsored Vesalius Award. No connection between the startups in the morning and the startups in the afternoon. The day ended with a round-up – but I don’t think there were many conclusions from the 21 examples we had seen (at least, I didn’t hear anything of note). The only (and valiant) attempt to draw a general message from looking at all the startups was by Jignesh Bhate, founder and MD of Molecular Connections, who proposed three rules for any startup to follow (based on his own experience):
- Keep innovating.
- “Guard your moat”, meaning, define your competitive advantage and guard it.
- Perseverance: Be persistent.
None of this can have been exactly news to the participants, but on reflection, several of them might have benefited from reviewing their service or product and deciding if they had any competitive advantage in the first place. Some of them didn’t have much of a private moat to guard.
So, can we make some sense of the day? Were there many insights to communicate?
The keynote, by Elena Simperl of Kings College London, included a few interesting points, but seemed mainly to comprise a description of the several projects she was involved in. There were phrases such as “data is the most undervalued aspect of AI today”, but no explanation of why that should be so. Equally, there were some startling insights, such as that Stack Overflow traffic has declined by 50% since ChatGPT was launched, but it wasn’t clear to me why generative AI should reduce the need for forums and conversations between peers; my experience of using generative AI has been an urgent need to ask other people about their experiences, and to compare notes with them. There was a mention of the benefit of human-AI interaction, with a screenshot from Microsoft, but I didn’t get an idea of how this human-AI interaction would work; there is little interaction in the generative AI interfaces I have seen to date. To be fair, one or two of the startups mentioned a human link in the chain.
The presentations by the 16 startups, with just two minutes per presentation, left me scratching my head, although I did notice that some of the so-called startups seemed to be mature companies that had been around for many years. By these criteria, I am almost a startup myself.
One way to assess the 16 participatns was to see how many of them were doing the same thing as other entrants. Last year, the most common theme among the startups was image manipulation checking; this year, it was submissions tools for publishers: to be precise, no fewer than six of the 16 presentations looked to be addressing a very similar market. However good each of these six might be, they have the challenge to create a market and at the same time differentiate themselves from their competitors. Hence, in a beauty parade with many entrants, companies that did only one thing looked at least more distinctive and more memorable, and probably more likely to survive for at least twelve months. Examples included Appetence, who do repository identifier checking, and Visual Abstract, who produce visual representations of an article.
Given only two minutes, some of the startups didn’t really communicate in the time available what they were about (Hum). Others described what they did but the solution seemed very niche (Kriyadocs). In contrast, potentially game-changing services that offered an entire submissions workflow (Molecular Connections, Morressier, MPS) had so little time to describe what they did that the audience were left little the wiser at the end.
Several of the presentations stated rather defensively that they weren’t using generative AI tools – and so they didn’t have the problem of hallucinations. However, since half the world is using Chat-GPT and similar tools, the problem of hallucinations doesn’t seem to be worrying too many users. That sounded like a dangerous argument for defending why you don’t use generative AI yourself.
Not being much the wiser after all the startup lightning sessions, I looked to the panel discussion in the afternoon to get some new insights – and insights there were. First, Avi Staiman described how he runs a free online course on AI tools for research, which sounds like an excellent initiative. As for the STM Association, they have created some “Guidelines for the use of Generative AI”, which sounded exciting (even if probably in response to the threat of government intervention in the use of generative AI). Yet the Guideline statements they provided were not exactly new:
- Authors are responsible for content.
- Peer review should be done by humans.
- Generative AI tools should provide references and sources.
- Publishers could create their own LLMs.
Apart from the last, all three principles are, I think, generally agreed. What might be more interesting to consider is why the generative AI companies produced the black-box tools they did, providing answers and advice but ignoring attribution and provenance, and even accuracy. The focus is to provide an answer at all times – not dissimilar, in these respects, to the Google search engine.
There was some interesting discussion about whether we should publicly reveal generative AI as a source of our content when we write an article, but it’s true that we don’t, as a rule, bother about mentioning Google search as one of our sources, even though it plays a significant part in the scholarly workflow.
At this point, there was a potentially interesting live poll of the audience to see which of eight topics lent themselves most to the use of generative AI tools, but the results were hardly conclusive. All eight topics scored fairly similar results. But perhaps asking end users where the most impact will be is never going to produce the hoped-for results.
So, still little the wiser, we headed to the main afternoon session: the selection of the winner for the Vesalius Innovation Award. Each of the five finalists had the luxury of a five-minute presentation(!). Yet here again, I didn’t find it easy to draw any conclusions, or even to spot a winner. Of the five finalists, two of them (and one was the competition winner) were medical applications with, as far as I could see, no publishing relevance. I’m sure the medical devices were excellent examples of their kind, but how could they be compared with a peer reviewer finder, or an educational medical knowledge quiz? The winner, Pipra, provided “pre-interventional preventive risk assessments”, but I don’t imagine any of the audience taking insights from this product back to their daily academic practice in the coming week.
So, did we learn about innovation? You recognise innovation when you see it, and you admire the way that, to take some simple examples, drones have replaced aerial photography for many purposes, and delivery robots successfully negotiate traffic and pavements to complete the last mile of delivering a parcel. But getting beyond a long list of potential exemplars of new ideas, and drawing any actions or conclusions, or becoming more innovative ourselves, is rather different. If this event didn’t quite capture the essence of innovation, well, there’s always next year, and a new format to try. In other words, try your own innovation.