The STM Innovations Day (London, December 2018)

What were the innovations that emerged from the STM Innovations Day? Well, there weren’t any robots on display. This annual event is an excellent opportunity to take stock of academic publishing and its current concerns, although as we shall see below, some of the real innovations and controversies emerged in a rather roundabout way, rather than as the focus of presentations. The challenge for the organisers of an innovation event is to try to capture issues that are of importance to the whole community, while preventing the day becoming a set of product pitches that only provide very partial (or very proprietary) solutions to these concerns. Inevitably, and quite correctly, the day seeks to highlight examples of co-operation between publishers for the good of the industry as a whole (and for the good of the researcher, who is the whole point of the exercise).

Hence, it was not surprising that the day opened with a keynote about a pan-industry initiative, that of Open Science. Paul Wouters, professor of scientometrics at Leiden, chairs the EU Expert Group on Research Indicators, so you would expect his talk to look at metrics. However, far from providing us with new metrics, he stated quite openly that the goal of his group was not to replace any existing metrics. As his talk progressed, he talked more and more about long-term goals, but I was left struggling to report on any short-term practical propositions. The goal of Open Science is to make science more inclusive and more global, all well and good, but how? In practical terms, he talked about measures such as “engagement with stakeholders”, or “public communications”, or “dissemination for a specialised audience” (measured as the number of blogs on a professional society website). All very commendable, but hardly ground-shaking. (There is an interesting post on the challenge of measuring research and the needs of international development by Valeria Izzi here.)

There was one moment of engagement with the audience when he talked about academics. Academics, he stated, should be the happiest people on the planet. They can read books while being paid by their boss, they can teach beautiful young people [his words], and they can do research. Yet they are not happy people! His solution suggested the blue-sky nature of his thinking. He argued that academics are not happy because they are stressed – they have insufficient time and too much bureaucracy. The answer, then, is to take responsibilities away from them. It seemed an odd recommendation to promote something by taking away other responsibilities – I could even argue that what many academics claim as bureaucracy might well be things that ensure their teaching is inclusive and egalitarian, such as looking at student feedback to their teaching, but that’s a subject for another blog.

Eefke Smit asked a very sensible question: Open Science is a great idea, but does open science lead to better science? I think her question summed up the problem with this initiative – how is Open Science to be measured? Can we equate quality in research with open knowledge practices? Without a clear set of metrics for Open Science, I didn’t hear anything in this presentation to indicate how (or if) this quality issue would be addressed. It raised several questions – can elite science be inclusive? – but these were not addressed in this presentation.

“The future of access” was a similarly broad theme that in practice generated few fireworks. The people invited (Elsevier, Kopernio, Web of Science) were never going to be radical unless challenged, despite the fact that Roger Schonfeld was chairing (he seems to have a reputation as someone who can manage potentially warring factions). This was a session by service providers, aimed squarely at publishers (Elsevier presenting itself here via Mendeley as a service provider, not a publisher). Gaby Appleton, CEO of Mendeley, suggested “taking some simple steps together” – hardly very radical. Such a small-scale idea seemed a million miles away from the tone of presentations by Mendeley before they were bought by Elsevier; then, they were going to change academic publishing. Overall, I didn’t feel that any of the three presentations demonstrated any fundamental change in the way that researchers accessed content, apart from the increasing usage for publishers by pointing them to the version of record (VoR), which would benefit publishers, but I don’t think academics would even notice it. You couldn’t help feeling that the solutions on offer were tinkering with a system rather than a fundamental change. The biggest initiative discussed at the event, RA21, is a rare example of a cross-industry collaboration that no individual or company on the platform could take credit for – although, to Jan Reichelt’s credit, he acknowledged its importance.  “Our universe is too complicated, compared with Spotify”, stated Reichelt. But is that really the case? Music streaming has changed the way many of us access music, despite the horrendous problems of music licensing. There are no similar initiatives for a common platform for academic content delivery.

RA21 is a genuine step forward; a way of tightening up access by switching from IP address to a federated SAML log-in system. This has the potential benefit of simplifying access to content on multiple platforms and in multiple locations. When RA21 was presented, there was barely a murmur from the audience; had Anthony Watkinson not asked a question about questions about RA21 from the library community, you would never have known that RA21 had generated anything but universal approval when initially presented. It appears that the concerns of librarians, expressed at the Charleston Library Conference, were perhaps simply that the new access system is considerably more complex than simply via IP address (which of course should not disqualify it, if it provides other advantages). For me, as a publisher, I welcomed the new system’s ability to potentially prevent the misuse of institutions’ IP addresses, plus the capability of RA21 to identify users, not personally, but by role, so for the first time, publishers (and libraries) will be able to differentiate usage between undergraduates, postgrads, and faculty. This sounds like a good thing; currently, all that a publisher or library knows is that (say) 250 people at institution XYZ accessed one of their journals in the last week, which doesn’t tell you much.

For me, the most valuable new things I discovered were from the presentation of the 2018 STM Report. It didn’t surprise me from this invaluable report that the focus of academic publishing is moving eastwards, or that China is today the largest producer of scholarly papers, above the USA, but I was astonished that India is now in third place. The other disappointing surprise is the lack of precision about quite fundamental industry statistics – how there is widespread disagreement over even the total number of journals or articles. But there are clear signs of an industry in fundamental change. The presenters revealed that 20% of the preprint articles in ArXiv are never published in journals. Here is a statement more radical than any other I heard on that day: in other words, there is a lot of real science going on that simply by-passes the standard manuscript submission – publisher – commercial hosting platform workflow. Maybe next year’s Innovations Day could look at that 20% and examine what it represents. There is some real innovation going on here, if we can understand it.

Books of the Year 2018

This is the time of year when the newspapers announce their books of the year. Unfortunately, the reviewers often praise each other’s books, or simply state they couldn’t put it down, without explaining why. Things are a bit different here; every one of the books on my list have been reviewed on this site during the past 13 months (bending the rules slightly to allow myself a wider selection).

Weapons of Math Destruction

The Signal and the Noise

Metaphors we Live by

Crossing the Chasm

Algorithms to Live by

The Book of Why?  (a review of this will follow in December).

The winner is Algorithms to Live by?  Why this book?

Because it provides an excellent overview of how AI tools and a machine-based approach can facilitate content creation and dissemination. I won’t get involved in details about what is or is not machine learning, so I’ll simply say a machine-based approach. In other words, the book praises a way of thinking, rather than slavishly following the adoption of one or two algorithms.  Essentially, the authors claim, there are ways in which machines can be used to assist human processes. There are thousands of processes that can benefit from such an approach; all it takes is a reasonable awareness of what machines are good at (such as remembering numbers, and carrying out operations at vast scale) and what humans are good at (such as making subtle judgements and fine distinctions, and reviewing many disparate factors at once).

Nate Silver’s The Signal and the Noise was a close runner-up. It is perhaps the clearest explanation of Bayesian reasoning I’ve yet read, but his almost universal recommendation to put Bayesian into operation in all circumstances leaves out some essential human caveats to take into account (“I’ve deliberately picked some challenging examples – terror attacks, cancer, being cheated on – because I want to demonstrate the breadth of problems to which Bayesian reasoning can be applied”). He’s a wonderfully entertaining writer, but his rather cavalier attitude to things outside probability (such as historical events) to be somewhat alarming.

In contrast, I found Algorithms to Live by provides practical, immediately applicable advice, an informed perspective that enables you to take a decision straight away, without feeling it was a blanket-like adoption of a single approach. Whether it’s matching socks, or filing old correspondence, this book provides straightforward and well-informed answers. Because it doesn’t restrict itself to a single algorithm, but makes it clear there might be several possible computing-inspired approaches to solving a problem, it is a book that inspires, which is as much as I can ask from any book. It’s  a book I would happily put into the hands of someone wanting to start in the area of content enrichment to see what the fuss is all about, and coming away with an informed idea of the possibilities of this technology.

Weapons of Math Destruction: more math please

The full title of this book is Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy, which sums up the book, certainly, and at the same time the author’s  approach. In practice she is less concerned about the math, more about the social and political implications and situation. She writes from a liberal standpoint, identifying political consequences of big data, and I agree with her sentiments completely; It’s a wonderful book, yet it is perhaps slightly churlish of me to want to know more about how to apply the maths sensibly, that is,  how to use algorithms that include some human component. For O’Neil, it’s not the maths, it’s the inequality; I optimistically want to write the most democratic, equal-opportunity algorithms, in support of the liberal values she support.

Read More

Is there such a thing as Digital Humanities? 

The Digital Humanities Stack (from Berry and Fagerjord, Digital Humanities: Knowledge and Critique in a Digital Age) (from the Wikipedia article on Digital Humanities)

At this year’s ALPSP annual conference (Windsor, October 2018), there was an entertaining session called “What’s New in Digital Humanities”. Of the three speakers, one was by Peter Berkery, the Director of the Association of University Presses, one by an academic, Paul Spence – a lecturer in a department of digital humanities, no less, so he should know – and the last, the most entertaining, was by Etienne Posthumus of Brill Publishers, a commercial publisher. Mr Posthumus was entertaining because he was prepared to question some givens; during his talk he roundly criticized XML, for the very good reason that the slightest error in an XML document makes the whole thing fail to process, unlike HTML which at least makes an effort to produce a result.

The other two talks described no doubt fascinating Digital Humanities initiatives, each of which was funded by a philanthropic institution. To be fair, these initiatives are not just about publishing one title, but providing tools for humanities titles to be published. Why not just publish a monograph like any scientific author would do? Alan Harvey, of Stanford University Press, explained that the goal of digital humanities is “not to publish a book in digital form … [but] embedding the scholarly argument within the digital object”. Whatever that means.

In the context of that kind of statement, Posthumus asked why there should be such a thing as digital humanities. After all, we don’t talk about digital physics, or digital astronomy – we just talk about physics or astronomy. This made me ponder; now I think about it, in my experience, the term “digital humanities” has sometimes been used to describe, not mainstream publishing projects, but some pet project by an academic to use vast swathes of highly specialised technology to create a niche publication, such as an annotated edition of an author’s manuscripts, including all the annotations and corrections, with custom functionality that might not be relevant for any other publication. To check my theory, I asked a publisher responsible for humanities publishing at one of the major academic publishers what his view of digital humanities was. No such thing, he said – there is just publishing. Whatever tools and techniques are used for publishing apply also to humanities; digital humanities is not a special case. Next time I attend a conference session on digital humanities, I will ask why digital humanities should be different. Paul Spence defined digital humanities as “the interpretation and visualisation of highly structured content”, but I fear such a definition argues for humanities becoming more side-tracked when it should be stressing what it has in common with other academic disciplines.

Text Analytics Forum: a valuable guide to what is happening

The second year of the Text Analytics Forum took place Washington, DC, last week, and it was good to see the conference building from its first iteration and developing in range.  Tom Reamy, the conference chair and organiser, has shaped this event very much around his view of the text analytics market, and around his book Deep Text (reviewed here).  The event had a great mix of practitioners, users, and an increasing range of solutions were discussed – on which more below.

His view of text analytics is very much an experiential one, based on many assignments with content owners, and has the great value of being thoroughly thought-through, but I think he limits the scope of text analytics in his own presentations rather more than the event as a whole might suggest. Tom is strongly in favour of a rule-based approach, although I tried to show in my presentation that not all analytics problems needed to be solved by rules (and in fact he even suggested himself there were some circumstances where an unsupervised approach works perfectly well).

In his keynote presentation, Tom was very hard on text mining, describing it as “treating words as things rather than understanding them” – but Solr and Elastic Search don’t understand words either, being entirely string-based, and rules are typically dependent on matching strings (and it’s not easy to build a rule that finds “renal” when you search for “kidney”). In any case, text mining can be combined (as UNSILO does) with NLP to provide some meaning-based capability as well, so even if basic text mining is at times meaning-free, it doesn’t mean the result is necessarily inferior to other approaches.

Nor is it true to say that text analytics is primarily auto-classification, since the finding of a peer reviewer for a manuscript of an academic article is not a classification exercise.

But my biggest question was his statement that “deep learning is a dead end in terms of accuracy”, stating it can only reach 60% to 70% accuracy, while rule-based systems could deliver up to 92% accuracy. Such statements as these are meaningless without some description of context, and miss one key point, which is that where a human index is involved in the measurement of a trial, it will inevitably be limited to human agreement – which is rarely above 70%.

The conference itself was admirably wide-ranging, and suggested a number of ways of using text analytics that left everyone at the conference thinking about how they use (or plan to use) this technology. For me, one of the most valuable presentations was not technical at all; it was from the Cognitive Computing Consortium, and showed a simple way of assessing a text analytics application against a number of criteria. Any project could be mapped to a number of sliding scales, such as accuracy or discovery. Key to this visual presentation was the recognition that you could have either one or the other, but moving towards greater accuracy made discovery of necessity more limited. This looks to me like a very simple and effective way of presenting some of the trade-offs from these machine tools. Another useful slogan from Susan Feldman was “no AI without IA”, which I understand to mean there is a very real need for a human managing the AI-based analytics process, to make sure it delivers sensible and effective results. I couldn’t agree more with that.

All in all, an excellent conference, leader (like Tom Reamy’s own book) in a field of its own, and all the more valuable for that. I look forward to next year’s event.

Do baseball stats help you understand climate change?

Nate Silver likes a bet, so much so that he quit his job and became a full-time poker player for some years. So perhaps it is not surprising that this is an author with a passion for probability, whose interpretation of Bayesian reasoning is gambling related: “the most practical definition of a Bayesian prior might simply be the odds at which you are willing to place a bet.”

He sets out to show how many situations comprise a mixture of signal (valuable information that enables us to learn about it) and noise (confusing messages that make it difficult for us to interpret). Paradoxically, it is something of a challenge to determine just what is signal and what is noise in Nate Silver’s own book.

Read More

Facts vs. Fake News: Who decides what is True?

The impressive title, “Facts vs. Fake News: Who Decides What is True?”, suggested a session that perhaps with hindsight was difficult to live up to. It was one of those topics that sounded self-evident, yet when people started discussing it turned out to be rather one-dimensional, or at least limited to the perspective of the panellists.

Let’s face it, the issue with fake news is that it’s only a problem for the minority of people who are bothered about it. All the people in the room were bothered about it, but the world outside (as can be seen from Trump’s USA and the UK that voted for Brexit) is not. So the issue is not “who decides what is true”, since the majority of the population have voted with their feet and decided what the truth is. So this discussion was never going to address the main issues. Panels of experts are not what the populace is looking for to combat fake news. They are quite happy with the news they get, even if it is (partly or wholly) fake. The seminar risked at times revealing itself as a coterie of left-wing liberals who could not understand how anyone could be taken in by fake news. In fact, one questioner asked just that question, if there was any evidence that fake news had really affected, say, the Clinton – Trump election. Nobody in their right mind, you might think, could be taken in by such tricks.

So it was somewhat irrelevant that two of the four panellists represented what I will describe as the voice of authority: they offered expert opinions, even if nobody was going to listen to them. However worthy they may be, they reminded me of the online encyclopedia initiative called Scholarpedia (it’s still running). It was intended to serve the same function as Wikipedia, but to be compiled by authorities in their field. It never took off, and it has, I would guess, less than 1% of the users of Wikipedia.

The third panellist, Heather Staines of hypothes.is, was again very well-meaning, but sounded to me like someone in the wrong session. Hypothes.is is a service to provide annotations. Those annotations could potentially be fake news, but clearly hypothes.is had not considered such a possibility, and did not have any tools in preparation for such an eventuality.

That left the fourth speaker to address the topic in a balanced way. She, Jennifer Pybus, is based at Kings College London, and seemed to carry the responsibility of the entire university on her shoulders. However, although she gave an interesting and informed talk, she did not have any recipe when asked at the end by the chair what to do. The most chilling answer to “who decides what is true” was Heather Staines, who said, as a historian, that history tells you that the victors decide what is true. It’s not truth, it’s power. That final note revealed perhaps there is a whole discussion that did not take place at this event. Ms Pybus often gave the impression from her slides that more than one conclusion could be drawn. For example, she talked about three challenges facing us all, two of them uncontentious, but the third worth thinking about:

  1. Loss of personal data
  2. Need for more accountability around political advertising
  3. Concentration of ownership in algorithmic practices leading to spread of misinformation

In my view, having access to understand the algorithm is a very necessary step in evaluating machine-based solutions, but the concentration of ownership of algorithms is not in itself a problem. Nor is the “algorithmic practices” themselves an issue, unless treated as a black box and never examined for bias. In conclusion, this was a discussion that, given the panel, would never reach a very pragmatic solution. Nonetheless, the presentations and the evening were fascinatin; if only they could have addressed the issue in a more informed way.

What semantic enrichment means for academic publishing

Anyone reading the latest issue of the invaluable Research Information (The Meaning of Semantics, Four industry figures discuss the latest developments around semantic enrichment with Tim Gillett) would be left little the wiser about semantic enrichment after reading it. Although the line-up of people interviewed is impressive, each respondent answered the questions in a very different way, which revealed perhaps how little agreement we really have when we talk to humans about what we believe to be one topic – especially when that topic includes the word “semantic”. 

What is semantic enrichment?

This question seemed simple enough. Babis Marmanis of Copyright Clearance Center and Giuliano Maciocci of eLife answered in broadly similar ways: “the enhancement of content with information about its meaning”. But Donald Samulack pointed out the enhancement could be by something visual, such as an infographic, and Jordan White of ProQuest described “adding disparate pieces of content … sharing certain metadata” (sounds suspiciously like using keywords to me). It could all be described as enrichment, although not perhaps what interviewer Tim Gillett perhaps had in mind when he asked the question.

What are the key industry developments of semantic enrichment in the last ten years?

Here again there was a range of answers. Babis Marmanis outlined entity extraction, which today can be achieved in a far richer way than ten years ago. Giuliano Maciocci described quite correctly the uncharismatic but vital steady progress of XML coding of articles and chapters, most recently to the JATS XML standard, plus ORCIDs for persistent identifiers. JATS and ORCID may be very unsemantic, but without these tools the clever semantic stuff can hardly begin.  Donald Samulack talked again about adding visual elements. Jordan White intriguingly described“taste clusters”, or “featured snippets”, which I thought was provided by the journal abstract, although on reading a few abstracts I am often left none the wiser on what the article is really about. Perhaps this reveals that leaving semantic enrichment to humans isn’t always wholly successful. 

Next, we came to the big question: how do these developments benefit the academic community?

Marmanis talked enthusiastically about synthesising, drawing inferences from and taking action from research articles – the holy grail that corresponds to the way a human would deal with an article. Sadly, it is still some way off realisation. Maciocci, practical and pragmatic as before, talked about providing more APIs to link content to other sites. Samulack described saving time for the researcher by delivering the “stopping power” of an article – a summary of what it means.  

 

The final question was about the future: what comes next?

Of course such a wide-ranging question, with no single correct answer, will produce considerable variety in the responses, but here perhaps the results were least satisfactory. Marmanis, interestingly, pinned his hope on “well-curated semantic lexicons … with minimal human intervention”. I would have thought that a true AI-based semantic enrichment might not result in a lexicon, which is a means, rather than an end; researchers don’t want a lexicon, they want answers. Maciocci talked about “automatic inference”, which seems a much more fruitful way of proceeding. Samulack described the need for images to have plain-text descriptions – I would have thought advances in image processing mean that most images would have automatically generated text descriptions of their contents before long. Jordan White talked about the end of relevance-ranked search, and not a moment too soon, being replaced by “trusted seminal works at the top of search results, vetted by evidence of usage”. It seems odd that the journey of semantic enrichment, which after all is about revealing the meaning of a text, should require usage results, just like Amazon provides when you buy something on their platform: since you bought X, most users then bought Y.  It works, but it’s hardly very semantic.

Are there good and bad metaphors?

George Lakoff is famous (according to Wikipedia) for the “conceptual metaphor theory”, which is that people are influenced by the metaphors they use.

Intrigued by this claim, I read the short book Metaphors We Live By (1980), by Lakoff and co-author Mark Johnson.  Sure enough, by the end of the book, the authors demonstrate (to their satisfaction, if not to mine) that if you choose the wrong metaphor, then who knows what might happen. “Drastic metaphorical differences can result in marital conflict”, state the authors, a claim I never expected to encounter in a book about linguistics. If Adam thinks marriage a haven, but Eve thinks marriage is a journey, then problems lie ahead. No doubt there will be disagreements, but not, I think, because their metaphors have landed them in different places.

How did metaphors become a yardstick (nice metaphor, that) of the good life? In terms of argument, the book proceeds as follows: it’s what I call the “slyly introduced hammer blow”. If you want to say something controversial, don’t say it upfront, but dress up your argument in the most persuasive terms that nobody could disagree with. Then repeat the process two or three times until, when the reader is lulled into acceptance of your drift, you insert something highly contentious. Don’t say it is contentious; simply state it follows logically, as night follows day.

Hence, Metaphors We Live By begins by saying much of human discourse uses metaphor – I can’t deny that. The metaphors we use can often be grouped, and Lakoff and Johnson capitalise the names of these groups,  a charming gesture. Thus, we have groups such as

TIME IS MONEY, e.g. “I’ve invested a lot of time in her.”

TIME IS A LIMITED RESOURCE, e.g. “Do you have much time left?”

The argument proceeds without controversy, in easily understood steps such as these, until suddenly signs of metaphors are linked to morality.  For some strange reason, Lakoff and Johnson object to metaphors that do not fit into one of their metaphor groups. Hence the seemingly inoffensive phrase “the foot of the mountain”, which is condemned outright:

“Examples like the foot of the mountain are idiosyncratic, unsystematic, and isolated. They do not interact with other metaphors, play no particularly interesting role in our conceptual system, and hence are not metaphors that we live by.”

Where did this argument come from?  Commentators have been complaining for years about “stale” use of language, and “dead” metaphors, but they are not usually trained linguists.  For Lakoff and Johnson, the metaphors we use have to be those that “enter into our everyday lives” – otherwise they are dubious.

The claims about the moral value of choosing the right metaphor are only fully stated in the book’s final chapter, when the authors become positively lyrical. I bet you had no idea that by adopting the correct use of metaphor, as described by Lakoff and Johnson, your life will be less “impoverished”.

I completely agree that in a conversation, “meaning is negotiated: you slowly figure out what you have in common.” But to say that the well-known “conduit” principle, which states that ideas are objects, linguistic expressions are containers, and communication is sending, is “pathetic” or even “evil” seems to be overstating the case.  “When a society lives by the Conduit metaphor on a large scale, misunderstanding, persecution, and much worse are the likely products”.

How then should we use metaphor? That isn’t so clearly described, but there is a reference in Chapter 30 to “appropriate personal metaphors that make sense of our lives”, in other words, that provide “self-understanding”. If self-understanding is possible through metaphor, why not then claim that metaphor enables “ritual”, “aesthetic experience”, and “politics”- and the authors have a section dedicated to each of these topics. Why politics, for example? Because “a metaphor in a political or an economic system, by virtue of what it hides, can lead to human degradation.” Lakoff & Johnsen quote the metaphor “Labor is a resource” and point out the labor could be “meaningful” or “dehumanized”. This seems to be an extreme version of the Sapir-Whorf hypothesis, that words in a language determine the way we think. It recalls the famous Roman proverb quoted by Marx in Capital, “Pecunia non olet”, meaning that cash – coins and, today, notes –  carry no associations from the possibly illegal and devious things they might have been involved with in an earlier transaction. For Lakoff, metaphors seem to be like coins, in that coins do not carry associations of the situations in which they have been transacted, but some moralists believe that they should. For Lakoff, the simple phrase “labor is a resource” is deeply suspicious: “the blind acceptance of the metaphor can hide degrading realities”, since phrases such as these are often used in a context where labour is seen as a cheap, undervalued resource.  Yet Lakoff himself earlier in the book has no difficulties with TIME IS A LIMITED RESOURCE.

Next time I use a metaphor, I’ll think carefully about how it can enter my everyday life.

Page 1 of 14

Powered by WordPress & Theme by Anders Norén