It was a moment of pure geekdom. Guglielmo Iozzia, director of machine learning at Merck, Dublin, proposed in a presentation how to run an LLM on a Commodore 64. It reminded me of my computer-mad kids trying to squeeze every game onto the puny Amstrad computer we had at home.
I don’t know if Iozzia ever succeeded in porting the LLM, but there was a serious point to it all. To succeed in such an exercise, you start with the technical requirements, considering uch aspects as quantization and the underlying architecture of the device you are working with. But as you think more about what you are trying to do, the implications of the business case become clearer; specifically, what the use of LLMs is imposing on us, the users. How did we get hooked on the use of LLMs for AI? Why do LLMs demand the largest-scale computing resources, larger than the resources of entire governments? Why do they follow the seemingly simple assumption that more scale delivers more value? Iozzia produced some telling responses to the LLM model, notably, that the assumption that what works in a San Francisco office works in a real-world production facility.
His list of problems with commercial LLMs included:
- Per-inference pricing
- Vendor lock-in
- Auditability and governance
- Energy and sustainability
This was a professional computer scientist giving reasons for not using LLMs, mind you. It reminded me of traditional computing exercises for students, where an essential part of the solution is to examine not just that the problem can be solved, but if the computing resources required to deliver the solution are appropriate. Donald Knuth’s The Art of Computer Programming is all about finding the algorithm that combines elegance with efficiency – but the latter seems to have been ignored in the creation of LLMs.
However, Iozzia was not simply telling us to abandon LLMs. Instead, he recommends (and has written a book about it) the use of SLMs, small language models. These are trained not for general use, but for a specific domain, and are already being used with great success in some medical contexts. His recipe:
Take a lightweight architecture, train it exclusively on domain-specific knowledge from scratch. It uses no general internet knowledge, typically no natural-language, no multilingual capability, no general reasoning.
An SLM can run on a local system (even if a PC, rather than a Commodore 64). The input and output can (if you wish) be entirely private. The benefits include:
- From renting intelligence to owning it
- From general capability to specific mastery
- From centralized intelligence to distributed intelligence
I suddenly remembered a history of AI I read some years ago, by Pedro Domingos. Called The Master Algorithm, and subtitled “how the quest for the ultimate learning machine will remake our world”. What struck me at the time was why computer scientists should seek universality without considering the side-effects, and in the process seemingly ignore if a more domain-specific solution might at times be more appropriate.
Of course, there are drawbacks. SLMs typically don’t use a natural-language interface, and don’t come with ready-to-use general benchmarks – you have to create your own metrics of success. But on the other hand, they can enable much more specific results; Iozzia stated you can get SLMs with higher reasoning capability in a single domain than an LLM used for the same domain.
So I haven’t quite dusted down my Sinclair Spectrum in the attic, but I have every intention of trying to build a SLM on my local files and images.
Leave a Reply