I’ve been interested in Neuromorphic Systems for a long time: I helped hold the first and the second European Workshops on Neuromorphic systems at Stirling University (Scotland, UK) way back in the 1990’s. I kept working on this area for some time, but then became Head of Department, and became more interested in Neuroinformatics (and was, for a time, the UK representative at the International Neuroinformatics Co-ordinating Forum). But the Medical Research Council pulled the plug on official UK membership, and much water has flown under many bridges since then… Now I’m Emeritus Professor (with the freedopm that brings), and more importantly, there’s been a huge increase in interest in this overall area.
What is meant by Neuromorphic Systems has moved on. At the very beginning (following Carver Mead’s book) this often meant based on MOS transistors in the subthreshold domain, because of their exponential transfer function. But this specific meaning was widened to include more normal analogue circuitry, and spiking systems as well. These days, the meaning has move on some more. Here, I attempt to redefine neuromorphic systems, primarily to avoid the term becoming associated with all the different types of neural networks, and thus becoming more or less meaningless!
What is meant by (or rather, what do I mean by) Neuromorphic systems?
I consider that there are two main branches of Neuromorphic systems:
1: Hardware (or hardware and software) that models neural systems. Examples are
- modelling ion channels,
- modelling patches of active membrane,
- modelling single neurons or neural microcircuits, and
- modelling larger-scale aspects of a brain.
2: Neurobiologically inspired hardware (or hardware/software) for solving real problems, particularly sensory or cognitive problems. Examples are
- auditory, visual, tactile (etc.) sensors designed for interpretation (rather than reproduction),
- systems for processing sensory data (whether from neuromorphic sensors or other sources),
- brain/computer interface systems processing real neural data.
One important aspect of these systems (whether implemented in analogue, mixed signal or digital domains) is real-time operation.
I have been trying to avoid neuromorphic systems becomeing snowed under by all the other large-scale applications of neurally inspired systms, such as neural networks, reinforcement learning systems, and all the systems that process huge volumes of data off-line to build recognition and generative sytems.
Why do this now?
There is renewed interest in neuromorphic systems for at least four reasons.
Firstly, although the large language models (GPTs) work extremely well, they only do so after being trained on extremely large volumes of data, and this training takes a very long time on a large number of processors. This means that training this type of AI system is only possible for those with large numbers of processors (google, microsoft, for example). Further, these systems are “in the cloud”, so that information has to be sent to them, and the results received. There is real interest in building stand-alone systems that sit “at the edge”, rather than “in the cloud”, and neuromorpphic systems are one possible way of achieveing this.
Secondly, there are advances in hardware, as well as in hardware design. Novel devices, specifically memristors are being developed by many different groups, and are being integrated into existing digital designs. This is still difficult but is becoming commercially viable. Such devices make adaptable memory possible in analog, mixed signal, and digital systems without either (relatively) large capacitors or complex digital circuitry.
Thirdly, there is increasing interest in incorporating neuromorphics into robotic systems. This needs not only the first reason above, but also effective real-time sensory systems that can enable the robotic system to co-exist with humans in real environments. There has been interest in neuromorphic cameras right from the start, (there’s a chapter in Mead’s 1989 book on this), but newer systems, like those from Inivation , are now commercially available. There are ideas for neuromorphic microphones and olfactory sensors too, though real neuromorphic microphones are still difficult. The primary aim is sensors that work for interpretation, rather than reproduction.
Fourthly, there have been major advances in neuroscience and neurophysiology, leading to new ideas about how neurons and neural circuitry work. There are many different types of neuron, and our understanding of their operation (both singly and in local microcircuits) has moved beyond the earlier leaky integrate-and-fire neuron. It is still early days for implementing these relatively new ideas in electronics.
As a result, there are more researchers working in the neuromorphic area than ever before. At the same time, there is a much larger community working on neural networks, large language models, big data, and so on, one one aim of this blog article is to identify the Neuromorphic systems community.
We need to be able to meet up and share ideas (as well as taking part in large conferences that include aspects of neuromorphic systems, such as neural net conferences (like NIPS, ICANN, etc.) and ISSCC and other chip design conferences. There are excellent workshops on the area (Telluride and Cappocaccia), but I’d like to start a discussion on how we might meet up and share ideas on a less formal basis.
Lestheprof's Blog 