Teuvo Kohonen (Helsinki), “Is the SOM a biological model?”
Barbara Hammer (Clausthal), “ Self Organizing Maps for Time Series”
Jose Principe (Univ. de Floride), “ SOM as general infrastructure for signal processing and controls”
Takeshi Yamakawa (Kitakiushu), “Brain-Inspired SOR Network with Fuzzy Inference Based Evaluation and Its Application to Controller Design”
The SOM paradigm was originally motivated by an attempt to explain some functional structures of the brain. Quite surprisingly, however, the SOM turned out to be very useful in exploratory data analysis and started to live a life of its own: some 6000 scientific articles have been based on it. Contrary to that, it is still not quite sure to what extent the SOM could be regarded as a biophysical model. At least there does not exist complete physiological evidence for the following partial functions: 1. The winner-take-all, in particular since several activity clusters may occur in brain maps concurrently; 2. Implementation of the neighborhood function, in particular because some newer biochemical findings contradict with the older hypotheses; 3. The Hebbian law of synaptic plasticity, since the conditions for synaptic modification can be very different; and 4. The biological mechanisms for the temporal control of the learning laws, where newer findings also contadict with older biochemical hypotheses.
The topic of this talk is to survey the status of the biological foundations of the SOM. After that I suggest how the WTA could be replaced by a concurrent, nonlinear enhancement process that also leads to the ordering of the connections, especially in the somatosensory maps. With this revision the implementation of the neighborhood function becomes very simple, and the Hebbian law takes a biologically more plausible form. Notwithstanding, the original SOM algorithm has still a superior organizing power, and may be regarded as an abstract idealization of the biologically more plausible self-organizing processes.
Since the discovery of the SOMs by T. Kohonen many results have been found in order to get a better description of their behaviour. Most of them are very convincing but ona mathematician point of view only a few are actually proved. Let's mention two wellknow facts that have been obseved a million
times but still unproved :
- the 8 neighbours square grid Som with uniform stimuli on [0,1]2 has the square grid as (unique?) stable equilibrium.
- it is possible to choose a slowly decreasing step so as it converge to this equilibrium
In this talk we will list the results actually established and point out some nice results which it would be satisfying to prove.
The self-organizing map (SOM) as proposed by Kohonen constitutes one of the most prominent data mining and visualization tools which has successfully been used in different application areas such as web-mining, telecommunication, image processing, or, generally speaking, unsupervised detection of information in large vectorial data sets.
Many real life data structures possess an inherent temporal characteristics: speech signals, biological time series such as EEG data, sensor streams as they occur in robotics etc. Since these data are nonvectorial, consisting of temporal sequences of possibly different and priorly unlimited length, a direct application of SOM is not possible. Several extensions of SOM for time series have been proposed in the literature, including biologically plausible but restricted models such as the temporal Kohonen map (TKM) or recurrent SOMs, or recent more complex models such as SOM for data structures or the recursive SOM.
In this talk, we present a new extension of SOM for temporal structures which is based on a very simple and sparse recurrence and which allows efficient training as well as a combination with arbitrary lattice structures.
We demonstrate its practical applicability in several examples and discuss its theoretical properties, in particular the internal representation of sequences within the model and its principled representation ability. It turns out that the model can be seen as a more efficient and powerful realization of the encoding scheme intended by TKM.
Afterwards, we put the approach into a general framework of recursive unsupervised models which also covers a variety of alternative approaches including the approaches mentioned above and standard supervised recurrent networks. Based on this formulation, mathematical properties, benefits and drawbacks of the models can be investigated in a uniform way. Interestingly, the formulation can be generalized from sequences to more general tree structures thus opening
the way to unsupervised processing of general data structures.
The results presented in this
talk are based on joint work with Marc Strickert (IPK
Gatersleben), Alessandro Sperduti
We have recently introduced new kinds of data fusion techniques, where the goal is to find what is shared by data sets, instead of modeling all variation in data. They extend our earlier works on learning of distance metrics, discriminative clustering, and other supervised statistical data mining methods. In the new methods the supervision is symmetric, which translates to mining of dependencies. We have introduced methods for associative clustering, for extracting dependent components which generalize classical canonical correlations, and for non-parametric dependency mining.
We will discuss the SOM as a general infrastructure to extract valuable information from the data in several relevant tasks in signal modeling. Specificaly, we will describe how the SOM can be used to create competitive multiple models for system identification and controls of aircrafts, as well as to create time saving algorithms for information theoretic distances in biomedical pattern recognition applications.
The self-organizing relationship (SOR) network has been proposed as a brain-inspired neural network model. The SOR network realizes an approximation of a desired Input/Output (I/O) relationship of a target system as a result of learning. The SOR network is effective in case that a set of learning data (desired I/O relationship) is unavailable but the input-output data obtained by trial and error can be evaluated. In learning process, I/O vector pairs with their evaluations are used. The structure of the SOR network and evaluation for the I/O vector pairs are equivalent to the lower brain function and the higher brain function, respectively. In the human brain, the lower brain function facilitates the mapping from sensory inputs to motor outputs, and the higher one controls the overall behavior of the learning process of the mapping. In fact, the evaluation criterion becomes the key to whether the desired I/O relationship is successfully approximated or not. Human beings evaluate the I/O relationship linguistically as opposed to mathematically when the target system is complex.
In this paper, we introduce a new evaluation method for the I/O relationship of the SOR network. In the proposed method, the evaluations are linguistically described with fuzzy if-then rules. After that, the controller is self-organizingly established by learning of the SOR network. In these processes, only experts’ commonsense knowledge is required. The proposed method is inspired by the process in which a human being obtains desirable motions by accumulating trial and error based on one’s evaluation criterion.
The back-up control of a trailer-truck is very difficult due to its nonlinear mechanism. The proposed method is successfully applied to the trailer-truck back-up controls both in computer simulations and in practical experiments with a radio controlled trailer-truck.