Below the video of the conference. And, in case you wonder… yes, a few Neurdons attended!

The results showed that an SSVEP BCI system was an effective form of communication for most of their 106 subjects. Who performed best? (drumroll please) Young people! Shocked? No, me neither. The authors note, however, that the result was not statistically significant since older subjects are known to have smaller evoked potentials for visual attention tasks. Another confounding issue stems from the fact that a majority of these types of studies are performed with young male populations from that particular study’s university.

Another problem is that many subjects find the use of SSVEP BCIs to be highly annoying. One could safely predict that having a white light flashing in your eyes for an hour is not enjoyable. This particular study was only 20 minutes in length, so most subjects had no problem with the visual stimuli presented. What made this particular study interesting was the use of a questionnaire given to each subject that addressed attentional strategies, mood, and personality factors. Understanding how users interact with BCI tasks is an important element in refining future BCI software and paradigms, since factors such as personality and mental strategy often correlate with higher (or lower) performance. There are numerous methods for improving BCI performance, many of which are user-dependent and cannot be generalized to the entire population of BCI users. However, knowing the key demographics for certain BCI applications, and continuing to garner feedback from users can assist researchers and software/hardware developers to find which factors can be either ignored or bolstered.

The second part of the article is about memristors, and HP. Cox does not provide a clear cause-effect link between the introduction of memristors and his forecasted “boom” in robotics, but Neurdon can fill in the blanks for you. Memristors, for the first time, provide the medium to allow modelers such as the ones that populate Neurdon to implement those large-scale circuits that can power the next generation robotic platform. And, thanks to the memristors, we will soon be able to do it at low cost and without the need to attach a power plant to the unfortunate robot.

Large-scale, more intelligent artificial neural systems and portability will be the two key innovations that will contribute to the advancement of robotics, as Cox (and Neurdon) hope…

Cliff Click did an excellent job covering many of the mechanisms used to increase commodity CPU performance in his recent talk at the 2009 JVM Languages Summit. The first half-hour or so is most relevant:

A Crash Course in Modern Hardware

As Click points out, a modern general-purpose processor includes a number of highly effective mechanisms for mitigating the Von Neumann bottleneck. Chief among these strategies is aggressive caching. Current processor generations have multiple levels of extremely high-speed memory integrated directly on-chip. The bandwidth to the lowest level of this memory is typically at least an order of magnitude higher than main memory. The latency is typically at least an order of magnitude shorter. As the processor requests information from main memory, the cache memory holds a local copy of the most recently-requested data. If the processor re-requests data in the cache, this is called a cache hit. The data is delivered to the processor directly from the high-speed cache memory. If the data isn’t available in the cache, a cache miss occurs. In a cache miss, the processor has to request the necessary data all the way from main memory.

The most important point of Click’s talk was the notion that performance of general-purpose processors is typically dominated by cache misses. Main memory is so much slower than cache that simply fetching data from it consumes the majority of processing time for many workloads. This is where more exotic processor architectures have been able to find market opportunities.

Caching is highly effective for computational workloads that require relatively low amounts of working memory and low parallelism. Graphics rendering is a canonical case of a computational workload where such assumptions are extremely undesirable. The rendering pipeline is highly parallel and requires exceptionally high memory bandwidth. Memory latency, however, is much less of an issue than with more conventional workloads. Hardware graphics accelerators offer a massive performance benefit because they make a different set of design trade-offs more suited to this workload. For graphics rendering and similar computational problems, a graphics accelerator can run up to a hundred times faster than a high-end conventional processor. For poorly-suited workloads, however, the conventional processor could be many times faster than the accelerator.

Graphics accelerators aren’t the only exotic architecture to have found a significant market niche. Digital signal processors can handle many audio and video tasks with far less power and cost than a standard CPU. Another example is the Cell processor, which is designed for certain types of video game and supercomputing workloads.

Neuromorphic computation is far beyond graphics in terms of parallelism and memory demands, as well as far less sensitive to numerical precision. Conventional graphics processors, however, are already a very fast and efficient means for simulating many neuromorphic algorithms. For neuromorphic hardware to carve out a market niche, it will have to offer dramatic benefits for a meaningful subset of computational workloads. This benefit will be measured against the (extremely impressive) computational prowess of modern commodity processors and hardware graphics accelerators, not a Von Neumann straw man.

(Image from Flickr user IsErik)

Though this information may surprise one at first, the example is but one of a string of revelations accompanying the journey of intelligence research.  Thirty years ago very few scientists expected to run into difficulties in building cameras that understood what they were ‘looking at’ similarly to a person, but the challenge remains.  The emerging theme from varied engineering projects is that biological intelligence trades off some performance abilities for adaptability to a wide range of difficult conditions.  Becoming a professional baseball pitcher may take a good fifteen to twenty years, but it doesn’t take long for someone to learn to play catch, throw trash away, flip a pancake, and so on.  One of the primary goals of neuromorphic technology is to fill in the spectrum between the mathematical but rigid perfection of traditional controllers (and models), and the messy, cheap and rugged heuristics of the animals.

Among the most likely reasons that vision, aerobatics, and many other complicated problems become difficult very quickly is the inherent nonlinearity of the relationship between the input and output of a control system.  Let’s say that someone wants to design a controller that regulates a car’s throttle and transfers power to the different wheels while driving.  The controller knows how fast the car is going, how hard the driver is turning the wheel, and the weather conditions.  The controller probably does a pretty good job of keeping the car from sliding when the road is dry and clean.  In the rain, however, the tires will be more prone to slipping, so the controller shouldn’t allow the throttle to drive the wheels as hard in case they break loose.  The rule ‘more rain → less throttle’ is a simple and linear relationship that could be easily trained into the controller .  During the first fifteen minutes of rain, however, the road is especially slick because rainwater brings oils to the surface of the asphalt, after which times it gets washed away.  In other words, the danger of slipping has a nonlinear relationship to the amount of water detected by the controller; it goes up for about fifteen minutes and then settles back down to something worse than a dry road.  This small difference might not be a problem for a conservative controller designed to keep a family safe, but what if its purpose is to maximize turning speed in a race?  If it were trained only on very dry and very rainy roads (which seems reasonable enough), then a linear controller couldn’t anticipate a good throttle position for the start of a rainstorm, and a small error in throttle could result in a huge difference for the driver!

Rain danger is an example of nonlinearity in a single input dimension, but the situation becomes more complicated with interacting conditions.  Turning quickly is safer on a cement road than uneven asphalt because the uneven surface allows for less traction; dry roads are safer than wet ones because of hydroplaning.  A wet asphalt road may be better than a wet cement road, however, because the uneven surface lifts the tire and allows water to escape from under it (unlikely, don’t test this).  Rather than two bad conditions simply adding together to become worse, the combination may actually be safer than either one alone.

(The situation is a contrived example of the XOR problem, which was shown to be impossible for a single layer perceptron by Minsky and Papert.  In a mathematical sense, all nonlinearities are expressed as multiplicative interacting terms; the previous single variable case arises from a variable’s interactions with itself.  There’s also an interesting connection between nonlinear interactions and the ability of people to categorize and structure the world hierarchically.  Hofstadter mentions that expert chess players, when performing a task of quickly memorizing and recalling chess boards, make recall errors with groups of pieces rather than the single piece errors that novices make.  The grouping process, the recognition that some arrangements of pieces affect the outcome of the game more than others, makes steps towards capturing the nonlinearity of such a complex system.)

A controller can either learn how the process it’s controlling (the plant) works, or it can learn the inverse plant.  Learning the inverse problem can be easier because the algorithm is straightforward to apply; the input to the network is the desired outcome and the output is the action for the controller to take.  This is exemplified in robotic models of inverse kinematics, where a controller must learn to rotate the joints of a limb to get the limb’s end to a particular place in space.  Undoing a process may be difficult, however, when the space of possible plant outcomes is much larger than the number of knobs the controller can twist; in that case learning the forward model may give better performance.  The DIRECT model is an example of how the human brain can act as a limb controller by learning both the forward and inverse plant for a wide range of motions.  An interesting hypothesis put forward here is that ‘babbling’, a childlike exploration of all kinds of movements to gain experience, is essential for fine tuning the network in preparation for precise movement.  For high performance movements, Marr and Albus independently both suggested that the cerebellum is a forward kinematic model of human movement which could be used to apply small corrective signals to those coming straight from the cortex to the muscles.  All of these are examples of neural networks that already exist as controllers in biological bodies, and their applicability both to specific high-performance and robust general-purpose tasks has only increased with continued research.

Some help to solve this technological problem comes from neuroscience. The question of how can the cerebral cortex develop stable memories while at the same time incorporating new information through an organism lifetime has been a central theme in many research groups. The talk posted on Neurdon describes one of these approaches.

This talk addresses a central issue on designing intelligent, stable memory systems: how to coordinate multiple levels of (cortical, or artificial) neural processing steps to rapidly learn, and stably remember, important information about a changing environment.

The Synchronous Matching Adaptive Resonance Theory (SMART) model begins to clarify how this can be done by coordinating bottom-up and top-down processes work. The model links processing of learning, expectation, attention, resonance, and synchrony in laminar circuits of spiking neurons obeying realistic membrane equations. The model also predicts how the generality of learned rcategories may be controlled by neuromodulation, and how the same circuit may explain challenging visual perceptual grouping experiments.

SLIDES

PART 1

PART 2

PART 3

In the 7th Framework Program of the European Community a project has started in 2008, in which modular robots are developed by many research parties in Europe (Universität Stuttgart, Universität Graz, Universität Karlsruhe, Scuola Superiore Sant’Anna, Sheffield Hallam University, Fraunhofer Gesellshaft, Institut Mikroelektronickych Aplikaci, Ubisense, Ceske Vysoke Uceni Technicke v Praze and Almende B.V., see http://www.replicators.eu) that go beyond the swarm mode and are able to form robot organisms.  This is possible by sophisticated docking devices on several sides of the individual robot modules. By interlocking the docking units of a pair of robot modules, a rigid connection is made and robotic body forms – like snakes, scorpions, 4-legged spiders and wheels – are created.
The robot organisms can crawl around by shifting their center of gravity and are not restricted to the tiny screw drives or continuous tracks on the individual modules anymore. For that reason the Replicator robots can navigate through environments with obstacles that are impossible to overcome by robots that only have wheels. There is a wild range of applications to think of. A bucket of robots can be emptied in a collapsed mine (or a building after an earthquake). Then the robots start to assemble and to search for survivors. Or the robots swarm over the surface of a to be inspected planet in a NASA or ESA mission, a few fall in a cavity in the landscape and the rest comes to rescue, making it possible to continue to carry out the mission.

There have been several modular robot projects in the last decade, but this one is special because the individual robots are gifted with highly sophisticated sensors as has never seen before in modular robotics. In the current prototype each of the robot modules carries a laser scanner, infrared and ultrasound sensors, microphones and multiple cameras (and for example, a camera streaming images at a rate of 10 Hz, of a size of 256*256 pixels, coded by 3 8-bit RGB values amounts to a bandwidth of almost 16 Mbit/s.). A robot organism consisting out of say 10 modules has to cope with an extraordinary amount of sensor data. For that reason this type of modular robots, far more than before, needs elaborate sensor data processing and fusion mechanisms in place.

Sensor fusion is an advanced area of research that comes across new challenges when it is confronted with the agility of robots not just carrying the sensors around, but anticipating changes in perceptual input because they know they are driving their own selves around, and which are moreover also capable of proactive search for some preferred sensory input. This highly autonomous nature asks for sensor fusion techniques that enable the robots to categorize and classify the to be known world for themselves. The robots need to learn how another robot looks like, how a power outlet looks like, what sounds a robot typically makes, what sound a power outlet typically makes (none…) and preferably combinations of those different sensory modalities.

At Almende B.V. (http://www.almende.com) we implemented an adaptive resonance theory (ART) model to auto-classify visual-acoustic data. The acoustic data had to be preprocessed by an echo state network. This is a recurrently connected “reservoir” of neurons, where the only connections that can change are the ones that go to the output (or readout) neurons. So, within the reservoir we will have neurons having all kind of firing patterns, and we simply select at the output the ones we want to combine. The output of the reservoir will abstract representations of the original raw audio input. The visual data is also preprocessed. In this case a spatial attention mechanism is implemented using a combination of the saliency model of Itti and Koch and that of Frintrop (for performance purpose). Color, intensity and other characteristics can be used to have a certain blob of pixels “pop out” of the scene. This salient entities can subsequently be used by SIFT (scale-invariant feature transform) to come with a set of features (key points). Similar to audio, we now have an abstract representation of the original image input.

The visual as well as the audio abstractions are fed into a multi-directional unsupervised ARTMAP, an adaptive resonance theory (ART) variation. The reason that an ART variant is used, is because this allows the robot to learn fast new visual-acoustic objects without forgetting the old ones it already knows. Standard ARTMAP uses two ART models. A pattern to the supervisor network activates a node in the so-called “map field”, as well as an incoming pattern in the supervised network. If those nodes are different, the latter starts to search for a better match. The crux is to see both networks as presenting just another modality. The sound profile that is heard when a robot encounters another robot, can be used as “supervised” information for the visual profile, or the other way around. The actual implementation allows for dynamic binding of this information, or the robot will always expect a unique visual-acoustic object, while in the real world different visual objects might be able to make the same sounds, and the other way around. The necessity for this becomes clear in experiments in the physical realistic Delta3D simulator. A robot was wandering around and detected an interesting object, a power outlet! In the meanwhile the robot heard a sound from a robot, but sadly one that was sneaking up behind him! Thanks to the dynamic binding of visual-acoustic classes the robot learned later (by encountering silent power outlets) that this sound was not made by outlets at all, but by the robots in the arena. The robot is smarter than a gosling classifying its mother early in life… To go into more details about the implementation would lead too far for now. A lot of information can be found on the tech blog http://replicator.almende.com. Enjoy!

HP memristor

The HP Labs research team headed by Stan Williams recently unveiled a two-terminal titanium dioxide nanoscale device that exhibited memristor characteristics. The UC Berkeley symposium is an impressive collection of talks on memristors and memristive systems. Below a talk by Greg Snider from HP Labs.