Start-Up Works On Learning Processor

Welcome Guest. | Log In| Register | Membership Benefits

September 15, 1999 (7:50 AM EDT)

Axeon, a start-up led by software entrepreneur HamishGrant, is attempting to resurrect the silicon implementation of artificial neural networks.

The company is aiming what it calls its "Learning Processor" at tasks within such diverse applications as third-generation mobile communications, inertial navigation, and automated image analysis.

Axeon was formed to commercialize neural-network technology developed at the University of Aberdeen and is waiting for the return of first silicon on a prototype of its processor: an array of 256 simple 8-bit RISC processors connected in parallel on a single chip.

Academic research into neural networks in the 1970s and early 1980s seemed to show that neural networks could be useful in finding solutions to problems that are based on uncertain or incomplete data sets or uncertain data relationships.

In the mid- to late 1980s a number of companies developed hardware implementations, usually cascadable, that could implement neural networks of varying complexity in digital form. Some companies also offered analog or hybrid analog-digital devices in an attempt to pack more "neurons" onto a single piece of silicon.

Artificial neural networks, however, were almost universally rejected in favor of either standard single-processor computation or neural-network simulation software running on conventional single-microprocessor computers.

"They failed for two reasons," Grant said. "First, the actual improvement in performance over software running on a conventional processor was not that great. Secondly, five to 10 years ago, you could not implement sufficiently large neural networks in silicon.

"We will succeed partly due to timing -- silicon technology has moved on a long way since then -- and partly due to the uniqueness of our architecture."

Axeon's work is based on an academic thesis on self-organizing maps written by Neil Lightowler while he was a Ph.D. student at Aberdeen University. Lightowler is a cofounder of Axeon and is head of research at the company.

Scaling Networks

The main feature of Lightowler's work is the development of methods to improve the scaling of neural networks so more neurons can be implemented on a single chip and so larger multichip networks can be trained and operated without an exponential increase in computational overhead.

Axeon said it has achieved that by dividing the generalized neural network into a hierarchy of modules. As a result, the computational and communication overheads are limited to the size of the neural network module. At the same time, the neural network's performance can be scaled with the absolute number of modules and, therefore, neurons combined in the hierarchy.

Lightowler has further enhanced the system's parallelism and density by replacing the multiplier, which is expensive in terms of die area, with a simpler arithmetic shifter, providing processors with local memory; arranging them in a single-instruction, multiple-data stream array; and creating an external module controller. The module controller calculates and stores global information, passes data and instructions to the array, and passes I/O to the external world asynchronously.

Simulations show Axeon's Learning Processor should be capable of 2.4 giga connections per second when running at 100 MHz with an average training time of 0.45 seconds.

"The prototype has a separate controller implemented as an FPGA, but in subsequent iterations that will be integrated into the ASIC," Grant said.

Version 01 chip sets will be available this December, although Grant said this should be considered a prototype of the Learning Processor and that Axeon has a technology road map to take the design to 0.35-micron and then 0.18-micron manufacturing processes over the next two years. Grant said the company plans to produce PCI card-based board-level accelerators, chip sets, and, ultimately, to make the Learning Processor available as an intellectual property core.

"For a simple problem, we would never suggest replacing a RISC or DSP processor with our Learning Processor," he said. "But there are certain problems where we can show a phenomenal performance improvement."

Grant cited a particular image feature extraction problem that takes 10 hours running on a Silicon Graphics workstation, but only about one second on the Learning Processor, according to simulations.

"We are targeting four specific areas, making use of academic expertise available to us from the university," Grant said. "These are mobile communications, automated image analysis, inertial navigation sensors, and network management for routers."

He said Axeon was looking for commercial partners in each of those four areas and that he is already talking with a number of systems companies.

"In the area of mobile communications, neural networks are particularly suited to channel equalization, which essentially is how the mobile phone holds onto the signal as you move around," Grant said. "It becomes exponentially more difficult as the data rate goes up."

"In theory, the third-generation UMTS [universal mobile telecommunications system] is capable of 2-Mbit/second data rates, but developers will struggle to get the bit error rate down using conventional DSPs, particularly indoors where there are multipath effects," he said. "Conventional algorithms require DSPs to improve one hundredfold, while we can already achieve double the performance using the same conventional algorithms."

"The areas which work are mainly telecommunications and banking, where we apply neural network software to detect fraud or highlight churn," said Simon Hancock, vice president of technology at Neural Technologies, Petersfield, England, a supplier of neural network simulation software. "The time scales of these problems mean we don't require high speed." This was true for most neural network problems and was one reason hardware implementations of neural networks did not succeed in the 1980s, he said.

"Similarly, training is often not an issue," Hancock said. "It can be done off-line. You do need higher speeds for video and audio processing.

"Most neural network applications have been met by the rapidly increasing performance of computer microprocessors. There are still some problems that could benefit from higher-performance networks, but one has to suppose that microprocessors and DSPs will continue to improve."

Axeon's Grant said the design of Version 02 of the Learning Processor would be available for license by September 2000 in the form of silicon intellectual property.

"The drive toward system-on-chip within the mobile communications sector, where customers may wish to integrate more than one type of processor or device onto one ASIC, requires us to be able to license the intellectual property," Grant said. "That wouldn't be appropriate in 0.6-micron process technology. We need to get the feature size down to 0.35-micron and 0.25-micron."

A MatLab-compatible simulation engine will be released this month to let researchers and product developers evaluate the Learning Processor.