Copyright © 2013 Fangyan Yang et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

This paper studies a small neural network with three neurons. First, the activation function takes the sign function. Although the network is a simple hybrid system with all subsystems being exponentially stable, we find that it can exhibit very complex dynamics such as limit cycles and chaos. Since the sign function is a limit case of sigmoidal functions, we find that chaos robustly exists with some different activation functions, which implies that such chaos in this network is more related to its weight matrix than the type of activation functions. For chaos, we present a rigorous computer-assisted study by virtue of topological horseshoe theory.