Neural mechanisms of feature extraction for the analysis of shape and behavioral patterns

Abstract

The human visual system segments 3D scenes in surfaces and objects which can appear at different depths with respect to the observer. The projection from 3D to 2D leads partially to occlusions of objects depending on their position in depth. There is experimental evidence that surface-based features (e.g. occluding contours or junctions) are used as cues for the robust segmentation of surfaces. These features are characterized by their robustness against variations of illumination and small changes in viewpoint. We demonstrate that this feature representation can be used to extract a sketch-like representation of salient features that captures and emphasizes perceptually relevant regions on objects and surfaces. Furthermore, this representation is also suitable for learning more complex form patterns such as faces and bodies in different posture. In this thesis, we present a biologically inspired model which extracts and interprets surface-based features from a 2D grayscale intensity image. The neural model architecture is characterized by feedforward and feedback processing between functional areas in the dorsal and ventral stream of the primate visual system. In the ventral pathway, prototypical views of head and body poses (snapshots) as well as their temporal appearances were learned unsupervised in a two-layer network. In the dorsal pathway, velocity patterns are generated and learned from local motion detectors. Activity from both pathways is finally integrated to extract a combined signal from motion and form features. Based on these initial feature representations we demonstrate a multi-layered learning scheme that is capable of learning form and motion features utilized for the detection of specific behaviorally relevant motion patterns. We show that the combined representation of form and motion features is superior compared to single pathway based model approaches.