In this paper an unsupervised scheme for stereoscopic video object extraction is presented based on a neural network classifier. More particularly, the procedure includes: (A) A retraining algorithm for adapting neural network weights to current conditions and (B) An active contour module, which extracts the retraining set. The retraining algorithm takes into consideration both the former and the current network knowledge in order to achieve good generalization and reduce retraining time. The retrained network performs video object tracking to the rest of the frames within a shot. Retraining set extraction is accomplished by utilizing depth information, provided by stereoscopic video analysis and incorporating an active contour. Finally results are presented which illustrate the promising performance of the proposed approach in real life experiments.