The increasing popularity of multimedia applications creates an increasing need for secure storage and transmission techniques. Especially wireless communications (satellite, mobile, RF), which can be easily intercepted, should be protected by unauthorized persons. Towards this direction several encryption schemes have been proposed in literature, however most of them do not consider regions of interest (video objects-VOs). These regions may need better protection, may be the only regions that need protection, or should be accessed separately according to different privileges. For these reasons in this paper a human video objects encryption system is proposed based on chaos. Initially videoconference/videophone sequences are analyzed and human video objects are automatically extracted using a face and body detection method. Next the pixels of each human video object are properly arranged and the chaotic cipher module is activated to encrypt them. Finally the background area is also encrypted using a different key. The system presents robustness against known cryptanalytic attacks and enables efficient rate control as semantic information is separated from the background.