A robust video object based watermarking scheme, based on Zernike and Hu moments, is proposed in this paper. Firstly, a human video object detector is applied to the initial image. Zernike and the Hu moments of each human video object are estimated and an invariant function for watermarking is incorporated. Then, the watermark is generated modifying the moment values of each human video object. In the detection scheme, a neural network classifier is initially used in order to extract possible watermarked human video objects from each received input image. Then, a watermark detection procedure is applied for video object authentication. A full experiment confirms the promising performance of the proposed scheme. Furthermore, the performances of the two types of moments are extensively investigated under several attacks, verifying the robustness of Zernike moments comparing to Hu moments.