In this paper, a segmentation-based coding technique as applied to video sequences is proposed. A proposed method separates an image into contour and texture parts, then the visually-sensitive contour part is represented by chain codes and the visually-insensitive texture part is reconstructed by a representative motion vector of a region and mean of the segmented frame difference. It uses a change detector to find moving areas and adopts variable blocks to represent different motions correctly. For better quality of reconstructed images, the displaced frame difference between the original image and the motion compensated image reconstructed by the representative motion vector is segmented. Computer simulation with several video sequences shows that the proposed method gives better performance than the conventional ones in terms of the peak signal to noise ratio(PSNR) and compression ration.