A three dimensional-two spatical dimensions plus time-image segmentation is widely used in a very low bit rate image sequence coding because it can solve the region correspondence problem. Mathematical morphology is a very efficient tool for the segmentation because it deals well with geometric features such as size, shape, contrast and connectivity. But if the motion in the image sequence is large in time axis, the conventional 3D morphological segmentation algorithm have difficulty in solving region correspondence problem. To alleviate this problem, we propose the hierarchical image sequence segmentation algorithm that uses the region motion information. Since the motion of a region in previous level affects that in current level uses the previous motion information to increase region correspondence. Simulation result shows improved performance for sequence frames with large motion.