This paper proposes an efficient moving picture compression for video sequences with luminance variations. In the proposed algorithm, the luminance variation parameters are estimated and local motions are compensated. To detect the frame required luminance compensation, we employ the frame classification based on the cross entropy between histograms of two successive frames, which can reduce the computational redundancy. Simulation results show that the proposed method yields a higher peak signal to noise ratio (PSNR) than that of the conventional methods, with a low computational load, when the video scene contains large luminance variations.